Signs You Need a Microsoft AI Consultant Not Just an IT Vendor
We get a lot of inbound that starts the same way. "We asked our IT vendor to build us a Copilot agent / Azure AI Foundry app / Power Platform AI workflow. Six months in, it's not working. Can you take a look?"
The honest answer is usually yes, but only because the original vendor was never the right shop for the job. They're competent at what they do. AI just isn't it.
This post is for the person sitting on a stalled Microsoft AI project, or the one about to commission one and trying to work out who to call. I'll be specific about the signs that mean you've outgrown a general IT vendor and need an actual AI consultant, and I'll be honest about the cost of getting it wrong. The goal isn't to tell you to fire your existing partner. Most of the time the right answer is to keep them for what they're good at and bring in specialist help for the parts they aren't.
The vendor types we keep running into
Before the signs, a quick map of who's pitching for your Microsoft AI work in Australia in 2026. The labels matter because they predict what you'll get.
Managed services providers (MSPs). Great at running your tenant, your endpoints, your licences. They'll sell you Copilot licences and help you roll out M365 Copilot at the organisation level. They're not engineering shops. If your AI need stops at "enable Copilot for our staff and run some training," they're fine.
Traditional Microsoft Gold/Solutions Partners. These are the firms that built your SharePoint intranet, your Dynamics implementation, your Power BI reporting. Strong at the Power Platform low-code layer. Most are weak at custom AI engineering, prompt design, evaluation, and the Azure AI Foundry stack. They will absolutely take the work if you offer it.
System integrators. The big ones. They have AI practices on paper. In practice you get whoever's on the bench, which in 2026 means a lot of people who finished a Microsoft Learn module last quarter.
AI-first consultancies. Smaller shops where AI is the only thing they do. They tend to be expensive per day but cheap per outcome because they've shipped the pattern you're trying to build many times. Team 400 is one of these. So are a handful of others.
The signs below help you work out which of these you actually need.
Sign 1 - Your vendor is treating the AI as a feature, not a system
When an IT vendor scopes a Microsoft AI project, they tend to write a statement of work that looks like a normal software project. Requirements, designs, build, test, deploy.
AI projects don't work that way. The model is non-deterministic. The data is messy. The user behaviour you'll see in production is different from what you imagined. A real AI consultant scopes for iteration, builds an evaluation pipeline before they build the feature, and assumes the first three versions will be wrong.
If your statement of work has a single "AI build" line item with a fixed price and no mention of evals, prompt iteration, or red-teaming, you're working with someone who thinks this is a SharePoint project. It isn't.
What this costs you when it goes wrong: usually about 60% of the budget is consumed before anyone realises the output quality isn't what was promised, because there was no evaluation framework to catch it earlier. Rebuild from there typically costs another 80-120% of the original budget.
Sign 2 - Nobody can answer "how do you know it's working?"
Ask your vendor this question directly. "How will you measure that the AI is producing good answers in production?"
A good answer involves an evaluation dataset, automated scoring, human review samples, and feedback loops from real users. A bad answer is "we'll do user acceptance testing" or "the users will tell us."
User acceptance testing for an AI feature is not the same as UAT for a CRM module. The model can pass UAT by getting the easy 80% of cases right and then fail in production on the 20% that actually matter to your business. We've cleaned up several Copilot Studio deployments where this exact thing happened. The bot demoed beautifully and then quietly gave wrong answers to about a third of real customer queries for four months before anyone noticed.
If your vendor doesn't have a serious answer to this question, they're not building an AI system. They're building a demo that runs in production.
Sign 3 - The recommendation is always "more Microsoft licences"
I have nothing against Microsoft. Most of our work is on the Microsoft stack. But there's a specific failure mode where an MSP's answer to every problem is "buy more Copilot licences" because that's what their commission structure rewards.
Some examples of when this goes badly:
- You need a customer-facing agent. Copilot Studio published to the public web is rarely the right answer at enterprise scale. You usually want Azure AI Foundry with a custom front end.
- You need an internal AI assistant that searches your company data. M365 Copilot does this for some content. For anything that isn't indexed by Graph, you need a real RAG implementation, not another licence.
- You need to automate a complex workflow. Power Automate with an AI Builder action is fine for the simple cases. For anything multi-step with branching logic and tool calls, you want a proper agent framework like Microsoft AI Agent Framework or LangChain.
When the same product is the answer to every question, the person answering isn't engineering. They're selling.
Sign 4 - There's no one on the team who's actually built an AI product before
This sounds obvious. It isn't, because vendors are very good at presenting AI experience that isn't really AI experience.
Things that aren't AI experience:
- Power BI dashboards using "AI insights" features
- Configuring M365 Copilot for tenants
- Running Microsoft Learn paths on AI
- A demo that calls the OpenAI API once
- Doing a hackathon project last year
Things that are AI experience:
- Shipping a RAG system to production with users
- Building and maintaining evaluation pipelines for model quality
- Designing prompts that hold up over thousands of calls
- Wiring up agent frameworks with tool use, memory, and guardrails
- Surviving the move from one model version to the next without breaking the product
Ask to speak to the engineer who'll actually be doing the work. Ask them what they shipped last quarter. If the answer is vague or the person is suddenly unavailable, you have your answer.
Sign 5 - Your project keeps stalling at "the data isn't ready"
This one is a bit unfair to vendors because the data usually genuinely isn't ready. But there's a difference between a vendor who treats data readiness as an unsolvable obstacle and a consultant who treats it as the first phase of work.
An AI consultant scopes data readiness up front. They look at what's in SharePoint, Dataverse, your line-of-business systems, your Fabric workspace. They know what RAG performs well on (clean, chunked, well-titled documents) and what it performs badly on (scanned PDFs from 2007, contradictory documents, content with implicit assumptions). They will tell you which use cases are realistic with your current data and which need data work first.
An IT vendor will start the build and then surface the data problem in month three when it's already cost you a hundred grand.
If your vendor hasn't asked detailed questions about your data sources, content quality, and metadata structure before quoting, that's a sign.
Sign 6 - You're being sold "the platform" instead of "the outcome"
This is the most expensive failure mode in Microsoft AI work right now.
Several vendors are running playbooks where they sell you the foundational work to "AI-enable" your tenant. Fabric standup, Purview rollout, Copilot governance, identity model, tagging strategy. All of this is real work that you do eventually need. None of it produces a business outcome on its own.
We've seen organisations spend $400,000 to $800,000 AUD on foundational platform work and have zero AI features in production at the end of it. The vendor delivers exactly what was scoped. The CFO looks at the bill and asks what they got for it. Nobody has a good answer.
An AI consultant works backwards from a specific business outcome. What will the users do differently in six months because of this work? If your vendor can't answer that question in one sentence, you're paying for platform, not product.
A decision framework
Use this if you're trying to work out whether you need to bring in a Microsoft AI consultant or stay with your existing partner.
| Situation | Stay with current IT vendor | Bring in an AI consultant |
|---|---|---|
| Rolling out M365 Copilot at the organisation level | Yes, this is their wheelhouse | Only for change/training |
| Standing up Fabric for analytics | Usually yes | If AI is the goal, yes |
| Building a customer-facing AI agent | No | Yes |
| Building internal RAG over company documents | Only for the simplest cases | Yes for anything serious |
| Multi-step agent automating a business process | Rarely | Yes |
| Stalled Copilot Studio project that "isn't quite working" | No | Yes |
| Project scoped without evaluation framework | No | Yes |
| Project where data quality hasn't been assessed | No | Yes first, build second |
The pattern is straightforward. Platform work and configuration usually belongs with your existing partner. AI engineering belongs with a specialist.
What it actually costs to switch
If you're reading this with an existing stalled project, the honest news is the rescue isn't free. We typically run a paid two-week diagnostic ($25,000 to $40,000 AUD) before recommending whether to rebuild, refactor, or kill the project. That's substantially cheaper than burning another two quarters and finding out for free.
A fresh Microsoft AI engagement with a specialist consultant runs in the same ballpark as your existing vendor's pricing. Day rates for senior AI engineers in Australia in 2026 sit at around $1,800 to $2,500 AUD. A first production use case typically lands between $120,000 and $300,000 AUD. The difference isn't the price per day. It's whether the work ships.
Our Microsoft AI consultants page has more detail on engagement models. Our Azure AI Foundry consultants and Copilot Studio consultants pages cover the specific platforms.
What to do if you're seeing these signs
If three or more of the signs above describe your project, the kindest thing to do for everyone (including your current vendor) is to have an honest conversation about scope. Most vendors will admit they're not the right shop for a piece of work if you ask directly. We've had several engagements that started with a current vendor sitting in the room and saying out loud that they should hand the AI build to us and keep doing the M365 admin work they're actually good at. That's a healthy outcome.
If you're earlier and just trying to scope the project, get a second opinion before you sign the statement of work. A one-hour call with a specialist will tell you whether the vendor's approach is sound. It will also tell you whether the scope is realistic, which is more useful than the price comparison most people focus on.
If you want that second opinion from us, get in touch. We do diagnostic calls without obligation, and if the work is genuinely the right fit for your current vendor we'll say so. We have no interest in taking on rescue projects we can't actually rescue.