What Is Azure AI Foundry and When Should You Use It
Eighteen months on from the Foundry rebrand, the questions we get from Australian business leaders have shifted. It used to be "what even is this thing?" Now it's "we know we need a platform, but does Foundry actually fit our shop, or is it overkill?" That is the right question, and this post is the answer I usually give before we ever open up the portal.
Azure AI Foundry is Microsoft's umbrella for building, deploying, and operating AI applications on Azure. It pulls together what used to live across Azure OpenAI Service, Azure Machine Learning, Azure AI Search, the old Cognitive Services catalogue, and a model-evaluation layer that didn't really exist before. If your team has been juggling four or five different Azure portals to ship one AI feature, that pain is exactly what Foundry was built to solve.
That said, it is not the right answer for every project. About one in three engagements we start with Foundry on the table, we end up steering the client somewhere lighter.
What Foundry actually gives you
I'll keep this concrete. When you log in to Foundry, you get four things that matter.
A model catalogue. GPT family models from OpenAI, the Claude family from Anthropic, Llama, Mistral, plus the small Phi models from Microsoft itself. You can run them as serverless endpoints, deploy them to dedicated capacity, or in the case of the open weights options, host them on your own compute inside Foundry. The serverless pricing is the easiest to start with, and for most Australian mid-market clients it is also where they stay.
A project workspace. Each project bundles your models, your data connections, your prompts, your evaluation results, your agents, and your deployed endpoints into one logical unit. RBAC, networking, content safety policies, and budget alerts attach at the project or hub level. This is the bit that actually matters for enterprise governance, and it is genuinely better than the chaos of the pre-Foundry world.
An agent builder. Foundry Agent Service is now generally available in Australia East and Australia Southeast. You can define an agent with a system prompt, give it tools (functions, OpenAPI specs, code interpreter, file search across grounded data), and deploy it behind an endpoint. Integration with Microsoft 365 Copilot and Copilot Studio is reasonable, though the wiring is fiddly the first time.
Evaluation and observability. You can run automated evaluations against ground-truth datasets, log prompt traces, look at token usage, and chase down where a bad response came from. This is the area Foundry has improved the most in 2026, and it is the reason we now recommend it for any project that needs to pass internal audit.
Where Foundry is genuinely the right call
We deploy Foundry confidently in a few situations.
You are already deep in Microsoft. If your data sits in Microsoft Fabric, your users live in Microsoft 365, and your platform team runs Azure, Foundry removes integration friction. You can ground an agent on a SharePoint library, push usage metrics into Fabric, and let an admin manage everything from one place. That alone is worth the platform tax.
You need to put guardrails on multiple AI workloads. A bank we work with had eleven separate AI proof-of-concepts running across business units, all on different stacks. Foundry gave them a single place to enforce content filters, prompt logging, and PII redaction policies. None of that was glamorous, but it got them through their internal risk review in weeks rather than months.
You are building agents that need tool calling, retrieval, and evaluation. If your project is "wire an LLM to your data and let users ask questions," and you also need to prove the answers stay accurate over time, Foundry's evaluation pipeline saves real engineering effort. Rolling your own version of this is doable, but it is several months of work that you do not need to do.
You want a serverless start with a path to scale. The pay-as-you-go endpoints let you build a prototype on a low spend, and the same code path keeps working when you move to provisioned throughput. We had a logistics client go from a $400 a month proof-of-concept to a $40,000 a month production workload without changing the application code.
Where Foundry is overkill or the wrong fit
Foundry is a platform. Platforms cost money even when you are not using them heavily, in the sense of cognitive load and operational overhead. A few situations where we steer clients elsewhere.
You only need one thing, like a transcription pipeline. If your whole AI requirement is "transcribe support calls and run sentiment analysis on them," the standalone Azure AI Speech and Azure AI Language services are cheaper, simpler, and faster to ship. You do not need a Foundry project to call a REST API.
Your data lives outside Azure and is staying there. We have AWS-native clients and GCP-native clients. Foundry can call out to external data, but the value of the platform drops sharply once you are not benefiting from native Azure integration. In those shops, a direct OpenAI API integration or AWS Bedrock often wins.
You want a fully customised training pipeline. Foundry includes Azure Machine Learning under the hood, but if your team is doing serious model training, custom MLOps, and bespoke infrastructure, you might find the Foundry abstraction layer gets in your way. Drop down to AML directly.
Budget is tight and the use case is bounded. For a small business that wants a chatbot on their website, Foundry is rarely the right starting point. Copilot Studio or even a direct API call to Azure OpenAI is faster to ship and cheaper to run.
A rough pricing picture in AUD
Foundry itself does not have a separate platform fee. You pay for the underlying compute, storage, and model inference. Approximate ranges we see in 2026 (these change, so treat as a guide rather than a quote):
| Workload | Typical monthly spend (AUD) |
|---|---|
| Pilot agent on serverless GPT-4o-mini, low traffic | $300 to $1,500 |
| Production agent on GPT-4.1 with retrieval, mid-volume | $4,000 to $15,000 |
| Heavy retrieval workload with reranking and evaluation | $10,000 to $40,000 |
| Custom fine-tuned model on provisioned throughput | $25,000 to $120,000+ |
| Internal Copilot for 1,000 employees | $15,000 to $35,000 |
Add on top: storage in Azure AI Search or Blob (usually $200 to $2,000 a month), networking, plus engineering time to keep it healthy. Most production Foundry workloads need at least a fractional engineer keeping an eye on them.
If you are getting quotes that are wildly outside these ranges in either direction, ask hard questions about what is included.
A decision framework before you commit
Here is the checklist I run through with clients before recommending Foundry.
- Do you have at least two AI workloads either live or in the next twelve-month roadmap? If only one, consider going direct to the underlying service.
- Do you need centralised governance, prompt logging, or evaluation? If yes, Foundry pays for itself in audit prep alone.
- Is your data already in Azure, or will it be? If it is sitting in AWS S3 and staying there, the integration win shrinks.
- Will you build agents with tool calling and retrieval, or just one-shot prompt responses? Foundry's strength is the agent and evaluation story.
- Does your team know Python and Azure, or are they Power Platform users? Foundry sits at the developer end. Power Platform users may be better served by Copilot Studio with Foundry sitting underneath.
- Are you comfortable with a 2 to 4 week setup phase before any business value lands? That is the realistic timeline for a properly configured project.
If you answer yes to four or more of these, Foundry is probably right. If you answer yes to two or fewer, you are likely better off with a lighter touch.
Common misconceptions we keep hearing
"Foundry is just a rebrand of Azure OpenAI." It includes Azure OpenAI but is much broader. The agent service, model catalogue, and evaluation tooling are genuine additions.
"We need to migrate everything to Foundry now." No, you don't. Existing Azure OpenAI deployments keep working. Move when there is a reason, not because Microsoft renamed something.
"It will replace our data science team." It absolutely won't. Foundry makes it faster to ship AI applications. It does not replace the people who understand your data, the people who decide what to build, or the people who keep it honest in production.
"We can build it ourselves cheaper." Sometimes true for a single workload. Almost never true once you are running three or more AI applications with any sort of governance requirement.
What an engagement with us usually looks like
When clients come to us about Foundry, we typically start with a two-week assessment. We look at the existing data estate, the current AI use cases (real ones and wishful ones), the team's skills, and the budget envelope. We come back with a recommendation that is either Foundry, a lighter touch, or a hybrid. About 60% of the time it ends up being Foundry, but the upfront honesty about when it is not the right tool has built a lot of long-term client relationships.
If Foundry is the right call, we then move into a build phase that usually runs 6 to 12 weeks for a first production workload, including the governance setup that pays dividends across every workload after it.
We have shipped Foundry projects across financial services, healthcare, mortgage broking, logistics, and government in Australia. The pattern that holds across all of them: the platform investment is worth it when there are multiple workloads to amortise across, and the early discipline on governance saves you from painful retrofits later.
Where to go from here
If you are evaluating Azure AI Foundry for your business, the next step is honest scoping. Our team works with Australian organisations to assess fit, plan implementation, and run the build. Have a look at our Azure AI Foundry consultants page for the specifics of how we engage, or read about our broader Microsoft AI consulting work if you want to see the full Azure picture.
If you want to talk through whether Foundry is the right fit for your specific situation, you can get in touch with us directly. We will tell you honestly if it is the wrong tool, and we will point you somewhere better if it is. That is how we have always run it, and it is why our clients tend to stick around.
For broader strategy work before you commit to a platform, our AI strategy consultants practice runs short, sharp scoping engagements that often save organisations from picking the wrong tool in the first place. And if you already know you want help shipping production agents on top of Foundry, our enterprise AI agents team has shipped more of these than anyone else we know of in the Australian market.