Azure AI Foundry for Enterprise - Governance and Security Considerations

Most Azure AI Foundry projects don't die because the model performs badly. They die because someone on the risk committee asks a question the project team can't answer. "Where does the prompt data sit? Who can see it? What happens if a customer asks us to delete it? How do we prove the model didn't discriminate against this loan applicant?"

If you're at the stage of evaluating Foundry for an enterprise rollout - not a pilot, an actual production system used by staff or customers - this is the post I wish I'd written eighteen months ago. We've shipped Foundry workloads for Australian banks, insurers, healthcare providers and a couple of government departments, and the security and governance conversation looks very different from the one you have during a POC.

This isn't a feature list. Microsoft's docs already do that job. This is what we tell clients to actually focus on before they spend serious money.

Why governance is a buying decision, not a project task

There's a common pattern we see. The technology team gets excited about Foundry, runs a six-week POC, builds something genuinely useful, and then the project sits in approval limbo for four months while legal, privacy, security and compliance work out whether the thing can go live.

The fix is to put governance into the buying decision, not the post-build approval. Before you commit to Foundry as your enterprise AI platform, you need confidence on five things:

1. Where your data goes and who can read it
2. How identity and access work across projects, models and tools
3. How you'll evaluate and prove model behaviour
4. How content filtering and abuse monitoring fit your obligations under Australian privacy law and APRA CPS 230
5. How you'll handle the audit trail when something goes wrong

If any of those are unclear, the platform decision isn't finished yet. We've seen organisations choose Foundry, get partway through, and then discover their data residency requirements meant they needed to redesign half the architecture. That's an expensive lesson.

Data residency and the Australia question

Foundry runs on Azure, which means the standard Australian regions (Australia East and Australia Southeast) are available. The detail that catches people out is which models are actually deployed in those regions. Not every model in the Foundry catalogue is available in every Australian region at every point in time. The big GPT models usually are. Some of the specialist or newer models may only be in US regions when they first launch.

For an enterprise with hard data residency requirements - and that's most Australian banks, government departments and healthcare providers - you need to:

• Lock your Foundry projects to Australian regions during creation
• Disable cross-region failover where you can't tolerate data leaving the country
• Use the data zone deployment options when you need broader model access but want to keep traffic within the Australia data zone
• Get written confirmation from Microsoft (via your account team) about which specific models are committed to Australian regions and which are best-effort

The Australia data zone is a useful middle ground. It guarantees inference data stays within Australian Azure infrastructure even when you're using models that span the broader region. For most regulated industries that's enough. For a few - particularly government and certain healthcare workloads - you'll want strict single-region deployment.

One thing worth knowing: the prompts and completions are not used to train Microsoft's models. That's contractually clear. But your monitoring data, the abuse detection logs, the content filter results - those have specific retention and access policies you need to read. We've had two clients want abuse monitoring data shortened from the default retention period because of internal data minimisation policies. It's configurable, but you have to ask.

Identity, access and the role model

The single biggest mistake we see in Foundry deployments is treating it like a flat platform. Foundry has a proper RBAC model with project-level scoping, and you should use it from day one.

Here's the access model that actually works at enterprise scale:

Role	Who	What they can do
Subscription owner	Cloud platform team	Manages Foundry resource provisioning, networking, billing
Foundry Hub admin	AI platform team	Creates projects, manages shared resources, sets policies
Project admin	Project lead	Manages a single project, deploys models, configures evaluations
AI developer	Builders	Creates and tests applications, no production deploy rights
Data scientist	Model team	Trains and fine-tunes, runs evaluations
Reader	Auditors, risk team	Views configurations and evaluations, no changes

The point isn't that you need exactly these roles. The point is that you need a role model that maps to your organisation's separation of duties. The default "everyone is a contributor" approach falls apart the moment auditors get involved.

For authentication, Entra ID (formerly Azure AD) does the heavy lifting. Use managed identities everywhere you can - for the apps that call Foundry, for the agents that pull from data sources, for the connections to storage and search. Avoid API keys for production workloads. They get leaked, they get committed to repos, and they're harder to rotate cleanly. We've gone into client environments and found Foundry API keys sitting in a Power Automate flow with three years of staff turnover. That's a finding.

If you're working through this in detail, our Microsoft AI Consultants team has a standard access design we adapt to each client's existing Entra structure.

Content filtering and the responsible AI question

Foundry ships with content filters. They're on by default. Most clients leave them on default settings and don't think about them again. That's fine for general business assistants. It's not fine for regulated industries or anything customer-facing.

The four content categories Foundry filters - hate, violence, sexual, self-harm - are configurable per project and per deployment. You can dial them up or down on input and output independently. There's also a protected material filter, a jailbreak detection layer, and groundedness checks if you're using retrieval.

Where this gets interesting for Australian enterprises:

• Healthcare - default filters can refuse to discuss self-harm or sensitive medical topics. If your app is meant to help clinicians or patients, you need to configure annotations that let the model engage appropriately, with strict logging
• Financial services - jailbreak detection is critical when you're exposing an assistant to customers who might try to extract confidential information or push the model into giving financial advice it shouldn't give
• Government and legal - the protected material filter and groundedness checks matter more here because you need to prove provenance for anything the AI produces

The thing risk committees want to see is not "we have content filters." They want to see "we have a documented threat model, we know which filters address which threats, we have evidence the filters work, and we have monitoring when they fail." That's a lot more work than ticking a box.

This is the area where our Azure AI Foundry consultants spend a disproportionate amount of project time, because getting it right is hard and getting it wrong is visible.

Network isolation and the private endpoint conversation

For an enterprise deployment, public network access to Foundry resources is a hard no for most security teams. You'll be using private endpoints.

The standard pattern:

• Foundry Hub on a private endpoint inside your vNet
• Project workspaces on private endpoints
• Storage accounts (for prompt flow, evaluations, fine-tuning data) on private endpoints
• Azure AI Search (if you're doing retrieval) on private endpoints
• Egress to model endpoints via private link

This works, but the network design is genuinely fiddly. We've seen projects get stuck for weeks because someone enabled "disable public network access" before configuring all the private endpoints, and suddenly nobody could connect to anything. The order of operations matters.

A pragmatic suggestion: build your reference network architecture for Foundry once, document it, and reuse it for every project. Don't let each project team reinvent the network design. The cost saving and consistency benefit is significant.

Auditing, evaluation and the "prove it" problem

Risk committees and auditors will ask three questions you need answers for:

Question 1: What is the model doing? Foundry logs every prompt and completion to your storage account when you enable diagnostics. You should always enable diagnostics. Always. The default is off, and that's the wrong default for an enterprise. Pipe the logs into Azure Monitor and your SIEM.

Question 2: How do you know the model is good enough? This is where Foundry's evaluation framework matters. You can run evaluations against test sets - groundedness, relevance, coherence, safety - and track scores over time. The discipline of running these evaluations before every model or prompt change is what separates real enterprise AI from "we vibed it into production." If you're not doing this, you don't have a defensible system.

Question 3: What happens when it gets it wrong? You need a feedback loop. Users should be able to flag bad outputs. Those flags should go somewhere that humans actually look at. Patterns should drive prompt or model updates. We help clients build this loop because most internal teams underestimate how important it is until the first incident.

Where Foundry is the right fit, and where it isn't

After all of that, here's our honest take on when to choose Foundry for an enterprise rollout.

Foundry is a strong fit if:

• You're already a Microsoft shop (Entra, Azure, Microsoft 365)
• You have regulated data and need the data residency, content filtering and audit story to stand up
• You want to centralise model access across multiple business units
• You're going to use a mix of OpenAI, Microsoft and open-source models
• You need fine-tuning and evaluation as part of your delivery model

Foundry is overkill or the wrong fit if:

• You're running a single small use case and don't need centralised governance
• You're building a consumer product where every millisecond and dollar matters
• Your team has deep ML engineering capability and prefers to build on raw infrastructure
• You're committed to an AWS or GCP environment and have no reason to bring Azure in

For most Australian mid-market and enterprise clients, Foundry is the right answer. The governance story is genuinely strong, the integration with the rest of the Microsoft stack saves enormous amounts of work, and the platform is mature enough to put in front of regulators.

A practical pre-deployment checklist

Before you go live with any Foundry workload in production, run through this list with your security and risk people:

• Region and data zone configuration documented and approved
• RBAC model designed and applied at hub and project level
• Managed identities used for all service-to-service auth
• Content filter configuration matched to use case, with rationale documented
• Private endpoints configured for all Foundry resources and dependencies
• Diagnostic logging enabled and piped to SIEM
• Evaluation framework set up with test sets and baseline scores
• User feedback mechanism designed and built
• Incident response runbook for AI-specific incidents
• Data retention policy reviewed for monitoring and abuse data
• Privacy Impact Assessment completed (most Australian regulated industries need this)
• Sign-off from privacy, security and risk obtained in writing

This isn't a comprehensive list, but if you can tick all of these, you're ahead of about 80% of the AI deployments we audit.

Getting started

Most clients we work with start by getting a clear-eyed picture of what good looks like, then back-fit it to their existing controls. If you're at the stage of planning an enterprise Foundry deployment, the governance work is the work. The model deployment is the easy bit.

Team 400 specialises in Azure AI Foundry for Australian enterprises with real compliance requirements. We do the architecture, the governance design, the production deployment and the ongoing support through our managed services. If you want a second opinion on your Foundry strategy, or you need help putting it together properly the first time, get in touch through our contact page. Our AI strategy consultants can also help with the broader governance framework if you're starting from scratch.

The technology side of AI is increasingly straightforward. The governance side is where most enterprises win or lose. Get this right and you'll ship; get it wrong and you'll be the project that never made it past the risk committee.