Responsible AI for Microsoft Foundry - What It Means in Practice

Whenever an Australian organisation engages us to build AI into something that customers or staff will actually use, the same question comes up in week two. Sometimes earlier. "What about responsible AI?" Sometimes the question comes from the CIO. More often it comes from legal, risk, or a board member who read something over the weekend.

The good news is that Microsoft Foundry (the platform formerly known as Azure AI Foundry, which evolved out of Azure AI Studio and Azure OpenAI) takes this seriously. The platform ships with a stack of responsible AI tooling that, used properly, gets you a long way. The honest news is that the platform doesn't do the work for you. Responsible AI is a practice, not a feature you toggle on.

I want to walk through what Microsoft means by responsible AI in the Foundry context, what the tooling actually does, and where we see Australian teams trip up.

The principles, briefly

Microsoft anchors its responsible AI work in six principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. They're sensible, broadly aligned with what Australian regulators are looking at, and not particularly controversial.

What matters in practice is that these aren't just policy statements. Each one maps to a set of engineering decisions you make when you build a Foundry application. Fairness means you've thought about what happens if the model treats different demographic groups differently. Reliability means you have a plan for when the model is wrong (because it will be). Privacy means you've decided what data goes in and what doesn't.

If you've ever worked through the Australian government's AI Ethics Framework or the NSW AI assessment, this maps fairly cleanly. The vocabulary is different but the intent is similar.

What Foundry gives you out of the box

The Foundry platform bundles a few responsible AI capabilities directly into the model deployment and orchestration experience. Worth knowing what they are before you start a project.

Content filters. Every model deployment in Foundry comes with content filtering for hate, sexual, violence, and self-harm categories. They run on both inputs and outputs. You can tune the severity threshold per category. For most enterprise use cases the defaults are sensible. For some regulated use cases (think mental health support, anything with vulnerable users) you'll want to tighten them and add custom blocklists.

Prompt shields. These detect prompt injection attempts, both direct (someone trying to override your system prompt) and indirect (a model reading a document that contains malicious instructions). The indirect case is the one most teams haven't thought about. If your AI reads emails or documents, you need this on.

Groundedness checks. These look at whether the model's output is actually supported by the data you gave it as context. It's not a perfect hallucination detector but it catches a meaningful share of cases. For retrieval augmented generation systems, which is most enterprise AI work we do, this is one of the more useful checks available.

Protected material detection. Flags whether the model is regurgitating copyrighted text or code. Niche but matters if you're building anything that produces longer text outputs.

Evaluation tools. Foundry has built-in evaluators for things like coherence, fluency, similarity, and groundedness, plus the ability to run safety evaluations against test datasets. The tooling has improved significantly over the last twelve months and is now genuinely useful for systematic testing.

Tracing and monitoring. You can log every call, every response, and the safety metrics for them, and analyse the data in Azure Monitor or export it to your own tools.

What the platform doesn't do

This is where we see teams get caught out. The Foundry stack handles a lot of mechanical safety checks. It does not handle:

Defining what 'responsible' means for your use case. A chatbot for an internal HR team has different risks than a customer-facing claims assistant for an insurer. The thresholds, the failure modes, and the human-in-the-loop decisions are different. Nobody at Microsoft can make these calls for you.

Stakeholder consultation. If your AI is going to make decisions that affect customers, you need to talk to the people who'll be affected, the people who'll deal with the complaints, and the people responsible for the business outcome. The platform doesn't do this.

Documentation. The Australian Privacy Act, the upcoming AI safety legislation, and various sector specific regulations expect you to document how the system was built, what data was used, what tests were run, and what risks were assessed. Foundry can generate some of this but most of it still has to be written.

Ongoing monitoring of real-world behaviour. The platform logs the calls. Someone still has to look at the logs, notice when behaviour has drifted, and decide what to do about it.

What we actually do on engagements

When we run an AI build that requires a proper responsible AI process (which is most of them now), the work breaks roughly into five chunks.

First, an impact assessment at the start. Who uses this? Who's affected by its decisions? What could go wrong, in what ways, and how bad would each failure be? We use a lightweight template that maps to the Microsoft impact assessment template but cut down for Australian contexts. This usually takes a couple of workshops and produces a document that becomes the anchor for everything else.

Second, threat modelling for the AI specifically. Prompt injection, data exfiltration, jailbreaks, sensitive data exposure. The Foundry content filters and prompt shields cover a good chunk but not all of it. We add custom filters for sector-specific concerns (PHI for healthcare, customer financial data for FS, and so on).

Third, an evaluation plan and dataset. Before you ship, you need to know how the system behaves on cases that matter. Real customer questions, edge cases, adversarial cases. We build evaluation datasets that get reused for regression testing. Foundry's evaluation tools make this much less painful than it used to be but the dataset still has to come from somewhere.

Fourth, a human review loop for high stakes outputs. For anything that affects a customer (a recommendation, a decision, an automated response), there's almost always a human in the loop somewhere. The design question is how to make their review efficient without making the system pointless. We usually build dashboards that surface low confidence or flagged outputs for review.

Fifth, monitoring after release. Cost monitoring, safety incident monitoring, drift monitoring, and a defined incident response process. The platform gives you the data. The process is on you.

Where Australian organisations get this wrong

A few patterns we see repeatedly.

Treating it as a tick box at the end. Responsible AI as a thing you do in the last sprint, where someone runs the safety evaluator once and signs off. This doesn't work. The decisions that determine whether your AI is responsible are made at the start, when you choose what to build and how it fits into a business process. Bolting on safety at the end is much more expensive and much less effective.

Trusting the content filters as the only line of defence. The default filters catch a lot but not everything. We've seen enterprise deployments where the only safety measure was the default Foundry filter, and the application would happily say things the organisation would be appalled by, because the categories the filter knew about didn't match what the organisation cared about.

Skipping the documentation. Australian regulators and auditors increasingly want to see the paperwork. If you're in financial services, health, education, or government, you'll be asked. Building the documentation as you go is much easier than retrofitting it later.

Not investing in evaluation. Teams that don't build proper evaluation datasets end up flying blind. Every model update, every prompt change, every new piece of context becomes a guess about whether the system got better or worse. The Foundry evaluators are right there. Use them.

A practical starting point

If you're early in an AI build on Foundry and want to do this properly without getting paralysed by frameworks, our short list is:

Do an impact assessment in week one. Even a one-page version. Know what you're building and what could go wrong.

Turn on prompt shields. The indirect injection case is real and growing.

Build an evaluation dataset with at least 50 realistic prompts before you go to production. More if the stakes are higher.

Wire up tracing to Azure Monitor or your SIEM. You'll want this data when something goes wrong.

Decide who reviews flagged outputs and how. Don't ship without this defined.

Write down what the system can and can't do. Put it where users can see it.

None of this is exotic. None of it requires specialist data science skills. It just requires treating responsible AI as part of the build, not a separate thing.

If you'd like a hand designing the impact assessment, building the evaluation pipeline, or just getting a sober view of what your AI system actually does versus what the marketing slides say, our Azure AI Foundry consultants and broader AI strategy team work on exactly this. We'd rather help you get it right at the start than be the people you call when something goes wrong six months in.

Reference: Responsible AI for Microsoft Foundry