The Azure AI Services Security Baseline - What It Covers and Where You Still Have Work to Do
There is a moment in almost every AI project where someone from security or risk asks the question that stops the room: "so where does the data go, and who can get at it?" If you are building on Azure AI Services, that is a fair question and you should have a good answer ready. The trouble is that a lot of teams stand up a Cognitive Services or Azure OpenAI resource, get a demo working, and never go back to lock it down properly. Then the security review happens, and suddenly the fun stops.
Microsoft publishes a security baseline for Azure AI Services that maps the service against the Microsoft cloud security benchmark. It is a solid document and if you are the person actually configuring these resources, you should read it end to end. What I want to do here is give you the consultant's version: what the baseline is really telling you to do, which controls carry the most weight, and the traps we keep watching Australian teams fall into when they go from proof of concept to something that handles real data.
What the baseline is, and what it is not
The security baseline is a checklist of controls, grouped into areas like network security, identity management, data protection, logging and threat detection, and so on. Each control tells you whether the feature is supported by Azure AI Services, whether Microsoft enables it by default, and what your responsibility is. That last column is the one that matters. A control being "supported" does not mean it is on. Most of the important ones are yours to configure.
This is the bit people miss. The baseline is not a description of how secure your resource is out of the box. It is a description of how secure you could make it if you did the work. A fresh Azure AI Services resource with default settings is convenient and open in ways that are fine for a sandbox and genuinely dangerous for production. The baseline is the map from one state to the other.
The controls that actually matter
If I had to pick the handful of things that move the needle most, here is where I would start.
Get the resource off the public internet. By default an Azure AI Services endpoint is reachable over a public endpoint with a key. Keys are convenient and they are also the thing that ends up pasted into a notebook, committed to a repo, or shared in a Teams message. The baseline points you at private endpoints and network restrictions, and this is the single biggest improvement most teams can make. Lock the resource to a virtual network, use Azure Private Link so traffic never touches the public internet, and set the firewall to deny by default. We have reviewed environments where a perfectly good private network design existed on paper while the AI resource itself sat wide open with a public endpoint. The attacker does not care about your network diagram.
Use managed identities instead of keys. This follows straight on. The baseline pushes you towards Microsoft Entra ID authentication and managed identities, and you should take that push. When your application authenticates with a managed identity, there is no key to leak, no secret to rotate, and no credential sitting in a config file. If you must use keys somewhere, at least keep them in Azure Key Vault and rotate them, but the goal is to stop using them at all. I have lost count of the incidents that trace back to a key that should never have existed.
Turn on diagnostic logging and send it somewhere useful. Logging is off by default, and a resource with no logs is a resource you cannot investigate when something goes wrong. Send diagnostic logs to a Log Analytics workspace, keep them for a sensible retention period, and actually build a couple of alerts on top of them. When your security team asks "who called this model and when", you want an answer that is not a shrug. This is also where a lot of the compliance value sits, which matters a great deal for anyone working under the Privacy Act or the Notifiable Data Breaches scheme.
Encrypt with customer-managed keys if your data warrants it. Azure encrypts data at rest by default, which is fine for a lot of use cases. If you are in a regulated sector or you have a contractual requirement to control your own encryption keys, the baseline covers customer-managed keys through Key Vault. Do not reach for this unless you need it, because it adds operational overhead, but know it is there.
The Australian angle
For Australian organisations this is not an abstract security exercise. If you are processing personal information through an AI service, you are on the hook under the Privacy Act, and a leak can turn into a notifiable breach with public disclosure and the reputational damage that comes with it. Sectors like health, financial services, and government carry extra layers on top.
The baseline helps you on two fronts. It reduces the chance of an incident, and it gives you evidence that you had reasonable controls in place and were monitoring. When something does go wrong, being able to show a regulator that you had private networking, identity-based access, and logging switched on is a completely different conversation to admitting you ran everything on a public endpoint with a shared key. We do a fair bit of this work with clients through our Azure AI consulting service, and the pattern is consistent: the technical controls are the easy part once someone decides they matter.
Data residency is the other question that comes up in almost every Australian engagement. Where is the model actually running, and where does the data get processed? For a lot of Azure AI Services you can pin the deployment to an Australian region, which keeps data in-country and makes the residency conversation with your compliance team much shorter. Check this early, because retrofitting a region change after you have built everything is painful.
Where teams keep going wrong
The most common failure is treating the security baseline as a one-time checklist for go-live and never looking at it again. Cloud environments drift. Someone spins up a new resource for a quick experiment, it never gets the hardening, and six months later it is quietly in production holding customer data with none of the controls. The baseline is only useful if you turn it into policy that is enforced continuously, which is exactly what Azure Policy is for, and worth pairing this reading with.
The second failure is the false comfort of a private endpoint with sloppy identity. Locking down the network feels like the big win, and it is important, but if your access model is still a shared key that half the team has, you have moved the risk rather than removed it. Network and identity have to move together.
The third is logging that goes nowhere. Plenty of teams enable diagnostic settings, point them at a workspace, and never build a single alert or run a single query. Logs you never look at are not security, they are storage costs. Decide up front what you want to be told about, and wire up the alerts.
How we approach it
When we harden an Azure AI Services deployment for a client, the order is roughly: get it off the public internet with private endpoints, switch authentication to managed identities and Entra ID, turn on diagnostic logging into a workspace we will actually watch, confirm the deployment region matches the data residency requirement, and only then look at customer-managed keys if the risk profile calls for it. None of it is exotic. It is disciplined application of controls that Microsoft has already documented, done in the right order so you are not blocking your own team while you tighten things.
The teams that get this right treat security as part of the build, not a gate at the end. They bake the baseline into how they provision resources so a new deployment is hardened by default rather than by remembering. If you are further along than you would like and need someone to come in, review what you have, and get it to a state you can defend in an audit, that is bread and butter for our Microsoft AI consultants. It is also the kind of thing we build into any custom AI work from the start, because retrofitting security is always more expensive than doing it as you go.
The bottom line
The Azure AI Services security baseline is a genuinely good piece of documentation and the controls it describes are the right ones. But it describes what is possible, not what is switched on. A default resource is convenient and exposed. Getting it to production-grade means private networking, identity-based access, logging you actually use, and a region that keeps your data where it needs to be. None of it is hard. It just has to be done, and then kept done as your environment grows.
If you have Azure AI in production and you are not confident it would survive a security review, have a chat with us and we will give you a straight assessment of where you stand and what to fix first.
Reference: Azure security baseline for Azure AI Services, Microsoft Learn.