Using Azure Container Registry for AI Services - A Get Started Guide

There is a moment in a lot of AI projects where the conversation stops being about models and starts being about plumbing. You have got a working model, maybe a containerised Azure AI service, maybe a custom inference image your team built, and now someone asks the unglamorous question: where does this container actually live, and how does it get to production safely? Nine times out of ten the answer involves Azure Container Registry, and nine times out of ten nobody on the team has thought about it until that exact moment.

That is a shame, because getting your container registry right early saves a surprising amount of grief later. This is the piece of infrastructure that sits quietly between "it works on my machine" and "it runs in production for real users." Get it wrong and you get mystery deployment failures, security review nightmares, and that special kind of 4pm-on-a-Friday panic when nobody can work out why the AI service container will not pull.

So let me walk through what Azure Container Registry is, why it matters specifically for AI workloads, and how to get started with it using the Azure CLI. This draws on the Microsoft Azure Container Registry get started with the Azure CLI documentation, plus the lessons we have collected running these deployments for Australian clients.

What it is, without the jargon

Azure Container Registry, usually shortened to ACR, is a private place to store and manage container images. If you have used Docker Hub, it is the same idea, except private to your organisation and sitting inside your Azure tenant where your security and networking rules apply.

A container image is a packaged-up bundle of your application and everything it needs to run. For AI work, this matters more than people expect. Several Azure AI services ship as containers you can run yourself - think certain Vision, Language, and Speech capabilities that you can pull down and run on your own infrastructure for data residency or latency reasons. And if your team builds custom models, you are almost certainly packaging the inference code and dependencies as a container too. All of those images need somewhere trusted to live. That somewhere is ACR.

Why not just use a public registry? Because your AI containers often embed proprietary code, custom models, or configuration you do not want on the public internet. A private registry keeps them inside your security boundary, lets you control exactly who can pull them, and gives you the audit trail your risk team will eventually ask for. For any Australian organisation with data residency obligations, this is not optional. It is the baseline.

Why AI workloads care about this more than most

Container registries are general infrastructure, but AI workloads lean on them harder for a few specific reasons.

The images are big. AI containers frequently bundle model weights, GPU libraries, and chunky frameworks. We routinely see images in the multiple-gigabyte range. That makes where the registry lives and how fast it serves images a real performance question, not a footnote. Pulling a 6GB image across regions every time a pod restarts is a genuine cost and latency problem.

You version models like you version code. When you retrain a model, you produce a new image. Good registry hygiene - sensible tags, a clear story about which version is in production - becomes part of your model governance, not just your DevOps. The registry quietly becomes the record of which model served which prediction, which matters enormously the day someone audits a decision your AI made.

And the supply chain is a security surface. An AI container that has been tampered with is a serious problem, because the model inside it makes decisions that affect real people. A private registry with proper access controls and image scanning is part of how you keep that supply chain honest. This is the sort of thing our Azure AI consulting work bakes in from the start rather than retrofitting after a security review goes badly.

Getting started with the Azure CLI

The Azure CLI is the most reliable way to set this up, because it is scriptable and repeatable. Clicking around the portal is fine for learning, but you will want this in a script eventually so your environments stay consistent. Here is the shape of it.

First, make sure you are logged in and pointed at the right subscription. This sounds obvious. It is also the single most common reason a setup script does the right thing in the wrong place.

az login
az account set --subscription "Your Subscription Name"

Create a resource group to hold the registry, assuming you do not already have one you want to use. Pick an Australian region for data residency.

az group create --name ai-registry-rg --location australiaeast

Now create the registry itself. The name has to be globally unique and alphanumeric, and the SKU determines your features and throughput.

az acr create --resource-group ai-registry-rg \
  --name myaiteamregistry --sku Standard

That SKU choice matters more than it looks. Basic is fine for development and small workloads. Standard is the sensible default for most production AI work, giving you more storage and better throughput for those big images. Premium unlocks geo-replication, private link, and higher throughput, which you want once you are serving large images across regions or have serious networking requirements. We usually start clients on Standard and move to Premium only when there is a concrete reason, because Premium is a noticeable step up in cost.

Once the registry exists, log in to it so you can push images:

az acr login --name myaiteamregistry

From here you tag your local image with the registry's login server and push it, the same as any Docker workflow:

docker tag my-ai-service:latest myaiteamregistry.azurecr.io/my-ai-service:v1
docker push myaiteamregistry.azurecr.io/my-ai-service:v1

And you can confirm what is sitting in the registry with:

az acr repository list --name myaiteamregistry --output table

That is genuinely the core loop. Create, log in, push, list. The complexity that follows is all about doing this securely and at scale.

The gotchas we keep hitting

The commands are simple. The problems show up in the parts the quickstart skims over. Here are the ones that bite real teams.

Authentication in production is not az acr login. The interactive login is great for a developer at a keyboard. It is useless for your Kubernetes cluster or App Service trying to pull an image at 3am. For production you want a managed identity granted the AcrPull role on the registry. No passwords, no admin user, no secrets to rotate or leak. Whenever I see a team using the registry's admin account credentials in a deployment, I know there is a tidy-up job coming, because that admin user is a single shared password that should not exist in a serious setup.

Image size will surprise you. The first time someone pushes a GPU-enabled inference container and watches it crawl up at six gigabytes, there is always a pause. Spend the time building lean images. Multi-stage Docker builds, slim base images, and not bundling your entire training pipeline into the serving container. Smaller images pull faster, cost less to store, and have a smaller attack surface. This is boring work that pays off every single deployment.

Geo-replication is a Premium feature for a reason. If you are serving an AI service to users across regions, or even just want resilience, pulling images from a single far-away registry adds latency and a point of failure. Premium tier geo-replication puts copies closer to where they run. For an Australian deployment serving local users this might not matter. For anything multi-region, it does.

Cleanup does not happen by itself. Every model retrain, every CI build, produces another image. Without a retention policy, your registry quietly fills with hundreds of old tags, your storage bill creeps up, and finding the actually-current image becomes a guessing game. Set retention rules early. Future you will be grateful.

Where this fits in the bigger picture

A container registry is not exciting. Nobody puts "we set up ACR" in a board update. But it is the foundation that makes containerised AI deployment repeatable and safe, and the teams who treat it as a first-class part of their architecture have noticeably calmer production environments than the teams who bolted it on at the last minute.

If you are running Azure AI services in containers, building custom inference images, or moving from a proof of concept toward something real, the registry deserves a proper conversation early. How big are your images going to be, where do they need to live, how will production authenticate to pull them, and who is allowed to push? Answer those four questions up front and you avoid most of the pain.

This is exactly the kind of production-readiness work our machine learning and Microsoft AI consultants teams handle so the data scientists can stay focused on the models rather than the plumbing. If you have got models working in a notebook and you are staring down the path to production, get in touch and we can help you map the route.

For the full command reference and the official walkthrough, see the Microsoft documentation on getting started with Azure Container Registry using the Azure CLI.