Setting Up an Azure Blob Storage Connection in Microsoft Fabric Data Factory - Auth, Networking and Ownership
The connection dialog for Azure Blob Storage in Microsoft Fabric looks like a two-minute job. A URL, an authentication kind, a few credentials, click Connect. And most of the time it really is two minutes.
The trouble is what happens later. I've lost count of the support calls that start with "the pipeline was fine yesterday" and end with one of three answers: a SAS token expired, the storage account firewall got tightened by the infrastructure team, or the person who created the connection left and their account was disabled. All three come back to decisions made in that two-minute dialog.
Microsoft's guide to setting up your Azure Blob Storage connection walks through the fields. This post is about the choices behind them, based on what we see in Australian Fabric environments. If you want the broader picture of what the connector does once it's connected, we covered that in an earlier post on the Blob Storage connector.
Where the connection gets created
You can create a Blob Storage connection from a couple of places in Fabric:
- Inside Dataflow Gen2, through Get Data, where you'll pick Azure Blobs and fill in the account details
- Inside a data pipeline, when you configure a Copy activity source or destination and choose a new connection
- From Manage connections and gateways in the settings menu, where you can create and administer connections centrally
All three end up in the same place: a cloud connection object that belongs to someone and can be shared. That's worth knowing because it means a connection made by an analyst inside a dataflow is just as real, and just as reusable, as one created by your platform team.
My preference is to create the important ones deliberately in Manage connections and gateways, give them a clear name, and share them with the people who need them. Ad hoc connections created from inside a dataflow tend to get names like "AzureBlobs 3" and nobody can tell which storage account they point at.
The account URL
The first field is the account name or URL. You'll normally use the full blob endpoint, something like https://contosodata.blob.core.windows.net. You can point at the account and browse containers, or go straight to a specific container.
Two small things trip people up here. One is pasting the Data Lake endpoint (dfs.core.windows.net) when they meant blob, or the other way around. If the storage account has hierarchical namespace turned on, you may be better off with the ADLS Gen2 connector anyway, which we've also written about. The other is a trailing path that doesn't match the container name exactly, which gives you a vague error that looks like an auth failure.
Choosing an authentication kind
This is the decision that matters. Depending on where you create the connection, you'll see some combination of these:
Anonymous. Only works if the container allows public read access. In practice we almost never see this in business environments, and if we do, it usually means something has been left open that shouldn't be. If you find yourself picking Anonymous, check with whoever owns the storage account first.
Account key. You paste in one of the storage account's access keys. It's fast and it works with everything. It also gives full access to the whole storage account, not just the container you care about, and the key is shared by everything else using it. When the infrastructure team rotates keys (which they should), your connection breaks. I'm fine with account keys for a proof of concept. I don't want them in production.
Shared Access Signature (SAS). A token scoped to specific permissions, resources and a time window. Better than an account key because you can limit it to read-only on one container. The catch is the expiry date. Teams generate a SAS with a twelve-month expiry, everything runs nicely, and then one morning in the following year every load fails at once. If you use SAS, put the expiry date in a calendar, and preferably generate it from a stored access policy so you can revoke or extend it without reissuing the token.
Organisational account. You sign in with your Entra ID account and the connection uses your identity. Great for exploring data and building a first version of a dataflow. Risky for anything scheduled, because the connection depends on you. If you go on leave and your password changes, or you leave the company, refreshes start failing. We see this one constantly in organisations where Fabric adoption grew from the business side.
Service principal. An Entra ID app registration with a client secret or certificate, granted a role like Storage Blob Data Reader on the storage account or container. This is the option we recommend for most production pipelines. The identity doesn't belong to a person, access is controlled through Azure RBAC and you can scope it tightly. The only thing to manage is the client secret expiry, and that's a known, single date.
Workspace identity. A managed identity tied to the Fabric workspace, which you then grant access to on the storage account. No secret to rotate at all. Where it's supported for your scenario, it's the cleanest option. Support across Fabric items has been growing, but check it works for the specific item type you're using before you design around it.
One practical note: for role-based options, the identity needs a data plane role like Storage Blob Data Reader or Storage Blob Data Contributor. A management role like Reader or Contributor on the storage account isn't enough. I'd estimate half the "access denied" errors we troubleshoot on new service principal setups come down to this.
Networking - the part that breaks quietly
Plenty of Australian organisations, especially in financial services, government and healthcare, have locked their storage accounts down so they only accept traffic from selected networks or private endpoints. That's good practice. It also means Fabric's cloud services can't reach the account by default, and the connection either fails on test or, more annoyingly, works today and fails after someone adds a firewall rule next month.
You have a few options:
- Trusted workspace access. Fabric workspaces with a workspace identity can be granted access through resource instance rules on the storage account firewall. This lets a specific workspace in, without opening the account to all of Azure. It needs Fabric capacity (F SKU) and some setup on the storage side, but it's the approach I'd look at first.
- On-premises data gateway or virtual network data gateway. Route the connection through a gateway that sits inside a network the storage account trusts. This works, but it's another component to patch and monitor.
- Managed private endpoints. For Spark workloads in Fabric, you can create managed private endpoints to reach locked-down storage. Useful if notebooks are part of your pattern.
Whichever you pick, talk to the people who own the storage account firewall before you build. A five-minute conversation now saves a confused afternoon later. And keep an eye on data residency: if the storage account is in Australia East and your Fabric capacity is in a different region, you're adding cost and latency, and possibly a compliance question too.
Privacy levels and other settings
In Dataflow Gen2 you'll also be asked for a privacy level. For most internal storage accounts, Organizational is the right choice. If you combine sources with different privacy levels in a dataflow, Power Query may stop folding or refuse to combine them, which shows up as slow refreshes or odd firewall errors in the query. Set it consistently across your connections and you'll mostly avoid this.
You'll also want to decide whether this connection should be allowed for use with on-premises gateways, and who it's shared with. Which brings me to ownership.
Ownership and sharing
Connections in Fabric have owners and users. Owners can edit and reshare. Users can use the connection in their own items without seeing the credentials. The pattern that works well for our clients:
- Platform or data team members own production connections, ideally two or more people so nobody becomes a single point of failure
- Analysts and engineers get User access, so they can build on the connection without handling secrets
- Connection names include the storage account, container and environment, like
blob-contosodata-raw-prod-sp - Secret and SAS expiry dates are tracked somewhere the whole team can see
None of this is exciting. All of it saves you from the 6am "everything failed" message.
A quick checklist
Before you mark a Blob Storage connection as production ready, I'd check:
- Is it using a service principal or workspace identity rather than a personal account or account key?
- Does that identity have a Storage Blob Data role, scoped to the container where possible?
- Will the storage account firewall still allow it if someone tightens the rules?
- Is the storage account in an Australian region if your data needs to stay onshore?
- Are there at least two owners on the connection?
- Is any secret or SAS expiry written down with a reminder set?
Where we help
Setting up one connection is easy. Setting up forty, across dev, test and prod workspaces, with sensible identities and firewall rules, is where it becomes a platform design question. Our Microsoft Fabric consultants and Microsoft Data Factory consultants do this work regularly, often as part of cleaning up an environment that grew faster than its governance. If you're planning to feed that data into AI workloads too, our Azure AI consulting team can make sure the same storage and identity setup works for both.
My blunt advice: spend the extra fifteen minutes on a service principal and a proper name. It's the cheapest reliability improvement you'll make in Fabric.