How to Ingest Data from Azure Data Factory into Microsoft Fabric - A Practical Guide

A question I get asked at least once a fortnight from Australian organisations - "We have Azure Data Factory running in production. We are now adopting Microsoft Fabric. Do we throw ADF away or keep it?"

The short answer is - keep it, at least for now, and use it as the ingestion engine that lands data into Fabric. The longer answer is that there are three or four patterns for getting ADF data into Fabric, each with its own trade-offs, and picking the wrong one will cost you either money or time. Usually both.

This post is the explanation I give over the phone when someone is trying to work out the right pattern for their situation. It assumes you already have Azure Data Factory running and Microsoft Fabric provisioned. If you are still working out the Fabric foundation, our Microsoft Fabric consultants page covers the strategy side.

Why You Probably Should Not Migrate Your ADF Pipelines

Before talking about ingestion patterns, a quick rant. Most organisations we see have invested hundreds of engineering hours in their ADF estate. They have linked services, integration runtimes, self-hosted IR boxes inside the corporate network, complex orchestrations, custom triggers, alerting, the whole picture. That investment is real.

Fabric Data Factory exists and it overlaps with ADF, but it is not a full replacement yet. There are connectors ADF has that Fabric Data Factory does not. There is no self-hosted integration runtime in Fabric (you use on-premises data gateway instead, which has different limits). There is no equivalent of the ADF managed virtual network for some scenarios. SSIS package migration only works in ADF.

So the pattern that works is to leave the heavy lifting in ADF where it is, and use ADF to land the data in a place Fabric can consume cheaply. That is what this post is about. If you want a deeper comparison of when each tool wins, we covered that in our Azure Data Factory vs Fabric Data Factory work.

Pattern 1 - ADF Writes to ADLS Gen2, Fabric Shortcuts to ADLS

This is the pattern we recommend most often. ADF writes data into Azure Data Lake Storage Gen2 in the same format it always has, typically Parquet or Delta. In Fabric, you create a OneLake shortcut that points to the ADLS container. From Fabric's perspective, the data is in OneLake. From ADF's perspective, nothing has changed.

Why this works well - you do not duplicate data. The shortcut is a pointer, not a copy. You only pay for the ADLS storage, not the same data in OneLake. Fabric workloads (Spark, T-SQL, Power BI Direct Lake) can query the data through the shortcut as if it were native OneLake data.

Why it sometimes does not work - shortcuts have permission considerations. The Fabric capacity needs read access to the ADLS account. If your ADLS sits behind a private endpoint, you need to configure trusted workspace access or accept that shortcuts will not work for that account. Microsoft has been improving this, but it is still an area to test before you commit.

The other limitation is that some Fabric features prefer native OneLake storage. Direct Lake mode in Power BI works through shortcuts, but performance is best when the data is in OneLake natively. For semantic models where milliseconds matter, you might want a copy in OneLake. For most workloads, the shortcut approach is fine.

This is our default recommendation for organisations that already have ADF running. Keep your ADF pipelines, point them at ADLS, shortcut into Fabric.

Pattern 2 - ADF Copy Activity Writes Directly to a Fabric Lakehouse

In November 2024, Microsoft added Fabric Lakehouse as a sink connector for ADF Copy activities. This means an ADF pipeline can write directly into a Fabric Lakehouse table without going through ADLS first.

For new pipelines, this is a reasonable choice. You skip the ADLS layer entirely and the data lands as a Delta table in OneLake, ready for consumption. For existing pipelines, it is rarely worth changing them just for this. The shortcut pattern in Pattern 1 gets you the same outcome with less rework.

The watch-out here is throughput. ADF Copy activities against a Fabric Lakehouse sink go through a different path than writes to ADLS, and we have seen the Lakehouse sink be slower for very large loads. Not always, but enough that I would test with a representative file before committing to this for a 1TB+ daily load. If you are doing the small to medium daily refresh, it is fine.

The other consideration is that this pattern couples your ADF pipelines to your Fabric capacity. If your Fabric capacity is constrained, the ADF write can be throttled. With Pattern 1, ADF writes to ADLS at full speed regardless of Fabric capacity state.

Pattern 3 - ADF Triggers a Fabric Pipeline Through the Fabric REST API

This is a more recent pattern that has become viable as Microsoft has matured the Fabric REST APIs. You keep ADF as the orchestration layer for cross-cloud workloads, but instead of ADF doing the data movement directly, it triggers a Fabric Data Factory pipeline through the API to do the Fabric-native work.

Where this earns its keep is when you have orchestrations that span Azure services that Fabric does not handle well (SSIS packages, Azure Batch jobs, on-premises file movement through self-hosted IR) but the final landing zone is Fabric. ADF does the messy cross-system orchestration, then hands off to Fabric for the in-Fabric work.

It is more complex to set up than Patterns 1 or 2, so we only recommend it for organisations with serious orchestration needs. If your current ADF pipeline is "copy from on-prem SQL to ADLS, then transform in Synapse," you probably do not need this. If your pipeline is "wait for SAP, then orchestrate seventeen Azure services with conditional logic," then this is worth the setup cost.

Pattern 4 - The Mount Approach

OneLake supports mounts, which let you mount an external storage account into the OneLake namespace. Mounts are similar to shortcuts but have different semantics around caching and permissions. For the ingestion scenarios this post is about, shortcuts are usually the better choice. Mounts come into their own for compute scenarios that need filesystem-style access, which is not what we are discussing here.

I am mentioning mounts so you know they exist and so you can rule them out for ingestion. If someone tells you to use mounts for ADF-to-Fabric ingestion, push back and ask why.

How to Decide Between Patterns

A quick decision framework that has held up across the engagements we have done.

Use Pattern 1 (ADF to ADLS with OneLake shortcut) if you already have ADF pipelines running and they write to ADLS. This is the lowest-risk migration path. Cheapest. Easiest to roll back if Fabric does not work out.

Use Pattern 2 (ADF Copy direct to Lakehouse) if you are building new pipelines and want to avoid the ADLS layer entirely. Test throughput first if your loads are large.

Use Pattern 3 (ADF triggers Fabric pipelines via REST) if you have complex orchestration needs that cross system boundaries Fabric does not handle. Worth the setup cost for the right scenario.

Avoid Pattern 4 for ingestion. It is the wrong tool for the job.

The Bits Microsoft Documentation Skips

A few practical observations from doing this work.

Authentication is the first thing that goes wrong. ADF writing to a Fabric Lakehouse needs the right managed identity setup. The Fabric workspace needs to grant the right role to the ADF managed identity. Shortcuts in Fabric pointing at ADLS need either the workspace identity or a service principal with the right RBAC on the ADLS account. None of this is hard, but it is fiddly and the error messages when it goes wrong are unhelpful. Build a connectivity test pipeline as the first thing you do, before you start migrating real workloads.

File formats matter more than people expect. Parquet is the default and is fine for most analytical workloads. If you can write Delta from ADF, do so. Delta tables in OneLake unlock Direct Lake mode, time travel, and easier upserts. Parquet works but you lose some of these capabilities. If your ADF version does not support Delta sinks natively, you can use a Mapping Data Flow to write Delta, or use a downstream Fabric notebook to convert Parquet to Delta after ingestion.

Refresh orchestration becomes a question. If ADF lands data in ADLS at 2am and you have Fabric semantic models that need to refresh at 3am, how do you trigger the Fabric refresh? Options include - have ADF call the Fabric REST API at the end of the pipeline to refresh the semantic model, use Power Automate, or use a scheduled Fabric pipeline that watches for new data. All work. Pick one and stick with it. Avoid hybrid approaches where different pipelines use different mechanisms, because debugging at 2am is bad enough already.

Cost. ADF Copy activity costs are reasonably well understood and predictable. Fabric capacity consumption when ingesting data is less predictable, particularly for first-time users. If you are moving to Pattern 2 (direct Lakehouse writes), measure the CU consumption on a representative load before committing. We have seen organisations get surprised by Fabric CU bills after switching to Lakehouse-native ingestion. Our data factory cost guidance covers the cost models in more detail.

Where to Start

If you have ADF in production and you are introducing Fabric, the safest first step is - keep your ADF pipelines exactly where they are, set up OneLake shortcuts to your existing ADLS containers, and let your Fabric workloads consume through the shortcuts. Get comfortable with that pattern before you change anything else.

Then, for new workloads, decide whether you want them in ADF or Fabric Data Factory based on the connector requirements and orchestration needs. For most new analytical workloads where the destination is Fabric, Fabric Data Factory is the right home.

For the original Microsoft guidance, see the Fabric Data Factory documentation. It covers the official patterns and connector configurations.

If you are working through this decision for your own organisation and would like a sounding board, get in touch. We have helped enough Australian organisations through the ADF-to-Fabric transition to spot the avoidable mistakes early.