Apache Airflow Job Pricing in Microsoft Fabric - What You Actually Pay For

A client in Melbourne rang me a few months ago with a question that sounded simple. Their Fabric capacity was running hot, they hadn't added any new reports, and the Capacity Metrics app showed a steady flat line of consumption from something called "Data Workflows" that never dropped, day or night. Someone on the data team had set up an Apache Airflow job to test a migration from their old self-hosted Airflow, picked a custom pool, got distracted, and left it running for three weeks.

Nothing was broken. Nothing was even running DAGs. It was just sitting there, billing.

That's the thing about Airflow in Fabric. The pricing model is not complicated once you see it, but it's different enough from pipelines and notebooks that people get caught out. If you're an Australian organisation already paying for an F SKU and you're thinking about moving Airflow workloads into Fabric, it's worth understanding how the meter works before you start, not after the capacity alerts start firing.

Microsoft's reference page is here: Pricing for Apache Airflow job in Fabric Data Factory. What follows is how we explain it to clients, plus what we've seen go wrong.

The short version

An Apache Airflow job in Fabric runs on a managed Airflow environment that Microsoft hosts for you. You don't pay per DAG run or per task. You pay for the time the Airflow environment is up, measured in capacity units (CUs) drawn from your Fabric capacity.

So the question isn't "how many DAGs do we run?" It's "how many hours is the environment alive, how big is it, and how many extra worker nodes does it have?"

That's a mental shift for teams coming from Fabric pipelines, where you mostly think in terms of activity runs and data movement. With Airflow you're paying for a running cluster. Idle time costs money.

How the billing is structured

There are a few moving parts.

The pool type. When you create an Airflow job you pick between a starter pool and a custom pool.

• A starter pool is the "just get going" option. It spins up quickly, comes with default settings, and pauses itself after a period of inactivity (20 minutes at the time of writing). If nothing is running and nobody is poking at it, it stops consuming.
• A custom pool gives you control. You choose the compute node size, you can enable autoscale, add extra nodes, install private packages, and keep it always on. That last bit is exactly why it can quietly drain your capacity.

The compute node size. You choose between small and large node sizes. Large costs more per hour, and it's billed as a separate meter from small. Most of the DAGs we see in client environments are orchestration-heavy rather than compute-heavy, so small is usually fine. If your DAGs are doing heavy Python processing inside the Airflow workers themselves, that's a design smell worth fixing before you reach for large nodes (more on that below).

Extra nodes. On top of the base environment, additional worker nodes are billed separately, again per hour. Autoscale will add nodes when the scheduler has work queued and remove them when it doesn't. Handy, but it means your hourly cost isn't a single fixed number.

All of these roll up into CU consumption against your Fabric capacity. In the Capacity Metrics app you'll see the operations labelled under data workflows (the earlier name for the feature), split by small and large runtime and extra nodes. Microsoft's pricing doc lists the exact CU rate for each meter, and those rates do get revised, so I'm not going to quote numbers here that might be stale by the time you read this. Go to the doc, grab the current rates, and do the maths for your scenario.

Working out what it will cost

Here's the rough calculation we run with clients. It's not clever, but it catches most surprises.

1. Work out how many hours per day the environment needs to be up. Be honest. If DAGs run hourly from 6am to 8pm, that's at least 14 hours. If you have one DAG at 2am and another at 11pm, a custom pool that stays on all day is paying for 22 hours of nothing.
2. Multiply by the CU rate for your node size.
3. Add the expected extra node hours. If you have autoscale on, look at how many concurrent tasks you actually run at peak and for how long.
4. Convert CU-hours into a share of your capacity. An F64 gives you 64 CUs. If your Airflow environment consumes a steady chunk of that around the clock, that's capacity your Power BI users, notebooks and pipelines can't use.

That last point matters more than the dollar figure for most of our clients. On a reserved F SKU you're not getting a separate invoice line for Airflow. You're paying for it in contention. When a large semantic model refresh collides with a busy Airflow environment and a few Spark notebooks, the capacity smooths and throttles, and suddenly the CFO's dashboard is slow on Monday morning. Nobody connects that back to the Airflow environment someone left running.

If you're on pay-as-you-go and pausing your capacity overnight, keep in mind the Airflow environment can't run anything while the capacity is paused either. Plan your schedule around it.

Starter pool or custom pool

My honest take is that most organisations should start on the starter pool and stay there longer than they think.

The auto-pause behaviour is the big win. For dev and test workspaces it's a no-brainer. Developers poke at the environment, it goes idle, it stops billing. You don't need a policy document reminding people to turn things off.

The trade-off is cold starts. If the pool has paused, the first trigger has to wait for the environment to come back up. For scheduled batch work that runs a few times a day, a short delay before the first task almost never matters. For anything that needs to respond within seconds, it might.

Move to a custom pool when you have a real reason:

• You need private Python packages or specific requirements that the starter pool doesn't support
• You've got enough DAG activity across the day that the environment would rarely pause anyway
• You need autoscale for genuinely bursty concurrency
• Cold start latency is actually causing problems, not just theoretically

And when you do move to a custom pool, put someone's name against it. Seriously. The single most common Airflow cost issue we see is an always-on custom pool in a workspace nobody is watching.

The design mistake that inflates Airflow costs

Airflow is an orchestrator. It's very good at deciding what runs when, handling dependencies, retries and backfills. It's not a great place to do heavy data processing.

We regularly see DAGs that pull a few million rows into a pandas DataFrame inside a PythonOperator, reshape them in memory, and write them back out. On a self-hosted Airflow that was "free" because the VM was already paid for. In Fabric it means you need bigger nodes, more nodes, and longer runtimes, and you're paying for all of it.

The better pattern in Fabric is to have Airflow trigger the work and let the right engine do it. Kick off a Fabric notebook for Spark transforms, trigger a pipeline for bulk data movement, run SQL in the warehouse. The Airflow workers then spend most of their time waiting on API calls, which is cheap and doesn't need large nodes. Yes, the notebook and pipeline consume CUs too, but they consume them for exactly as long as the work takes, on engines built for it.

When we've refactored client DAGs this way, the Airflow environment often drops from large to small nodes with no extra nodes, and the overall capacity footprint goes down even after counting the Spark usage.

Should you use Airflow in Fabric at all?

This is the question I get most often, and the answer depends on where you're coming from.

If your team already has a substantial Airflow codebase, either self-hosted or on another managed service, moving it into Fabric makes a lot of sense. You keep your DAGs, you drop the infrastructure maintenance, and everything sits inside the same capacity, security model and workspace structure as the rest of your data platform. We've helped a couple of clients do this and the "we don't have to patch the Airflow VM anymore" relief is real.

If you're starting fresh and your team doesn't already know Airflow, I'd usually point you to Fabric Data Factory pipelines first. They cover most orchestration needs, they're easier for mixed-skill teams to maintain, and their cost profile is more predictable for low-frequency batch work. Airflow earns its place when you need code-first orchestration with complex dependency logic, dynamic DAG generation, or you've got Python-heavy engineers who'd rather write code than drag boxes around a canvas. Nothing wrong with either choice, but don't pick Airflow just because it sounds more serious.

There are also still some rough edges. The Fabric Airflow experience has improved a lot over the last year, but some of the provider and plugin support that seasoned Airflow users expect is narrower than a self-managed install, and debugging environment issues gives you less visibility than when you own the box. Test your actual DAGs, especially ones relying on unusual providers, before committing to a migration plan.

Practical controls we put in place

For clients running Airflow jobs in Fabric, this is our usual checklist:

• Default to starter pools in dev and test workspaces, no exceptions without a reason written down somewhere
• Tag custom pools with an owner and review them monthly in the Capacity Metrics app
• Check the Capacity Metrics app weekly for the first month after go-live, specifically filtering on the Airflow-related operations
• Keep heavy processing out of Airflow workers and push it down to notebooks, pipelines or the warehouse
• Consolidate DAGs into fewer environments where it makes sense, rather than one Airflow job per team, because every environment has its own base cost
• Align schedules so a custom pool can actually be paused overnight if your workloads allow it

None of this is sophisticated. It's just the operational hygiene that stops a reasonable architecture from turning into an annoying capacity conversation three months later.

Where we fit in

We spend a lot of time helping Australian organisations design and tune Fabric platforms, including working out which orchestration tool fits which workload and keeping capacity spend under control. If you're planning an Airflow migration or trying to work out why your F SKU is always busy, our Microsoft Data Factory consultants can help, and our broader Microsoft Fabric consulting work covers capacity planning across the whole platform. If you'd rather just talk it through, get in touch.

The short version I give every client: Airflow in Fabric bills by uptime, not by work done. Keep that in your head and most of the cost surprises go away.