Troubleshooting Power BI Incremental Refresh and Real-Time Data - What Actually Goes Wrong

Incremental refresh is the feature that turns a Power BI model from something that falls over at a few million rows into something that comfortably handles hundreds of millions. When it works, it's brilliant. You refresh only the data that changed, the nightly job that used to take forty minutes drops to two, and your capacity stops groaning under the weight of reloading years of history nobody edited. When it doesn't work, it fails in ways that are genuinely hard to reason about, because the thing that's broken is usually invisible in Power BI Desktop and only shows up once the model is published and running against real data.

We get called in on this a lot. A client sets up incremental refresh following a tutorial, it seems fine, and then three weeks later the numbers are wrong, or the refresh is somehow taking longer than before, or today's sales just aren't showing up. This post is the field guide: the failure modes we actually see, how to tell them apart, and what to check first. Microsoft's own reference on troubleshooting incremental refresh and real-time data is worth having open alongside this, but the docs describe the mechanics, not the pattern of what breaks in the wild.

First, understand what's actually happening under the hood

Most incremental refresh problems come from a shaky mental model of how the feature works, so it's worth thirty seconds on the mechanics before we get to the failures.

When you configure incremental refresh, you're telling Power BI to slice your table into partitions based on a date column. History sits in cold partitions that get loaded once and then left alone. A rolling window of recent data sits in warm partitions that get refreshed each time. Optionally, the newest slice can be a DirectQuery partition for near real-time data. The whole thing hinges on two things being correct: your RangeStart and RangeEnd parameters, and query folding. Get either wrong and you'll have a model that looks configured but behaves badly.

That's the crux of nearly every problem below. Keep it in mind.

The single biggest culprit - query folding that silently stops

If I had to bet on the cause of an incremental refresh problem sight unseen, I'd bet on query folding, and I'd usually be right.

Incremental refresh depends on Power Query being able to translate your RangeStart and RangeEnd filters into a WHERE clause that runs on the source. That's query folding. When folding works, the source database returns only the rows in the partition being refreshed. When folding breaks, Power BI pulls the entire table across the wire and filters it in memory, which defeats the entire purpose. Your refresh isn't just slow, it's often slower than a plain full load, because now you're moving all the data AND doing partition bookkeeping on top.

The insidious part is that folding breaks quietly. Add a custom column, change a data type in the wrong place, use a Power Query function the source connector can't translate, and folding stops at that step with no error and no warning. The model still refreshes. It just stops being incremental.

How we check: right-click each step in Power Query and look for "View Native Query". If it's greyed out at or before your date filter, folding has broken and your incremental refresh is a fiction. The fix is to reorder your query so the folding-friendly steps, especially the date filter driven by RangeStart and RangeEnd, happen first, and push the non-folding transformations as late as possible. Sometimes the honest answer is that a particular source or transformation just won't fold, and you need to solve it upstream in a view or a proper data pipeline. This is exactly the kind of unglamorous plumbing our Power BI consultants end up fixing, and it's almost always folding.

The parameters have to be exactly right, and exactly typed

RangeStart and RangeEnd are not optional and they are not flexible. They must be named exactly that, they must be of type Date/Time, and they must be used to filter your date column with a specific pattern: greater than or equal to RangeStart, and strictly less than RangeEnd. Use greater-than on both ends, or less-than-or-equal on the wrong side, and you'll double-count or drop rows at partition boundaries. That produces the worst kind of bug, the one where the totals are only slightly wrong, so nobody notices for a month.

A common trap: your source date column is a Date type but the parameters are Date/Time, and the implicit conversion breaks folding or produces off-by-a-day boundary errors. Match the types deliberately. If your data has time components, account for them. If it doesn't, be consistent about midnight boundaries.

When the refresh works but the data is stale or wrong

This is the category that makes people distrust the whole feature, so it's worth pulling apart.

If new data isn't appearing, the first question is whether it falls inside your refresh window. Incremental refresh only reprocesses the warm partitions. If you set the window to refresh the last 3 days but your source system posts backdated transactions a week late, those late arrivals land in a cold partition that never gets touched again, and they'll never show up. This isn't a bug, it's the design, but it catches people constantly. The fix is either a wider refresh window or the "detect data changes" option pointed at a reliable last-modified column so Power BI knows which older partitions actually changed.

If historical numbers have shifted unexpectedly, suspect the opposite problem. Someone edited or deleted records in a cold partition, and because those partitions aren't refreshed, the model and the source have quietly diverged. When source data is genuinely mutable that far back, incremental refresh in its simple form is the wrong tool, or at least needs the change-detection option doing real work.

The real-time and DirectQuery partition gotchas

The near real-time story, where the newest partition is DirectQuery over a live source while history stays in import mode, is powerful and also where the sharpest edges live.

The hybrid table only makes sense if your source can actually handle interactive DirectQuery load. We've seen teams flip on the real-time partition against a transactional database that was never sized for analytics queries, and the result is a live report that hammers the production system every time someone opens it. Before you reach for this, make sure the source is a proper analytics store or a replica, not the system your operations team depends on staying up.

You also inherit every DirectQuery limitation on that partition: DAX that doesn't translate, slower visuals, and the need for a gateway that's correctly configured and, above all, running. A surprising share of "real-time data isn't updating" tickets come down to a gateway that's offline, out of date, or pointed at credentials that expired. Check the gateway before you check anything clever.

Publishing, partitions, and the tools that actually help

Here's something that trips up nearly everyone the first time. The full partition structure doesn't exist in Power BI Desktop. Desktop only ever loads a small sample defined by your parameters, so everything can look perfect locally. The partitions get created on the first refresh after you publish to the service. This means real incremental refresh testing only happens in the service, against real volume, and the first post-publish refresh is the moment of truth. Plan for it, watch it, and don't assume a clean Desktop experience means you're done.

When something does go wrong in the service, the built-in refresh history is a blunt instrument. It'll tell you a refresh failed and give you an error, but it won't show you the partition-level detail you often need. For anything serious, connect to the model through the XMLA endpoint with a tool like SQL Server Management Studio or Tabular Editor, where you can see the individual partitions, when each last refreshed, how many rows each holds, and refresh a single partition in isolation to test a theory. That partition-level visibility is the difference between guessing and diagnosing. It's the same tooling we lean on across the wider Microsoft Fabric and data platform work, and it turns most incremental refresh mysteries into a ten-minute answer.

A sane order of operations when it breaks

When a client hands us a misbehaving incremental refresh, this is roughly the sequence we run, and it resolves the large majority of cases:

Check query folding first, because it's the most common cause and it invalidates everything else. Confirm RangeStart and RangeEnd are named right, typed right, and filtering with the correct boundary operators. Check whether the missing or wrong data falls inside the refresh window, and whether "detect data changes" is needed. If real-time is involved, verify the gateway and the source's capacity to serve live queries. Then, and only then, drop to the XMLA endpoint to inspect partitions directly.

Nine times out of ten it's folding or a window that's too narrow for how the source actually behaves. The exotic causes are real but rare, and chasing them first is how people lose a day.

Where this fits

Incremental refresh is worth the trouble. For any Australian business with real data volume, a retailer with years of transactions, a logistics operation with millions of tracking events, a financial services firm with a long ledger, it's often the difference between a report that's usable and one that times out. But it rewards understanding and punishes cargo-culting a tutorial, and the failures are subtle enough that they erode trust in the numbers before anyone traces the cause.

If your refreshes have become slow, flaky, or quietly wrong, that's a very fixable situation and usually a quick one once you know where to look. It's a routine part of what we do on business intelligence and analytics engagements, often as the first step before any AI work, because good analytics has to sit on data you can trust. If your Power BI estate has reached the point where nobody's quite sure the refresh is telling the truth, get in touch and we'll help you sort it out.