Power BI Adoption Tracking - How to Measure Whether Your Rollout Is Working

Most Australian organisations that roll out Power BI can tell you how many licences they bought. Far fewer can tell you whether it's working.

That gap matters more than people think. When budget season comes around and someone in finance asks why you're paying for 400 Pro licences and an F64 capacity, "people seem to like it" isn't a great answer. Neither is a screenshot of the usage metrics report showing 12,000 report views last month. Views of what? By whom? Did anyone change a decision because of it?

Adoption tracking is the part of a Power BI rollout that almost everyone agrees is important and almost nobody does properly. Microsoft has a section of its Fabric adoption roadmap guidance dedicated to adoption tracking, and it's worth reading. Here's how we apply it with clients, and what we've learned about doing it without it turning into a reporting project of its own.

Adoption is more than usage

The first thing to get straight is that "adoption" in Microsoft's framing isn't one number. Their adoption roadmap breaks it into three levels:

• Organisational adoption - how mature your data culture, governance and support structures are across the whole organisation.
• User adoption - whether individuals are actually using Power BI, and using it well.
• Solution adoption - whether a specific report or semantic model is delivering what it was built to do.

These move at different speeds. You can have high user adoption (lots of people logging in) with poor organisational maturity (no governance, duplicated semantic models everywhere, three versions of "revenue"). We see this a lot. It feels like success for about 18 months, then someone notices that the board pack and the sales report disagree by $2 million and confidence collapses.

The opposite also happens. A beautifully governed tenant with a centre of excellence, a data catalogue and clear policies, and about forty people actually using it. Usually a sign the governance got ahead of the value.

Maturity levels - useful, if you're honest

The adoption roadmap uses a five-level maturity model for each area: 100 (initial), 200 (repeatable), 300 (defined), 400 (capable) and 500 (efficient). Areas include things like data culture, executive sponsorship, content ownership, the centre of excellence, governance, mentoring and user enablement, community of practice, user support, and system oversight.

Microsoft's advice on adoption tracking is essentially: assess where you are in each area, decide where you want to be, and reassess periodically. That's sensible. The trap is that self-assessment drifts optimistic. Every organisation we've run this exercise with initially rates itself a level higher than it is.

What works better in our experience:

Use evidence, not opinion. For each area, write down what you'd expect to see at each level, then go and look. "We have a community of practice" isn't evidence. "We have a Teams channel with 150 members and 30 posts last month, half of them answered by someone outside the BI team" is.

Get people outside the BI team to score it. The people building the reports are the least objective judges of how well they're being used. Ask a few business users and a couple of managers.

Don't aim for 500 everywhere. Honestly, most mid-sized Australian organisations would be in great shape at 300 across the board. Level 500 in every area is expensive and often not worth it. Pick the two or three areas that are actually holding you back and focus there.

Reassess on a fixed rhythm. Every six months works for most clients. Quarterly is too often to see real change, annually is too slow to course-correct.

The usage data worth watching

Maturity assessments give you the qualitative picture. You also need numbers. Here's what's available and what we've found useful.

The activity log

The Power BI and Fabric activity log records almost everything users do: viewing reports, exporting data, sharing, creating workspaces, refreshing semantic models. It's the foundation of any serious tracking. The catch is that it only keeps about 30 days of history. If you want trends over time (and you do) you need to extract it regularly and store it somewhere. We typically set up a small pipeline using the activity events REST API that lands the data into a lakehouse or SQL database daily, and build a semantic model on top. It's not glamorous, but it's the first thing we do in any adoption tracking engagement, because every week you don't do it is a week of history you can't get back.

Usage metrics reports

Each workspace has built-in usage metrics reports. They're fine for a content owner who wants to know whether anyone looks at their report. They're not good for tenant-wide analysis, because they're scoped per workspace and the history is limited. Use them for what they are.

The admin monitoring workspace

Fabric administrators get an admin monitoring workspace with a Feature Usage and Adoption report out of the box. It's a decent starting point and better than nothing, particularly for smaller organisations that won't build their own. It's also a bit rigid. You'll outgrow it if you want to slice adoption by business unit or job role.

Joining with HR data

This is where tracking gets genuinely useful. Activity data tells you that user ID such-and-such viewed a report. Joined with HR or Entra ID data, it tells you that 80% of the finance team are active weekly while only 15% of regional operations managers have opened anything this quarter. That's something you can act on. It's also where privacy questions come in, so talk to HR and whoever owns your privacy obligations before you start publishing named individual activity. We generally recommend reporting at team or role level, not person level, for anything widely shared.

Metrics we ignore (or at least distrust)

Total report views. Inflated by the BI team checking their own work, by automated testing, and by a handful of heavy users. A single analyst who refreshes a report 40 times a day looks like a lot of adoption.

Number of reports. More reports is often a sign of a problem. If you have 3,000 reports for 500 users, you probably have a duplication and trust problem, not a success story.

Licence count. Tells you what you're paying, not what you're getting.

Metrics we find more useful

Weekly active users as a percentage of licensed users. Simple, hard to fake, and the trend tells you a lot. If it's flat or falling after the launch bump, something's wrong.

Breadth of use. How many distinct business units or teams have active users. Adoption concentrated in finance and nowhere else is a different situation to adoption across the board.

Use of certified and promoted content. What share of views go to endorsed semantic models and reports versus personal workspaces and random copies? This is the best single indicator of whether governance is working.

Exports to Excel. Some export is normal and healthy. A lot of export usually means the reports don't answer the question and people are rebuilding things in spreadsheets. We had a client where one report accounted for nearly half of all exports in the tenant. Turned out it was missing a single column the accounts team needed. Twenty minutes of work fixed a behaviour that had been going on for a year.

Support and community signals. Volume of help requests, questions in the community channel, attendance at training. Rising questions early on is a good sign. People only ask questions about tools they're trying to use.

Solution-level tracking

For individual high-value solutions, set success criteria before you build. "The weekly ops report will replace the Monday spreadsheet, and the spreadsheet will stop being emailed within six weeks." Then check. It sounds obvious. Almost nobody does it. Without it, you have no way to tell whether the report you spent three months on is delivering anything, and you can't make the case for the next one.

The mistakes we see most

Tracking without anyone acting on it. The adoption report gets built, goes into a workspace, and nobody looks at it. Tracking only matters if it feeds a regular conversation, ideally with your executive sponsor, about what to change.

Turning it into a leaderboard. Publishing "top 10 users" or ranking teams by usage tends to backfire. People game it or resent it.

Starting too late. Because of the 30-day activity log window, organisations that start tracking 18 months after rollout have no baseline. Start extracting on day one, even if you don't build the reporting until later.

Measuring Power BI in isolation. Adoption of Power BI is usually part of a broader data and AI program now, with Fabric, Copilot and agents in the mix. Your tracking should be able to grow to cover those too.

Where this connects to AI adoption

The same discipline applies to AI tools. We're seeing organisations make exactly the same mistakes with Copilot and Claude rollouts that they made with Power BI five years ago: licences bought, launch email sent, no tracking, no idea whether it's working. If you've built decent adoption tracking for Power BI, you've got most of the pattern you need for AI tools too. Our work on AI strategy increasingly includes exactly this kind of measurement framework.

Getting started

If you're doing nothing today, start with three things this week. Set up a daily extract of the activity log into storage you control. Pick four or five metrics from the list above and agree them with your sponsor. Book a maturity self-assessment for next month and invite people outside the BI team.

That's enough to get you from guessing to knowing. If you want help setting it up, or an outside view on where your rollout really sits, our Power BI consultants have done this for organisations from 50 users to several thousand, and our Microsoft Fabric team can help get the data foundations in place.