How to Enable User Feedback on Your Microsoft 365 Copilot Agent

Most agent projects have the same blind spot. You build the thing, you test it against a handful of prompts you thought of yourself, it works, you ship it. Then real people start using it and you have almost no idea whether it is actually helping them or quietly frustrating them ten times a day. The agent does not complain. The users mostly do not complain either. They just use it less, and by the time that shows up in your usage numbers you have already lost their trust.

User feedback is the cheapest fix for that blind spot, and for declarative agents in Microsoft 365 Copilot it is close to free to turn on. Microsoft's documentation on enabling feedback walks through the manifest property. This post is about why it matters, what the signal actually tells you, and the mistakes we see teams make once they have it switched on.

What you are actually turning on

A declarative agent is the lightweight kind of Copilot agent. You describe its behaviour, point it at some knowledge sources and actions, and Copilot itself does the heavy lifting of orchestration. You are not writing the model logic, you are configuring it. That makes these agents fast to build, which is exactly why so many Australian businesses are starting here rather than with a full custom engine.

Feedback for these agents is a property in the manifest. Once it is set, Copilot renders the familiar thumbs-up and thumbs-down controls under the agent's responses. A user who got a good answer can click up. A user who got a bad one can click down, and depending on your configuration they can add a short comment explaining what went wrong. That feedback then flows into the reporting available to your admins through the Microsoft 365 admin surfaces, where you can see it aggregated rather than having to chase it down one conversation at a time.

That is the whole feature, mechanically. It is not complicated. What is worth thinking harder about is the layer on top: what the numbers mean, and what you change because of them.

Why the signal is worth having

Here is the thing about agents that a lot of people miss. A traditional application either works or throws an error. An agent has a third state, and it is the dangerous one: it produces an answer that looks fine, reads confidently, and is subtly wrong or unhelpful. No exception gets logged. No alert fires. The only person who knows something went wrong is the user, and they are usually not going to raise a ticket about it. They will just shrug and go do the task the old way.

Feedback is your window into that third state. A thumbs-down is a user telling you "this looked like an answer but it did not help me", and that is information you cannot get any other way at scale. When we run managed services for AI agents, the feedback stream is one of the first things we wire into our monitoring, because it is the earliest warning that an agent has drifted, that a knowledge source has gone stale, or that people are asking it questions it was never scoped to handle.

The comments, when you enable them, are often more useful than the ratings. A raw thumbs-down count tells you something is wrong. A comment that says "it keeps giving me the old leave policy" tells you exactly what is wrong and roughly how to fix it. We push teams to collect the comments, not just the up and down, because the qualitative stuff is where the actionable fixes live.

What the numbers really mean

A word of caution, because feedback data is easy to misread.

The first trap is treating the ratio as a grade. If your agent has 85 per cent positive feedback, that is not a mark out of a hundred. Feedback is heavily self-selected. Plenty of people who got a perfectly good answer never click anything, and a fair few who click thumbs-down are annoyed about something the agent was never meant to do in the first place. So do not obsess over the exact percentage. Watch the trend and watch the volume. A sudden spike in thumbs-down after you changed a knowledge source is a far stronger signal than the absolute number on any given week.

The second trap is silence. Very low feedback volume does not mean everything is fine. It usually means people are not using the agent much, or they have already decided it is not worth the effort of rating. Low engagement is its own red flag, and it is one you will only spot if you are looking at feedback volume alongside actual usage rather than in isolation.

The third trap, and this one bites teams that are otherwise doing everything right, is collecting feedback and then never acting on it. Nothing kills user trust faster than clicking thumbs-down on the same broken behaviour three weeks running and watching nothing change. If you turn on feedback, you are making an implicit promise to look at it. Make sure someone owns that.

How we build feedback into the loop

The manifest change is the easy five per cent. The other ninety-five is the process around it.

When we stand up an agent for a client, feedback is not a launch-day afterthought, it is part of the operating model from the start. We agree who reviews the feedback and how often. For a busy internal agent that might be a weekly fifteen-minute look at the thumbs-down comments, grouping them into themes. Is it a knowledge gap, where the agent simply does not have the information? Is it a scope problem, where people expect it to do things it was never designed for? Is it a genuine quality issue, where it has the right information but is presenting it badly? Those three buckets lead to three completely different fixes, and the comments are what let you tell them apart.

Then we close the loop visibly. When a batch of feedback leads to a change, we tell the users. "You told us the agent kept citing the wrong policy document, we have fixed the source it reads from." That single habit does more for adoption than almost anything else, because people who see their feedback change the product keep giving it. People who feel like they are shouting into a void stop.

This is the same discipline we bring to any AI project we manage. An agent is not a thing you ship once, it is a thing you run. Feedback is the cheapest, most direct instrumentation you have for running it well, and it is genuinely a waste to leave it switched off.

The honest limitations

I will not pretend the built-in feedback is a complete evaluation system, because it is not.

It tells you that a user was unhappy. It does not, on its own, tell you why in a structured way you can measure over time. If you want to know whether your agent is factually accurate across a defined set of questions, whether a prompt change made it better or worse, or whether it regressed after a platform update, thumbs-up and thumbs-down will not get you there. That is what proper evaluations are for, and Microsoft has been building tooling in that direction that is worth pairing with this. Think of user feedback as the smoke detector and evaluations as the diagnostic. You want both.

The reporting is also coarser than a dedicated analytics stack. You get aggregated views through the admin surfaces, which is enough to spot trends and triage, but if you want to slice feedback by user cohort, correlate it with specific agent actions, or track it against business outcomes, you will end up wanting to pull the data somewhere you have more control. For most businesses starting out, the built-in view is plenty. For a heavily-used agent that matters to the business, you will grow out of it, and that is fine.

None of that is a reason not to turn it on. It is the single easiest bit of instrumentation available to you, it costs nothing, and it will tell you things about your agent that you genuinely cannot learn any other way. My blunt advice: enable it on day one, even for a pilot. The worst outcome is that you learn your agent is worse than you thought, which is exactly the thing you want to learn early rather than late.

Where to start

If you already have a declarative agent running without feedback, add the manifest property this week and start watching. If you are still building, design the review process before you launch, not after. Decide who looks at the feedback, how often, and what happens when a theme emerges. That is the part that turns a feature into an actual improvement loop.

If you want a hand building Copilot agents that people actually keep using, feedback loops and all, that is squarely what we do. Have a look at what our AI agent builders work on, the broader Microsoft AI consulting we offer, and if you have got an agent you want to get right, get in touch.