User Experience Features for Custom Engine Agents in Microsoft 365 Copilot

Here's a pattern I've watched play out more than once. A team builds a custom engine agent, wires it up to their own orchestration and their model of choice, gets it answering hard questions accurately, demos it, everyone's impressed. Then it goes live inside Microsoft 365 Copilot, real people start using it, and the feedback isn't about the answers at all. It's "why does it just sit there for ten seconds doing nothing", "I couldn't tell if it was still working or had crashed", "it dumped a wall of text and I didn't know where the important bit was". The intelligence was fine. The experience felt broken.

That gap is what the user experience features for custom engine agents are about. When you build a custom engine agent, you own the brains - your orchestration, your logic, your model. But it renders inside the Copilot chat surface, and Copilot gives you a set of UX capabilities to make that rendering feel like a finished product rather than a raw API with a chat box bolted on. These features are the difference between something that demos well and something people actually keep using.

Why the experience is doing more work than you think

People judge a chat agent in the first few seconds, and they judge it on feel, not accuracy. An agent that starts responding immediately with a visible sign it's working feels smart even when it's slow. An agent that goes silent after you hit enter feels broken even when it's about to return a perfect answer. The user can't see your orchestration. All they have to go on is what the surface shows them, and if the surface shows them nothing, they assume nothing is happening.

This is the thing that trips up teams who come from a pure API background. When you're testing in Postman, latency is just a number. Inside a chat window with a human waiting, that same latency is dead air, and dead air reads as failure. A custom engine agent that takes eight seconds to answer with no feedback will get abandoned. The same agent showing "searching your documents" then "drafting a response" over those same eight seconds feels responsive and trustworthy. Identical speed, completely different verdict from the user.

So the UX layer isn't decoration on top of the real work. For adoption, it often is the real work. We've had projects where the model and retrieval were solid from early on, and the last stretch of effort - the part that actually moved usage - was entirely about making the thing feel right in the chat surface.

What the surface gives you to work with

The core idea is that Copilot lets your custom engine agent communicate its state and structure the response, rather than only returning a blob of text at the end.

Progress and streaming. The single most valuable thing you can do is show the user that work is happening. Streaming the response so text appears as it's generated, and surfacing what the agent is currently doing while it works, kills the dead-air problem. This matters most for exactly the agents worth building - the ones doing multi-step work, calling tools, retrieving from several sources. Those take real time, and that time needs to be visible. An agent that narrates "checking the policy documents" then "cross-referencing with your project" while it works turns a slow response into a reassuring one. Do not skip this. It's the highest-return piece of UX you'll implement.

Structured and rich responses. A wall of plain text is hard to use even when the content is good. Being able to return formatted content - proper markdown, cards, structured layouts - lets you put the important part where the eye lands and organise the rest so it's scannable. If your agent returns a recommendation with supporting detail, the recommendation should be visually obvious and the detail should sit underneath it, not be buried in paragraph four.

Citations and references. For any agent grounded in your organisation's content, showing where an answer came from is close to mandatory in a business setting. People won't act on an answer they can't check, especially for anything that carries consequences. A confident answer with no source is treated as a guess, and rightly so. Surfacing the documents and data behind a response is what earns the trust that gets people acting on it instead of double-checking everything manually, which defeats the point of the agent.

Suggested actions and follow-ups. Offering sensible next steps or follow-up prompts helps people who don't yet know what your agent can do. A lot of agent underuse isn't distrust, it's that users genuinely don't know what to ask. Good suggestions teach the agent's capabilities through use, which is far more effective than a help doc nobody reads.

This is the layer that separates the custom engine agent route from the simpler declarative agent route. You take on more build complexity, and in exchange you control the full experience. Whether that trade is worth it for a given project is exactly the sort of thing we work through with clients as part of our Copilot Studio consulting and broader AI agent development. Sometimes the answer is that a declarative agent does the job with a fraction of the effort. Sometimes the requirements genuinely need the custom engine, and then this UX layer becomes central.

The honest read on where this sits

A few things worth being straight about.

The UX features are genuinely good and clearly the right direction. Microsoft has understood that a custom engine agent isn't just a model endpoint, it's an experience, and they've given you real tools to shape that experience rather than making you fight the surface. That's the correct call.

But this is still more work than teams expect. The reason people reach for a custom engine agent is control, and control has a cost. You're not just implementing your orchestration, you're also implementing how it presents - the streaming, the state signals, the response formatting, the citations. Budget for that. The team that scopes only the "make it answer correctly" part and treats the experience as a quick finish at the end always runs over, because the experience turns out to be a real chunk of the work, not a garnish.

The features also give you room to make things worse, not just better. Rich formatting used badly is more confusing than plain text, not less. Progress messages that don't reflect what's genuinely happening are worse than none, because a spinner that says "searching" while the thing is actually stuck erodes trust fast when people notice. These tools reward restraint. Show real state, format to aid reading, cite honestly, and stop there. The temptation to use every capability because it exists is one to resist.

And the whole thing is still evolving. Custom engine agents and their surface are an area Microsoft is actively developing, so specifics will move. Build against the current capabilities, but keep the presentation layer loosely coupled from your core logic so that when the surface changes, you're adjusting a layer rather than unpicking your whole agent.

How I'd approach it

Treat the experience as a first-class requirement, not a polish phase. When we scope a custom engine agent, streaming, state feedback and citations go in the plan from the start, alongside the retrieval and orchestration, because retrofitting them onto a design that assumed a single blob response at the end is genuinely painful. Design for progressive, cited, structured responses from day one and the build is straightforward. Bolt them on afterwards and you'll be re-plumbing.

Test with real users early, and watch their faces during the wait, not just their reaction to the answer. The dead-air problem is invisible in your own testing because you know the agent works and you know how long it takes. A first-time user knows neither, and their experience of those silent seconds is the thing that decides whether they come back. That feedback is worth more than any amount of internal QA on answer quality.

The pattern underneath all of this is one we keep coming back to with clients: an AI feature succeeds or fails on adoption, and adoption is mostly about how the thing feels to use, not how clever it is underneath. A brilliant agent nobody trusts or enjoys using is a failed project. If you're building agents inside Microsoft 365 and want them to actually get used rather than quietly ignored after launch, that's the work we do - come and talk to us.