How to Evaluate Multi-Turn Conversations in Microsoft 365 Copilot Extensions

Most Copilot agents that fail in production do not fail on the first question. They fail on the third. Someone asks a follow-up, the agent forgets what they were talking about, loses the thread, contradicts what it said two messages ago, or confidently answers a question the user never actually asked because it misread the context. And the reason these failures reach production is almost always the same: the agent was tested one prompt at a time, and real people do not talk one prompt at a time.

If you are building a Microsoft 365 Copilot extension, whether that is a declarative agent, a plugin, or a custom engine agent, this is the gap that will bite you. You test it by typing a good clear question, it gives a good clear answer, you ship it. Then a real user has an actual conversation with it, with all the back-and-forth and half-finished thoughts and "no I meant the other one" that real conversations contain, and it falls apart. Evaluating multi-turn conversations is how you catch that before your users do.

Microsoft's guidance on evaluating multi-turn conversations sets out the approach. Here is what it looks like in practice, from building and testing these agents rather than just reading about them.

Why single-turn testing lies to you

A single-turn test asks the agent one question and checks the answer. It is easy to write, easy to automate, and it tells you almost nothing about how the agent behaves in a real session. Because the hard part of a conversation is not answering a question. It is answering a question in the context of everything that came before it.

Think about how people actually use these things. "What were our sales in Queensland last quarter?" Good, it answers. "What about Victoria?" Now the agent has to understand that "what about Victoria" means "same question, different state", carrying forward "sales" and "last quarter" from the previous turn. "And compared to the year before?" Now it has to hold two things in context at once. A single-turn test never exercises any of this, because every question in a single-turn test arrives with no history. You are testing the agent on the one input shape it will rarely receive.

The failures that live in the gaps between turns are the expensive ones. An agent that answers each isolated question perfectly but cannot follow a conversation feels broken to a user even when every individual answer is technically correct, because humans experience the conversation as a whole, not as a list of independent Q&A pairs.

What multi-turn evaluation actually measures

When you evaluate across turns, you are testing a different and harder set of things. Does the agent maintain context, remembering entities, filters, and the subject of discussion as the conversation moves. Does it handle references, understanding that "it", "that one", "the second option" point back to something said earlier. Does it stay consistent, not contradicting an answer it gave three turns ago. Does it recover gracefully when the user corrects it or changes direction mid-conversation.

There is also the harder question of whether it knows when it has lost the thread. A good agent that is unsure whether "Victoria" means the state or a person named Victoria should ask, not guess. Testing across turns is how you find out whether it guesses, and whether it guesses wrong in a way that sends the whole conversation off a cliff.

For agents that call tools or actions, and most useful Copilot extensions do, there is another layer. The agent has to decide when to call an action and with what parameters, and those parameters often depend on earlier turns. If the user said "the Sydney office" four messages ago and now says "book a room there", the agent has to carry "Sydney" into the action call. Multi-turn evaluation is where you catch the agent that calls the right action with the wrong parameter because it lost track of context.

Building conversations that reflect reality

The practical work here is designing test conversations that look like real ones, not idealised ones. This is where a lot of teams go wrong, because it is tempting to write clean, well-formed test conversations where every turn is a perfectly phrased question. Real users do not do that.

Real conversations have follow-ups that assume context. They have vague references. They have corrections: "no, I meant the March figures". They have topic changes and then returns to the earlier topic. They have the user testing the agent, being terse, being ambiguous, asking two things at once. Your test conversations should include all of that, because that is the input distribution your agent will actually face. A test suite full of polite, complete, unambiguous prompts will pass happily and tell you nothing about the messy reality.

Build a set of representative multi-turn scenarios grounded in what your users actually do. For an HR agent, that might be someone asking about leave, then asking a follow-up about their specific balance, then asking how to apply, then getting a detail wrong and being corrected. Run the whole conversation, not each turn in isolation, and evaluate whether the agent held it together end to end.

How to judge the answers

Evaluating a conversation is harder than evaluating a single answer, because "correct" now depends on context and there is often more than one acceptable response. This is where using a capable model as a judge earns its place. You give the judge the full conversation and a rubric, did the agent maintain context, was the answer grounded and accurate, did it handle the reference correctly, and it scores each turn with the benefit of seeing the whole exchange.

Model-as-judge is not flawless and I would not treat its score as gospel, but for evaluating conversational quality at any scale it is far more practical than a human reading every transcript, and far more meaningful than exact-string matching, which is useless the moment there is more than one right way to phrase an answer. My honest position: use automated judging to cover breadth and catch regressions across hundreds of conversations, and use human review on a sample and on anything the judge flags as borderline. The combination is what works. Either one alone leaves gaps.

Build these evaluations so they run repeatedly, not once. The reason is that Copilot agents are not static. The underlying models get updated, you change a prompt, you add a new action, you tweak the grounding data, and any of those can quietly change conversational behaviour in ways a single manual test will never surface. An evaluation suite you can re-run turns "we think it still works" into "we checked it still works", which is the difference between shipping with confidence and shipping and hoping.

What to watch out for

A few honest cautions. Multi-turn evaluation is more effort to set up than single-turn, and the temptation is to skip it and rely on a few manual chats before launch. That manual testing is better than nothing but it does not scale and it is not repeatable, so it catches almost nothing on the next change.

The other trap is testing only the happy path. The conversations where the user is clear and cooperative are the ones least likely to break. The value is in the awkward ones: ambiguity, correction, topic-hopping, the user who asks something slightly outside what the agent is meant to handle. Those are where agents fail and where evaluation earns its keep. If your test conversations are all polite and tidy, you have built a suite that passes and a product that breaks.

And do not forget that grounding and context are related but different. An agent can maintain conversational context perfectly and still give a wrong answer because its underlying data is stale or its retrieval missed. Evaluate both, and when something is wrong, work out which layer failed before you start changing prompts.

Where this fits

Getting Copilot agents to work over a real conversation, not just a clean demo prompt, is most of the difference between a pilot that impresses in a meeting and something people actually use every day. The evaluation work is not glamorous, but it is what stops you shipping an agent that looks great in the demo and frustrates everyone by Thursday.

This is central to how we build and ship agents in our Microsoft Copilot and Copilot Studio work, and it connects to the broader discipline we bring as AI agent builders where evaluation and reliability matter more than the flashy first demo. If you are building Copilot extensions and want them to hold up in real use rather than just the sales pitch, that is exactly the kind of work we do. Happy to have a chat about getting proper evaluation into your agent development.