Microsoft 365 Copilot Retrieval API - What It Is and When to Use It
Almost every Australian organisation we talk to has the same problem. Years of policies, contracts, project files and procedures sit in SharePoint and OneDrive, and they want an AI assistant that can answer questions from that content. Then someone works out what it would take to build that properly: extracting documents, chunking them, embedding them, keeping a vector index in sync, and (the bit that kills most projects) making sure nobody can see an answer drawn from a file they don't have permission to open.
The Microsoft 365 Copilot Retrieval API exists to remove most of that work. It's one of the more useful things Microsoft has shipped in the Copilot extensibility space, and it gets far less attention than the agent builders and the flashy demos. Here's how it works, where we've found it useful, and where it still falls short.
What the Retrieval API actually does
At its core, the Retrieval API lets your application send a natural language query to Microsoft Graph and get back relevant text extracts from content the calling user can already access. It's the same semantic index and retrieval stack that Microsoft 365 Copilot uses to ground its own answers, exposed as an API so you can use it in your own apps.
You call an endpoint (it lives under the Microsoft Graph copilot namespace), pass a query string, tell it which data source to search, and get back a ranked list of results. Each result includes the web URL of the source item, one or more text extracts, and relevance scores. You can also ask for specific metadata fields like title or author to come back with each hit.
The supported data sources at the moment are:
- SharePoint content
- OneDrive for Business content
- Copilot connectors (the external items you've ingested from other systems through Microsoft Graph connectors)
What you do with those extracts is up to you. You can pass them into your own model in Azure AI Foundry, into an agent built with Semantic Kernel or the Microsoft Agent Framework, or into a non-Microsoft model entirely. The API does retrieval only. It doesn't generate an answer for you.
That separation is the whole point. You get Microsoft's retrieval and permission trimming, and you keep control over the model, the prompt and the user experience.
Why permissions matter more than anything else here
I've lost count of the RAG prototypes we've reviewed where the team indexed an entire SharePoint tenant into a vector database using a service account. The demo works beautifully. Then someone in security asks what happens when a graduate asks the bot about executive salary bands, and the project stops.
Rebuilding SharePoint's permission model in your own index is hard. You have to capture ACLs at index time, map Entra ID groups, handle sharing links, and keep it all current as permissions change. Most teams either get it wrong or quietly give up and restrict the bot to a single "safe" library.
The Retrieval API runs with delegated permissions in the context of the signed-in user. If the user can't open a document, it won't come back in their results. Sensitivity labels and the rest of your existing Microsoft Purview configuration still apply. For a lot of organisations we work with, particularly in financial services and government, that single property is what gets an AI project past the risk committee.
How it compares to building your own RAG pipeline
To be fair, a custom pipeline still makes sense in some situations, so here's my honest comparison.
The Retrieval API wins when:
- Your content already lives in SharePoint and OneDrive and changes often
- Permissions are complex and you can't afford to get them wrong
- You don't want to own an indexing pipeline, a vector store and the sync jobs that go with them
- Your users already have Microsoft 365 Copilot licences
A custom pipeline still wins when:
- You need control over chunking strategy, for example splitting legal contracts by clause rather than by token count
- The content lives mostly outside Microsoft 365 and you'd have to ingest it through connectors anyway
- You need hybrid search with your own ranking logic, or domain-specific embeddings
- You're serving users who don't have Copilot licences
That last point catches people. Using the Retrieval API generally requires the calling user to have a Microsoft 365 Copilot licence, though Microsoft has been experimenting with pay-as-you-go options for some scenarios. Check the current licensing terms on the documentation page before you design anything around it. If you're building something for 2,000 frontline workers who don't have Copilot, the economics may look very different.
Filtering and scoping your queries
One feature I'd encourage every team to use from day one is the filter expression. The API accepts KQL (Keyword Query Language) filters, so you can scope retrieval to particular SharePoint sites, paths, file types or author fields.
This matters because tenant-wide retrieval is noisy. If you're building an HR policy assistant, you don't want it pulling extracts from a project team's draft meeting notes that happen to mention "leave entitlements". Scoping the query to the HR policy site with a path filter improves answer quality a lot more than any prompt tweak.
A pattern we've used a few times:
- Classify the user's question first (HR, IT, finance, operations)
- Map each category to a set of SharePoint sites
- Call the Retrieval API with a KQL filter restricting results to those sites
- Pass the top extracts to the model with clear instructions to cite the source URL
It's simple, and it beats sending every question to the whole tenant.
Limits and rough edges
The API isn't perfect, and you should know about these before committing to it.
Result counts are capped. You get a small, fixed maximum number of results per request (25 at the time of writing). For most question-answering scenarios that's fine. For anything that needs broad coverage, like "summarise every incident report from the last quarter", it's the wrong tool. Retrieval APIs are built for finding the most relevant passages, not for enumerating everything.
Query strings have a length limit. If you're tempted to stuff the user's entire conversation history into the query, don't. Rewrite the user's question into a focused search query first. This is good practice for any RAG system anyway.
Rate limits apply per user. For interactive assistants this rarely matters. For batch jobs that run on behalf of a single service identity, it will. Design accordingly.
You don't control the chunking. The extracts come back the way Microsoft's index has broken them up. Usually they're sensible. Occasionally you'll get an extract that cuts off mid-table or lacks the heading that gives it context. You can mitigate this by fetching more of the source document when a result looks promising, but that adds latency.
Content quality still matters. This is the one people least want to hear. If your SharePoint is full of duplicates, outdated versions named "Policy_FINAL_v3_USE_THIS.docx" and scanned PDFs with no OCR, the Retrieval API will faithfully return that mess. No retrieval system fixes bad content. We usually recommend a content clean-up of the highest-value sites before any assistant goes live.
Where this fits with Copilot Studio and declarative agents
There's some confusion about how the Retrieval API relates to the other ways of extending Copilot, so here's the short version.
If you want an agent that lives inside Microsoft 365 Copilot (in Teams, Outlook, the Copilot app), you'd usually build a declarative agent or use Copilot Studio, and point it at SharePoint knowledge sources. Microsoft handles retrieval for you behind the scenes.
The Retrieval API is for when you're building something outside that experience. A custom web app, a line-of-business system with an embedded assistant, a multi-agent workflow running in Azure, or a solution that uses a model Microsoft 365 Copilot doesn't offer. It's the "bring your own orchestration, borrow our retrieval" option.
In our own projects, it's been most useful for customers who already have a custom AI application and want to add Microsoft 365 knowledge to it without building a second indexing pipeline. One professional services firm we worked with had an internal proposal-writing tool built on Azure OpenAI. They wanted it to pull relevant past proposals and capability statements from SharePoint. Before the Retrieval API, that meant a nightly export job, a separate search index and an awkward permissions workaround. With it, the tool queries SharePoint directly in the context of the consultant using it, and the export job is gone.
Practical advice before you start
A few things we'd tell any team about to build on this:
- Prototype with Graph Explorer first. You can call the API interactively and see exactly what comes back for real queries against your tenant. Do this before writing any code. It'll tell you quickly whether your content is in good enough shape.
- Log queries and results during pilot. You'll learn more about retrieval quality from a week of real usage logs than from any amount of upfront testing.
- Always show citations. The API gives you source URLs. Put them in front of the user. It builds trust and makes wrong answers obvious instead of dangerous.
- Plan for licensing early. Work out who will use the app and whether they're licensed before you build. Finding out at go-live is painful.
- Don't skip the security review just because permissions are handled. Permission trimming is only as good as your permissions. If half the company has accidental access to a sensitive site through an "Everyone except external users" grant, the API will respect that grant. Copilot projects have a habit of surfacing oversharing that's been there for years.
My take
The Retrieval API is the right default for grounding custom AI apps in Microsoft 365 content. It removes the hardest and riskiest part of enterprise RAG (permissions) and saves you from running your own indexing infrastructure. The licensing dependency and the lack of control over chunking are real constraints, but for most organisations already invested in Microsoft 365, they're a fair trade.
Where I'd be cautious is treating it as a replacement for proper content governance. The organisations getting the best results are the ones that cleaned up their SharePoint first.
If you're working out whether to use the Retrieval API, Copilot Studio or a fully custom RAG build, we help teams make that call and build the result. Have a look at our Microsoft AI consulting work, or get in touch and we can talk through your scenario.
Reference: Microsoft 365 Copilot Retrieval API overview