Most organizations already have the answers. They're in SharePoint, spread across hundreds of documents, and it takes an analyst hours to find the right paragraph in the right version.
A retrieval assistant can cut that to seconds. But only if people can trust it. That takes three things: it shows its source, it never shows a document the user isn't allowed to see, and it admits when it doesn't know.
Here's how we build one.
1. Decide what's in scope first
Don't index everything. Pick the libraries people actually research from and get the owner to approve the list.
For each source, record:
- Who owns it.
- Which version is authoritative when there are several.
- How long content stays current.
An assistant that confidently quotes a superseded draft does more damage than no assistant.
2. Keep the permissions with every chunk
This is the step teams skip, and it's the one that gets a project shut down.
When you pull a document from SharePoint, pull who can read it too. Store that list with every chunk you index. When someone asks a question, filter the search to chunks that person's account and groups can open, before the model sees anything.
Filter at search time, not after. If a restricted passage reaches the model, it can leak into the answer even if you hide the citation.
Then test it. Ask the same question as two users with different access and confirm the answers differ the way they should.
3. Chunk by structure, and keep the address
Split documents where the author did: headings, sections, tables. A fixed 500-word window cuts tables in half and separates a rule from its exception.
With each chunk, keep the document title, the section heading and the page number. That's what lets a citation point to the exact place, so a reviewer can check it in one click.
4. Search two ways, then rerank
Keyword search finds exact terms: an account code, a form number, a program name. Vector search finds meaning when the wording differs. Budget and policy questions need both.
Run both, merge the results, then use a reranker to put the best few passages on top. In our experience this step does more for answer quality than swapping models.
5. Make the model answer only from what it was given
The instruction is simple: answer from these passages, cite each claim, and if the passages don't cover the question, say so.
Then enforce it in code. If an answer comes back with no citation, don't show it. Show "not enough evidence in the approved sources" instead. Users forgive "I don't know." They don't forgive a confident wrong answer in a briefing.
6. Keep the index current
Documents change daily. Use change tracking to pick up edits, new files and deletions, and update permissions on the same schedule. A file someone lost access to yesterday shouldn't be answerable today.
7. Log every question and answer
Record who asked, what was retrieved, what was answered and which sources were cited. That log does three jobs. It's your audit trail. It shows you what people actually need. And the questions that failed become your next test cases.
How to know it works
Before release, build a fixed set of real questions with known answers and sources. Include questions that have no answer. Score whether the right source was retrieved, whether the answer matches it, and whether the assistant declined when it should have. Run that set again on every change. We cover the method in how to test an AI agent before production.
The short version
Scope it, carry the permissions, cite everything, refuse to guess, and log it all. None of that is exotic. It's just the work that separates a demo from a tool analysts rely on.
We've built this pattern for budget and policy research on a federal program. If you're planning something similar, see Knowledge & Research or tell us about your sources.