Knowledge a business has already written down can still be hard to find: prior support resolutions in a ticket system, configuration notes in a CRM, a policy PDF on a file share, the reason a customer gets handled a certain way buried in an email thread. A custom AI knowledge base can make those records answerable in plain language, with a citation back to the record.
It can only work from what was written down. When the person who knew why a decision was made leaves without recording it, no retrieval system recovers that reason. What a knowledge base can do is keep the supplied records findable, show which record an answer came from, and give people a way to correct it.
One Support Queue as the First Domain
A support team is a reasonable place to start because its sources are concrete and its mistakes are visible. The useful sources are closed tickets with their resolutions, product notes, the escalation path, and the configuration record for the customer on the line.
Those sources behave differently. A resolution from a closed ticket is a document that can be indexed and cited. The customer's current configuration changes, and this first version deliberately answers questions about it with a query to the CRM or entitlement record at the moment the question is asked, instead of indexing a copy that would need its own refresh. That choice is where software integration work comes in: the answer layer needs a live, permission-checked call into the system that holds the record.
When a Cited Source Goes Stale
A support agent sees a troubleshooting step that no longer matches the product and flags it from the result screen. The review item needs the question, the answer, the cited source and its version, and the name of the person responsible for that source document. The reviewer then replaces the source, retires it, or leaves it and records why.
Retiring a source only helps if the retraction reaches the retrieval index. Search indexes keep their own copy of each document, and removing the file from the source does not always remove it from the index. Azure AI Search is a documented example. Its Azure Storage indexers pick up new and changed files from their timestamps but do not track deletions; Microsoft's change and delete detection guidance relies on a soft-delete strategy, either native blob soft delete or a custom metadata flag, and says the deletion policy must be in place from the very first indexer run. Documents deleted before the policy existed stay in the index even if the policy is added and the indexer reset later. With native blob soft delete, which also requires blob versioning to be off, the indexer removes a document only if it processes the blob while it is still soft deleted, so the storage retention period has to outlast the wait for the next successful indexer run; Microsoft recommends setting it much higher than the indexer schedule. The check belongs in the review workflow itself: after a retirement, confirm after a successful indexer run inside that retention period that the retired document's ID no longer comes back in retrieval results.
Who Can See Which Answer?
A client portal might show account-specific setup guidance while hiding pricing exceptions, unresolved internal notes, and escalation history. A support agent may need troubleshooting notes a customer should never see. The answer layer has to respect the same boundaries as the systems it reads from, and an approval in the review queue should not widen what a user could already read in the source.
That means filtering at retrieval time using the identity of the person asking. In an indexed knowledge base the permissions are also a copy. Azure AI Search's native document-level access control, in preview as of September 2026, compares the caller's Microsoft Entra claims against permission metadata already stored in the index, and a permission change in the source shows up in results only after that metadata is synchronized again. Its generally available alternative, a security filter, is string matching on an identifier the application supplies, with no authorization behind the string. What Your Search Index Still Answers After You Revoke Access follows that refresh problem in detail.
A Retirement Test for the Support Queue
Use a known retired ticket as the test record. After a successful indexer run within the soft-delete retention period, query the retrieval layer under the support agent's identity and check that ticket's document ID. Repeat under the customer-facing identity if the same source feeds the portal. If the ID still appears, keep the retirement item open and inspect deletion tracking before accepting new answers from that source.