Production RAG: Access Control, Freshness & Multi-Tenancy
What naive RAG tutorials skip — keeping a shared index from leaking data across tenants, and keeping it from serving policies that were already replaced.
Beyond the Demo
The refund bot's RAG pipeline works well against one static policy PDF. Then the company grows: the knowledge base expands to include internal-only escalation procedures alongside the public refund policy, the policy itself gets revised every quarter, and the same bot gets sold to other companies who each need their own, isolated knowledge base. None of these are chunking or embedding problems — they're the production concerns naive RAG tutorials almost universally skip, because a single-tenant demo with one unchanging document never surfaces them.
Access Control: Retrieval Isn't Automatically Authorization
The uncomfortable default: a vector index has no concept of "who's allowed to see this" unless you explicitly build one in. If the internal escalation procedures get embedded into the same index as the public refund policy, a customer-facing query that happens to be semantically close to an internal document can retrieve it — the vector search doesn't know or care that one document is public and the other isn't, it only knows they're nearby in latent space.
Metadata filtering is the standard fix: attach access-control metadata to every chunk at ingestion time (audience: public vs internal, required role, tenant ID) and apply that filter before or during the vector search, not as a check on results afterward. Filtering after retrieval is a much weaker guarantee — it means the wrong document was retrieved and sat in context at least momentarily, and a policy that depends on catching that after the fact is fragile in exactly the way strict grounding was designed to avoid.
query_vector = embed("What's the escalation path for a VIP complaint?")
results = vector_index.search(
query_vector,
top_k=5,
filter={"audience": "internal", "tenant_id": "acme-support-team"}
)
Pinecone's metadata filtering guide covers the mechanics of filtering at query time rather than post-processing results — worth reading closely, since the difference between "filtered before search" and "filtered after search" is exactly the difference between a real access-control boundary and a leaky one.
Freshness: A Static Index Goes Stale the Moment the Policy Changes
The original hallucination this whole roadmap opened with was a model inventing a policy that never existed. A RAG pipeline with a stale index produces an eerily similar failure from the opposite direction: it confidently, accurately cites a policy excerpt that used to be true and was replaced last quarter. Grounding in a document doesn't help if the document itself is out of date — it just makes the wrong answer better-cited.
The fix is incremental indexing: track a version or last-modified timestamp per source document, and on each ingestion run, only re-chunk, re-embed, and re-index documents that actually changed — deleting or superseding the vectors tied to the old version, not just adding new ones alongside them. Re-embedding an entire corpus on every update works at small scale and becomes an expensive, slow habit the moment the knowledge base grows past a handful of documents, which is exactly when staying current matters most.
Multi-Tenancy: One Index, Many Customers, Zero Cross-Contamination
Once the bot serves more than one customer, each customer's documents need to stay isolated — Customer A's support bot should never retrieve so much as a chunk of Customer B's knowledge base, even by accident. There are two structurally different ways to get there:
- Separate indexes per tenant — the strongest isolation guarantee, since there's no shared index for a bug or misconfigured filter to leak across. The cost is operational: more indexes to provision, monitor, and keep in sync as tenant count grows.
- Shared index with a mandatory tenant-ID filter — cheaper to operate at scale, but the isolation guarantee is now only as strong as "every single query path correctly applies the tenant filter, with no exceptions." One missed filter in one code path is a real data leak, not a degraded-quality issue.
Weaviate's multi-tenancy documentation describes a middle ground worth knowing about: tenant-partitioned collections within one deployment, giving much of the operational simplicity of a shared system with stronger isolation boundaries than a bare metadata filter — the general pattern (partition at the storage layer, not just the query layer) is worth understanding even if you end up implementing it differently.
Check yourself
A shared vector index uses a tenant_id metadata filter for isolation, applied inside the application code that calls the vector search API. What's the main risk with this approach compared to separate indexes per tenant?
Back to the Refund Bot
Put together, this phase is the full story of how "a customer support bot confidently invented a policy" became "hallucination rate down 95%": chunk the real policy document sensibly, embed it with a model suited to the content, retrieve the right passages and force the model to cite them, filter that retrieval by who's actually allowed to see what, keep the index current as the policy changes, and — once the bot serves more than one customer — make sure none of that ever crosses a tenant boundary. None of these five phases individually would have solved the opening problem. Together, they replace "trust the model's training-time knowledge" with "trust a pipeline whose every stage is inspectable and correctable."
That's also the last piece before this roadmap moves past "make retrieval work at all" into "make retrieval work well at scale" — HNSW indexing, hybrid search, and query rewriting, starting next phase.
Frequently asked questions
Does metadata filtering slow down vector search?
It depends on the vector database and how the filter is implemented — some apply filters during the approximate nearest-neighbor search itself with minimal overhead, others filter a candidate set afterward, which can be slower and can also return fewer than top_k results if too many candidates get filtered out. This is exactly the kind of algorithmic detail Phase 5 covers when it gets into vector search internals.
How often should incremental indexing run?
It depends entirely on how often source documents actually change and how costly staleness is. A refund policy that updates quarterly doesn't need hourly re-indexing; a live inventory or pricing document might need near-real-time updates. The right cadence is a product decision about acceptable staleness, not a technical default to copy from a tutorial.
Is a shared index with metadata filtering ever the wrong choice regardless of cost savings?
Yes — for genuinely sensitive data (regulated industries, strict contractual data-isolation requirements), the operational savings of a shared index usually aren't worth betting compliance on 'every code path correctly applies the filter.' Separate indexes, or Weaviate-style partitioned collections, are the safer default whenever a cross-tenant leak would be a serious incident rather than a minor quality issue.