Interview: Multi-Tenant Enterprise RAG
Designing RAG for many customers or departments: tenant isolation, document-level permissions enforced at retrieval, authentication, prompt injection through documents, data residency and deletion, per-tenant cost and quotas, and the enterprise review questions that come with them.
What you will be able to do
- Compare isolation models for multi-tenant RAG
- Enforce document permissions during retrieval
- Defend against prompt injection in documents
- Plan deletion, residency, quotas and cost per tenant
The idea, in plain English
Enterprise RAG adds a hard requirement: one customer, team or employee must never see another's data. Interviewers check that you enforce this in the retrieval layer, not in the prompt or the interface.
Other enterprise topics follow: authentication, auditing, deletion rights, data location, noisy neighbours and cost per tenant.
FastAPI Lessons 6.1 and 6.2 cover authentication and authorization; the Agentic AI production lessons cover prompt injection and guardrails.
Worked example: Twelve questions from an enterprise design review of a shared RAG platform.
Intermediate: What is multi-tenancy in a RAG platform?
One platform serving many separate customers or departments (tenants), each with their own documents and users, whose data must stay separate.
- Isolation, permissions and fair resource use are the core problems.
Intermediate: How do you isolate tenants' data?
Three models: a separate index or database per tenant (strongest isolation, more overhead), a namespace or collection per tenant, or one shared index with a tenant ID filter on every query (cheapest, but one missing filter leaks data).
- Large or regulated tenants often get their own index.
- Make the tenant filter impossible to forget - set it in one place.
Intermediate: How do you enforce document-level permissions?
Store access information (groups, roles, owners) as metadata on every chunk, and filter retrieval by the current user's permissions on every query - so forbidden chunks never reach the prompt.
- Never retrieve first and hide later in the UI.
- Sync permission changes from the source systems.
Advanced: Why is it not enough to tell the model "do not reveal confidential data"?
Prompts are not security. If a forbidden chunk is in the prompt, the model can leak it - by mistake or because of prompt injection. Access control must happen before the text reaches the model.
- Retrieval filters are the security boundary.
Advanced: What is indirect prompt injection in RAG, and how do you defend against it?
A document contains hidden instructions ("ignore your rules and ...") that the model may follow when the chunk is retrieved. Defences: treat retrieved text as data, separate it clearly from instructions, limit what the system can do, filter outputs, and scan or restrict untrusted sources.
- Especially dangerous when RAG feeds an agent with tools.
Advanced: How do you authenticate users and services?
Users log in through the company's identity provider (SSO with OAuth/OpenID Connect); every API request carries a token that identifies the user and tenant; services use their own credentials with limited scopes.
- Derive the tenant from the token - never from a request field the client can change.
Advanced: A customer asks you to delete all their data. What must happen?
Delete their documents, chunks, vectors, caches, logs with content, conversation history and backups (within the backup policy), and confirm - which is much easier with per-tenant storage and stable IDs.
- Design deletion from day one; it is hard to add later.
Advanced: How do you stop one tenant from slowing down everyone else?
Per-tenant quotas and rate limits, separate ingestion queues or priorities, limits on document size and volume, and dedicated resources for the largest tenants.
- This is the "noisy neighbour" problem.
Advanced: How do you track and bill cost per tenant?
Record tokens, embeddings, storage and requests per tenant in traces and usage tables, aggregate monthly, and set budgets and alerts per tenant.
- Cost data also shows which tenants need optimisation.
Professional: What would you include in an enterprise security review of your RAG platform?
Data flow diagram, isolation model, permission enforcement and sync, authentication, encryption in transit and at rest, prompt-injection defences, logging and audit, deletion and retention, data residency, third-party model providers, and an incident response plan.
- Bring evidence: tests that prove one tenant cannot retrieve another's data.
Professional: How do you prove that tenants cannot see each other's data?
Automated tests that log in as one tenant and search for another tenant's unique content, run on every release; plus code review of the single place where the tenant filter is set, and periodic audits.
- Test the boundary, not just the happy path.
Professional: A large customer wants their data to stay in their own country or cloud. What are your options?
Deploy a dedicated stack (index, workers, and a local or regional model) in their region or cloud account, or use a provider with data-residency guarantees - and check every component that touches their data, including logs and the LLM provider.
- Model providers are often the hidden part of the data flow.
Key points
- Choose an isolation model: per-tenant index, namespace, or shared index with mandatory filters.
- Enforce permissions as retrieval filters on every query.
- Prompts are not security; documents can carry prompt injection.
- Plan deletion, residency, quotas and per-tenant cost from day one.
- Prove isolation with automated cross-tenant tests.
Interview questions
What is a tenant?
A customer or group whose data must be kept separate.
Where must permissions be enforced?
In retrieval, on every query.
What is a noisy neighbour?
A tenant whose heavy use slows others down.
What is data residency?
A rule about which country or region data may be stored in.
What is SSO?
Single sign-on - users log in once with the company identity provider.
Where should the tenant ID come from?
From the authenticated token, never from client input.
Why audit logs?
To show who accessed what, and when.
Strongest isolation model?
A separate index or database per tenant.
Comments
Sign in to leave a comment. Your name and photo come from Google; nothing else is shared.
Loading comments...