Mini Project 3: Multi-Tenant SaaS Support Bot
A support bot that survives its first real customer: hard tenant isolation at retrieval, plus per-tenant cost tracking before usage bankrupts the free tier.
The Demo That Almost Shipped With a Data Leak
A support-bot demo goes great in the sales call: upload a customer's help docs, ask questions, get grounded answers. Someone on the team notices the retrieval query has no WHERE clause on tenant — every customer's documents live in the same vector store, and the "isolation" is just that nobody's typed a query specific enough to surface another tenant's content yet. That's not isolation, that's luck. The fix has to land before the second paying customer signs up, not after an incident report names the first one.
This project hardens the RAG shape from this roadmap's third phase into something that's actually safe to sell: retrieval that's structurally unable to cross a tenant boundary, not just unlikely to, plus per-tenant cost tracking so one customer's usage spike doesn't silently eat the margin on everyone else's plan.
Setup
// build.gradle
dependencies {
implementation 'org.springframework.ai:spring-ai-ollama-spring-boot-starter'
implementation 'org.springframework.ai:spring-ai-rag'
implementation 'org.springframework.ai:spring-ai-pgvector-store-spring-boot-starter'
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation 'org.springframework.boot:spring-boot-starter-data-redis' // per-tenant usage counters
}Every document ingested carries a mandatory tenant_id field in its metadata from day one — added at ingestion, never left optional:
public Document toTenantDocument(String tenantId, Document raw) {
raw.getMetadata().put("tenant_id", tenantId);
return raw;
}There is no code path in this project that ingests a document without a tenant_id. Treat a missing tenant_id at ingestion as a hard failure (throw, don't default to a sentinel value) — a document that silently lands with no tenant tag is a document that can leak to every tenant once retrieval assumes the field is always present.
1. Isolation as a Filter Expression, Not a Prompt Instruction
The naive fix is a system-prompt line: "only discuss documents belonging to this customer." That relies on the model choosing not to use content it was handed — the same category of mistake as trusting a prompt instruction to stop prompt injection. The correct fix is identical in shape to the staleness filtering from this phase's Knowledge Base Copilot project, applied to tenant instead of age:
public Advisor supportAdvisorFor(String tenantId) {
DocumentRetriever retriever = VectorStoreDocumentRetriever.builder()
.vectorStore(vectorStore)
.similarityThreshold(0.65)
.filterExpression(new FilterExpressionBuilder()
.eq("tenant_id", tenantId)
.build())
.build();
return RetrievalAugmentationAdvisor.builder()
.documentRetriever(retriever)
.queryAugmenter(ContextualQueryAugmenter.builder().allowEmptyContext(false).build())
.build();
}A tenant's query is structurally unable to retrieve a chunk it doesn't own — the vector similarity search never even considers other tenants' rows, the same way a WHERE tenant_id = ? clause in SQL doesn't rely on application code remembering to filter correctly on every query path. Build this advisor per-request from the authenticated tenant's ID, never from anything the client sends as free-form input.
Don't take tenantId from a request body or a header the caller controls unchecked. Derive it from the authenticated session/JWT claim the same way you'd derive any other authorization decision — a tenantId request parameter is exactly the kind of thing that turns into a one-line IDOR vulnerability the first time someone edits it in devtools.
2. Per-Tenant Cost Tracking
Spring AI's ChatResponse carries token usage on every call — the raw material for cost attribution:
@Service
public class TenantCostTracker {
private final StringRedisTemplate redis;
public void record(String tenantId, ChatResponse response) {
Usage usage = response.getMetadata().getUsage();
String key = "tenant:%s:tokens:%s".formatted(tenantId, LocalDate.now());
redis.opsForValue().increment(key, usage.getTotalTokens());
redis.expire(key, Duration.ofDays(35)); // keep ~a month of daily buckets
}
public long todayUsage(String tenantId) {
String key = "tenant:%s:tokens:%s".formatted(tenantId, LocalDate.now());
String value = redis.opsForValue().get(key);
return value == null ? 0 : Long.parseLong(value);
}
}Wrap every support-bot call with a budget check before spending the tokens, not just a log line after:
@Service
public class TenantSupportBotService {
private final ChatClient chatClient;
private final TenantCostTracker costTracker;
private static final long DAILY_TOKEN_BUDGET = 200_000;
public String ask(String tenantId, String question) {
if (costTracker.todayUsage(tenantId) >= DAILY_TOKEN_BUDGET) {
throw new TenantBudgetExceededException(tenantId, DAILY_TOKEN_BUDGET);
}
ChatResponse response = chatClient.prompt()
.advisors(supportAdvisorFor(tenantId))
.user(question)
.call()
.chatResponse();
costTracker.record(tenantId, response);
return response.getResult().getOutput().getText();
}
}A daily reset via a date-suffixed Redis key is deliberately simple — it resets naturally at midnight with no scheduled job needed, and EXPIRE cleans up old keys automatically. It's not perfectly precise (a burst right at midnight can technically span two buckets), which is a fine trade-off for a soft usage guardrail; reach for a proper sliding-window rate limiter only if a customer contract requires exact enforcement.
3. Handling the Budget-Exceeded Case Gracefully
A tenant hitting their budget shouldn't get a raw 500 — it's an expected, plan-tier outcome, not a bug:
@RestControllerAdvice
public class TenantBudgetAdvice {
@ExceptionHandler(TenantBudgetExceededException.class)
public ResponseEntity<Map<String, Object>> handle(TenantBudgetExceededException e) {
return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS).body(Map.of(
"error", "daily_budget_exceeded",
"tenantId", e.tenantId(),
"dailyBudget", e.budget(),
"resetsAt", LocalDate.now().plusDays(1).atStartOfDay()
));
}
}A 429 with a machine-readable resetsAt lets the frontend show "You've used today's support-bot budget — resets at midnight" instead of a generic error, and lets a sales-facing dashboard build a real "upgrade for a higher budget" prompt off the same data.
Putting It Together
@Service
public class TenantSupportBotService {
private final ChatClient.Builder chatClientBuilder;
private final VectorStore vectorStore;
private final TenantCostTracker costTracker;
private static final long DAILY_TOKEN_BUDGET = 200_000;
public String ask(String tenantId, String question) {
if (costTracker.todayUsage(tenantId) >= DAILY_TOKEN_BUDGET) {
throw new TenantBudgetExceededException(tenantId, DAILY_TOKEN_BUDGET);
}
Advisor tenantScopedAdvisor = RetrievalAugmentationAdvisor.builder()
.documentRetriever(VectorStoreDocumentRetriever.builder()
.vectorStore(vectorStore)
.similarityThreshold(0.65)
.filterExpression(new FilterExpressionBuilder().eq("tenant_id", tenantId).build())
.build())
.queryAugmenter(ContextualQueryAugmenter.builder().allowEmptyContext(false).build())
.build();
ChatResponse response = chatClientBuilder.build().prompt()
.advisors(tenantScopedAdvisor)
.user(question)
.call()
.chatResponse();
costTracker.record(tenantId, response);
return response.getResult().getOutput().getText();
}
}Every request is scoped to exactly one tenant at the retrieval layer, and metered against that same tenant's budget before a token is spent — the two problems that turn a great demo into an incident report, closed at the same layer they'd actually break at.
What's Next
Mini Project 8 (Role-Aware HR & Policy Assistant) reuses this exact FilterExpressionBuilder-as-access-control pattern one level deeper — filtering by role and region within a single tenant, not just between tenants. Worth pairing with this project if your product needs both layers of isolation at once.
Frequently asked questions
Why filter by tenant_id in the vector store instead of running a separate vector store per tenant?
Separate vector stores (or separate collections) is a legitimate alternative and some vector databases (Weaviate, Pinecone) support it natively as a first-class multi-tenancy feature — it trades operational complexity (provisioning per tenant) for a stronger isolation guarantee (no filter to forget). Metadata filtering is the right default for a small-to-medium number of tenants sharing infrastructure; reconsider per-tenant collections once tenant count or a compliance requirement makes shared infrastructure itself the risk.
What happens if the filterExpression is accidentally omitted on one code path?
Exactly the incident this project opened with — a silent cross-tenant leak. Don't rely on every call site remembering the filter; build the tenant-scoped advisor in one factory method (as above) that every request path is forced to go through, so there's exactly one place isolation could break instead of one per call site.
Should the token budget be enforced server-side only, or also surfaced to the tenant in real time?
Both — server-side enforcement (this guide) is the actual security/cost boundary and must exist regardless. A real-time usage indicator in the product UI is a product decision on top of it, not a substitute for it; never trust a client-side counter as the enforcement mechanism.
Does this pattern extend to rate limiting requests per second, not just tokens per day?
Yes, and it's a natural addition — wrap the same Redis-backed pattern with a shorter TTL and a lower threshold (e.g. a sliding 60-second window) as a second, faster-tripping guardrail, layered on top of the daily token budget rather than replacing it. This roadmap's rate-limiting simulator covers the algorithm choices (token bucket, sliding window) in more depth if you want to go beyond a fixed daily cap.