Multi-Turn Conversations, Prompt Templates & Structured Output

Two Bug Reports From the Same Sprint

The fintech team from the previous guide shipped their /api/tickets/triage endpoint. It classified support tickets into BILLING, TECHNICAL, FRAUD, or GENERAL by returning a raw string and calling .trim() on it. Two bug reports landed within the same week:

1. "Triage returned 'This ticket appears to be about BILLING.' instead of BILLING." The model followed the spirit of the instruction, not the letter — and the routing queue's switch statement had no case for a full sentence, so the ticket silently fell through to GENERAL.
2. "The bot forgot what the customer just said." Product added a follow-up question — "can you also check if this affects my linked account?" — and the bot answered as if it had never seen the original ticket. Every call to chatClient.prompt() was, as the first guide put it, completely stateless.

Both bugs have the same root cause: the first guide got you talking to a model, but it didn't give the model a contract for output, or the application any memory of the conversation so far. This guide fixes both, and along the way properly introduces the tool that makes it possible — the ChatClient advisor chain.

1. Advisors: ChatClient's Interceptor Chain

You already used ChatClient.Builder in the last guide. What we skipped over is how ChatClient stays extensible without every feature — memory, RAG, logging, structured output — turning into constructor parameters. The answer is advisors: a chain of interceptors that sit between your .prompt() call and the model, each able to inspect or rewrite the request before it goes out, and the response before it comes back.

If you've written a Spring MVC HandlerInterceptor or a servlet Filter, this is the exact same shape: a request flows through a chain, each link does its job, and the chain unwinds on the way back. Every remaining skill in this guide — memory, structured output validation — is implemented as an advisor. You rarely write one yourself; you mostly attach the ones Spring AI ships.

@Service
public class SupportChatService {
 
    private final ChatClient chatClient;
 
    public SupportChatService(ChatClient.Builder builder, ChatMemory chatMemory) {
        this.chatClient = builder
            .defaultSystem("You are a support agent for Acme Fintech. Be concise and factual.")
            .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
            .build();
    }
}

.defaultAdvisors(...) attaches an advisor to every call this ChatClient makes. You can also attach one for a single call with .advisors(...) on the .prompt() chain — useful when only one endpoint needs, say, verbose logging.

2. PromptTemplate: Stop Concatenating Strings

The first guide built prompts with plain string concatenation ("Write a welcome message for a new customer named " + customerName). That's fine for a two-word substitution; it gets ugly and error-prone once a prompt has five variables, conditional sections, or needs to be reviewed by someone who isn't a Java developer. PromptTemplate — and ChatClient's built-in .param() support — solves this properly.

String reply = chatClient.prompt()
    .user(u -> u.text("""
        Draft a reply to this support ticket.
        Customer tier: {tier}
        Ticket category: {category}
        Ticket text: {ticketText}
        """)
        .param("tier", customer.tier())
        .param("category", category)
        .param("ticketText", ticketText))
    .call()
    .content();

Each {placeholder} is substituted before the prompt is sent — the same mental model as a SQL prepared statement, and for the same reason: it separates the fixed instruction from the variable data, which makes the prompt reviewable and testable independent of any specific customer or ticket.

Loading templates from a file

For anything longer than a few lines, keep the prompt out of your Java source entirely:

PromptTemplate template = PromptTemplate.builder()
    .resource(new ClassPathResource("prompts/support-reply.st"))
    .build();
 
Prompt prompt = template.create(Map.of(
    "tier", customer.tier(),
    "category", category,
    "ticketText", ticketText
));

# src/main/resources/prompts/support-reply.st
You are a support agent for Acme Fintech.

Customer tier: {tier}
Ticket category: {category}

Examples of the tone we want:
- "Thanks for flagging this — I've checked your account and..."
- "I'm sorry for the trouble. Here's what's happening..."

Draft a reply to this ticket: {ticketText}

The payoff: localization without a redeploy

Once a prompt lives in a resource file instead of a Java string, a new locale is a new file, not a new deployment:

@Value("classpath:prompts/support-reply-${spring.profiles.active:en}.st")
private Resource replyPrompt;
 
// prompts/support-reply-en.st
// prompts/support-reply-es.st
// prompts/support-reply-fr.st

This is the same separation-of-concerns argument as templating a Thymeleaf view instead of building HTML with StringBuilder — the prompt becomes an editable asset, not logic buried inside a class.

3. Structured Output: Fixing Bug #1

Here's the actual fix for .trim()-and-hope. Instead of asking for a string and parsing it yourself, describe the shape you want with a Java record, and let Spring AI's BeanOutputConverter handle both the instructions to the model and the parsing back.

public enum TicketCategory { BILLING, TECHNICAL, FRAUD, GENERAL }
 
public record TicketTriage(
    TicketCategory category,
    double confidence,
    String suggestedReply
) {}

@PostMapping("/triage")
public TicketTriage triage(@RequestBody String ticketText) {
    return chatClient.prompt()
        .user(u -> u.text("Triage this support ticket: {ticketText}")
            .param("ticketText", ticketText))
        .options(ChatOptions.builder().temperature(0.0).build())
        .call()
        .entity(TicketTriage.class);
}

.entity(TicketTriage.class) does three things automatically: generates a JSON schema from TicketTriage, appends formatting instructions to the prompt telling the model to reply in that exact shape, and parses the JSON response back into a real TicketTriage object — category is now a genuine TicketCategory enum value, not a string you hope matches one.

// Option 1: ask for schema validation with automatic retry on failure
TicketTriage result = chatClient.prompt()
    .user(u -> u.text("Triage this support ticket: {ticketText}").param("ticketText", ticketText))
    .call()
    .entity(TicketTriage.class, spec -> spec.validateSchema());
 
// Option 2: use the provider's native structured-output mode (OpenAI GPT-4o+,
// Anthropic Claude 3.5+, Gemini 1.5+, Ollama) — the model itself enforces the
// schema, so no format instructions are needed in the prompt at all.
TicketTriage result2 = chatClient.prompt()
    .user(u -> u.text("Triage this support ticket: {ticketText}").param("ticketText", ticketText))
    .call()
    .entity(TicketTriage.class, spec -> spec.useProviderStructuredOutput().validateSchema());

For a classification endpoint feeding an automated routing queue, reach for useProviderStructuredOutput() whenever your model supports it — it moves the guarantee from "the model was asked nicely" to "the provider's API enforces it."

Simpler shapes don't need a record

Not every response justifies a custom type. .entity() also accepts a ParameterizedTypeReference for generic shapes:

List<String> tags = chatClient.prompt()
    .user(u -> u.text("List 3 relevant tags for this ticket: {ticketText}").param("ticketText", ticketText))
    .call()
    .entity(new ParameterizedTypeReference<List<String>>() {});

ListOutputConverter and MapOutputConverter cover List<String> and Map<String, Object> this way. Reach for a record like TicketTriage the moment the shape has more than one field with a specific type — that's where you get compile-time safety back.

4. Conversation Memory: Fixing Bug #2

Spring AI splits memory into two separate concerns, and the split matters once you start debugging:

• ChatMemory decides which messages to keep and when to forget them — a policy.
• ChatMemoryRepository decides where they're stored — a storage backend.

The default ChatMemory policy is MessageWindowChatMemory: a sliding window (20 messages by default) that evicts the oldest messages once the window fills up. Critically, it evicts whole turns — a user message plus every assistant reply and tool call that followed it — never leaving a dangling tool response with no matching call. It also always keeps the SystemMessage, so your persona and constraints never age out even in a very long conversation.

Choosing a ChatMemoryRepository

Development: in-memory, zero setup

@Bean
ChatMemory chatMemory() {
    return MessageWindowChatMemory.builder()
        .maxMessages(20)
        .build(); // backed by InMemoryChatMemoryRepository by default
}

Production: persisted to a database you already run

// build.gradle — persist chat history across restarts and app instances
implementation 'org.springframework.ai:spring-ai-starter-model-chat-memory-repository-jdbc'

Add that starter and Spring Boot auto-configures a JdbcChatMemoryRepository, wired straight into the same ChatMemory bean — no code change in SupportChatService.

Wiring it up and using a conversation ID

this.chatClient = builder
    .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
    .build();
 
@PostMapping("/tickets/{ticketId}/messages")
public String reply(@PathVariable String ticketId, @RequestBody String message) {
    return chatClient.prompt()
        .user(message)
        .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, ticketId))
        .call()
        .content();
}

Using the ticket ID as the conversation ID is exactly what makes the second bug report disappear: every message on the same ticket now shares history, so "can you also check if this affects my linked account?" is answered with the original ticket's context still in view.

5. Putting It Together

The triage endpoint from the first guide and the stateful reply endpoint above are really two views of the same ChatClient, configured once with both advisors and reused everywhere:

@Service
public class SupportChatService {
 
    private final ChatClient chatClient;
 
    public SupportChatService(ChatClient.Builder builder, ChatMemory chatMemory) {
        this.chatClient = builder
            .defaultSystem("You are a support agent for Acme Fintech. Be concise and factual.")
            .defaultAdvisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
            .build();
    }
 
    public TicketTriage triage(String ticketId, String ticketText) {
        return chatClient.prompt()
            .user(u -> u.text("Triage this support ticket: {ticketText}").param("ticketText", ticketText))
            .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, ticketId))
            .options(ChatOptions.builder().temperature(0.0).build())
            .call()
            .entity(TicketTriage.class, spec -> spec.validateSchema());
    }
 
    public String reply(String ticketId, String customerMessage) {
        return chatClient.prompt()
            .user(customerMessage)
            .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, ticketId))
            .call()
            .content();
    }
}

Both methods share the same system persona, the same memory, and the same conversation thread — triage() runs once per ticket at temperature 0.0 for consistent routing, and reply() runs on every follow-up message, now with full context of everything said before it on that ticket.

What's Next

You've replaced fragile string parsing with typed, schema-validated output, and replaced stateless one-shot calls with a real conversation backed by a pluggable storage layer. The next guide in this roadmap gives the model access to data it was never trained on — loading documents, chunking them, embedding them into a vector store, and retrieving the right ones at query time with RAG.