Tool Calling: Giving the Model the Ability to Act

"What's the Status of My Refund?"

The policy bot from the last phase can explain Acme Fintech's refund policy flawlessly — it has the PDF, RAG retrieves the right chunk, the answer is grounded and accurate. Then a customer asks something the policy document was never going to contain: "What's the status of my refund for order #4521?"

There's no document to retrieve here. That answer lives in a live order-service database, changes by the minute, and is different for every customer who asks. RAG grounds a model in static knowledge; tool calling grounds it in live knowledge and lets it act — call a real Java method, hit a real REST endpoint, run a real calculation — as part of answering a single question.

1. Your First Tool: @Tool and @ToolParam

A tool is a plain Java method, annotated with @Tool. Spring AI generates a JSON schema from the method signature and gives it to the model; the model decides if and when to call it.

@Component
public class OrderTools {
 
    private final OrderRepository orderRepository;
 
    public OrderTools(OrderRepository orderRepository) {
        this.orderRepository = orderRepository;
    }
 
    @Tool(description = "Look up the current status of a refund by order ID")
    public RefundStatus getRefundStatus(
        @ToolParam(description = "The order ID, e.g. 4521") Long orderId
    ) {
        return orderRepository.findRefundStatus(orderId)
            .orElseThrow(() -> new IllegalArgumentException("No order found with ID " + orderId));
    }
}
 
public record RefundStatus(Long orderId, String status, LocalDate expectedDate) {}

Two things are doing real work here:

• description is not documentation, it's the prompt. The model decides whether to call this tool almost entirely based on how well description matches what the user is asking. A vague description ("Get status") gets skipped in favor of the model guessing; a specific one ("Look up the current status of a refund by order ID") gets picked reliably.
• The return type becomes the tool's contract, the same way a record became .entity()'s contract in the structured-output guide. RefundStatus gets serialized to JSON and handed back to the model, which then writes the actual reply to the customer in natural language.

2. Registering Tools With ChatClient

Same two-tier pattern you've seen for advisors: attach for every call, or attach for one.

// Every call this ChatClient makes can use OrderTools
ChatClient chatClient = builder
    .defaultTools(new OrderTools(orderRepository))
    .build();

// Just this one call
String answer = chatClient.prompt()
    .user("What's the status of my refund for order 4521?")
    .tools(new OrderTools(orderRepository))
    .call()
    .content();

You never write that loop. ChatClient auto-registers a ToolCallingAdvisor — one unified implementation shared across every provider as of Spring AI 2.0 — which runs the "call model, execute tool, send result back, call model again" cycle until the model stops requesting tools and returns a final answer.

3. Multiple Tools, One Class Each

Real agents rarely have exactly one tool. Group related tools into their own classes and register all of them:

@Component
public class CalculatorTools {
 
    @Tool(description = "Calculate the prorated refund amount for a partial billing period")
    public double calculateProratedRefund(
        @ToolParam(description = "Full monthly subscription price") double monthlyPrice,
        @ToolParam(description = "Number of unused days in the billing period") int unusedDays
    ) {
        return Math.round((monthlyPrice / 30.0) * unusedDays * 100.0) / 100.0;
    }
}
 
@Component
public class EscalationTools {
 
    private final TicketService ticketService;
 
    public EscalationTools(TicketService ticketService) {
        this.ticketService = ticketService;
    }
 
    @Tool(description = "Escalate the current conversation to a human support agent")
    public String escalateToHuman(
        @ToolParam(description = "Brief reason for escalation") String reason
    ) {
        String ticketId = ticketService.createEscalation(reason);
        return "Escalated. A human agent will follow up under ticket " + ticketId + ".";
    }
}

ChatClient chatClient = builder
    .defaultTools(
        new OrderTools(orderRepository),
        new CalculatorTools(),
        new EscalationTools(ticketService)
    )
    .build();

One customer message — "I cancelled 12 days into a $30/month plan and support hasn't responded in 3 days" — can now trigger calculateProratedRefund, then escalateToHuman, in the same turn, with the model deciding both the order and the arguments. This is the actual mechanism behind "the bot can look things up, calculate things, and take action," not three separate features.

4. returnDirect: Skipping the Round-Trip Back to the Model

By default, a tool's result goes back to the model, which then composes the final reply in its own words. Sometimes you don't want that — escalateToHuman's confirmation message above is already exactly what the customer should see, verbatim, with no risk of the model paraphrasing a legally meaningful sentence.

@Tool(description = "Escalate the current conversation to a human support agent", returnDirect = true)
public String escalateToHuman(@ToolParam(description = "Brief reason for escalation") String reason) {
    String ticketId = ticketService.createEscalation(reason);
    return "Escalated. A human agent will follow up under ticket " + ticketId + ".";
}

With returnDirect = true, the tool's return value becomes the response — the model never gets a chance to rewrite it. Reach for this for confirmations, legal/compliance text, or any agent loop where a specific tool call should be a hard stop, not a soft one.

5. ToolContext: Data the Model Should Never See

getRefundStatus above trusts whatever orderId the model passes it. That's fine for an order ID the customer typed themselves — but what about the customer's own account ID, used to make sure they can only ever look up their own orders? You don't want that in the prompt where the model could echo it, and you definitely don't want to trust the model to pass the right one.

@Tool(description = "Look up the current status of a refund by order ID")
public RefundStatus getRefundStatus(
    @ToolParam(description = "The order ID") Long orderId,
    ToolContext toolContext
) {
    String authenticatedCustomerId = (String) toolContext.getContext().get("customerId");
    return orderRepository.findRefundStatus(orderId, authenticatedCustomerId)
        .orElseThrow(() -> new IllegalArgumentException("No matching order found"));
}

String answer = chatClient.prompt()
    .user("What's the status of my refund for order 4521?")
    .toolContext(Map.of("customerId", currentUser.getId())) // from your auth layer, never the model
    .call()
    .content();

ToolContext parameters are invisible to the model — they never appear in the generated tool schema, and the model can't set or influence them. This is the correct place for anything security-sensitive: tenant IDs, authenticated user IDs, feature flags. Treat the model as untrusted input for anything that needs an authorization check, the same way you'd treat a request body.

6. Error Handling: What the Model Sees When a Tool Fails

getRefundStatus throws IllegalArgumentException when the order doesn't match. By default, Spring AI catches RuntimeExceptions thrown from a tool and sends the exception message back to the model as the tool's result — the model gets a chance to react sensibly ("I couldn't find that order — could you double check the order number?") instead of your controller throwing a raw 500.

// spring.ai.tools.throw-exception-on-error=false is the default:
// RuntimeExceptions become tool results the model can react to.
// Checked exceptions and Errors still propagate normally.

7. Putting It Together

Tool calling and RAG compose on the same ChatClient exactly like memory and RAG did in the last phase — neither one needs to know the other exists:

@Service
public class SupportAgentService {
 
    private final ChatClient chatClient;
 
    public SupportAgentService(
        ChatClient.Builder builder,
        ChatMemory chatMemory,
        VectorStore vectorStore,
        OrderTools orderTools,
        CalculatorTools calculatorTools,
        EscalationTools escalationTools
    ) {
        this.chatClient = builder
            .defaultSystem("""
                You help Acme Fintech customers with policy questions and order-specific
                requests. Use the provided tools for anything account- or order-specific.
                Never guess an order status or refund amount — always call a tool.
                """)
            .defaultAdvisors(
                MessageChatMemoryAdvisor.builder(chatMemory).build(),
                QuestionAnswerAdvisor.builder(vectorStore).build()
            )
            .defaultTools(orderTools, calculatorTools, escalationTools)
            .build();
    }
 
    public String ask(String conversationId, String customerId, String message) {
        return chatClient.prompt()
            .user(message)
            .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
            .toolContext(Map.of("customerId", customerId))
            .call()
            .content();
    }
}

One ask() method now answers policy questions grounded in documents (RAG), remembers the conversation (memory), and can look up, calculate, or act on a specific customer's data (tools) — scoped to that customer, safely, via ToolContext.

What's Next

The model can now call one tool, or several, per turn. The next guide covers what happens when a single tool call isn't enough — multi-step reasoning loops where the model needs to think, act, observe, and think again, hand-rolled with the same ChatClient you already know.