Spring AI Roadmap
Build intelligent apps — chat, RAG, agentic orchestration & the JVM AI ecosystem
Your Journey at a Glance
💡 How to use this roadmap
Work through each phase in order. Click on a skill to expand it — you'll find a description and curated resources. Don't rush; understanding beats speed. Complete one phase before moving to the next.
Foundations & Setup
Understand what Large Language Models are and how Spring AI abstracts them. Configure a provider (OpenAI, Ollama, Azure OpenAI, Anthropic) and make your first AI call with minimal boilerplate.
Chat, Prompts & Structured Output
Move beyond simple question-answer calls. Build multi-turn conversations, template complex prompts, parse model output into typed Java objects, and store conversation history.
Embeddings, Documents & Vector Stores
Give your AI access to your own data. Load documents, split them into chunks, convert to vector embeddings, store in a vector database, and retrieve semantically similar chunks at query time.
Tool Calling, MCP & Agent Evaluation
Go beyond single-turn Q&A. Build agents that call tools (functions, APIs, databases), reason across multiple hand-rolled steps, interoperate with other agents via MCP, and evaluate their own outputs.
Embabel: Goal-Oriented Agent Orchestration
When hand-rolled ReAct loops stop scaling, reach for a real planner. Embabel is Rod Johnson's JVM-native agent framework, built on Spring AI, that plans multi-step agent behaviour deterministically instead of asking an LLM to orchestrate itself.
LangChain4j: The Alternative JVM AI Stack
Spring AI isn't the only serious option on the JVM. LangChain4j is the older, framework-agnostic, community-driven library — understand its declarative style and where it beats or complements Spring AI.
Production, Observability & Staying Current
Ship AI features to production safely. Stream responses for interactivity, instrument AI calls with Micrometer, build Grafana dashboards, control cost with caching and guardrails, and build a system for staying ahead of a fast-moving release train.
AI Security, Resilience & Multimodal
Everything so far assumed a well-behaved model, a healthy provider, and text in, text out. Real deployments don't get those guarantees for free: retrieved documents and tool results can carry malicious instructions, providers time out or rate-limit, and a growing share of real use cases are images, scanned documents, and audio, not just chat.
Real-World Mini Projects
Concepts don't stick from reading alone. Eleven self-paced project ideas, each built from a different combination of skills across this roadmap, each with a distinct enterprise angle — pick the ones closest to problems you actually have. This complements the capstone above rather than replacing it; the capstone is one deep build, this phase is breadth.
Roadmap Complete!
You now have the foundations of a production-ready Java engineer. Apply by building real projects.
AI-Powered Customer Support Agent
A production-ready intelligent support agent backed by a private knowledge base. Users ask questions via a streaming chat UI; the agent retrieves answers from ingested PDF documentation, calls live tools to look up order status, and streams responses back in real time.
What you'll build
- ChatClient with system prompt persona and persistent JdbcChatMemory for multi-turn sessions
- PDF knowledge base: ingested with PagePdfDocumentReader, chunked via TokenTextSplitter, stored in PGVector
- RAG pipeline: QuestionAnswerAdvisor retrieves top-5 relevant chunks per query
- Tool calling: @Tool methods for order status lookup (REST), product search (DB), and ticket creation
- MCP server: exposes tools to Claude Desktop / other MCP clients for cross-model interoperability
- Streaming SSE endpoint: /api/chat/stream returns Flux<String> consumed with EventSource
- Micrometer AI observability: token usage counter, request latency histogram per model
- Grafana dashboard: tokens/s, p99 latency, error rate, cost estimate panel
- Agent evaluation suite: RelevancyEvaluator + FactCheckingEvaluator run in CI via @SpringBootTest
- Stretch goal: re-implement the escalation/refund workflow as an Embabel @Agent so it re-plans automatically when a tool call fails, instead of a hand-rolled retry loop
Tech stack
Key highlights
- ✦Full RAG pipeline from PDF ingestion to grounded streamed answer
- ✦Zero hallucination guardrail: FactCheckingEvaluator fails CI if score < 0.8
- ✦MCP server makes tools available to Claude Desktop without code changes
- ✦Demonstrates when to hand-roll agent orchestration vs delegate planning to Embabel
- ✦Once this is built, Phase 9's eleven mini-projects are where to go next for breadth — this capstone is depth on one scenario, Phase 9 is variety across eleven
Key Resources for Spring AI
Handpicked books, websites, and YouTube channels to go alongside this roadmap.
- Building AI Applications with Spring AIOfficial Spring AI project page — start here for guides, docs, and release notes
- AI Engineering (Chip Huyen)Chip Huyen — production ML systems, RAG, agents, and LLM evaluation
- Prompt Engineering for LLMsJohn Berryman & Albert Ziegler — practical prompting patterns and techniques
- Spring AI ReferenceOfficial, always up-to-date Spring AI documentation
- Spring AI — GitHubSource, releases, changelog, and community discussions
- Spring BlogOfficial Spring team blog — Spring AI release posts and deep dives
- Model Context ProtocolOpen standard for connecting AI models to tools — increasingly relevant for agents