Coder Legion - Recent posts tagged springai

The Bug That's Been Breaking Software for 40 Years Is Now Breaking Your AI Agents, Too

Tue, 25 Aug 2026 00:00:00 +0000

Picture a Java developer shipping an AI agent into production. The agent gets a prompt and starts working through a task, 50 steps, each one a call to a model like OpenAI or Anthropic, a database write, maybe a few dollars of compute. At step 49 of 50, something fails: a bug, a network blip, an AWS outage, a Kubernetes scale-down mid-run. What happens next, according to Yaron Schneider, CTO and co-founder of Diagrid, is the part nobody budgets for. "The agent is going to need to pick back up all the way from the beginning, and you're going to be paying for all of those LLM calls right from the very beginning," he said. "Which is a great way to go bankrupt."

That failure mode is what Diagrid's new integration, diagrid-spring-ai, is built to close. Announced this week, it brings durable execution to Spring AI, the framework Java developers already use to build agents, without asking them to rethink how they write one.

The same 40-year-old problem, with worse consequences

Schneider doesn't frame this as a new category of failure. It's the same reliability problem distributed systems have always had. "The exact same problems that made software unreliable for the past 40 years are still true for AI agents," he said. "Only with AI agents, the consequences are much bigger." A crashed script used to mean lost time. A crashed agent means lost time and a re-billed stack of model calls, at a cost that scales with however many steps the agent had already completed.

Schneider said the companies Diagrid works with see failures on roughly 30% of production agent runs, on average, not rare edge cases, but a routine cost of running agents at scale.

Zero code changes, by design

The integration's core pitch is unusually low-friction for infrastructure-level tooling. "With Spring AI, literally nothing," Schneider said, when asked what a developer has to change to adopt it. "All you need to do is pull in the package. There is zero code changes. You do not need to recompile your code." Underneath, the integration runs on Dapr Workflow, the open-source project Schneider originally built at Microsoft and later spun into Diagrid, now used by more than 30,000 companies, including over half the Fortune 500 and, notably, NASA.

How it actually avoids re-running side effects

The obvious follow-up question for anyone who's built distributed systems: how do you resume an interrupted agent without re-executing a step that already sent an email or wrote to a database? Diagrid's answer is an immutable ledger. "We keep an immutable record of everything that the agent did. We're essentially journaling," Schneider said. When an agent fails mid-run, the platform reads the journal, recovers the state, exactly what was completed, its inputs, outputs, and memory, and resumes from that point rather than replaying anything already finished.

Schneider was direct about the confidence customers should have in that recovery: "If they're using our technology, they know for 100%, for a fact, that we will always resume where they left off mid-run." He added that customers can and do test this themselves, running failure simulations in their own environments to verify the behavior before trusting it in production.

Why this isn't just a Spring AI feature

A natural question is why durable execution lives as a separate integration rather than something Spring AI ships with directly. Schneider's answer comes down to what a workflow engine actually requires. "Workflow engines are incredibly hard to get right. They require their own separate infrastructure, and Spring is just not that," he said. Bundling a database and deployment model into a lightweight developer framework would hurt adoption, not help it. Building that infrastructure in-house isn't a reasonable alternative either: Schneider pointed to Microsoft's own investment, a team of 15 engineers spending more than $10 million a year maintaining its workflow engine, as a sense of the actual cost most teams would take on trying to solve this themselves.

The claim Schneider thinks is overrated

Asked what's overstated in the broader AI agent conversation right now, Schneider didn't hesitate: model accuracy. "Models are going to continue getting better. They're going to continue being cheaper," he said. "I see a lot of emphasis being put on that instead of, you know, how do you orchestrate multiple agents needing to solve a task and all of them failing at the same time." In his view, the infrastructure problem, keeping agents reliable when things break, is a bigger and less-discussed risk than whatever incremental accuracy gain the next model release brings.

Beyond Spring

diagrid-spring-ai is one piece of a broader set of integrations Diagrid maintains across roughly ten agent frameworks spanning Python, C#, and Java, including LangChain, LangGraph, and CrewAI. Developers can try the platform directly at diagrid.io, where Schneider said quick-start guides let teams test the failure-recovery behavior themselves before committing to it in production.

Is Java Still Viable in the Era of Agentic AI?

Fri, 06 Mar 2026 15:33:07 +0000

If you only watched conference keynotes and social feeds, you might assume the future of agentic AI belongs entirely to Python notebooks and TypeScript frontends. Meanwhile, most systems that agents actually need to reason about and act on are still written in Java. That mismatch is a huge opportunity for Java developers.

Agentic AI isn’t about yet another chatbot; it is about autonomous systems that understand goals, plan multi‑step workflows, call tools, and adapt based on feedback. The more those agents touch mission‑critical workflows—payments, logistics, fraud detection, core banking—the more they need predictable, observable, transactional backends. That’s exactly where modern Java shines.

So the real question isn’t “Is Java still viable?” It is “Why would you not want your agentic AI to live as close as possible to your existing Java estate?”

Java is still the backbone where agents act

Despite the hype around newer languages, multiple analyses show Java remains the backbone of enterprise applications. Across banking, healthcare, logistics, and government, the systems of record and high‑value workflows agents will operate on are overwhelmingly Java/JVM‑based.

That matters for agentic AI because:

Agents need reliable tools: tool‑calling means hitting APIs, services, and workflows that already exist—often Spring Boot services, Jakarta EE apps, or JVM‑backed microservices.
Latency and consistency matter: when agents orchestrate multi‑step workflows, you care about transactional guarantees, idempotency, and consistent state—not just “good enough” responses.
Domain skills live in Java: the most valuable agents encapsulate deep domain knowledge, and that knowledge is usually encoded in existing Java services and libraries.

The more “real work” agents do, the closer they get to your Java services. You can wrap those services from the outside—or you can make Java a first‑class home for the agents themselves.

The emerging Java agentic AI stack

Over the last 18–24 months, a serious Java‑native ecosystem for agentic AI has started to materialize. Instead of bolting Python micro‑projects onto your stack, you can increasingly stay in the JVM world.

Notable examples:

Spring AI & agentic patterns: Spring AI and the new agentic patterns series show how to build agents, define “skills,” orchestrate tasks, and even wire up agent‑to‑agent protocols using familiar Spring idioms.
Quarkus + LangChain4j: Community content demonstrates building reasoning, tool‑using agents with Quarkus and LangChain4j, turning annotated Java methods into callable agent tools.
Dedicated Java agent learning paths: New courses and specializations focus on “AI agents in Java,” covering goal management, tool discovery, and multi‑agent collaboration entirely on the JVM.

The message: agentic AI no longer has to be a foreign body that lives in a different language. You can build and run it where your code, tooling, and operational muscle already exist—inside Java.

Why agentic AI plays to Java’s strengths

Agentic systems are long‑lived, stateful, and deeply integrated. That aligns naturally with Java’s historical strengths in enterprise backends.

Key advantages Java brings:

Mature concurrency and scaling: Agent workflows involve background tasks, parallel tool calls, streaming IO, and backpressure. Java’s concurrency primitives and modern structured concurrency patterns give you robust ways to manage that complexity.
Stability and backward compatibility: Enterprises rely on Java because they can modernize runtimes without rewriting entire codebases, which is critical when agents touch long‑lived systems.
Battle‑tested ecosystem: Observability (Micrometer, Actuator), messaging (Kafka, Pulsar), security, and transaction management are all mature in the Java world—exactly what you want when autonomous systems start making changes in production.

Agentic AI doesn’t reduce the need for these qualities; it magnifies it.

A minimal Spring AI agent example

To make this concrete, here is a simplified example inspired by Spring AI agent samples. Imagine a Java “incident triage” agent that can read a goal and decide which internal tools to call. github

@Service
public class IncidentTriageAgent {

    private final AgentClient agentClient;

    public IncidentTriageAgent(AgentClient agentClient) {
        this.agentClient = agentClient;
    }

    // Tool the agent can call
    @Tool("fetch current JVM error rate")
    public double getErrorRate(String serviceName) {
        // call Prometheus/Grafana/observability backend
        return metricsClient.queryErrorRate(serviceName);
    }

    public String triage(String description) {
        String goal = """
            You are an SRE assistant for a Java microservices platform.
            Use available tools to decide:
            1) Is this incident urgent?
            2) Which service is most likely impacted?
            3) What is the first remediation step?
            Incident: %s
            """.formatted(description);

        AgentClientResponse response = agentClient
            .goal(goal)
            .timeout(Duration.ofSeconds(30))
            .run();

        return response.getResult();
    }
}

Here, the agent runs inside a Spring Boot service, calls Java methods as tools, and uses your existing metrics stack to reason about production incidents. You get agentic behavior without leaving the JVM or duplicating operational infrastructure.

What this means for Java developers

If you are a Java developer wondering how to stay relevant in the era of agentic AI, the answer is not “switch to Python.” It is to lean into the parts of agentic systems that Java is uniquely good at.

Practical next steps:

Pick a JVM‑friendly framework like Spring AI or LangChain4j and build one small agent that calls tools you already own.
Model “tools” as well‑typed Java methods and services, not ad hoc shell commands.
Double down on observability, safety, and rollback paths—areas where Java and Spring Boot already excel.

Agentic AI is moving from buzzword to architecture, and the systems it will depend on are overwhelmingly Java. That doesn’t make Java obsolete; it makes Java more strategically central than ever.

Building a RAG Powered Assistant with Spring AI and LM Studio

Tue, 17 Feb 2026 03:15:30 +0000

How to Create an Intelligent Document Q&A System Using Spring AI, PostgreSQL, and Vector Embeddings

Imagine having an AI assistant that can instantly answer questions about hundreds of financial documents, quarterly reports, market analyses, policy papers without you having to manually search through pages of text. That's exactly what Retrieval Augmented Generation (RAG) enables, and in this tutorial, we'll build one from scratch using Spring Boot.

By the end of this guide, you'll have a fully functional application that:

Ingests PDF documents and extracts their content
Converts text into semantic embeddings using AI models
Stores embeddings in a PostgreSQL vector database
Answers natural language queries with contextual accuracy

What is RAG and Why Does It Matter?

Retrieval Augmented Generation (RAG) is a technique that enhances Large Language Models (LLMs) by providing them with relevant context from external knowledge sources. Instead of relying solely on the model's training data, RAG systems:

Retrieve relevant documents based on semantic similarity
Augment the LLM's prompt with retrieved context
Generate accurate, grounded answers

This approach is particularly powerful for:

Enterprise knowledge bases with proprietary information
Financial document analysis and compliance
Customer support systems with extensive documentation
Research paper exploration and literature reviews

Architecture Overview

Our FinanceRag application follows a straightforward yet powerful architecture:

1. Document Ingestion Pipeline

PDF documents are read from the classpath and processed by Spring AI's PagePdfDocumentReader, which extracts text while preserving structure.

2. Text Chunking

The TokenTextSplitter divides the extracted text into manageable chunks (800 tokens each). This is crucial because:

Embedding models have token limits
Smaller chunks provide more precise semantic matching
Context windows in LLMs benefit from focused, relevant information

3. Vector Embedding Generation

Each text chunk is converted into a high-dimensional vector (embedding) using the nomic-embed-text model. These embeddings capture semantic meaning similar concepts cluster together in vector space.

4. Vector Storage with pgvector

Embeddings are persisted in PostgreSQL using the pgvector extension, which enables efficient similarity searches. We use HNSW indexing for fast approximate nearest neighbor (ANN) queries.

5. Query Processing

When a user asks a question:

The question is embedded using the same model
Vector similarity search retrieves the most relevant document chunks
The QuestionAnswerAdvisor augments the LLM prompt with this context
The LLM generates a contextual answer

Building the Application: Step by Step

Prerequisites

Before diving into code, ensure you have:

Java 17+ installed
PostgreSQL 12+ with pgvector extension
LM Studio (or another OpenAI-compatible LLM endpoint)
Maven 3+ for dependency management

Setting Up PostgreSQL with pgvector

You have two options for setting up PostgreSQL:

Option 1: Using Docker (Recommended for Quick Start)

Your repository includes a compose.yaml file for easy setup:

services:
  postgres:
    image: pgvector/pgvector:pg16
    ports:
      - "55419:5432"
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: finance
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

Simply run:

docker-compose up -d

This spins up PostgreSQL with pgvector pre-installed on port 55419.

Option 2: Manual Installation

First, create a database and enable the vector extension:

CREATE DATABASE finance;
\c finance
CREATE EXTENSION IF NOT EXISTS vector;

The pgvector extension adds a new vector data type to PostgreSQL, enabling efficient storage and querying of high-dimensional vectors.

Configuring Spring Boot

Your application.properties file should include:

# Application Name
spring.application.name=finaceRag

# Database Configuration (Docker setup)
spring.datasource.url=jdbc:postgresql://localhost:55419/finance
spring.datasource.username=postgres
spring.datasource.password=postgres

# LLM Configuration (LM Studio)
spring.ai.openai.base-url=http://localhost:1234/
spring.ai.openai.api-key=dummy

# Embedding Model
spring.ai.openai.embedding.options.model=nomic-embed-text

# Chat Model
spring.ai.openai.chat.options.model=google/gemma-3-4b

# Vector Store Configuration
spring.ai.vectorstore.pgvector.initialize-schema=true

# Ingestion Control (IMPORTANT!)
financerag.ingest.enabled=true

Key Configuration Notes:

Port 55419 matches the Docker Compose setup
The initialize-schema=true automatically creates the vector store table
nomic-embed-text is a lightweight, high-quality embedding model
google/gemma-3-4b is the chat model served by LM Studio

Important: Ingestion Control

The financerag.ingest.enabled property is a smart optimization:

First Run (Initial Setup):

financerag.ingest.enabled=true

This processes your PDFs and populates the vector store.

Subsequent Runs:

financerag.ingest.enabled=false

This skips ingestion and starts the application immediately. The embeddings are already in PostgreSQL, so there's no need to re-process documents every time!

This design prevents:

Duplicate embeddings in the database
Slow startup times on every restart
Unnecessary LLM API calls

Setting Up LM Studio

Download and Install LM Studio from lmstudio.ai
Download the Required Models:
- Embedding Model: Search for "nomic-embed-text" in LM Studio and download it
- Chat Model: Search for "google/gemma-3-4b" (or similar) and download it
Start the Local Server:
- Open LM Studio
- Go to the "Local Server" tab
- Select your chat model (gemma-3-4b)
- Click "Start Server" (it will run on http://localhost:1234 by default)
- Ensure the embedding model is also loaded
Verify the Connection:
```
curl http://localhost:1234/v1/models
```
You should see your loaded models listed.

The Ingestion Service

The heart of our document processing pipeline is the IngestionService. Here's how it works:

@Component
@ConditionalOnProperty(
    name = "financerag.ingest.enabled",
    havingValue = "true",
    matchIfMissing = false
)
public class IngestionService implements CommandLineRunner {
    private static final Logger logger = LoggerFactory.getLogger(IngestionService.class);
    
    private final VectorStore vectorStore;
    
    @Value("classpath:/docs/article.pdf")
    private Resource pdfResource;
    
    public IngestionService(VectorStore vectorStore) {
        this.vectorStore = vectorStore;
    }
    
    @Override
    public void run(String... args) throws Exception {
        logger.info("Starting data ingestion process...");
        
        // 1. Read PDF using paragraph-based reader
        var pdfReader = new ParagraphPdfDocumentReader(pdfResource);
        
        // 2. Split text into chunks
        TextSplitter splitter = new TokenTextSplitter();
        
        // 3. Process and store in vector database
        vectorStore.accept(splitter.apply(pdfReader.get()));
        
        logger.info("Vector store updated with PDF content.");
    }
}

Key Implementation Insights:

1. Conditional Ingestion
The @ConditionalOnProperty annotation is brilliant - it only runs ingestion when you explicitly enable it:

# Enable ingestion on first run
financerag.ingest.enabled=true

# Disable after initial setup to avoid re-ingesting
financerag.ingest.enabled=false

This prevents re-processing documents on every application restart!

2. CommandLineRunner Interface
By implementing CommandLineRunner, the ingestion happens automatically after Spring Boot starts, but before the application begins serving requests.

3. ParagraphPdfDocumentReader vs PagePdfDocumentReader
Your code uses ParagraphPdfDocumentReader which:

Preserves document structure better by respecting paragraph boundaries
Creates more semantically meaningful chunks
Better suited for financial documents with structured content

4. Simplified API
The vectorStore.accept() method elegantly handles:

Embedding generation for each chunk
Batch insertion into PostgreSQL
All the complexity hidden behind a clean API

The Chat Controller

Now let's expose a REST endpoint for queries:

@RestController
public class ChatController {

    private final ChatClient chatClient;

    public ChatController(ChatClient.Builder chatClient, PgVectorStore vectorStore) {
        this.chatClient = chatClient
            .defaultAdvisors(QuestionAnswerAdvisor.builder(vectorStore).build())
            .build();
    }
    
    @GetMapping("/chat")
    public String chat(@RequestParam String question) {
        return chatClient.prompt()
            .user(question)
            .call()
            .content();
    }
}

The Magic of QuestionAnswerAdvisor:

The QuestionAnswerAdvisor is where RAG happens. Behind the scenes, it:

Converts the user's question into an embedding
Performs a similarity search against the vector store
Injects the most relevant document chunks into the prompt
Sends the augmented prompt to the LLM

Key Implementation Details:

The advisor is built using the builder pattern: QuestionAnswerAdvisor.builder(vectorStore).build()
Spring AI automatically handles the vector search and context injection
The controller method is elegantly simple - just pass the question through the chat client

Real World Considerations

Choosing the Right Chunk Size

The 800-token chunk size is a starting point. Consider:

Smaller chunks (200-400 tokens): Better precision, but may lose context
Larger chunks (1000-1500 tokens): More context, but less precise matching

Experiment with your specific use case. Financial reports might need larger chunks to preserve numerical context, while FAQs work better with smaller, focused chunks.

Scaling to Production

For production deployments, consider:

Async ingestion: Move document processing to background jobs
Caching: Cache embeddings for frequently accessed documents
Metadata filtering: Add tags (date, category, source) to narrow searches
Monitoring: Track query latency and similarity scores

Hybrid Search Strategies

Pure vector search isn't always optimal. Combine it with:

Full-text search: For exact keyword matches
BM25 ranking: Traditional relevance scoring
Re-ranking: Use a cross-encoder model to refine top results

Testing Your RAG System

Start the application and test with curl:

curl "http://localhost:8080/chat?question=What%20were%20the%20key%20trends%20in%20Q4%20earnings?"

You should see an answer grounded in your ingested documents. Compare responses with and without RAG to appreciate the difference in accuracy and relevance.

Common Pitfalls and Solutions

1. Embedding Dimension Mismatch

Problem: Embeddings fail to store with dimension errors.

Solution: Ensure spring.ai.vectorstore.pgvector.dimensions matches your embedding model. For nomic-embed-text, use 768.

2. Poor Retrieval Quality

Problem: Answers don't align with document content.

Solution: Adjust chunk size, increase topK, or lower the similarity threshold. Also verify your embedding model is appropriate for your domain.

3. Memory Issues During Ingestion

Problem: Application crashes with OutOfMemoryError.

Solution: Process documents in batches, increase JVM heap size (-Xmx4g), or limit the maxNumChunks parameter.

Extending FinanceRag: Ideas for Enhancement

This project is a foundation. Here are some powerful extensions:

Multi Document Support

Instead of hardcoding a single PDF, scan a directory or accept uploads via REST API. Add metadata (filename, upload date) to enable filtered searches.

Conversational Memory

Implement session based chat history so users can ask follow up questions without repeating context. Spring AI supports this with MessageChatMemoryAdvisor.

Source Attribution

Return not just the answer but citations showing which document chunks were used. This builds trust and allows users to verify information.

Advanced Analytics

Track which documents are queried most frequently, average similarity scores, and query patterns to identify knowledge gaps.

Conclusion

You've now built a production-ready RAG system that can intelligently answer questions about your documents. This architecture scales to thousands of documents and can be adapted for countless use cases such as customer support, legal document analysis, medical research, and more.

The beauty of Spring AI is how it abstracts the complexity of embeddings, vector stores, and LLM orchestration, letting you focus on business logic. With just three components IngestionService, ChatController, and pgvector we've created a powerful AI assistant.

The full source code for FinanceRag is available on GitHub. Clone it, experiment with different models and chunk sizes, and adapt it to your domain. The future of enterprise AI is built on foundations like these combining the power of LLMs with your organization's proprietary knowledge.

Special Thanks: This project was inspired by the excellent Spring AI content from Dan Vega, whose tutorials have helped countless developers understand the power of RAG architectures.

Happy coding, and may your AI assistants always retrieve the right context!

About the Author

This tutorial is brought to you by the Abhijith Rajesh

Links: