Creating a chatbot with Spring AI, Java, and OpenAI

What’s Spring AI?

According to Spring AI documentation:

"Spring AI is an application framework for AI engineering. Its goal is to apply to the AI domain Spring ecosystem design principles such as portability and modular design and promote using POJOs as the building blocks of an application to the AI domain."

It offers:

• Support for the main AI providers such as OpenAI, Microsoft, Amazon, Google, and Ollama.

• Structured outputs.

• Support for the main vector databases such as Apache Cassandra, Azure Vector Search, Chroma, Milvus, MongoDB Atlas, Neo4j, Oracle, PostgreSQL/PGVector, PineCone, Qdrant, Redis, and Weaviate.

• Tools/Function calling.

• Observability.

• Chatbot support: chat client, conversation memory, Retrieval Augmented Generation (RAG).

Connecting your Spring AI chatbot to an LLM

Let’s create our Spring Boot application using the Spring Initializr.

1. Navigate to https://start.spring.io/

2. Add the dependencies: Spring Web, OpenAI, PGvector Vector Database, Docker Compose Support, and Markdown Document Reader.

3. Download your project and open it using your favourite IDE.

4. Add a controller to your project:

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;


@RestController
public class ChatController {

    private final ChatClient chatClient;

    public ChatController(ChatModel chatModel) {
        this.chatClient = ChatClient.builder(chatModel).build();
    }

    @PostMapping(path = "/chat", consumes = "text/plain", produces = "text/plain")
    public String chat(@RequestBody String message) {
        return chatClient.prompt().user(message).call().content();
    }
}

5. Generate an API Key in the OpenAPI Platform and add it to your application.yml file.

spring:
  ai:
    openai:
      api-key: "your-api-key-here"

Let’s test our chatbot using curl. Start your application and execute this code in your terminal:

curl -H "Content-type: text/plain" \
     http://localhost:8080/chat \
     --data "Hello, who are you?"

You should see a response like this:

Hello! I'm an AI language model created by OpenAI. I'm here to help answer 
your questions and assist with a variety of topics. How can I assist you 
today?%

Pretty nice! But this chat is not that useful for now because it doesn’t have a memory to keep the context of a conversation. Let’s see this lack of memory in action:

# curl -H"Content-type: text/plain"  http://localhost:8080/chat --data "Hello, my name is JP" 
Hello JP! How can I assist you today?%                                                                                            
# curl -H"Content-type: text/plain"  http://localhost:8080/chat --data "What's my name?"      
I'm sorry, I can't determine your name.%

Adding memory to your chatbot using Spring AI

1. Add a new Java class to your project called ChatBotConfiguration with this content:

import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.InMemoryChatMemory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ChatBotConfiguration {
    @Bean
    public ChatMemory chatMemory() {
        return new InMemoryChatMemory();
    }
}

2. Update the controller constructor as follows:

public ChatController(ChatModel chatModel, ChatMemory chatMemory) {
    this.chatClient = ChatClient.builder(chatModel)
            .defaultAdvisors(new MessageChatMemoryAdvisor(chatMemory))
            .build();
}

3. Test it again:

# curl -H"Content-type: text/plain"  http://localhost:8080/chat --data "Hello, my name is JP" 
Hello JP! How can I assist you today?%  
# curl -H"Content-type: text/plain"  http://localhost:8080/chat --data "What's my name?"      
You mentioned that your name is JP%

How does it work? If we enable some detailed logs, the following is what we see:

The first time, one request with the content “Hello, my name is JP” was sent, and a response with the content “Hello JP! How can I assist you today?” was received.

In the second request, the content was “What’s my name?” but there was also an array with the previous messages, something like:

- User: “Hello, my name is JP”

- Assistant: “Hello JP! How can I assist you today?”

This is how ChatMemory works: It always sends the previous messages to give more context to the chat model.

Restricting the conversation to a specific context

When you create a chatbot, you want it to answer questions based on your company’s information, but OpenAI was not trained on that data.

To achieve that, the first step is to convert the information to something searchable that can be included in the request context before calling the OpenAI API. Something searchable in the AI world is a vector of floating-point numbers. Example: [0.123, 0.456, 0.789, 1.012, 2.890, 3.123, 3.456, 3.788, 5.001, 5.234, 5.567, 5.890]. We will use the Postgres Vector database we added as a dependency when creating the project.

We will work with markdown files in the resources folder for this scenario. These markdown files have information about a fictitious pizza place. Before continuing, let’s ask a question about this pizza place to our chatbot:

# curl -H"Content-type: text/plain"  http://localhost:8080/chat --data "Which pizzas are available?"
Could you please specify the restaurant or chain you are referring to? Pizza offerings can vary widely depending on the location and the specific menu of a restaurant or pizzeria.%  

Let’s make our pizza place known by the chatbot. First, enable the creation of the table that will hold the vectors. Add this to your application.yml file:

spring:
  ai:
    vectorstore:
      pgvector:
        initialize-schema: true

After that, let’s create the class to transform our markdown files into vectors.

package com.johnowl.demo_spring_ai;

import org.springframework.ai.reader.markdown.MarkdownDocumentReader;
import org.springframework.ai.reader.markdown.config.MarkdownDocumentReaderConfig;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.stereotype.Component;

@Component
public class MarkdownLoader implements InitializingBean {

    private final VectorStore vectorStore;

    public MarkdownLoader(VectorStore vectorStore) {
        this.vectorStore = vectorStore;
    }

    private final MarkdownDocumentReaderConfig config = MarkdownDocumentReaderConfig.builder().build();


    @Override
    public void afterPropertiesSet() throws Exception {
        final var resolver = new PathMatchingResourcePatternResolver();
        final var markdownFiles = resolver.getResources("*.md");

        for (var file : markdownFiles) {
            final var reader = new MarkdownDocumentReader(file, config);
            final var documents = reader.read();
            vectorStore.add(documents);
        }
    }
}

Lastly, let’s change our chatbot to be aware of the context. Update the controller constructor to be like this:

public ChatController(ChatModel chatModel, ChatMemory chatMemory, VectorStore vectorStore) {
    this.chatClient = ChatClient.builder(chatModel)
            .defaultAdvisors(
                    new MessageChatMemoryAdvisor(chatMemory),
                    new QuestionAnswerAdvisor(vectorStore)
            )
            .build();
}

Let’s test it:

# curl -H"Content-type: text/plain"  http://localhost:8080/chat --data "Which pizzas are available?"

The available pizzas are:

1. Classic Margherita
2. Pepperoni Delight
3. Four Cheese
4. Vegetarian Supreme
5. BBQ Chicken
6. Hawaiian
7. Meat Lovers
8. Spinach & Feta
9. Seafood Special
10. Buffalo Chicken
11. Truffle Mushroom
12. Prosciutto & Arugula%

Awesome! Now it knows what to answer! Using the Tika Document Reader dependency, Spring AI offers document readers for formats like PDF, HTML, DOCX, and PPTX.

Conclusion

Spring AI is almost production-ready at this moment. Its current version is v1.0.0-M6. Even so, we can already create a fully functional chatbot using OpenAI.

Spring AI also supports Ollama, so you can run a local version of this chatbot if you replace the dependency “OpenAI” with “Ollama”. This way, you will not spend money on API calls to create your chatbot.

You can find the source code of this example on GitHub.

Do you think you have what it takes to be one of us?

At WAES, we are always looking for the best developers and data engineers to help Dutch companies succeed. If you are interested in becoming a part of our team and moving to The Netherlands, look at our open positions here.

WAES publication

Our content creators constantly create new articles about software development, lifestyle, and WAES. So make sure to follow us on Medium to learn more.