Kickstarting AI Agent Development in Kotlin With Koog

Sure. Before joining Koog, I worked in native engineering and backend services. I have been with JetBrains for about five years.

And I have been with JetBrains for around eight years. I started my career as a Kotlin developer, working on the Kotlin IDE and libraries. I was also part of the Ktor team and worked as a backend developer on Kadana, a cloud-based code analysis tool. After several AI initiatives, I began working on Koog. Now, let’s jump straight to the point.

In contrast, Koog provides a comprehensive range of tools, starting with basic LLM connections and including out-of-the-box building blocks and agents that can be easily used and run. It also allows for customization through complex graph workflows. We plan to expand Koog further by enabling the development of non-graph-based strategies for agents and offering additional out-of-the-box solutions, including prompts and techniques developed in collaboration with our machine learning colleagues.

Our vision for Koog is informed by the understanding that there is no one-size-fits-all solution. We have explored various products and approaches, discovering that while some work well in certain scenarios, they may fail in others. Therefore, instead of focusing on a single paradigm, we aim to empower developers with a diverse set of tools to ensure flexibility. Now, let’s briefly cover some benefits of Koog.

Koog is also integral to real product development at JetBrains. This is significant because, in a recent episode of the Talking Kotlin YouTube channel featuring Rod Johnson, the creator of the Spring Framework, he discussed the reasons behind Spring’s success. He noted that their team was composed of practitioners rather than framework developers. They were focused on building practical solutions for real companies and creating production-ready microservices. This hands-on experience informed their framework’s development, making it both popular and useful.

I believe we are experiencing something similar with Koog. We are actively building agents at JetBrains, conducting numerous experiments and product developments. As we gain experience, we refine our approaches, retaining only what works in the framework. This process leads us to what we refer to as high-level solutions. Let me elaborate on that.

I also want to highlight Kotlin Multiplatform. With the power of Kotlin Multiplatform and Koog, you can develop AI solutions not only for the JVM and server-side but also for end-user devices like browsers, iOS, and Android devices, with more features coming soon. Koog is the first AI framework with native deep IDE integration, and we are actively working on extensive IDE support, including an AI debugger and advanced code analysis. We are heavily invested in this aspect, which I believe is a significant benefit of Koog.

It’s important to note that this is still in progress, and we will soon publish something exciting. While it is not yet ready for trial, we believe it is crucial to build agents with IDE support. Lastly, Koog enables you to create flexible and predictable systems by defining specific steps in workflows, ensuring that only what you programmed occurs in your system. Instead of building a random guesser, Koog allows you to create reliable systems you can trust. This is a key advantage of Koog, enhanced by Kotlin as a language.

From the user’s perspective, the agent functions as a black box. Users provide input and receive the final result. Internally, the agent communicates with the LLM by sending requests and receiving responses, which can be one of two types: assistant messages, which are plain text, or tool calls. If the LLM provides a tool call, the agent logic delegates this to the environment, calls the tool, and retrieves the tool’s result.

Within the agent, the strategy component is available for immediate use; you don’t need to create it from scratch, as Cook provides working strategies. However, Cook also allows for custom implementations tailored to your specific application.

For example, to create an AI agent, you would connect it to your application, such as by integrating search and email functionalities. You would have a search function in your code that you bind to the tool registry and register the sending email tool from your application. Next, you provide the agent strategy, which you can implement yourself or leave out since Cook includes some strategies by default. You also define the agent configuration, which includes the prompt, the task for the agent, the specific model connection, and parameters like maximum agent iterations. Finally, you establish connections to the LLMs, such as connecting to OpenAI. Now, let’s look at a concrete and simple example.

You might wonder if you need to manage the state of the LLM, including the history and messages sent previously. The answer is no. When developing Cook applications, think of it like chatting with ChatGPT in a browser window, which remembers the conversation history. As a user, you focus on the last response and then formulate your next question, while the window manages the history automatically. You don’t need to close the window or copy the entire conversation; it happens seamlessly. Similarly, Cook handles all that logic, allowing you to think in a type-safe manner about previous and next steps and how to compose them.

First, we need web access, which I’m achieving through pre-made APIs. I’m using Bright Data, but other providers offer similar functionalities. The code is straightforward. We define API schemas that we will receive as responses from Bright Data. We’ll use two tools: a search tool for Google searches and a page scraping tool that reads the page, removes HTML tags and JavaScript, and retains only the content. Essentially, we are defining response schemas here.

The more interesting part is how we implement the web access logic. We have a simple class that takes an API key and the cater client. I’m making simple REST requests without using SDKs. We have two tools, and as mentioned earlier, tools are just functions. One function is a regular Kotlin suspend function called search, which takes a query parameter, for example, “pizza.” It’s about 10 to 20 lines of code, where we make an API request and receive a response.

Now, we need to create an agent configuration. The agent configuration allows us to set presets, values, and settings for our agent. The most important aspect is the prompt itself, which defines what the agent does and what the LLM needs to accomplish. Here, I’m using a simple string: “You’re a helpful assistant that helps users research information on the web.” In real-world scenarios, you would likely expand on this and provide more examples.

We also specify the initial model to be used, which in our case is GPT-4. The maxAgentIterations setting is important to prevent the agent from entering an infinite loop, which can happen during complex implementations. To avoid excessive credit usage and compute time, we can set a maximum limit for agent iterations, which I’m setting to 50.

To finalize, we instantiate the agent itself, specifying input and output types. Although I’m explicitly mentioning the strategy here for clarity, we are using the default strategy, which is a tool calling loop. This means you provide input to the agent and a set of tools, and it queries the LLM, executes tools, and returns results until it decides it’s done. The default strategy is string-string, similar to a regular chat interface where you input a string and receive a string output.

We provide our prompt executor, the connector to the LLM, along with the API key, the strategy, the tool registry, and the agent configuration. This is the final step to piece all these configurations together to create a ready-to-go instance of the agent. We can then call agent.run and provide a string input, which will yield a string output. I can specify this explicitly, and as you can see, it’s string in, string out, and we simply print the result.

To enhance observability and confirm that the agent can call the provided tools, we can use the HandleEvents DSL to specify different event listeners, such as printing to the console or logging. In this case, I’m using the onToolCall listener to track which tools the LLM calls and when. I will print this information to the console each time the LLM calls a tool. Now, let me run this and see if it works; I’m asking the agent to tell me about the Kube framework.

I will demonstrate how to build a precise, custom strategy that remains simple and effective. First, I will create a strategy using the strategy DSL builder. For both the strategy and the nodes, I need to specify their types. While types can be automatically deduced from context, I will explicitly state that I am building a strategy to convert a string to an integer. I will name this strategy “my strategy.”

Inside the strategy, I will create a few nodes. The first node will take a string and convert it to an integer using the .toInt() function. I apologize for the slowness of my IDE. This node takes a string input and produces an integer output. Next, I will create another node for incrementing the integer. It’s important to use “by” instead of an equal sign for the nodes, as this allows for deducing node names from variable names, which is one of the strengths of the Kotlin DSL.

You can provide different names for the nodes, which represent the steps in your agentic algorithm. However, with Kotlin, you can leverage the delegating operator to automatically use the variable name as the node name. The increment node takes an integer and returns the next integer.

Now, I will connect the nodes by creating edges. The node start, which is always present in the strategy, will connect to the “toInt” node. From “toInt,” I will connect to the increment node, and finally, from the increment node, I will connect to the node finish. This strategy performs a simple operation: it takes a string, parses it, and produces the next integer. For instance, if you pass the string “100” to this agent, it will output “101.”

You must provide the strategy name, “my strategy,” and everything is type-safe. If I change the type of the strategy, the nodes will become incompatible. For example, if the start node outputs an integer while “toInt” expects a string, the code will not compile. This illustrates the DSL capabilities of Kotlin, which are advantageous for agentic development.

Now, let’s move on to a more complex example: building the out-of-the-box default strategy that you have seen in the previous visuals. This strategy includes three required nodes: “ask LLM,” which initiates the process; “call tool,” which executes any tool provided to the agent; and “send result,” which returns the tool’s result back to the LLM.

I will create edges starting from the node start to “ask LLM,” as this is where the process begins. The LLM will then determine the next step. Let me copy this line. While I wait for my IDE to respond, I can explain the roles of the node start and node end. These are special nodes that serve as the input and output for our strategy.

The issue arises because the ASCII LLM produces a response of type message.response, while the call tool node expects a tool call, which is a type that inherits from message.response. If you look at the definition, you’ll see that it’s a sealed interface. The response from the LLM can be either a system message or a tool call, as shown in the presentation images.

To connect this to the node, the IDE suggests a solution. In a typical scenario, you would need to write a condition. There is a DSL method called onCondition, and I’ll copy the full type name. If it’s a tool call, I will proceed to calling a tool. However, I just checked the condition without transforming the value. There is a DSL method for that, which I can write as transform it to message.toolCall. This is a lengthy line, but you usually don’t have to do this because there’s better syntactic sugar available for common use cases.

You can use the DSL method called onIsInstance with the class. While this is convenient, it may not be ideal since there are many other types in your strategy depending on your application. The message.toolCall check is crucial. Fortunately, there’s even more syntactic sugar in the framework, allowing you to write something like onToolCall, where you can specify a lambda. For example, you might want to target specific tool calls by name. However, in this example, we are using a generic strategy, so it will evaluate to true for any tool call.

After asking the LLM, you will proceed to calling the tool. If the ASCII LLM does not provide a tool call but instead gives an assistant message, you will finish at the node finish. There is a similar method called onAssistantMessage, which also evaluates to true for type checking.

Initially, I implemented a string to integer strategy. The node finish has the output type of the strategy, which is integer, but I was providing the assistant message converted to a string. I will correct that. Kotlin helps prevent these types of mistakes. From asking the LLM, I either branch to calling the tool or finish the strategy. After calling the tool, I will forward the tool results back to the LLM, which are already compatible since it provides and receives the tool result.

Finally, the sending result is a hanging node. If I run this code, it will throw an exception indicating there is no path from this node to finish. I need to copy this part as well. Let me… Why is my PC freezing again? Sorry, I have to reshare the screen one more time.

Now, let me illustrate how to implement a similar node. For this example, I’ll create a node that doesn’t add anything to the history but simply calls the LLM. Inside the node, it functions from an input type to an output type, but there is also a receiver type available. Within this context, various elements are accessible, including configuration, environment, LLM, strategy name, and agent ID. Among these, the LLM object is crucial, as it contains the tools linked to your LLM, the prompt with the message history, and other parameters.

Every time you execute the node, the context, including the list of messages, gets updated. You primarily focus on the input and output, but you also have access to the entire context, allowing you to request the LLM without altering anything. To do this, you would need to acquire a write session, as the LLM is a shared object with mutable state.

You can then request the LLM, which will return the message response. For instance, if I want to request the LLM with the current state, I would save the response in a variable. To update the prompt, I could copy the existing messages and use a DSL method to update it. The assistant message from the LLM will then be appended at the end, regardless of the type, treating it as an assistant message.

Although this node may seem trivial, it demonstrates how strategies allow you to control interactions with LLMs. In future sessions, we will explore these concepts in greater depth. For now, I will stop sharing this part of the screen and move on to the final slide of my presentation.

However, transforming the answer into a visually appealing UI is somewhat outside the scope of KUK. With KUK, you can retrieve the response content as text and use structured output to obtain results that can be parsed into classes, rather than just strings. With this structured output, you can create your own custom UI or utilize a library like Juni to help design it. The demo Android application was created using Juni, which excels at rendering composed UIs.

Additionally, if a KUK agent or any AI agent needs to call a critical tool—such as in a banking app where user confirmation is required—you have several options. You can incorporate this logic into the event handler, as shown previously, by handling events and printing the output. Alternatively, since tool calls are suspend functions, you can send a message to a coroutine channel and wait for user confirmation. You can also include UI elements as part of a tool call.

Another interesting idea is to personalize UI using AI agents. You can describe UI components as JSON, and LLMs are effective at generating these JSON structures. By providing context about the user and their workflow, the LLM can create various types of windows, buttons, and messages, allowing for a highly personalized UI for each user. We would be excited if someone implemented this idea and submitted a pull request to add it to our examples.