JBang Quick Start

A first LLM call from one Java file, with no build tool and no project.

Table of contents

  1. Before you start
  2. The files
  3. Run it
  4. If something goes wrong
  5. Trying the code on main
  6. Next steps

JBang runs a Java file with its dependencies declared in the file itself, so you can try LLM4S from Java without creating a project. llm4s-java-api is on Maven Central from 0.5.0. When you want a real project, use Maven or Gradle.

Before you start

  • JBang, installed as described on its download page.
  • JDK 21 or newer: the file’s //JAVA 21 line makes JBang use JDK 21, and download it the first time if you do not have it. LLM4S needs JDK 21 (see Installation).
  • A local Ollama, which needs no key: run ollama serve and ollama pull llama3.

The files

Make a directory with these three files. The same files are in modules/samples/jbang.

HelloLlm4s.java:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

///usr/bin/env jbang "$0" "$@" ; exit $?
//JAVA 21
//DEPS org.llm4s:llm4s-java-api_3:0.5.1
//DEPS org.llm4s:llm4s-ollama_3:0.5.1
//DEPS ch.qos.logback:logback-classic:1.5.34
//FILES application.conf
//FILES logback.xml

import org.llm4s.javaapi.JLlmClient;
import org.llm4s.javaapi.Llm4s;
import org.llm4s.javaapi.LlmResult;

public class HelloLlm4s {

    public static void main(String[] args) {
        LlmResult<JLlmClient> created = Llm4s.createDefaultClient();
        if (created.isFailure()) {
            System.err.println("Could not create a client: " + created.getError().getMessage());
            System.exit(1);
        }

        try (JLlmClient client = created.get()) {
            LlmResult<String> answer = client.complete("In one sentence, what is Scala?");
            if (answer.isFailure()) {
                System.err.println("The call failed: " + answer.getError().getMessage());
                System.exit(1);
            }
            System.out.println(answer.get());
        } catch (Exception e) {
            System.err.println("Closing the client failed: " + e.getMessage());
            System.exit(1);
        }
    }
}

The first //DEPS line is llm4s: Maven does not add the Scala suffix, so the artifact is llm4s-java-api_3, and its POM brings llm4s-core and llm4s-agent. The second is the provider module the file calls, llm4s-ollama_3; for another provider, name its module instead (llm4s-openai_3, llm4s-anthropic_3, …). Up to 0.5.x the facade brought every provider module (#1496). The third //DEPS line is the logging backend: llm4s logs through SLF4J and ships none, so without it SLF4J prints a “no SLF4J providers were found” warning and drops the logs. //FILES puts a file on the classpath.

application.conf names the provider, as a named section; the Configuration guide covers the others:

1
2
3
4
5
6
7
8
9
10
11
12

llm4s {
  providers {
    provider = "ollama-local"

    ollama-local {
      provider = "ollama"
      model    = "llama3"
      baseUrl  = "http://localhost:11434"
      baseUrl  = ${?OLLAMA_BASE_URL}
    }
  }
}

logback.xml keeps the library’s own logging out of your first answer (without it, logback prints every debug message):

1
2
3
4
5
6
7

<configuration>
  <appender name="STDERR" class="ch.qos.logback.core.ConsoleAppender">
    <target>System.err</target>
    <encoder><pattern>%d{HH:mm:ss} %-5level %logger{20} - %msg%n</pattern></encoder>
  </appender>
  <root level="WARN"><appender-ref ref="STDERR"/></root>
</configuration>

Run it

From the directory that holds the three files:

1

jbang HelloLlm4s.java

The first run downloads the dependencies, which takes a while. You should see one sentence; the wording depends on the model. To compile without running, use jbang build HelloLlm4s.java.

For OpenAI instead of Ollama, change the section in application.conf to provider = "openai" with model = "gpt-4o-mini", and set OPENAI_API_KEY in the shell.

If something goes wrong

Could not create a client: ...: the message names what is missing, for example a provider key. Check application.conf and the environment.

The call failed: ... connection failed: no Ollama server is listening. Start it with ollama serve, or point OLLAMA_BASE_URL at the one you have.

A wall of DEBUG lines before the answer: logback.xml is not beside the file, or //FILES logback.xml is missing.

The version: change the number in both //DEPS org.llm4s lines to move to another release.

Trying the code on main

llm4s-java-api ships with 0.5.0. Before a change is released, publish it to your local Maven repository from a checkout of this repository, as the Gradle sample does:

1

sbt 'set ThisBuild / version := "0.1.0-SNAPSHOT"' publishM2

Then change the two //DEPS org.llm4s lines to version 0.1.0-SNAPSHOT and add //REPOS mavenlocal,mavencentral above it.

Next steps