org.llm4s.repl
Members list
Type members
Classlikes
What one model call cost: the model that answered, the tokens the provider reported, how long the call took, and the estimated cost in US dollars when llm4s knows the model's price. Printed as one line:
What one model call cost: the model that answered, the tokens the provider reported, how long the call took, and the estimated cost in US dollars when llm4s knows the model's price. Printed as one line:
gpt-4o-mini: 12 prompt + 8 completion = 20 tokens, 312 ms, $0.000007
Value parameters
- estimatedCost
-
the estimated cost in US dollars, when the model's price is known
- latency
-
the time from the request to the last byte of the answer
- model
-
the model that answered, as the provider named it
- usage
-
the token counts, when the provider reports them
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
The conversation a session has had so far, as ask and chat() recorded it: each question, each answer, and the tool calls and results in between. A REPL prints it as a transcript, one message per line:
The conversation a session has had so far, as ask and chat() recorded it: each question, each answer, and the tool calls and results in between. A REPL prints it as a transcript, one message per line:
user: What is the weather in Paris?
assistant: -> get_weather({"city":"Paris"})
tool: <- {"temperature":21}
assistant: It is 21 degrees in Paris.
Value parameters
- messages
-
the messages, oldest first; the session's system prompt is not among them
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
What a REPL helper returns: the value, or the error that stopped it. A REPL prints it as the value itself, or as error: <kind>: <message and hint>, never as a stack trace. A script that wants a Result calls toResult, or uses the helpers of strict.
What a REPL helper returns: the value, or the error that stopped it. A REPL prints it as the value itself, or as error: <kind>: <message and hint>, never as a stack trace. A script that wants a Result calls toResult, or uses the helpers of strict.
val answer = ask("What is a monad?")
answer.getOrElse("") // the text, or "" when the call failed
answer.toResult // Right(text) or Left(error)
Attributes
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
What the helpers of org.llm4s.repl talk to: a client, its completion options, the conversation so far and the statistics of the last call. Set it once as a context parameter and every helper finds it:
What the helpers of org.llm4s.repl talk to: a client, its completion options, the conversation so far and the statistics of the last call. Set it once as a context parameter and every helper finds it:
import org.llm4s.repl.*
given Session = Session.fromConfig() // the `llm4s.providers.provider` section of application.conf
ask("Explain variance in Scala 3 in two sentences")
A session whose client cannot be built (no application.conf, a missing API key, an unregistered provider) is still a session: every helper prints the configuration error with its troubleshooting hint and returns a failure, so a worksheet never stops on a stack trace. failure says whether that is the case.
A session is for a REPL, a worksheet or a script, not for production code: it keeps the conversation in memory, prints to the console, and its helpers are not meant to be called concurrently. Each with* method returns a new session with the same client and an empty history.
Attributes
- Companion
- object
- Supertypes
-
class Objecttrait Matchableclass Any
The helpers for scripts: the same calls, returning Result so that a script can stop on the first failure with a for comprehension. They print nothing on failure; strict.ask still streams the answer.
The helpers for scripts: the same calls, returning Result so that a script can stop on the first failure with a for comprehension. They print nothing on failure; strict.ask still streams the answer.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
strict.type
Value members
Concrete methods
Asks the model prompt, after the conversation so far, and streams the answer to the session's output as it arrives. The question and answer are added to the history.
Asks the model prompt, after the conversation so far, and streams the answer to the session's output as it arrives. The question and answer are added to the history.
With tools, each tool call the model makes is run, printed (-> name(arguments), then <- result) and sent back, until the model answers in text.
A failed call prints error: <kind>: <message>, with the troubleshooting hint llm4s attaches, and returns Outcome.Failed; nothing is thrown. strict.ask returns a Result instead.
Value parameters
- prompt
-
the question
- tools
-
the tools the model may call; none by default
Attributes
- Returns
-
the answer's text, or the error
A small read-eval loop over the session: each line read is asked with ask, so the conversation carries on from one line to the next. :q (or the end of the input) leaves, :history prints the conversation so far and :reset forgets it.
A small read-eval loop over the session: each line read is asked with ask, so the conversation carries on from one line to the next. :q (or the end of the input) leaves, :history prints the conversation so far and :reset forgets it.
Attributes
Asks the model for an A read from text, with the JSON schema derived from A (completeStructuredOf):
Asks the model for an A read from text, with the JSON schema derived from A (completeStructuredOf):
case class Invoice(vendor: String, amount: Double, currency: String) derives SchemaOf, ReadWriter
val invoice = extract[Invoice]("ACME, 12.50 EUR, two widgets")
The call is not added to the history. A failure is printed and returned as Outcome.Failed.
Type parameters
- A
-
a case class or sum type that
derives SchemaOfand has a uPickleReader
Value parameters
- text
-
the text to read the value from