llm4s-docs
API
Generated with
llm4s-docs/org.llm4s/org.llm4s.jev

org.llm4s.jev

package org.llm4s.jev

Members list

Type members

Classlikes

final case class ChoiceAnswer(choice: String, probabilities: Map[String, Double], confidence: Double) extends JevAnswer

The answer to a JevQuestion.Choice.

The answer to a JevQuestion.Choice.

Value parameters

choice

the highest-probability option

confidence

how certain the model is, from 0 to 1, derived from the distribution (not the same thing as the probability of the choice)

probabilities

every option with its probability; the API says they sum to 1, which is not re-checked here because the values are rounded on the wire

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait JevAnswer
class Object
trait Matchable
class Any
Show all
sealed trait JevAnswer

Jev's answer to one question. Its shape follows the question's type, and the API reports the type with every answer (type on the wire): see JevQuestion.

Jev's answer to one question. Its shape follows the question's type, and the API reports the type with every answer (type on the wire): see JevQuestion.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class ChoiceAnswer
class NoulAnswer
class ScoreAnswer
final class JevClient extends AutoCloseable

A client for TypeSafe's Jev decision model (https://docs.typesafe.ai/api.md).

A client for TypeSafe's Jev decision model (https://docs.typesafe.ai/api.md).

Jev is not a chat model. It takes a state and named, typed JevQuestions and answers each with a typed JevAnswer: the probability of a yes, the selected option with a distribution, a score along ordered levels. So this client is not an LLMClient and returns no Completion; it has no streaming and no conversation.

val decision = for
 config   <- JevConfigLoader.default()
 client   <- JevClient(config)
 response <- client.evaluate(JevRequest("Charged twice; please refund", Map("urgent" -> JevQuestion.noul("Is this urgent?"))))
 urgent   <- response.noul("urgent")
yield urgent.probability

evaluate is blocking and returns every failure as a org.llm4s.types.Result: an invalid request is a ValidationError before anything is sent, a rejected key an AuthenticationError, a rate limit a RateLimitError carrying the delay the server asked for, an outage a ServiceError, a transport failure a NetworkError or TimeoutError, a response that does not match the API a ProcessingError, and an interrupt a CancelledError. Transient failures are retried as the config's JevRetryPolicy says. The API key is never printed or logged, and is masked in an error wherever the server echoed it verbatim, JSON-escaped or URL-encoded (a partial echo of the key is not recognised).

'''No idempotency key.''' TypeSafe's documentation describes no idempotency key or other de-duplication mechanism, so this client does not invent one: each attempt of a retried request is a new, billable call. A header the caller attaches with JevRequest#withHeader is sent unchanged on every attempt.

The client is thread-safe. close releases the HTTP connections it owns.

Attributes

Companion
object
Supertypes
trait AutoCloseable
class Object
trait Matchable
class Any
object JevClient

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
JevClient.type
final case class JevConfig

What JevClient needs to call TypeSafe's API.

What JevClient needs to call TypeSafe's API.

Load it from configuration with org.llm4s.config.JevConfigLoader, which reads the llm4s.jev block and the API key from TYPESAFE_API_KEY, or build one in code:

val config = JevConfig(apiKey = key).withModel("jev-1.13.0")

A config is checked by JevConfig#validate, which JevClient runs when it is built.

Value parameters

apiKey

the API key; never printed by toString or logged, and masked in an error wherever the server echoed it verbatim, JSON-escaped or URL-encoded (a partial or otherwise transformed echo of the key is not recognised)

baseUrl

the API root, without a path. It must be https: the key is sent as a bearer token, so a plain http URL is accepted only for a loopback host (localhost, 127.0.0.0/8, ::1), which is how a test points the client at a local server.

headers

extra HTTP headers for every request (a request can add its own); the client sets the credentials, content type and Accept header itself and refuses to have them replaced

model

the model a request uses unless it names one: jev-latest, or a versioned id such as jev-1.13.0 to pin the answers

retry

how a transient failure is retried

timeout

how long each HTTP attempt may take. The API documents no figure: 30 s is this client's choice, matching the retry budget of TypeSafe's SDKs. An attempt never waits past what is left of the retry budget, so the budget bounds the whole call.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object JevConfig

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
JevConfig.type
sealed trait JevQuestion

A typed question for Jev, TypeSafe's System One decision model.

A typed question for Jev, TypeSafe's System One decision model.

Jev evaluates one state against a map of named questions and answers each with a typed answer: a JevQuestion.Noul with the probability of "yes", a JevQuestion.Choice with the selected option and a distribution over the options, a JevQuestion.Score with a score along ordered levels. See JevAnswer for the answers.

Wire format (https://docs.typesafe.ai/api.md): every question has a type and instructions; instructions and every criterion is a string, an object or an array (JSON structure is allowed so a question can carry the data it refers to). The limits the API documents are checked before a request is sent: at most JevQuestion.MaxChoiceOptions options in a Choice, between JevQuestion.MinScoreLevels and JevQuestion.MaxScoreLevels levels in a Score.

ujson.Value is the JSON type the rest of LLM4S's public API uses (tool parameters, HTTP responses). It is mutable: do not change a value after it has been handed to a question.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class Choice
class Noul
class Score
object JevQuestion

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class JevRequest

One evaluation: a state and the named questions to ask about it.

One evaluation: a state and the named questions to ask about it.

val request = JevRequest(
 "Charged twice; please refund",
 Map(
   "route"  -> JevQuestion.choice("Which team should handle this?", "billing" -> "Payments and refunds", "technical" -> "Product issues"),
   "urgent" -> JevQuestion.noul("Does this need immediate human attention?")
 )
)

The answers come back under the same ids as the questions, which the model never sees. All questions are answered against the one state in a single call, so ask everything you might need at once.

Value parameters

headers

extra HTTP headers for this request, sent unchanged on every retry of it; one replaces a configured header of the same name, whatever its case. The client sets the credentials, content type and Accept header itself and refuses to have them replaced.

model

the model to use, or None for the client's configured model (jev-latest by default)

questions

the questions, by the id their answers come back under

state

what to evaluate: a string, or a JSON object or array for structured data

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object JevRequest

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
JevRequest.type
final case class JevResponse

Jev's answers to one JevRequest.

Jev's answers to one JevRequest.

Value parameters

answers

one answer per question, under the id the question was asked with

model

the versioned model that answered (jev-1.13.0 for the alias jev-latest): log it, because an alias moves when a new release ships

requestId

the API's request id (x-typesafe-request-id), for support, when it sent one

usage

tokens used

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object JevResponse

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class JevRetryPolicy

How JevClient retries a transient failure.

How JevClient retries a transient failure.

The defaults are those of TypeSafe's own client SDKs (https://docs.typesafe.ai/sdk/python/api/retries.md): two retries after the first attempt, a first delay of 0.5 s that doubles up to 5 s with a quarter of each delay randomly taken off, and a budget of 30 s for the whole call. A delay the server asked for (Retry-After or retry-after-ms) replaces the computed one. What is retried is LLM4S's one retry rule, org.llm4s.reliability.RetryPolicy.isTransient: a 408, a 429, a 5xx (the API's 529 Overloaded too), and a connection failure or timeout. A rejected credential (401, 403), an invalid request (400, 422) and a cancellation are never retried.

Value parameters

backoffInitial

the first delay, doubled for each further retry

backoffMax

the longest computed delay

budget

the longest a call may take, attempts and delays together: each attempt's HTTP timeout is capped at what is left of it, and no retry is started whose delay would reach what is left

jitter

the fraction of each computed delay that is randomly taken off, from 0 to 1

maxRetries

retries after the first attempt; 0 sends each request once

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object JevRetryPolicy

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class JevUsage(inputTokens: Int, outputTokens: Int)

Tokens billed for a request. The API charges per input token; output tokens are reported but free.

Tokens billed for a request. The API charges per input token; output tokens are reported but free.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class NoulAnswer(probability: Double) extends JevAnswer

The answer to a JevQuestion.Noul.

The answer to a JevQuestion.Noul.

Value parameters

probability

the probability that the answer is yes, from 0 (no) to 1 (yes). The API reports no confidence for a Noul.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait JevAnswer
class Object
trait Matchable
class Any
Show all
final case class ScoreAnswer(score: Double, levels: Seq[ScoreLevel], confidence: Double) extends JevAnswer

The answer to a JevQuestion.Score.

The answer to a JevQuestion.Score.

Value parameters

confidence

how certain the model is, from 0 to 1

levels

every level with its description and probability, in level order

score

the probability-weighted level, which can land between levels (1.05 is just above level 1). It lies within the levels: a score the API sends a rounding error outside them is clamped in

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
trait JevAnswer
class Object
trait Matchable
class Any
Show all
final case class ScoreLevel(index: Int, description: Value, probability: Double)

One level of a ScoreAnswer.

One level of a ScoreAnswer.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Generated with