JlamaConfig

org.llm4s.llmconnect.config.JlamaConfig
See theJlamaConfig companion object
@Experimental
final case class JlamaConfig extends ProviderConfig

Configuration for a model run in-process by Jlama: provider = "jlama".

Build one with JlamaConfig.fromValues; a provider = "jlama" section builds one for you.

Value parameters

contextWindow

the tokens a prompt and its reply may take together.

model

the Hugging Face model id, owner/name, e.g. "tjake/Llama-3.2-1B-Instruct-JQ4".

modelDir

where models are downloaded to and loaded from: one directory per model below it.

reserveCompletion

the tokens held back for the reply: the reply's length when a request sets no maxTokens.

threads

how many threads Jlama computes with; None leaves Jlama's default, the physical core count. Jlama's thread pool is shared by the whole JVM, so the first model loaded sets it.

Attributes

Companion
object
Source
JlamaConfig.scala
Graph
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Members list

Value members

Concrete methods

override def endpointUrl: Option[String]

The endpoint this config will contact, when it is known statically.

The endpoint this config will contact, when it is known statically.

Used by policy checks and diagnostics that need to know where traffic will go without knowing which provider it belongs to. None means the config carries no single URL — not that it makes no network calls.

Attributes

Definition Classes
Source
JlamaConfig.scala
override def withModel(model: String): JlamaConfig

The same provider and credentials, pointed at a different model.

The same provider and credentials, pointed at a different model.

Attributes

Definition Classes
Source
JlamaConfig.scala
def withModelDir(modelDir: Path): JlamaConfig

Attributes

Source
JlamaConfig.scala
def withThreads(threads: Int): JlamaConfig

Attributes

Source
JlamaConfig.scala
def withThreads(threads: Option[Int]): JlamaConfig

Attributes

Source
JlamaConfig.scala

Inherited methods

Where the client gets the API key it sends with each request, when not from the key this config holds: a section's apiKeyFile, or a provider set with withApiKeyProvider (#2313).

Where the client gets the API key it sends with each request, when not from the key this config holds: a section's apiKeyFile, or a provider set with withApiKeyProvider (#2313).

The default is None, and so is the value of a config that does not support a provider.

Attributes

Inherited from:
ProviderConfig
Source
ProviderConfig.scala

The proxy and TLS settings this provider's HTTP calls use: llm4s.http, overridden key by key by the section's own http block.

The proxy and TLS settings this provider's HTTP calls use: llm4s.http, overridden key by key by the section's own http block.

The default is org.llm4s.http.HttpNetworkSettings.default - no proxy, no extra TLS material - so a config that does not carry the setting, such as one supplied by another module, behaves as before.

Attributes

Inherited from:
ProviderConfig
Source
ProviderConfig.scala

Attributes

Inherited from:
Product

Attributes

Inherited from:
Product

How long this provider's HTTP calls may take: the timeouts block of its section.

How long this provider's HTTP calls may take: the timeouts block of its section.

The default is ProviderTimeouts.default, which leaves every client on its own default, so a config that predates the block, or a provider supplied by another module that does not read it, behaves as before. A config that carries the setting overrides this and its client reads it with ProviderTimeouts.requestOr / ProviderTimeouts.streamOr.

Attributes

Inherited from:
ProviderConfig
Source
ProviderConfig.scala

The same config, its client asking provider for the key on each request, so a rotated key is used without rebuilding the client.

The same config, its client asking provider for the key on each request, so a rotated key is used without rebuilding the client.

The loader calls this for a section that sets apiKeyFile. The default returns this: a config whose client cannot take a provider ignores it, and its apiKeyProvider stays None - which the loader reports as an error for apiKeyFile and code setting a provider should check. A config that supports it overrides this with its own return type.

Attributes

Inherited from:
ProviderConfig
Source
ProviderConfig.scala

The same config with the given proxy and TLS settings.

The same config with the given proxy and TLS settings.

The loader calls this on every config a descriptor builds, as it does withTimeouts, so a descriptor does not read the http block itself. The default returns this: a config that does not carry the setting ignores it. A config that does overrides this with its own return type.

Attributes

Inherited from:
ProviderConfig
Source
ProviderConfig.scala

The same config with the given timeouts.

The same config with the given timeouts.

The loader calls this on every config a descriptor builds, passing the section's timeouts block, so a descriptor does not read the block itself. The default returns this: a config that does not carry timeouts, such as one supplied by another module, ignores them. A config that does overrides this with its own return type.

Attributes

Inherited from:
ProviderConfig
Source
ProviderConfig.scala

Concrete fields

override val providerId: ProviderId

Canonical id of the provider this config addresses, e.g. ProviderId("openai").

Canonical id of the provider this config addresses, e.g. ProviderId("openai").

Attributes

Source
JlamaConfig.scala