Configuration for a model run in-process by Jlama: provider = "jlama".
Build one with JlamaConfig.fromValues; a provider = "jlama" section builds one for you.
Value parameters
- contextWindow
-
the tokens a prompt and its reply may take together.
- model
-
the Hugging Face model id,
owner/name, e.g."tjake/Llama-3.2-1B-Instruct-JQ4". - modelDir
-
where models are downloaded to and loaded from: one directory per model below it.
- reserveCompletion
-
the tokens held back for the reply: the reply's length when a request sets no
maxTokens. - threads
-
how many threads Jlama computes with;
Noneleaves Jlama's default, the physical core count. Jlama's thread pool is shared by the whole JVM, so the first model loaded sets it.
Attributes
- Companion
- object
- Source
- JlamaConfig.scala
- Graph
-
- Supertypes
-
trait Serializabletrait Producttrait Equalstrait ProviderConfigclass Objecttrait Matchableclass AnyShow all
Members list
Value members
Concrete methods
The endpoint this config will contact, when it is known statically.
The endpoint this config will contact, when it is known statically.
Used by policy checks and diagnostics that need to know where traffic will go without knowing which provider it belongs to. None means the config carries no single URL — not that it makes no network calls.
Attributes
- Definition Classes
- Source
- JlamaConfig.scala
The same provider and credentials, pointed at a different model.
The same provider and credentials, pointed at a different model.
Attributes
- Definition Classes
- Source
- JlamaConfig.scala
Attributes
- Source
- JlamaConfig.scala
Attributes
- Source
- JlamaConfig.scala
Attributes
- Source
- JlamaConfig.scala
Inherited methods
Where the client gets the API key it sends with each request, when not from the key this config holds: a section's apiKeyFile, or a provider set with withApiKeyProvider (#2313).
Where the client gets the API key it sends with each request, when not from the key this config holds: a section's apiKeyFile, or a provider set with withApiKeyProvider (#2313).
The default is None, and so is the value of a config that does not support a provider.
Attributes
- Inherited from:
- ProviderConfig
- Source
- ProviderConfig.scala
The proxy and TLS settings this provider's HTTP calls use: llm4s.http, overridden key by key by the section's own http block.
The proxy and TLS settings this provider's HTTP calls use: llm4s.http, overridden key by key by the section's own http block.
The default is org.llm4s.http.HttpNetworkSettings.default - no proxy, no extra TLS material - so a config that does not carry the setting, such as one supplied by another module, behaves as before.
Attributes
- Inherited from:
- ProviderConfig
- Source
- ProviderConfig.scala
Attributes
- Inherited from:
- Product
Attributes
- Inherited from:
- Product
How long this provider's HTTP calls may take: the timeouts block of its section.
How long this provider's HTTP calls may take: the timeouts block of its section.
The default is ProviderTimeouts.default, which leaves every client on its own default, so a config that predates the block, or a provider supplied by another module that does not read it, behaves as before. A config that carries the setting overrides this and its client reads it with ProviderTimeouts.requestOr / ProviderTimeouts.streamOr.
Attributes
- Inherited from:
- ProviderConfig
- Source
- ProviderConfig.scala
The same config, its client asking provider for the key on each request, so a rotated key is used without rebuilding the client.
The same config, its client asking provider for the key on each request, so a rotated key is used without rebuilding the client.
The loader calls this for a section that sets apiKeyFile. The default returns this: a config whose client cannot take a provider ignores it, and its apiKeyProvider stays None - which the loader reports as an error for apiKeyFile and code setting a provider should check. A config that supports it overrides this with its own return type.
Attributes
- Inherited from:
- ProviderConfig
- Source
- ProviderConfig.scala
The same config with the given proxy and TLS settings.
The same config with the given proxy and TLS settings.
The loader calls this on every config a descriptor builds, as it does withTimeouts, so a descriptor does not read the http block itself. The default returns this: a config that does not carry the setting ignores it. A config that does overrides this with its own return type.
Attributes
- Inherited from:
- ProviderConfig
- Source
- ProviderConfig.scala
The same config with the given timeouts.
The same config with the given timeouts.
The loader calls this on every config a descriptor builds, passing the section's timeouts block, so a descriptor does not read the block itself. The default returns this: a config that does not carry timeouts, such as one supplied by another module, ignores them. A config that does overrides this with its own return type.
Attributes
- Inherited from:
- ProviderConfig
- Source
- ProviderConfig.scala
Concrete fields
Canonical id of the provider this config addresses, e.g. ProviderId("openai").
Canonical id of the provider this config addresses, e.g. ProviderId("openai").
Attributes
- Source
- JlamaConfig.scala
