JlamaProvider
Registration for a model run in-process by Jlama: provider = "jlama". No API key and no server:
local {
provider = "jlama"
model = "tjake/Llama-3.2-1B-Instruct-JQ4"
}
The model, a Hugging Face id, is downloaded on the first call to modelDir (default ~/.cache/llm4s/jlama) and loaded from there afterwards. threads sets how many threads Jlama computes with. Jlama needs the JDK's Vector API: run the JVM with --add-modules=jdk.incubator.vector --enable-preview (without the first, Jlama falls back to much slower scalar code). Streaming, no tool calling.
Attributes
- Source
- JlamaProvider.scala
- Graph
-
- Supertypes
- Self type
-
JlamaProvider.type
Members list
Value members
Concrete methods
Constructs the client for a ProviderConfig this provider produced.
Constructs the client for a ProviderConfig this provider produced.
config is whatever the caller passed, so implementations must check the type rather than cast — ProviderDescriptor.expectConfig does that and produces the standard mismatch error.
A client that sends its own HTTP requests sends them through options.httpClient when it is set, and does not close that client, and declares so with features.customHttpClient = true (it defaults to false, which a provider built on a vendor SDK keeps).
Attributes
- Source
- JlamaProvider.scala
Turns a validated config section into the runtime ProviderConfig for this provider.
Turns a validated config section into the runtime ProviderConfig for this provider.
The section has already been checked against configSpec, so required fields are present and section.extras holds exactly the declared provider-specific keys that have a value (defaults applied, deprecated aliases resolved). A Left here means something configSpec cannot express (a malformed value, say).
Value parameters
- providerName
-
the user's instance name (
llm4s.providers.<name>), for error messages.
Attributes
- Source
- JlamaProvider.scala
Streams; tool calling is not offered, since a small local model does not call tools reliably.
Streams; tool calling is not offered, since a small local model does not call tools reliably.
Attributes
- Definition Classes
- Source
- JlamaProvider.scala
Inherited methods
Alternative spellings accepted in provider = "...", folded onto id.
Alternative spellings accepted in provider = "...", folded onto id.
For example Gemini accepts "google". Aliases are matched after the same trim/lowercase canonicalisation as id itself.
Attributes
- Inherited from:
- ProviderDescriptor
- Source
- ProviderDescriptor.scala
Live model discovery, when the provider's API offers it.
Live model discovery, when the provider's API offers it.
Attributes
- Inherited from:
- ProviderDescriptor
- Source
- ProviderDescriptor.scala
Concrete fields
The key naming the directory models are downloaded to.
The key setting how many threads Jlama computes with.
Which config fields this provider requires, its default endpoint, and the provider-specific keys (ProviderConfigSpec.extras) its section accepts.
Which config fields this provider requires, its default endpoint, and the provider-specific keys (ProviderConfigSpec.extras) its section accepts.
Attributes
- Source
- JlamaProvider.scala
Canonical id, e.g. ProviderId("openai"). Must be unique within a registry.
Canonical id, e.g. ProviderId("openai"). Must be unique within a registry.
Attributes
- Source
- JlamaProvider.scala