Cost controls
Thinking tokens are output tokens: a model that thinks before it answers can cost several times as much as one that does not. llm4s gives you two controls for it, and tells you when a provider could not send one as you asked.
Thinking and reasoning effort
CompletionOptions has two separate controls (RFC 0008):
thinking: Option[Thinking]says whether the model thinks, and within what bound:Thinking.Off,Thinking.Adaptive(the model decides whether and how much), orThinking.budget(n)(a token budget).reasoning: Option[ReasoningEffort]says how hard to think:Minimal,Low,Medium,High,XHighorMax.
1
2
3
4
5
6
7
8
9
10
import org.llm4s.llmconnect.model._
// let the model decide whether to think, at low effort
val adaptive = CompletionOptions().withThinking(Thinking.Adaptive).withReasoning(ReasoningEffort.Low)
// think within a budget; a budget of 0 or less is a ValidationError, so this is a Result
val budgeted: Result[CompletionOptions] = Thinking.budget(8000).map(CompletionOptions().withThinking)
// do not think
val off = CompletionOptions().withThinking(Thinking.Off)
From Java, JCompletionOptions.builder().thinking(JThinking.adaptive()).reasoning(JReasoningEffort.LOW);
JThinking.budget(n) throws IllegalArgumentException for n <= 0. From Kotlin, the completionOptions { } DSL
has a thinking property.
Which one wins
An explicit thinking wins. Without one, reasoning decides as it always has: a provider that thinks within a
budget gets the level’s default budget, so a request that sets only reasoning sends what it sent before thinking
existed. A pair that contradicts itself on a provider with one control for both, such as thinking = Off with
reasoning = High on OpenAI, is resolved by this rule and the level is reported as Adjusted.
The exception. Some current models reject a manual thinking budget with a 400 (Claude Opus 4.7, Opus 4.8 and
Fable 5 among those in the bundled model metadata). For a model whose capabilities say so
(ModelCapabilities.supportsBudgetThinking = Some(false)), a reasoning-only request is sent as adaptive thinking at
that effort instead, reported as Adjusted(Reasoning, "budget_tokens", "output_config.effort", ...). Every request
this changes is one that failed before.
What each provider sends
reasoning |
thinking |
|
|---|---|---|
| Anthropic | output_config.effort where the model takes it and you set thinking or a new level; otherwise a budget_tokens budget, as before. A level Anthropic lacks (minimal) is sent as the nearest. |
Off → disabled (a model that cannot turn thinking off gets its default, reported); Adaptive → adaptive (a model from before adaptive thinking gets a budget, reported); Budget(n) → budget_tokens, fitted to [1024, max_tokens), or adaptive on a model that rejects a budget, each reported when changed |
| OpenAI, Azure | reasoning_effort; a level the model lacks is sent as the nearest it accepts |
Off → the lowest effort the model accepts (none where it can turn thinking off); Adaptive not needed; Budget dropped |
| OpenRouter | as before: a budget for Claude models, reasoning_effort for OpenAI o-series |
Claude models: Budget(n) → thinking.budget_tokens, Off → no thinking block; otherwise dropped |
| Z.ai, xAI | by model family; see Providers | Off → thinking.type: disabled or the lowest effort, as the model allows (reported where the model still thinks) |
| Perplexity, watsonx | the Agent API’s or chat API’s levels, the nearest where one is missing | Off → the lowest effort; the others are not sent |
| Gemini, Vertex AI, Bedrock, Ollama, other OpenAI-compatible | dropped | dropped |
A provider’s own mapping, and which models take which levels, is on its section of Providers. Which
levels a model accepts comes from the bundled model metadata (ModelCapabilities.supportedReasoningEfforts,
supportsAdaptiveThinking, supportsOutputConfig, supportsThinkingOff, supportsBudgetThinking); a model the
metadata does not know is sent what you asked.
Seeing what was sent
Whatever a provider could not send as asked is on Completion.adjustments (and on an agent’s
ModelCallCompleted events), and logged at WARN once per client, model and option:
1
2
3
4
5
6
7
client.complete(conversation, CompletionOptions().withReasoning(ReasoningEffort.Max)).foreach { completion =>
completion.adjustments.foreach {
case RequestAdjustment.Adjusted(option, requested, sent, reason) => println(s"$option: $requested -> $sent ($reason)")
case RequestAdjustment.Dropped(option, reason) => println(s"$option not sent: $reason")
case RequestAdjustment.NotNeeded(option, reason) => println(s"$option not needed: $reason")
}
}
A caller that must not run with an option it did not get treats a Dropped as a failure. An empty list means
nothing was reported, not that every option was sent: a few providers (Perplexity, watsonx) report nothing yet.