WatsonXClient

org.llm4s.llmconnect.provider.WatsonXClient
See theWatsonXClient companion object
class WatsonXClient(config: WatsonXConfig, val metrics: MetricsCollector, exchangeLogging: ProviderExchangeLogging, httpClient: Llm4sHttpClient, val nowSeconds: () => Long)(using val registryService: ModelRegistryService) extends BaseLifecycleLLMClient

LLMClient for IBM watsonx.ai chat (POST /ml/v1/text/chat and /ml/v1/text/chat_stream).

'''Beta - never run against the live service.''' There is no watsonx account behind this project, so nothing here has been checked against the real service (see https://github.com/llm4s/llm4s/issues/1314). The request and response shapes are those IBM's public pages report, cross-checked against IBM's own open-source client code (see ''Evidence'' below). The module's API is not frozen. It replaces the text-generation endpoints (/ml/v1/text/generation and /generation_stream), which IBM's February 2026 release notes deprecate without giving a removal date.

== Evidence ==

A detail is ''SDK-evidenced'' when IBM's own code shows it. That proves IBM's client sends it, which is strong evidence the service accepts it, but it is not a live check. The sources, pinned:

The @Cloud suite WatsonXAssumptionProbeSpec (modules/it) re-checks each of these against a real account and prints which held.

== Authentication ==

The IBM Cloud API key is exchanged at the IAM endpoint for a bearer token that lives an hour. The token is cached and refreshed lazily, five minutes before it expires.

== Request format ==

The conversation is sent as structured messages with roles (system, user, assistant, tool), so content is data and cannot forge a turn: there is no prompt string with role markers, and no stop sequences standing in for them. The model goes in model_id, the project or space in project_id or space_id, and the API version in the version query parameter (WatsonXConfig.apiVersion, by default the date PY sends: org.llm4s.llmconnect.config.WatsonXConfig.DEFAULT_API_VERSION). temperature is always sent, max_tokens when set (SDK-evidenced: NODE types/vml_v1.ts:790-797, a deprecated alias of max_completion_tokens), and top_p when not 1.0.

== Tools ==

CompletionOptions.tools are sent as tools (type: function, function: {name, description, parameters}; the OpenAI strict flag is dropped) with tool_choice_option: "auto". Tool calls in a reply (message.tool_calls, whole, or streamed as delta.tool_calls pieces merged by index) become ToolCalls. An assistant turn's calls go back as tool_calls, and a ToolMessage as a tool message with its tool_call_id. arguments is a JSON string on the wire (SDK-evidenced: NODE types/messages.ts:19-24) and a JSON object in a ToolCall. A call without an id gets a generated one, defensively: the SDK's type makes id required (NODE types/messages.ts:29-30), and a stream sends it on a call's first piece only (LCIBM chat_models.py:324). Whether the model calls tools, or supports them at all, depends on the model.

== Penalties and response format ==

presencePenalty and frequencyPenalty are sent as presence_penalty and frequency_penalty when not 0.0, their default, so a request that leaves them alone is unchanged (SDK-evidenced: NODE vml_v1.ts:2808,2815 for textChat and 3032,3039 for textChatStream; PY foundation_models/schema/_api.py:270,273; LCIBM chat_models.py:939,969,1664,1670). Their range is the service's to check.

responseFormat is sent as response_format (WatsonXClient.encodeResponseFormat): ResponseFormat.Json as {"type":"json_object"} (NODE types/vml_v1.ts:3283-3310; PY schema/_api.py:225; LCIBM chat_models.py:2147-2149), and ResponseFormat.JsonSchema as {"type":"json_schema","json_schema":{"name","schema","strict"}} (PY schema/_api.py:226,231-234,255-257; LCIBM chat_models.py:2160-2182). The Node SDK's type names only json_object but takes any string, so json_schema rests on the Python SDK and IBM's LangChain integration. IBM's SDK notes that JSON mode needs the prompt to ask for JSON as well (NODE types/vml_v1.ts:3284-3292); this client adds no instruction. Whether a model honours either format depends on the model.

== Reasoning ==

CompletionOptions.reasoning is sent as reasoning_effort (low, medium or high; SDK-evidenced: NODE types/vml_v1.ts:860-864 and vml_v1.ts:2828,3053 for textChat and textChatStream; PY schema/_api.py:289; LCIBM chat_models.py:1010), but only to the models known to take it (WatsonXClient.takesReasoningEffort): OpenAI's gpt-oss models, openai/gpt-oss-*, which IBM's clients send it to (JAVA ChatServiceThinkingTest; LCIBM chat_models.py:657-690). Any other model is sent nothing, and a warning is logged once per process: IBM's Model Gateway rejects the parameter for models without it (JAVA ModelGatewayChatServiceIT), and whether the chat API does is unchecked. The chat API has no value that turns reasoning off, so ReasoningEffort.None is sent as low, the least, with a warning logged once per process (WatsonXClient.reasoningEffortValue). include_reasoning is not sent: it defaults to true (NODE types/vml_v1.ts:858).

A reply's reasoning is read as thinking, from any model: message.reasoning_content on complete (NODE types/vml_v1.ts:3383-3384) and delta.reasoning_content on a stream, passed on as StreamedChunk.thinkingDelta (JAVA gpt_oss_thinking_response.txt and gpt_oss_thinking_streaming_response.txt, recorded replies of openai/gpt-oss-120b); reasoning, which IBM's LangChain integration also reads (LCIBM chat_models.py:150-153, 332-335), when reasoning_content is absent. Thinking is not sent back on a later turn: the chat API's assistant message has no field for it. Reasoning tokens are not reported: the chat API's usage has only prompt_tokens, completion_tokens and total_tokens (NODE types/vml_v1.ts:3496-3503, and the recorded replies above), so TokenUsage.thinkingTokens stays empty. budgetTokens has no counterpart and is ignored.

== Finish reasons and stream endings ==

A stream must end with a choice that carries a finish_reason. One that ends without it, or whose reason is in WatsonXClient.ErrorFinishReasons, is a Left(ServiceError) naming the reason; text received so far is not returned as a success. Every other reason (stop, length, tool_calls, unknown values) is a normal stop. complete applies the same rule to choices[0].finish_reason (a missing one is fine). The usage arrives in a final chunk that may have no choices (SDK-evidenced: LCIBM chat_models.py:379-383). Streamed tool calls are reported once, whole, after the last delta, never as fragments.

An event: error frame ends the stream with IBM's error: its data: line holds IBM's error JSON, whose status_code picks the LLMError as an HTTP status would (401/403 AuthenticationError, 429 RateLimitError, 400 ValidationError, otherwise ServiceError; none is ServiceError(502)) and whose first errors entry's code and message, redacted, are the message (SDK-evidenced: PY foundation_models/inference/fm_model_inference.py:683-689 raises on that frame).

== Timeouts ==

A chat call times out after two minutes and a streamed one after ten, unless the section's timeouts { request = ..., stream = ... } block sets either (WatsonXConfig.timeouts). The IAM token exchange keeps its own 30 seconds.

Value parameters

config

model, credentials, project or space and endpoints.

exchangeLogging

optional provider exchange logging.

httpClient

used for both the model calls and the IAM exchange.

metrics

receives per-call latency and token-usage events.

Attributes

Companion
object
Source
WatsonXClient.scala
Graph
Supertypes
trait LLMClient
class Object
trait Matchable
class Any
Show all

Members list

Value members

Concrete methods

override def complete(conversation: Conversation, options: CompletionOptions): Result[Completion]

Executes a blocking completion request and returns the full response.

Executes a blocking completion request and returns the full response.

Sends the conversation to the LLM and waits for the complete response. Use when you need the entire response at once or when streaming is not required.

Value parameters

conversation

conversation history including system, user, assistant, and tool messages

options

configuration including temperature, max tokens, tools, etc. (default: CompletionOptions())

Attributes

Returns

Right(Completion) with the model's response, or Left(LLMError) on failure

Definition Classes
Source
WatsonXClient.scala
override def getContextWindow(): Int

Returns the maximum context window size supported by this model in tokens.

Returns the maximum context window size supported by this model in tokens.

The context window is the total tokens (prompt + completion) the model can process in a single request, including all conversation messages and the generated response.

Attributes

Returns

total context window size in tokens (e.g., 4096, 8192, 128000)

Definition Classes
Source
WatsonXClient.scala
override def getReserveCompletion(): Int

Returns the number of tokens reserved for the model's completion response.

Returns the number of tokens reserved for the model's completion response.

This value is subtracted from the context window when calculating available tokens for prompts. Corresponds to the max_tokens or completion token limit configured for the model.

Attributes

Returns

number of tokens reserved for completion

Definition Classes
Source
WatsonXClient.scala
override def streamComplete(conversation: Conversation, options: CompletionOptions, onChunk: StreamedChunk => Unit): Result[Completion]

Executes a streaming completion request, invoking a callback for each chunk as it arrives.

Executes a streaming completion request, invoking a callback for each chunk as it arrives.

Streams the response incrementally, calling onChunk for each token/chunk received. Enables real-time display of responses. Returns the final accumulated completion on success.

Value parameters

conversation

conversation history including system, user, assistant, and tool messages

onChunk

callback invoked for each chunk; called synchronously, avoid blocking operations

options

configuration including temperature, max tokens, tools, etc. (default: CompletionOptions())

Attributes

Returns

Right(Completion) with the complete accumulated response, or Left(LLMError) on failure

Definition Classes
Source
WatsonXClient.scala

Inherited methods

override def close(): Unit

Releases resources and closes connections to the LLM provider.

Releases resources and closes connections to the LLM provider.

Call when the client is no longer needed. After calling close(), the client should not be used. Default implementation is a no-op; override if managing resources like connections or thread pools.

Attributes

Definition Classes
Inherited from:
BaseLifecycleLLMClient
Source
BaseLifecycleLLMClient.scala
def completeStructured[A](conversation: Conversation, schema: ObjectSchema[A], options: CompletionOptions)(implicit reader: Reader[A]): Result[A]

Sends the conversation and parses the response into a typed value using the provided schema.

Sends the conversation and parses the response into a typed value using the provided schema.

Sets ResponseFormat.JsonSchema on the options. OpenAI, Azure OpenAI and Gemini enforce the schema at generation time, as does Ollama 0.5 or later through its format field. Requesty and the OpenAI-compatible providers (including Cohere) send it as response_format, but whether it is enforced is up to the server - for Requesty, a router, the backend model it routes to: one that ignores the field returns unconstrained text. Anthropic falls back to a best-effort system-prompt instruction, which is not schema-enforced. Clients that do not read responseFormat (watsonx, Bedrock) send no schema at all. Because models may wrap JSON in markdown code fences or surround it with prose, the response is normalised (fence stripped, first balanced {...} or [...] extracted) before being deserialised with uPickle into the expected type A. A is read from that text, so a Long field receives an integer above 2^53 exactly; a reader that goes through ujson.Value (such as a bridge to another JSON library) sees it rounded to the nearest double.

The reply is '''not''' validated against the schema: it is only deserialised. A constraint the reader does not check - an enum, a numeric or string bound, additionalProperties = false (extra keys are ignored) - can be violated by a reply that still returns Right(A). Check such constraints on the result yourself.

The schema is derived with strict = true, which lists '''every''' property as required, including a property declared optional with required = false. Only responseFormat is overridden: every other option you pass is forwarded unchanged to complete, where the provider client may adjust or drop options the model does not support, as for any other complete call. name and strict on the format are left at their defaults ("response" and true); call complete with your own ResponseFormat.JsonSchema to set them.

Type parameters

A

target type; must have a corresponding upickle.default.Reader[A]

Value parameters

conversation

conversation history

options

additional completion options (default: CompletionOptions())

reader

implicit uPickle reader for deserialising the JSON into A

schema

JSON-Schema description of the expected response object

Attributes

Returns

Right(A) on success. Left(ValidationError) with field structured_output when the reply is not JSON, is JSON null, or cannot be deserialised as A; any other Left is the provider call's own error, returned unchanged

Inherited from:
LLMClient
Source
LLMClient.scala
protected def completeWithMetrics(operation: => Result[Completion]): Result[Completion]

Validates that the client is open, executes the operation, and records standard completion metrics (latency, token usage, estimated cost).

Validates that the client is open, executes the operation, and records standard completion metrics (latency, token usage, estimated cost).

Use this in complete and streamComplete implementations to avoid repeating the lifecycle-check + metrics-wrapping boilerplate.

An interrupted call - one that throws InterruptedException, or fails while the thread is interrupted - is returned as Left(CancelledError) with the interrupt flag kept, whatever the provider SDK did with it. A call made with the flag already set returns Left(CancelledError) at once, without running operation: an SDK that ignores the flag would otherwise send the (billed) request anyway.

Value parameters

operation

The provider-specific completion logic to execute. Called only when the client is open and the thread is not interrupted.

Attributes

Returns

The completion result with metrics recorded as a side-effect.

Inherited from:
BaseLifecycleLLMClient
Source
BaseLifecycleLLMClient.scala

Calculates available token budget for prompts after accounting for completion reserve and headroom.

Calculates available token budget for prompts after accounting for completion reserve and headroom.

Formula: (contextWindow - reserveCompletion) * (1 - headroom)

Headroom provides a safety margin for tokenization variations and message formatting overhead.

Value parameters

headroom

safety margin as percentage of prompt budget (default: HeadroomPercent.Standard ~10%)

Attributes

Returns

maximum tokens available for prompt content

Inherited from:
LLMClient
Source
LLMClient.scala
override def providerId: Option[ProviderId]

This client's providerName, as the provider id tracing reports.

This client's providerName, as the provider id tracing reports.

Attributes

Definition Classes
Inherited from:
BaseLifecycleLLMClient
Source
BaseLifecycleLLMClient.scala
def validate(): Result[Unit]

Validates client configuration and connectivity to the LLM provider.

Validates client configuration and connectivity to the LLM provider.

May perform checks such as verifying API credentials, testing connectivity, and validating configuration. Default implementation returns success; override for provider-specific validation.

Attributes

Returns

Right(()) if validation succeeds, Left(LLMError) with details on failure

Inherited from:
LLMClient
Source
LLMClient.scala
protected def validateNotClosed: Result[Unit]

Attributes

Inherited from:
BaseLifecycleLLMClient
Source
BaseLifecycleLLMClient.scala
protected def withMetrics[A](provider: String, model: String, operation: => Result[A], extractUsage: A => Option[TokenUsage], extractCost: A => Option[Double]): Result[A]

Executes operation and records metrics for the call.

Executes operation and records metrics for the call.

Latency and outcome (success or classified error) are recorded for every call regardless of result. Token counts and cost are recorded only on success — a Left result emits an Outcome.Error (org.llm4s.metrics.Outcome) event whose kind is derived from the org.llm4s.error.LLMError subtype via ErrorKind.fromLLMError.

Value parameters

extractCost

Extracts the pre-computed cost (USD) from a successful result; return None to skip cost recording.

extractUsage

Extracts prompt/completion token counts from a successful result; return None to skip token recording.

model

Model identifier forwarded to the collector.

operation

The LLM call to time and observe.

provider

Provider label forwarded to the collector (e.g. "openai").

Attributes

Returns

The result of operation, unchanged.

Inherited from:
MetricsRecording
Source
MetricsRecording.scala

Givens

Givens