ContractCase

org.llm4s.testkit.ContractCase
See theContractCase companion object

One way a model's reply can end, which a chat client must report the same way on complete and on a stream (RFC 0009, Part 5). A provider module's spec answers each case with its provider's own bytes, in ProviderModuleChecks.assertModelCallContract, or declares it Applicability.NotApplicable with a reason, so a gap shows in the spec instead of being missed.

A case added later fails every spec that does not declare it, until it does.

Attributes

Companion
object
Source
ModelCallContract.scala
Graph
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Members list

Type members

Enum entries

case CleanText extends ContractCase

The model ended its turn with text: Right, FinishReason.Stop, the text, no tool call.

The model ended its turn with text: Right, FinishReason.Stop, the text, no tool call.

Attributes

Source
ModelCallContract.scala

The provider's content filter stopped the reply: Left(ContentFilteredError), where the provider has a filter.

The provider's content filter stopped the reply: Left(ContentFilteredError), where the provider has a filter.

Attributes

Source
ModelCallContract.scala
case ErrorEvent extends ContractCase

Stream only: the provider reported an error in the stream, after the 200 status and some text: a Left whose message carries ContractCase.ProviderErrorMessage, which the reply's error must state. The reply answers streamComplete only; build it with RecordedReply.streamOnly.

Stream only: the provider reported an error in the stream, after the 200 status and some text: a Left whose message carries ContractCase.ProviderErrorMessage, which the reply's error must state. The reply answers streamComplete only; build it with RecordedReply.streamOnly.

Attributes

Source
ModelCallContract.scala

A tool call's arguments are not a JSON object (invalid or cut JSON, or JSON of another type), in a reply that otherwise ended cleanly: a Left, the malformed-tool-call ProcessingError, never {}, never dropped (RFC 0009, Part 3, rule 1).

A tool call's arguments are not a JSON object (invalid or cut JSON, or JSON of another type), in a reply that otherwise ended cleanly: a Left, the malformed-tool-call ProcessingError, never {}, never dropped (RFC 0009, Part 3, rule 1).

Attributes

Source
ModelCallContract.scala

The model called a tool that takes no parameters, its arguments absent or blank (or {}), and the reply ended cleanly: Right, FinishReason.ToolCalls, each call's arguments {} (RFC 0009, Part 3, rule 2).

The model called a tool that takes no parameters, its arguments absent or blank (or {}), and the reply ended cleanly: Right, FinishReason.ToolCalls, each call's arguments {} (RFC 0009, Part 3, rule 2).

Attributes

Source
ModelCallContract.scala

Stream only: the stream ended without the provider's end marker, after some text: a Left that a retry may fix, a NetworkError or a ServiceError with status 502. The reply answers streamComplete only; build it with RecordedReply.streamOnly.

Stream only: the stream ended without the provider's end marker, after some text: a Left that a retry may fix, a NetworkError or a ServiceError with status 502. The reply answers streamComplete only; build it with RecordedReply.streamOnly.

Attributes

Source
ModelCallContract.scala

The output limit cut the text short: Right, FinishReason.Length, the partial text.

The output limit cut the text short: Right, FinishReason.Length, the partial text.

Attributes

Source
ModelCallContract.scala
case ToolCall extends ContractCase

The model asked for a tool call: Right, FinishReason.ToolCalls, each call's arguments a JSON object.

The model asked for a tool call: Right, FinishReason.ToolCalls, each call's arguments a JSON object.

Attributes

Source
ModelCallContract.scala

The output limit cut the reply while it made a tool call: a Left, the malformed-tool-call ProcessingError. No call is run on intent the model did not finish stating, even one whose arguments happen to parse (RFC 0009, Part 3, rule 3).

The output limit cut the reply while it made a tool call: a Left, the malformed-tool-call ProcessingError. No call is run on intent the model did not finish stating, even one whose arguments happen to parse (RFC 0009, Part 3, rule 3).

Attributes

Source
ModelCallContract.scala