llm4s-core
API
Generated with
llm4s-core/org.llm4s/org.llm4s.model

org.llm4s.model

package org.llm4s.model

Members list

Type members

Classlikes

class DefaultRequestTransformer(overrides: Map[String, ModelCapabilities]) extends RequestTransformer

Default implementation that uses ModelRegistry for capability lookups.

Default implementation that uses ModelRegistry for capability lookups.

Value parameters

overrides

Optional map of model-specific capability overrides

Attributes

Supertypes
class Object
trait Matchable
class Any
case class ModelCapabilities(supportsFunctionCalling: Option[Boolean], supportsParallelFunctionCalling: Option[Boolean], supportsVision: Option[Boolean], supportsPromptCaching: Option[Boolean], supportsReasoning: Option[Boolean], supportsResponseSchema: Option[Boolean], supportsSystemMessages: Option[Boolean], supportsPdfInput: Option[Boolean], supportsAudioInput: Option[Boolean], supportsAudioOutput: Option[Boolean], supportsWebSearch: Option[Boolean], supportsComputerUse: Option[Boolean], supportsAssistantPrefill: Option[Boolean], supportsToolChoice: Option[Boolean], supportsNativeStreaming: Option[Boolean], supportedRegions: Option[List[String]], disallowedParams: Option[Set[String]], temperatureConstraint: Option[(Double, Double)])

Model capabilities and features.

Model capabilities and features.

Value parameters

disallowedParams

Set of parameter names that are not supported by this model

supportedRegions

List of supported deployment regions

supportsAssistantPrefill

Whether the model supports assistant message prefill

supportsAudioInput

Whether the model supports audio inputs

supportsAudioOutput

Whether the model supports audio outputs

supportsComputerUse

Whether the model supports computer use

supportsFunctionCalling

Whether the model supports function/tool calling

supportsNativeStreaming

Whether the model supports native streaming (false = needs fake streaming)

supportsParallelFunctionCalling

Whether the model supports parallel tool calls

supportsPdfInput

Whether the model supports PDF file inputs

supportsPromptCaching

Whether the model supports prompt caching

supportsReasoning

Whether the model supports reasoning (O-series, Claude thinking)

supportsResponseSchema

Whether the model supports structured response schemas

supportsSystemMessages

Whether the model supports system messages (false for some O-series)

supportsToolChoice

Whether the model supports tool_choice parameter

supportsVision

Whether the model supports vision/image inputs

supportsWebSearch

Whether the model supports web search

temperatureConstraint

Temperature constraint: None = any, Some((min, max)) = restricted range

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ModelCapabilities

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
case class ModelMetadata(modelId: String, provider: String, mode: ModelMode, maxInputTokens: Option[Int], maxOutputTokens: Option[Int], inputCostPerToken: Option[Double], outputCostPerToken: Option[Double], capabilities: ModelCapabilities, pricing: ModelPricing, deprecationDate: Option[String])

Comprehensive metadata for an LLM model. Based on litellm's model_prices_and_context_window.json schema.

Comprehensive metadata for an LLM model. Based on litellm's model_prices_and_context_window.json schema.

This provides a centralized, type-safe representation of model capabilities, pricing, and constraints that can be queried at runtime.

Value parameters

capabilities

Model capabilities and features

deprecationDate

Optional deprecation date (YYYY-MM-DD)

inputCostPerToken

Cost per input token (in dollars)

maxInputTokens

Maximum input tokens supported

maxOutputTokens

Maximum output tokens supported

mode

The model mode (chat, embedding, image_generation, etc.)

modelId

The unique identifier for this model (e.g., "gpt-4o", "claude-3-7-sonnet-latest")

outputCostPerToken

Cost per output token (in dollars)

pricing

Detailed pricing information

provider

The LLM provider (openai, anthropic, azure, etc.)

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ModelMetadata

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
sealed trait ModelMode

Model operation mode.

Model operation mode.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object AudioSpeech
object Chat
object Completion
object Embedding
object Moderation
object Rerank
object Search
object Unknown
Show all
object ModelMode

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
ModelMode.type
case class ModelPricing(inputCostPerToken: Option[Double], outputCostPerToken: Option[Double], cacheCreationInputTokenCost: Option[Double], cacheReadInputTokenCost: Option[Double], inputCostPerTokenBatches: Option[Double], outputCostPerTokenBatches: Option[Double], inputCostPerTokenPriority: Option[Double], outputCostPerTokenPriority: Option[Double], outputCostPerReasoningToken: Option[Double], inputCostPerAudioToken: Option[Double], outputCostPerAudioToken: Option[Double], inputCostPerImage: Option[Double], outputCostPerImage: Option[Double], inputCostPerPixel: Option[Double], outputCostPerPixel: Option[Double])

Detailed pricing information for a model.

Detailed pricing information for a model.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ModelPricing

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
object ModelRegistry

Central registry for LLM model metadata.

Central registry for LLM model metadata.

This provides a singleton lookup service for model capabilities, pricing, and constraints. It loads data from the embedded litellm metadata file and supports runtime updates from external sources.

Example usage:

 val metadata = ModelRegistry.lookup("gpt-4o")
 metadata.foreach { m =>
   println(s"Context window: $${m.contextWindow}")
   println(s"Supports vision: $${m.supports("vision")}")
 }

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
trait RequestTransformer

Handles model-specific parameter validation and transformation.

Handles model-specific parameter validation and transformation.

Uses ModelCapabilities from ModelRegistry to apply constraints based on what each model supports. This mirrors LiteLLM's approach to handling model-specific quirks (e.g., O-series temperature restrictions).

Example usage:

 val transformer = RequestTransformer.default
 val result = transformer.transformOptions("o1", options, dropUnsupported = true)
 result match {
   case Right(transformed) => // use transformed options
   case Left(error) => // handle validation error
 }

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object RequestTransformer

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
case class TransformationResult(options: CompletionOptions, messages: Seq[Message], warnings: Seq[String], requiresFakeStreaming: Boolean, requiresMaxCompletionTokens: Boolean)

Transformation result containing both transformed options and any warnings.

Transformation result containing both transformed options and any warnings.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object TransformationResult

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Generated with