core
API
Generated with
core/org.llm4s/org.llm4s.context/org.llm4s.context.tokens/TokenizerAccuracy

TokenizerAccuracy

org.llm4s.context.tokens.TokenizerAccuracy
See theTokenizerAccuracy companion trait
object TokenizerAccuracy

Accuracy level variants for tokenizer mappings.

Attributes

Companion
trait
Graph
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type

Members list

Type members

Classlikes

case class Approximate(description: String, accuracy: Double) extends TokenizerAccuracy

Approximate tokenizer (Anthropic, Ollama). Token counts may differ by the specified accuracy percentage.

Approximate tokenizer (Anthropic, Ollama). Token counts may differ by the specified accuracy percentage.

Value parameters

accuracy

Expected accuracy as a decimal (0.75 = 75% accurate)

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
case class Exact(description: String) extends TokenizerAccuracy

Exact tokenizer available (OpenAI, Azure). Token counts will match actual API usage.

Exact tokenizer available (OpenAI, Azure). Token counts will match actual API usage.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
case class Unknown(description: String) extends TokenizerAccuracy

Unknown model, using fallback tokenizer. Token counts are best-effort estimates.

Unknown model, using fallback tokenizer. Token counts are best-effort estimates.

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Inherited and Abstract types

type MirroredElemLabels <: Tuple

The names of the product elements

The names of the product elements

Attributes

Inherited from:
Mirror
type MirroredLabel <: String

The name of the type

The name of the type

Attributes

Inherited from:
Mirror
Generated with