org.llm4s.trace

@Stable

Console-based Tracing implementation with colored, formatted output.

Prints trace events to standard output with ANSI color formatting for improved readability during development and debugging. Returns Result[Unit] to support functional composition.

== Content is withheld unless enabled ==

By default it prints metadata only: each event's type, timestamp, model, ids, tool names, success flags, durations, token counts and costs. Text that may hold personal or confidential data - the query of AgentInitialized, the content of CompletionReceived, the input and output of ToolExecuted, the field values of a CustomEvent (whose data is defined by whoever emits it, so may hold anything), and the message and context of an error - is replaced by [withheld: N chars] (a CustomEvent shows its field names). Standard output is often collected into logs kept outside the controls applied to the conversation itself (#2311).

Pass captureContent = true, or set llm4s.tracing.console.captureContent = true (LLM4S_TRACING_CONSOLE_CAPTURE_CONTENT=true) when the tracer is built from Llm4sConfig.tracing(), to print that text as well, truncated for readability.

== Features ==

Color-coded output by event type (errors in red, success in green, etc.)
Visual separators and headers for different trace sections
Formatted display of all TraceEvent types
Truncation of long JSON content for readability
Timestamps on all events

== Usage ==

val tracing: Tracing = new ConsoleTracing()            // metadata only
val verbose: Tracing = new ConsoleTracing(captureContent = true) // queries, answers, tool I/O too

// Trace events functionally
for {
 _ <- tracing.traceEvent(TraceEvent.AgentInitialized("query", Vector("tool1")))
 _ <- tracing.traceTokenUsage(TokenUsage(100, 50, 150), "gpt-4", "completion")
} yield ()

Value parameters

captureContent: print prompts, completions, tool input/output, custom event data and error messages; false (the default) prints metadata only

Attributes

See also: NoOpTracing for silent tracing

LangfuseTracing in llm4s-observability for production observability

AnsiColors for color constants used
Companion: object
Supertypes: trait Tracing

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: ConsoleTracing.type

Default implementation of LangfuseBatchSender using HTTP requests.

Sends trace events to Langfuse using basic authentication with the provided public and secret keys. A 2xx response is a success; a 207 Multi-Status response is read per event, and any events it lists as rejected are logged as an error.

Logs warnings if credentials are not configured and errors if the HTTP request fails.

Attributes

Companion: object
Supertypes: trait LangfuseBatchSender

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: DefaultLangfuseBatchSender.type

Abstraction for sending trace events to Langfuse in batches.

Implementations handle the HTTP communication with the Langfuse API, including authentication, error handling, and logging.

Attributes

See also: DefaultLangfuseBatchSender for the standard implementation
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class DefaultLangfuseBatchSender

Configuration for calling the Langfuse HTTP API.

Value parameters

langfuseUrl: Base URL of the Langfuse API endpoint
publicKey: Langfuse public key for authentication
secretKey: Langfuse secret key for authentication

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Langfuse Tracing implementation for production observability.

Sends trace events to Langfuse for centralized observability, debugging, and LLM application monitoring. Supports all trace event types with type-safe Result[Unit] return values.

== Features ==

Real-time event streaming to Langfuse
Hierarchical trace structure (traces with child spans)
Token usage and cost tracking
Error logging with stack traces
Agent state snapshots with conversation history

== Configuration ==

Configure via environment variables or direct instantiation:

// Environment variables
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_URL=https://cloud.langfuse.com (optional)

// Or programmatic configuration
val tracing = new LangfuseTracing(
 langfuseUrl = "https://cloud.langfuse.com",
 publicKey = Secret("pk-lf-..."),
 secretKey = Secret("sk-lf-..."),
 environment = "production",
 release = "1.0.0",
 version = "1.0.0"
)

== Usage ==

val tracing: Tracing = LangfuseTracing.from(config)

for {
 _ <- tracing.traceEvent(TraceEvent.AgentInitialized("query", tools))
 _ <- tracing.traceCompletion(completion, "gpt-4")
 _ <- tracing.traceTokenUsage(usage, "gpt-4", "completion")
} yield ()

Value parameters

environment: Environment name (e.g., "production", "staging")
langfuseUrl: Langfuse API URL (default: https://cloud.langfuse.com)
publicKey: Langfuse public key for authentication
release: Release/version identifier
secretKey: Langfuse secret key for authentication
version: API version

Attributes

See also: ConsoleTracing for local development

NoOpTracing for disabled tracing

TracingMode for configuration modes
Companion: object
Supertypes: trait Tracing

class Object

trait Matchable

class Any

Factory methods for LangfuseTracing.

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: LangfuseTracing.type

Registers LangfuseTracing for TracingMode.Named("langfuse") (TRACING_MODE=langfuse).

Declared in llm4s-observability's META-INF/services/org.llm4s.trace.spi.TracingBackend, so depending on the module is all it takes. It reads its settings from settings.extras - the llm4s.tracing.langfuse block, which this module's reference.conf binds to the LANGFUSE_* variables - as a provider descriptor reads its named section.

Without both keys it refuses to start, so Tracing.fromSettings reports the missing variables and Tracing.create falls back to NoOpTracing with that error logged, rather than returning a tracer that drops every batch.

A class rather than an object because java.util.ServiceLoader instantiates it through a public no-arg constructor.

Attributes

Supertypes: trait TracingBackend

class Object

trait Matchable

class Any

@Stable

No-operation Tracing implementation that silently discards all events.

All methods return Right(()) immediately without performing any operations. Use this implementation when tracing is disabled or not needed.

== Usage ==

val tracing: Tracing = new NoOpTracing()

// All operations succeed silently
tracing.traceEvent(TraceEvent.CustomEvent("test", ujson.Obj())) // Returns Right(())
tracing.traceError(new Exception("ignored")) // Returns Right(())

== Use Cases ==

Production environments where tracing overhead is undesirable
Unit tests that don't need trace output
Default fallback when no tracing is configured

Attributes

See also: ConsoleTracing for development/debugging

LangfuseTracing in llm4s-observability for production observability
Supertypes: trait Tracing

class Object

trait Matchable

class Any

Exports trace events as OpenTelemetry spans over OTLP/gRPC.

An event placed in a trace (TraceEvent.span, design 4.58) becomes a span with exactly the placement's ids: its trace and span id, its parent - which may be a remote parent, a span another process exported - its start time and, as its end, the event's timestamp. An agent's run is so one tree: invoke_agent <agent> (from AgentRunEnded) over chat <model> (agent.model_call_completed), execute_tool <tool> (agent.tool_executed) and the run's other events, and each delegated child's invoke_agent under the execute_tool task that started it, whichever process ran it. An event without a placement becomes a span of its own, as before. The agent's spans carry no message content (Stage 5 rule 10).

Model calls follow the OpenTelemetry GenAI semantic conventions, version OpenTelemetryTracing.SemconvVersion (#2275): a completion - plain or streamed - is a CLIENT span chat <model>, an embedding call one named embeddings <model>, with gen_ai.operation.name, gen_ai.provider.name, the request and response model, the response id and finish reasons, token usage, the request parameters and server.address; a failed call is that span with error.type and an ERROR status. Agent spans carry gen_ai.provider.name too.

Message content - prompts, completions, an agent's query and tool input and output - is recorded only with captureContent (llm4s.tracing.opentelemetry.captureContent, off by default), as the conventions' gen_ai.input.messages / gen_ai.output.messages, with secrets redacted first.

Attributes

Companion: object
Supertypes: trait Tracing

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: OpenTelemetryTracing.type

Registers OpenTelemetryTracing for TracingMode.Named("opentelemetry") (TRACING_MODE=opentelemetry or otel).

Declared in this module's META-INF/services/org.llm4s.trace.spi.TracingBackend, so depending on llm4s-observability-otel is all it takes; Tracing.create finds it by ServiceLoader. That replaced the Class.forName reflection core used to load this module (#1133).

A class rather than an object because java.util.ServiceLoader instantiates it through a public no-arg constructor.

Attributes

Supertypes: trait TracingBackend

class Object

trait Matchable

class Any

A Tracing implementation that converts every TraceEvent into a Span and persists it in the given TraceStore[F].

Composable with other backends via TracingComposer.combine(). The traceId field exposes the identifier assigned to this run so callers can retrieve spans from the store after execution completes.

Type parameters

F: effect type of the backing store; an implicit org.llm4s.util.LiftToResult instance must be available to bridge F[A] into Result[A]

Value parameters

initialTraceId: optional fixed trace ID; a random UUID is used when absent
store: destination for all recorded spans

Attributes

Companion: object
Supertypes: trait Tracing

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: TraceCollectorTracing.type

@Stable

A position in a distributed trace, in W3C Trace Context terms (design 4.58): the trace (traceId, 32 lowercase hex digits), one span in it (spanId, 16 lowercase hex digits) and whether the trace is sampled. Neither id is all zeros. It is the serializable carrier a caller stores or sends so that work done elsewhere - another thread, another process, another machine - continues the same trace: written as a traceparent header value (traceParent, 00-<traceId>-<spanId>-<flags>) and read back with TraceContext.parse; its JSON form is that string.

Whoever continues a context makes it the remote parent of the span it opens, so that span keeps the context's trace id. A backend that cannot express a parent ignores it.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: TraceContext.type

@Stable

Type-safe, sealed hierarchy of trace events emitted during LLM agent execution.

Each subtype carries the data relevant to one observable moment in the agent lifecycle (completion received, tool executed, token usage recorded, etc.). All subtypes are serialisable to JSON via toJson, which produces a flat object containing at minimum "event_type" and "timestamp" fields.

== Error truncation == ErrorOccurred.toJson includes only the first 5 frames of the stack trace to limit payload size.

== Timestamp default == Every event's companion apply defaults timestamp to Instant.now() at construction time. Override explicitly when replaying historical events or writing deterministic tests.

== Growth safety == Each event is a final case class with a private constructor and copy: build one with its companion apply and change a field with its with* setter (an Option field's setter takes the value or an Option), so a field can be added later without breaking binary compatibility.

Attributes

See also: Tracing for the interface that consumes these events
Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class AgentInitialized

class AgentRunEnded

class CacheHit

class CacheMiss

class CompletionReceived

class CostRecorded

class CustomEvent

class EmbeddingUsageRecorded

class ErrorOccurred

class ImageGenerationCompleted

class RAGOperationCompleted

class TokenUsageRecorded

class ToolExecuted
Show all

Attributes

Companion: trait
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: TraceEvent.type

@Stable

Where a traced event sits in a trace (design 4.58): the span it is (context), that span's parent, if any - which may be a span of another process, a remote parent - and when the span started. The event's own timestamp is when it ended. A backend that builds spans uses all three; one that does not ignores them.

A span is in its parent's trace: a parent of another trace is refused, by TraceSpan.of as a ValidationError and by apply and the with* setters as an IllegalArgumentException, so no backend ever sees a span whose trace id and parent's disagree.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: TraceSpan.type

@Stable

Type-safe tracing interface for observability and debugging.

Provides a functional approach to tracing with Result[Unit] return types for proper error handling and composition. Supports multiple backends including console output, Langfuse, OpenTelemetry and custom implementations.

== Implementations ==

ConsoleTracing - Colored console output for development (core); metadata only unless llm4s.tracing.console.captureContent is set
NoOpTracing - Silent implementation for testing/disabled tracing (core)
LangfuseTracing - Production observability via Langfuse (llm4s-observability)
OpenTelemetryTracing - OTLP export (llm4s-observability-otel)

== Usage ==

// Create from settings
val tracing = Tracing.create(settings)

// Or use directly
val tracing: Tracing = new ConsoleTracing()

// Trace events functionally
for {
 _ <- tracing.traceEvent(TraceEvent.AgentInitialized("query", tools))
 _ <- tracing.traceTokenUsage(usage, "gpt-4", "completion")
} yield ()

== Agent state ==

There is no traceAgentState(AgentState). An agent run reports its end with TraceEvent.AgentRunEnded; the agent's per-step events arrive as CustomEvents named graph.* and agent.*.

== Composition ==

Tracers can be composed using TracingComposer:

val combined = TracingComposer.combine(consoleTracer, langfuseTracer)
val filtered = TracingComposer.filter(tracer)(_.eventType == "error_occurred")

Attributes

See also: TraceEvent for available event types

TracingComposer for composition utilities
Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class ConsoleTracing

class LangfuseTracing

class NoOpTracing

class OpenTelemetryTracing

class TraceCollectorTracing[F]

Factory for creating Tracing instances.

// From TracingSettings: falls back to NoOp, with an error logged, if the backend is missing
val tracing = Tracing.create(settings)

// The same, but a missing or failing backend is an error the caller sees
val checked: Result[Tracing] = Tracing.fromSettings(settings)

// Direct instantiation
val console = new ConsoleTracing()
val noop = new NoOpTracing()

Attributes

See also: TracingMode for available modes

org.llm4s.trace.spi.TracingBackend for adding a backend
Companion: trait
Supertypes: class Object

trait Matchable

class Any
Self type: Tracing.type

@Stable

Utilities for composing multiple tracers.

Provides functional composition patterns for combining, filtering, and transforming trace events across multiple tracing backends.

== Combining Tracers ==

Send events to multiple backends simultaneously:

val combined = TracingComposer.combine(consoleTracer, langfuseTracer)
combined.traceEvent(event) // Sends to both

== Filtering Events ==

Only trace events matching a predicate:

val errorsOnly = TracingComposer.filter(tracer)(_.eventType == "error_occurred")

== Transforming Events ==

Modify events before tracing:

val enriched = TracingComposer.transform(tracer) {
 case e: TraceEvent.CustomEvent => e.withName("prefix_" + e.name)
 case other => other
}

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: object TracingComposer

Provides convenient access to tracer composition, event filtering, and event transformation.

Attributes

Companion: trait
Supertypes: trait TracingComposer

class Object

trait Matchable

class Any
Self type: TracingComposer.type

@Experimental

An org.llm4s.llmconnect.LLMClient that traces each of its calls (#2275): a completion - plain or streamed - as one TraceEvent.CompletionReceived carrying the call's provider, request model, server address, options, duration, finish reasons, usage and messages, and a failed call as one TraceEvent.ErrorOccurred naming the operation (chat), model, provider and error type. A backend that follows the OpenTelemetry GenAI conventions turns each into the call's chat <model> span; backends record message content only when told to.

val traced = TracingLLMClient(client, tracing, providerConfig)
traced.complete(conversation)   // one `chat <model>` span

A tracing failure never fails the call: the client returns what the wrapped client returned. close closes the wrapped client.

Attributes

Companion: object
Supertypes: trait LLMClient

trait AutoCloseable

class Object

trait Matchable

class Any

@Experimental

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: TracingLLMClient.type

@Stable

Selects a tracing backend.

NoOp and Console are built by core itself, and are the only modes core has a case for. Every other mode is a TracingMode.Named served by a org.llm4s.trace.spi.TracingBackend found on the classpath: Named("langfuse") by llm4s-observability, Named("opentelemetry") by llm4s-observability-otel, and any third-party backend the same way (D2, #1133). TracingMode.Langfuse and TracingMode.OpenTelemetry were removed when their backends left core.

The TRACING_MODE environment variable (llm4s.tracing.mode) is the standard way to select a mode; see Llm4sConfig.tracing.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all
Known subtypes: object Console

class Named

object NoOp

Defines built-in and named tracing modes and provides case-insensitive parsing of mode strings.

Attributes

Companion: trait
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: TracingMode.type

org.llm4s.trace

Members list

Packages

Type members

Classlikes

Value parameters

Attributes

Attributes

Attributes

Attributes

Attributes

Value parameters

Attributes

Value parameters

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Type parameters

Value parameters

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes