An LLM-as-Judge guardrail that rates whether content has one of a set of allowed tones.
What it evaluates: the judge is asked to rate whether the content has one of the tones in allowedTones, considering word choice and vocabulary, sentence structure and formality, and the overall impression. It is told that 1.0 means the tone clearly matches one of the allowed tones, 0.0 that it is completely different or inappropriate, and to use intermediate values for partial matches. Tone names are free text passed to the judge as they are, so "warm" or "playful" work as well as "professional".
When to use it: when the tones you need are not covered by the fixed categories of ToneValidator, which detects them from keywords and sentence shapes without any LLM call and is free, instant and repeatable. This guardrail judges meaning instead, at the cost of the points below.
Cost and side: every validation makes one extra llmClient.complete call, and the content goes to the provider of llmClient. It is an output guardrail only. The scoring rules and the other limits are described on org.llm4s.agent.guardrails.LLMGuardrail.
Details: the tones appear in the prompt joined with , in the iteration order of the Set, which is unspecified for larger sets. An empty set is not rejected: the prompt then names no tone, and the judge has nothing to match.
Failure:Left with a org.llm4s.error.ValidationError on field output, for example LLM judge score (0.30) below threshold (0.70) for LLMToneGuardrail. It does not name the tone the judge perceived. An unreadable reply or a failing llmClient is a Left too, never a pass.
Value parameters
allowedTones
the acceptable tones, for example Set("professional", "friendly")
llmClient
the client that makes the judge call; it can be the agent's own or a separate model
threshold
the lowest score that passes (a score equal to it passes), between 0.0 and 1.0 or validate fails; default 0.7
The options of the judge call. The default is temperature 0.0, which makes the judge as repeatable as the model allows, and a maximum of 10 tokens, enough for a bare number. Override to give a judge model more room or to set other options.
The options of the judge call. The default is temperature 0.0, which makes the judge as repeatable as the model allows, and a maximum of 10 tokens, enough for a bare number. Override to give a judge model more room or to set other options.
Judges value with one LLM call and compares the score with threshold.
Judges value with one LLM call and compares the score with threshold.
Value parameters
value
the text to judge, typically the agent's final answer
Attributes
Returns
Right(value) unchanged when the score reaches the threshold; otherwise Left: a org.llm4s.error.ValidationError on field threshold when the threshold is not between 0.0 and 1.0 (no judge call is made), on field output when the score is too low, on field llm_response when the reply has no readable score, or the client's own error when the call fails
The evaluation criteria in natural language. It is sent after Evaluation criteria: and before the content, which goes between triple quotes. The fixed system message already asks for a bare number between 0 and 1; say in the criteria what 0 and 1 mean, because a reply on any other scale is refused as unreadable and fails the guardrail (see the scoring notes on the trait).
The evaluation criteria in natural language. It is sent after Evaluation criteria: and before the content, which goes between triple quotes. The fixed system message already asks for a bare number between 0 and 1; say in the criteria what 0 and 1 mean, because a reply on any other scale is refused as unreadable and fails the guardrail (see the scoring notes on the trait).
Attributes
Example
"Rate if this response is professional in tone. Return only a number between 0 and 1."
The client that makes the judge call. It can be the agent's own client or a different one; a separate, cheaper model is a common choice. Every validation makes one blocking complete call on it.
The client that makes the judge call. It can be the agent's own client or a different one; a separate, cheaper model is a common choice. Every validation makes one blocking complete call on it.
The lowest score that passes (a score equal to it passes). The default is 0.7. It must be between 0.0 and 1.0, both included: validate returns a org.llm4s.error.ValidationError on field threshold, without calling the judge, for a value outside that range (including infinities) or NaN. It is checked when validate runs, not when the guardrail is built, so the constructors and factories keep their signatures and a subclass that overrides it is checked too. A threshold of 0.0 passes every reply that parses as a score. It is not a probability or a confidence: it is a cut-off on whatever number the judge returns.
The lowest score that passes (a score equal to it passes). The default is 0.7. It must be between 0.0 and 1.0, both included: validate returns a org.llm4s.error.ValidationError on field threshold, without calling the judge, for a value outside that range (including infinities) or NaN. It is checked when validate runs, not when the guardrail is built, so the constructors and factories keep their signatures and a subclass that overrides it is checked too. A threshold of 0.0 passes every reply that parses as a score. It is not a probability or a confidence: it is a cut-off on whatever number the judge returns.