Guardrails

Validate agent inputs and outputs for safety, quality, and compliance.

Table of contents

  1. Overview
  2. Built-in Guardrails
    1. Simple Validators
    2. LLM-as-Judge Guardrails
    3. RAG-Specific Guardrails
  3. Basic Usage
    1. Input Validation
    2. Output Validation
    3. Combined Input/Output
  4. LLM-as-Judge Examples
    1. Safety Check
    2. Factuality Check
    3. Tone Validation
  5. Composite Guardrails
    1. All Must Pass (AND)
    2. Any Must Pass (OR)
    3. Sequential (Short-Circuit)
    4. Using a composite in an agent
  6. Custom Guardrails
    1. Basic Custom Guardrail
    2. Custom Output Guardrail
    3. Custom LLM-Based Guardrail
  7. RAG Guardrails
    1. Basic RAG Setup
    2. Preset Configurations
    3. Individual RAG Guardrails
  8. PII Detection and Masking
    1. Detect PII
    2. Mask PII in Output
    3. Supported PII Types
  9. Prompt Injection Protection
  10. Error Handling
    1. Blocked Turns
    2. Validation Mode
  11. Best Practices
    1. 1. Layer Your Guardrails
    2. 2. Use Appropriate Guardrails for Each Use Case
    3. 3. Test Your Guardrails
  12. Examples
  13. Next Steps

Overview

Guardrails are validation functions that run before (input) and after (output) agent processing. They help ensure:

  • Safety - Block harmful or inappropriate content
  • Quality - Enforce response standards
  • Compliance - Meet business requirements
  • Security - Detect prompt injection and PII

Guardrails belong to the agent, not to a run: give them to the builder in a GuardrailMiddleware.

1
2
3
4
5
6
7
8
9
10
11
12

import org.llm4s.agent.Agent
import org.llm4s.agent.graph.middleware.GuardrailMiddleware

val agent = Agent.builder("assistant", client)
  .withTools(tools)
  .withMiddleware(
    GuardrailMiddleware(
      input = Seq(...),   // Validate each query before the LLM call
      output = Seq(...)   // Validate each final answer
    )
  )
  .build()

A guardrail that refuses does not make run return a Left: the turn ends with AgentStatus.Blocked(guardrail, reason) - the first failing guardrail’s name, and every failure’s error - and the thread stays usable, so the next run or continueConversation on it works. The blocked turn is never kept: an input block stores nothing for that turn, and an output block removes the whole turn - the query, any tool calls and results, and the answer - so result.messages is the conversation as it was before the turn and the model never sees the blocked answer again. usage still counts the blocked turn’s model calls. A guardrail that transforms (PIIMasker) changes the stored query or answer instead.

Underneath, a block is the graph runtime’s Block: the run ends as a finished failure (RunResult.Failed with a GuardrailBlocked error, the thread’s checkpoint Failed), which recover has nothing to continue and start accepts like a completed thread. A Left from another middleware’s beforeAgent or afterAgent blocks the same way, but is returned as that Left.

A list of guardrails runs completely: each guardrail, in order, on the value the previous one returned, even after one has failed. That is why the block carries every failure’s error, and why a guardrail that costs a call - an LLM judge - runs on every turn that reaches its list. To stop at the first failure, use CompositeGuardrail.sequential (adapted with InputGuardrail.from, see Using a composite in an agent); the block then names the composite as the guardrail. Running the whole list is the settled default (#1521): it reports every reason a value was refused in one block, and an early stop is one composite away, while the opposite default would hide the other failures with no way to ask for them.

A block sends one durable agent.guardrail_blocked event, naming the first failing guardrail and its phase (Input or Output), and no event comes from a guardrail that passed or that ran after the first failure. The run then ends with the kernel’s RunFailed; RunCompleted is never sent for a blocked turn. An input block comes before any model call. An output block comes after the turn’s ModelCallCompleted events, since the answer it refuses has been produced. See Streaming for the event stream.

Guardrails on the root agent guard its whole handoff family: they apply to every turn’s query and every final answer, whichever agent is active after a handoff. See Guardrails and Middleware Across Handoffs.

Built-in Guardrails

Simple Validators

These guardrails run locally without LLM calls:

Guardrail Purpose Example
LengthCheck Enforce min/max length new LengthCheck(1, 10000)
ProfanityFilter Block profane content new ProfanityFilter()
JSONValidator Ensure valid JSON output new JSONValidator()
RegexValidator Pattern matching new RegexValidator("\\d{3}-\\d{4}")
ToneValidator Simple tone detection new ToneValidator(Tone.Professional)
PIIDetector Detect PII (email, SSN, etc.) new PIIDetector()
PIIMasker Mask detected PII new PIIMasker()
PromptInjectionDetector Detect injection attempts new PromptInjectionDetector()

LLM-as-Judge Guardrails

These use an LLM to evaluate subjective qualities:

Guardrail Purpose Example
LLMSafetyGuardrail Content safety check new LLMSafetyGuardrail(client)
LLMFactualityGuardrail Verify factual accuracy new LLMFactualityGuardrail(client)
LLMQualityGuardrail Assess response quality new LLMQualityGuardrail(client)
LLMToneGuardrail Validate tone compliance new LLMToneGuardrail(client, "professional")

Each judge scores the content and passes it when the score reaches its threshold. The threshold must be between 0.0 and 1.0: for any other value, or NaN, validate returns a ValidationError on the field threshold without calling the judge.

RAG-Specific Guardrails

For retrieval-augmented generation:

Guardrail Purpose
GroundingGuardrail Verify answers are grounded in retrieved context
ContextRelevanceGuardrail Check context relevance to query
SourceAttributionGuardrail Ensure sources are cited
TopicBoundaryGuardrail Prevent off-topic responses

Basic Usage

Input Validation 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

import org.llm4s.agent.{ Agent, AgentStatus }
import org.llm4s.agent.graph.middleware.GuardrailMiddleware
import org.llm4s.agent.guardrails.builtin._

val result = for {
  agent <- Agent.builder("assistant", client)
    .withTools(tools)
    .withMiddleware(
      GuardrailMiddleware(
        input = Seq(
          new LengthCheck(min = 1, max = 10000),
          new ProfanityFilter(),
          new PromptInjectionDetector()
        ),
        output = Nil
      )
    )
    .build()
  result <- agent.run(userInput)
} yield result

result match {
  case Right(r) =>
    r.status match {
      case AgentStatus.Blocked(guardrail, reason) => println(s"Input rejected by $guardrail: $reason")
      case AgentStatus.Completed(answer)          => println(answer)
      case other                                  => println(s"Run ended: $other")
    }
  case Left(error) =>
    println(s"Run failed: ${error.message}")
}

Output Validation 

1
2
3
4
5
6

val agent = Agent.builder("assistant", client)
  .withTools(tools)
  .withMiddleware(GuardrailMiddleware(input = Nil, output = Seq(new JSONValidator(), new PIIMasker())))
  .build()

val result = agent.flatMap(_.run("Generate a JSON response with user data"))

Combined Input/Output 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

val agent = Agent.builder("assistant", client)
  .withTools(tools)
  .withMiddleware(
    GuardrailMiddleware(
      input = Seq(
        new LengthCheck(1, 5000),
        new ProfanityFilter()
      ),
      output = Seq(
        new LLMSafetyGuardrail(client),
        new ToneValidator(Set(Tone.Professional))
      )
    )
  )
  .build()

val result = agent.flatMap(_.run(userInput))

LLM-as-Judge Examples

Safety Check 

1
2
3
4
5
6
7
8
9

import org.llm4s.agent.guardrails.builtin.LLMSafetyGuardrail

val safetyGuardrail = new LLMSafetyGuardrail(client)

val agent = Agent.builder("storyteller", client)
  .withMiddleware(GuardrailMiddleware(input = Nil, output = Seq(safetyGuardrail)))
  .build()

agent.flatMap(_.run("Write a story"))

Factuality Check

Verify responses are grounded in source documents:

1
2
3
4
5
6
7
8
9
10
11
12

import org.llm4s.agent.guardrails.builtin.LLMFactualityGuardrail

val factualityGuardrail = LLMFactualityGuardrail.strict(
  client = client,
  referenceContext = "The capital of France is Paris. Paris has a population of 2.1 million."
)

val agent = Agent.builder("geography", client)
  .withMiddleware(GuardrailMiddleware(input = Nil, output = Seq(factualityGuardrail)))
  .build()

agent.flatMap(_.run("What is the capital of France?"))

Tone Validation 

1
2
3
4
5
6
7
8
9
10
11
12

import org.llm4s.agent.guardrails.builtin.LLMToneGuardrail

val toneGuardrail = new LLMToneGuardrail(
  llmClient = client,
  allowedTones = Set("professional", "helpful")
)

val agent = Agent.builder("support", client)
  .withMiddleware(GuardrailMiddleware(input = Nil, output = Seq(toneGuardrail)))
  .build()

agent.flatMap(_.run("Help with customer complaint"))

Composite Guardrails

Combine multiple guardrails with different strategies:

All Must Pass (AND) 

1
2
3
4
5
6
7
8
9

import org.llm4s.agent.guardrails.CompositeGuardrail

val strictValidation = CompositeGuardrail.all(Seq(
  new LengthCheck(1, 5000),
  new ProfanityFilter(),
  new PIIDetector()
))

// All guardrails must pass for input to be accepted

Any Must Pass (OR) 

1
2
3
4
5
6

val flexibleValidation = CompositeGuardrail.any(Seq(
  new RegexValidator("^[A-Z].*"),  // Starts with capital
  new RegexValidator("^\\d.*")     // Starts with digit
))

// At least one guardrail must pass

Sequential (Short-Circuit) 

1
2
3
4
5
6
7

val sequentialValidation = CompositeGuardrail.sequential(Seq(
  new LengthCheck(1, 10000),  // Check length first
  new ProfanityFilter(),      // Then profanity
  new PIIDetector()           // Then PII
))

// Stops at first failure, more efficient

Using a composite in an agent

A composite is a Guardrail[String], while GuardrailMiddleware takes InputGuardrails and OutputGuardrails, so adapt it with InputGuardrail.from or OutputGuardrail.from. A cast does not work: it throws a ClassCastException.

1
2
3
4
5
6
7

import org.llm4s.agent.graph.middleware.GuardrailMiddleware
import org.llm4s.agent.guardrails.InputGuardrail

val agent = Agent
  .builder("assistant", client)
  .withMiddleware(GuardrailMiddleware(Seq(InputGuardrail.from(sequentialValidation)), Seq.empty))
  .build()

from keeps the guardrail’s name and description and passes the hook’s context on, so an LLM judge inside the composite has its model calls counted in the thread’s usage (a hand-written wrapper that overrides only validate(value) loses that). An InputGuardrail passed to InputGuardrail.from comes back unchanged.

all runs every guardrail and reports every failure; any stops at the first guardrail that passes, so those after it do not run; sequential stops at the first failure.

Custom Guardrails

Basic Custom Guardrail 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import org.llm4s.agent.guardrails.InputGuardrail
import org.llm4s.error.ValidationError
import org.llm4s.types.Result

class KeywordRequirementGuardrail(requiredKeywords: Set[String]) extends InputGuardrail {
  val name: String = "keyword-requirement"

  def validate(value: String): Result[String] = {
    val found = requiredKeywords.filter(kw => value.toLowerCase.contains(kw.toLowerCase))
    if (found.nonEmpty) {
      Right(value)
    } else {
      Left(ValidationError.invalid(
        "input",
        s"Input must contain at least one of: ${requiredKeywords.mkString(", ")}"
      ))
    }
  }
}

// Usage
val guardrail = new KeywordRequirementGuardrail(Set("scala", "java", "kotlin"))

Custom Output Guardrail 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

import org.llm4s.agent.guardrails.OutputGuardrail
import org.llm4s.error.ValidationError
import org.llm4s.types.Result

class MaxSentenceCountGuardrail(maxSentences: Int) extends OutputGuardrail {
  val name: String = "max-sentence-count"

  def validate(value: String): Result[String] = {
    val sentenceCount = value.split("[.!?]+").length
    if (sentenceCount <= maxSentences) {
      Right(value)
    } else {
      Left(ValidationError.invalid(
        "output",
        s"Response has $sentenceCount sentences, max allowed is $maxSentences"
      ))
    }
  }
}

Custom LLM-Based Guardrail 

1
2
3
4
5
6
7
8
9

import org.llm4s.agent.guardrails.LLMGuardrail
import org.llm4s.llmconnect.LLMClient

class CustomLLMGuardrail(client: LLMClient) extends LLMGuardrail {
  val name: String             = "custom-llm-check"
  val llmClient: LLMClient     = client
  val evaluationPrompt: String = "Rate if this content is appropriate for a children's website (0-1)"
  override val threshold       = 0.8
}

RAG Guardrails

Basic RAG Setup 

1
2
3
4
5
6
7
8
9
10

import org.llm4s.agent.guardrails.rag._

val ragGuardrails = RAGGuardrails.standard(client)

val agent = Agent.builder("rag", client)
  .withTools(tools)
  .withMiddleware(GuardrailMiddleware(ragGuardrails.inputGuardrails, ragGuardrails.outputGuardrails))
  .build()

agent.flatMap(_.run(question))

Preset Configurations 

1
2
3
4
5
6
7
8
9
10
11

// Minimal - basic safety only
val minimal = RAGGuardrails.minimal()

// Standard - balanced for production
val standard = RAGGuardrails.standard(client)

// Strict - maximum safety
val strict = RAGGuardrails.strict(client)

// Monitoring - warn mode, doesn't block
val monitoring = RAGGuardrails.monitoring(client)

Individual RAG Guardrails 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

// Verify answer is grounded in retrieved context
val grounding = new GroundingGuardrail(
  client = client,
  retrievedContext = retrievedDocuments
)

// Check retrieved context is relevant to query
val relevance = new ContextRelevanceGuardrail(
  client = client,
  query = userQuery
)

// Ensure sources are properly cited
val attribution = new SourceAttributionGuardrail(
  client = client,
  sourceDocuments = sources
)

// Prevent off-topic responses
val topicBoundary = new TopicBoundaryGuardrail(
  client = client,
  allowedTopics = Set("programming", "software engineering")
)

PII Detection and Masking

Detect PII 

1
2
3
4
5
6
7
8
9
10
11

import org.llm4s.agent.guardrails.builtin.PIIDetector

val piiDetector = new PIIDetector()

// Detects: emails, SSNs, credit cards, phone numbers, etc.
val agent = Agent.builder("assistant", client)
  .withTools(tools)
  .withMiddleware(GuardrailMiddleware(input = Seq(piiDetector), output = Nil))
  .build()

agent.flatMap(_.run(userInput))

Mask PII in Output 

1
2
3
4
5
6
7
8
9
10
11

import org.llm4s.agent.guardrails.builtin.PIIMasker

val piiMasker = new PIIMasker()

// Replaces PII with [REDACTED_EMAIL], [REDACTED_SSN], etc. - the stored answer is the masked one
val agent = Agent.builder("assistant", client)
  .withTools(tools)
  .withMiddleware(GuardrailMiddleware(input = Nil, output = Seq(piiMasker)))
  .build()

agent.flatMap(_.run("Get user details"))

Supported PII Types

Type Pattern Masked As
Email user@domain.com [REDACTED_EMAIL]
SSN 123-45-6789 [REDACTED_SSN]
Credit Card 4111-1111-1111-1111, 3782 822463 10005 (15-digit American Express) [REDACTED_CARD]
Phone (555) 123-4567, +44 20 7946 0958 [REDACTED_PHONE]
IP Address 192.168.1.1 [REDACTED_IP]

A phone number is a US number (with an optional +1 or 1 prefix) or an international number that starts with + and has 8 to 15 digits, with spaces, dashes, dots or parentheses between them. Digits without a + are not treated as an international number. A card number is recognised by its shape (a Visa, MasterCard, Amex or Discover prefix and the right number of digits); the check digit is not validated. When two types match overlapping text (a plain card number is also account-shaped, and a phone number can run into the email address after it), the overlapping text is masked once, under the type whose match starts first. Detection does not merge them: detect returns one match per type, so summarize counts that text once for each type (123456789 gives SSN -> 1, Passport -> 1 with PIIType.all), and PIIDetector’s block message lists both types. A finding is one match of one type, so the sum of the counts is how often the patterns fired, not how many stretches of PII the text holds.

Prompt Injection Protection 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

import org.llm4s.agent.guardrails.builtin.PromptInjectionDetector

val injectionDetector = new PromptInjectionDetector()

val agent = Agent.builder("assistant", client)
  .withTools(tools)
  .withMiddleware(GuardrailMiddleware(input = Seq(injectionDetector), output = Nil))
  .build()

agent.flatMap(_.run(userInput))

// Detects patterns like:
// - "Ignore previous instructions..."
// - "System: You are now..."
// - "---\nNew instructions:"
// - Base64 encoded payloads

Error Handling

Blocked Turns

A block is an outcome, not an error: run returns Right with AgentStatus.Blocked, the blocked turn absent from result.messages, and the thread can take another turn.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

agent.run(query) match {
  case Right(result) =>
    result.status match {
      case AgentStatus.Blocked(guardrail, reason) =>
        println(s"Guardrail '$guardrail' blocked the turn: $reason")
        // Take appropriate action (rephrase, notify user, log); agent.continueConversation(result, ...) still works
      case AgentStatus.Completed(answer) =>
        println(answer)
      case other =>
        println(s"Run ended: $other")
    }

  case Left(error) =>
    // a provider error, a tool's failure, a blank query, or another middleware's beforeAgent/afterAgent Left
    println(s"Run failed: ${error.message}")
}

Validation Mode

Control how guardrail results are handled:

1
2
3
4
5
6
7
8
9
10

import org.llm4s.agent.guardrails.ValidationMode

// Block on failure (default)
val blocking = ValidationMode.Block

// Warn only, continue processing
val warn = ValidationMode.Warn

// Log and continue
val log = ValidationMode.Log

Best Practices

1. Layer Your Guardrails 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

// Fast, local checks first
val input = Seq(
  new LengthCheck(1, 10000),        // Cheapest first
  new ProfanityFilter(),             // Still fast
  new PromptInjectionDetector(),     // Pattern matching
  new PIIDetector()                  // More complex but local
)

// LLM checks for output only (expensive)
val output = Seq(
  new JSONValidator(),               // Fast, local
  new LLMSafetyGuardrail(client)    // Expensive, last
)

val agent = Agent.builder("assistant", client)
  .withMiddleware(GuardrailMiddleware(input, output))
  .build()

2. Use Appropriate Guardrails for Each Use Case

Use Case Recommended Guardrails
Customer support ProfanityFilter, ToneValidator, LLMSafetyGuardrail
Code generation LengthCheck, JSONValidator (for structured output)
RAG application GroundingGuardrail, SourceAttributionGuardrail
Content moderation PIIDetector, ProfanityFilter, LLMSafetyGuardrail
Form processing RegexValidator, LengthCheck

3. Test Your Guardrails 

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers

class GuardrailSpec extends AnyFlatSpec with Matchers {
  "LengthCheck" should "reject empty input" in {
    val guardrail = new LengthCheck(min = 1, max = 100)
    guardrail.validate("") shouldBe a[Left[_, _]]
  }

  it should "accept valid input" in {
    val guardrail = new LengthCheck(min = 1, max = 100)
    guardrail.validate("Hello") shouldBe Right("Hello")
  }
}

Examples

Example Description
BasicInputValidationExample Length and profanity checks
JSONOutputValidationExample JSON output validation
LLMJudgeGuardrailExample LLM-as-Judge patterns
CompositeGuardrailExample Combining guardrails
CustomGuardrailExample Building custom validators
FactualityGuardrailExample RAG factuality checking

Browse all examples →

Next Steps