> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Guardrails

> Create LLM-as-a-Judge and Python guardrails in the AI Gateway to validate requests and responses and block non-compliant generations.

**Guardrails** in the **AI Gateway** are LLM-as-a-Judge and Python evaluators that validate requests and responses passing through the gateway. Once created, they can be attached to [Guardrail Rules](/docs/ai-gateway/configuration/guardrail-rules.md) to block non-compliant generations before they reach the caller.

> [card] [LLM Guardrail](#llm-guardrail)
>
> Use a model to judge outputs against any criteria defined in a prompt.

> [card] [Python Guardrail](#python-guardrail)
>
> Write custom Python code for full flexibility: regex checks, length validation, HTTP calls, or JSON schema validation.

## Execution behavior

Set each Guardrail to **Input**, **Output**, or **Both** when adding it to a [Guardrail Rule](/docs/ai-gateway/configuration/guardrail-rules.md).

Input Guardrails, including Secret Detection, run synchronously on the complete input before the **AI Gateway** sends the request to the model. Streaming does not change this behavior.

Output Guardrails run after generation and before the response for non-streaming requests.

> [!WARNING]
>
> Output Guardrails do not run on streaming responses. A secret can span multiple chunks, so a chunk-by-chunk check cannot inspect the complete output. Use a non-streaming request when output enforcement is required.

## Tool calls

A tool call served through an [**MCP Gateway**](/docs/ai-gateway/mcp-portal/mcp-gateways.md#guardrails) is checked by the **Guardrails** attached to that gateway, on the tool arguments, the tool result, or both, and by the gateway's [plugins](/docs/ai-gateway/mcp-portal/mcp-gateways.md#plugins). An Agent's tool call is also governed by tool approval on the tool.

## Timeout

On the Responses endpoint, each entry in a request's `guardrails[]` array can set its own `timeout` in milliseconds. It bounds that one entry's execution and is independent of the request-level `call_timeout`, which bounds the model call only. See [Request timeouts](/docs/ai-gateway/features/timeouts.md).

An entry runs as a blocking Guardrail when `is_guardrail` is `true`, and as a monitoring Evaluator otherwise.

| Field | Type | Required | Description |
| - | - | - | - |
| `id` | string | Yes | Key of a published Guardrail or Evaluator. |
| `execute_on` | string | Yes | `input`, `output`, or `both`. An entry with an unrecognized value is skipped without an error. |
| `is_guardrail` | boolean | No | `true` runs the check as a blocking Guardrail, `false` as a monitoring Evaluator. Defaults to `false`. |
| `sample_rate` | number | No | Fraction in `[0,1]` of matching requests to run on. Ignored for a Guardrail, which always runs; a monitoring Evaluator defaults to `0`. |
| `timeout` | integer | No | Execution limit in milliseconds. Defaults to the evaluator's timeout, or `60000` (60 seconds) if it has none; values above `600000` (10 minutes) are capped at 10 minutes. |
| `options` | object | No | Guardrail-specific configuration, for example `language` and `threshold` for PII Detection. |

```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "model": "openai/gpt-5.6-sol",
  "timeout": { "call_timeout": 30000 },
  "guardrails": [
    { "id": "orq_pii_detection", "execute_on": "input", "is_guardrail": true, "timeout": 5000 },
    { "id": "response-quality", "execute_on": "output", "is_guardrail": true, "timeout": 15000 }
  ]
}
```

On the Responses endpoint and the OpenAI-compatible chat completions endpoint, a Guardrail that exceeds its `timeout` fails the request with HTTP `408` and `error.code` `guardrail_timeout`, alongside the other Guardrail failures. See Guardrail Error Response.

The OpenAI-compatible chat completions endpoint accepts the same `timeout` on each entry of its `guardrails[]` array, but rejects values outside `1000`-`600000` milliseconds with `400` rather than using them verbatim.

> [!NOTE]
>
> A timeout can also be set on the **Evaluator**, under **Guardrail configuration**, from 1 to 120 seconds. It applies wherever the evaluator runs, including [Guardrail Rules](/docs/ai-gateway/configuration/guardrail-rules.md). If a request also sets `timeout`, the lower value applies.

## LLM Guardrail

LLM Guardrails use a model to judge requests or responses against criteria defined in a prompt.

Open **Evaluators**, click <kbd>+ Evaluator</kbd>, and select **LLM-as-a-judge**. Fill in the following fields:

| Field | Description |
| - | - |
| **Key** | Unique identifier for the guardrail |
| **Description** | Optional context |
| **Model** | The model used as judge. Any model enabled in the [AI Gateway](/docs/ai-gateway/using-the-router.md) is available |
| **Guardrail configuration** | The pass rule the evaluator applies when it runs as a guardrail, for example **Pass If Greater Or Equal Than** |

### Configure Prompt

Reference the evaluated run with the following **string** variables. Type `{{` in the prompt editor to pick one from the full list.

| Variable | Description |
| - | - |
| `{{input.user_query}}` | The last message sent to the model |
| `{{input.all_messages}}` | The full conversation, including the graded turn |
| `{{input.system_instructions}}` | The system prompt used for the run |
| `{{input.retrievals}}` | Knowledge Base retrievals |
| `{{input.expected_output}}` | The reference used to compare output |
| `{{output.response}}` | The output response generated by the evaluated model |
| `{{output.tools_called}}` | The tool calls made during the run, with their results |

See Structured variable shapes for the fields available when indexing messages and tool calls.

> [accordion] Legacy log variables
>
> The `log.*` variables remain supported, so existing Guardrails keep working. Prefer the variables above for new ones.
>
> | Variable | Description |
> | - | - |
> | `{{log.input}}` | Same as `{{input.user_query}}` |
> | `{{log.output}}` | Same as `{{output.response}}` |
> | `{{log.retrievals}}` | Same as `{{input.retrievals}}` |
> | `{{log.reference}}` | Same as `{{input.expected_output}}` |
> | `{{log.tool_calls}}` | Same as `{{output.tools_called}}` |
> | `{{log.messages}}` | The conversation **without** the graded turn. `{{input.all_messages}}` keeps it. |

### Output and Guardrail Configuration

Select the output type and set the pass condition. The **Guardrail configuration** panel is visible directly in the settings.

> [tab] Boolean
>
> The model returns a **True** or **False** response. Use for binary pass/fail checks.
>
> **Pass condition**: Select **True** or **False**. The guardrail passes when the model returns the selected value.

> [tab] Number
>
> The model returns a numeric score. Use any scale that fits the use case (e.g. 1-5, 0-100).
>
> **Pass condition**: Enter a threshold in **Pass if greater or equal than**. The guardrail passes when the score meets or exceeds the threshold.

> [tab] Categorical
>
> The model classifies the output into one of the predefined labels.
>
> When **Categorical** is selected, a label editor appears. Add one label per row: enter a **Value** (the exact string the model must return) and an optional **Description** to guide the model.
>
> **Pass condition**: Select one or more values in **Pass if output is one of**. The guardrail passes when the model's output matches any selected label.

> [tab] String
>
> The model returns a free-form string response. Not available as a guardrail pass condition.

### Testing

The **Playground** panel provides an **Editor** for testing. Fill the payload manually:

```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "messages": [],
  "input": "",
  "retrievals": [],
  "output": "",
  "reference": ""
}
```

Click **Run test** to execute the guardrail. The result appears in the **Response** field.

> [!NOTE]
>
> The Dataset tab is not available for Guardrails.

## Python Guardrail

Python Guardrails accept custom **Python code** for full evaluation flexibility. The UI has three panels: **Settings**, **Code**, and **Playground**.

> [!NOTE]
>
> Python code is limited to 1 MB (1,048,576 bytes) per guardrail: roughly 1 million characters, or about 20,000 lines of typical Python. Larger code returns a `Code exceeds maximum size` error and does not run.

Open **Evaluators**, click <kbd>+ Evaluator</kbd>, and select **Python**.

The evaluation function receives a `log` object with the following fields:

* `log["input"]` `<str>`: the last message sent to generate the output
* `log["output"]` `<str>`: the generated response from the model
* `log["reference"]` `<str>`: the reference used to compare the output
* `log["messages"]` `list<str>`: all previous messages sent to the model
* `log["retrievals"]` `list<str>`: all Knowledge Base retrievals

The function must return a **Boolean** or **Number**:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def evaluate(log):
    return True
```

To explain a result, return a dict instead of a bare value: `{"value": <value>, "explanation": <str>}`. `explanation` is optional, must be a string, and appears as `reason` in the **Output** panel of the evaluator span in [Traces](/docs/docs/ai-studio/observability/traces.md). It does not change whether the guardrail passes. A dict carrying any other key is returned as-is, with no explanation.

> [!NOTE]
>
> Define helper functions anywhere in the code. The runner calls `evaluate(log)` by name, so keep that function name.

### Environment and Libraries

The Python Guardrail runs in **Python 3.13** with the following preloaded libraries:

```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
numpy==2.5.0
nltk==3.10.3
requests
pydantic
json
re
```

NLTK's `punkt_tab` tokenizer data is preloaded, so `nltk.word_tokenize()` works without downloading additional data.

### Guardrail Configuration

Set the pass condition based on the return type:

* **Boolean**: select **True** or **False**. The guardrail passes when the function returns the selected value.
* **Number**: enter a score threshold. The guardrail passes when the return value is greater than or equal to the threshold.

### Testing

The **Playground** panel provides an **Editor** for testing. Fill the payload manually and click **Run test** to execute the guardrail.

> [!NOTE]
>
> The Dataset tab is not available for Guardrails.

## Versions

Click <kbd>Publish</kbd> to save changes. Choose a version bump:

* **Patch** (e.g. `v1.0.0` → `v1.0.1`): small fixes, no behavior change
* **Minor** (e.g. `v1.0.0` → `v1.1.0`): new functionality, backwards compatible
* **Major** (e.g. `v1.0.0` → `v2.0.0`): breaking change or significant rework

The **Versions** tab shows the full history with author and publish timestamp for each version.

### Restoring a Version

Click the compare icon on any version in the **Versions** tab to open **Compare**, then click <kbd>Restore</kbd> next to an older version to load it into the current working draft.

Restore does not publish automatically: the guardrail is loaded into the draft as unpublished changes on the **Settings** tab, and <kbd>Publish</kbd> still needs to be clicked for it to become a real version. Earlier versions are never deleted, so restoring is always reversible.

> [!NOTE]
>
> If there are unpublished changes already, a confirmation dialog asks for confirmation before overwriting them.

## Using Guardrails

Guardrails created here are available for selection when configuring [Guardrail Rules](/docs/ai-gateway/configuration/guardrail-rules.md). To block on a decision from a policy engine or a third-party provider such as Guardrails AI or Lakera, see [External Guardrails](/docs/ai-gateway/configuration/external-guardrails.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.