> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Decisions

> Answer yes/no, multiple-choice and scored questions about any text or JSON state with a classification model through the AI Gateway.

See [HTTP URLs and authentication](/docs/reference/base-urls.md) and the API reference for complete request and response formats.

> [!NOTE]
>
> Decisions is in Beta, and its request and response contract matches the [TypeSafe Classify API](https://docs.typesafe.ai/api).
>
> * Answers through native decision APIs or structured outputs on supported chat models: see [Supported models](#supported-models).
> * Does not apply PII plugins, guardrails, or evaluators yet: see [Enforcement during the Beta](#enforcement-during-the-beta).
> * Requires an API key with the `classify` permission.
> * Available through the API and SDK only.

## Overview

**Decisions** answers typed questions about a piece of content in one call. Send the content as `state` and one or more named `questions` to [`POST /decisions`](/docs/reference/decisions/create-decisions) on the **AI Gateway** base URL, `https://router.toqan.ai/eu/v3/router`.

The response returns one structured answer per question, so nothing has to be parsed out of free-form text. A `choice` or `score` answer also carries the probabilities behind it and a confidence for the selected option or level.

A question pairs plain-language `instructions` with a `type` that fixes the answer shape, and `criteria` that define the rubric:

| `type` | What the answer is | Reach for it when |
| - | - | - |
| `noul` | The probability that the statement in `instructions` is true, in `[0, 1]` | Yes/no decisions, or a probability to threshold instead of a hard label |
| `choice` | One option from a named rubric, with a probability for each option | The content belongs to one of a known set: a queue, a topic, a category |
| `score` | A position on an ordered scale, with a probability for each level | The answer is a degree rather than a category: sentiment, quality, effort |

`state` is not limited to text: an object or array is serialized as JSON before classification, so the same call can judge a customer message, a tool payload, or a whole record.

Several questions can be asked at once, each with its own rubric, so a request costs one round trip instead of one call per question. See [Question types](#question-types) for the `criteria` each type expects.

The request is traced, priced, and rate-limited like other gateway traffic. See [Enforcement during the Beta](#enforcement-during-the-beta) for what applies today.

## Use cases

* **Triage and routing.** Ask `noul` whether the message reports a problem and `choice` for the topic or product, then route on the labels and escalate the rows whose confidence is low.
* **Building labelled data.** Turn a rubric into a `choice` question to label a corpus for Evaluators or Datasets, and use `probabilities` to pick the rows worth human review.
* **Scoring open feedback.** Use `score` on survey answers, support transcripts, or review text to get an ordinal reading without writing a scoring prompt.
* **Gating expensive work.** Use a `noul` question on the native model, whose output tokens are free, as a pre-check, and send only the requests that pass on to a larger model.
* **Enriching structured records.** Pass a record as `state` and ask one question per field to derive, instead of building a prompt per field.
* **Comparing models on one rubric.** Ask the same questions against `typesafe/jev-latest` and a chat model to see where they disagree. Thresholds tuned on one do not transfer, so compare answers rather than raw probabilities.

## Supported models

| Model | How it answers |
| - | - |
| `alibaba/decision-model-preview` | Native |
| `anthropic/claude-haiku-4-5` | Emulated |
| `anthropic/claude-haiku-4-5-20251001` | Emulated |
| `anthropic/claude-haiku-5-5` | Emulated |
| `aws/au.anthropic.claude-haiku-4-5-20251001-v1:0` | Emulated |
| `aws/au.anthropic.claude-haiku-5-5` | Emulated |
| `aws/eu.anthropic.claude-haiku-4-5-20251001-v1:0` | Emulated |
| `aws/eu.anthropic.claude-haiku-5-5` | Emulated |
| `aws/global.anthropic.claude-haiku-4-5-20251001-v1:0` | Emulated |
| `aws/global.anthropic.claude-haiku-5-5` | Emulated |
| `aws/jp.anthropic.claude-haiku-4-5-20251001-v1:0` | Emulated |
| `aws/jp.anthropic.claude-haiku-5-5` | Emulated |
| `aws/qwen.qwen3-32b-v1:0` | Emulated |
| `aws/us-east-1.openai.gpt-5.6-luna` | Emulated |
| `aws/us-east-2.openai.gpt-5.6-luna` | Emulated |
| `aws/us-west-2.openai.gpt-5.6-luna` | Emulated |
| `aws/us.anthropic.claude-haiku-4-5-20251001-v1:0` | Emulated |
| `aws/us.anthropic.claude-haiku-5-5` | Emulated |
| `azure/eu.gpt-5.6-luna` | Emulated |
| `azure/global.gpt-5.6-luna` | Emulated |
| `azure/microsoft-decision-1` | Native |
| `baseten/elephant-1` | Native |
| `baseten/glm-5.3-flash` | Emulated |
| `baseten/mammoth-1` | Native |
| `berget/Cloudflare/clef-flash` | Native |
| `berget/convaiinnovations/laya` | Native |
| `berget/Qwen/Qwen3.5-2B` | Native |
| `cerebras/qwen-3.8-27b` | Emulated |
| `cloudflare/clef` | Native |
| `cloudflare/clef-flash` | Native |
| `cloudflare/clef-omni` | Native |
| `fireworks/glm-5.3-flash` | Emulated |
| `fireworks/gpt-oss-120b` | Emulated |
| `google-ai/gemini-3.8-flash` | Emulated |
| `google-ai/gemini-4-argon` | Emulated |
| `google/claude-haiku-4-5` | Emulated |
| `google/eu.claude-haiku-5-5` | Emulated |
| `google/eu.gemini-3.8-flash` | Emulated |
| `google/eu.gemini-4-argon` | Emulated |
| `google/gemini-3.8-flash` | Emulated |
| `google/gemini-4-argon` | Emulated |
| `google/global.claude-haiku-5-5` | Emulated |
| `google/us.claude-haiku-5-5` | Emulated |
| `google/us.gemini-3.8-flash` | Emulated |
| `google/us.gemini-4-argon` | Emulated |
| `groq/openai/gpt-oss-120b` | Emulated |
| `groq/openai/gpt-oss-20b` | Emulated |
| `groq/openai/gpt-oss-safeguard-20b` | Emulated |
| `groq/qwen/qwen3.8-27b` | Emulated |
| `inception/mercury-decide` | Native |
| `inceptron/zai-org/GLM-5.3-Flash` | Emulated |
| `liquid/d1` | Native |
| `liquid/d1:free` | Native |
| `munito/blockbrain/cygnet@eu` | Native |
| `munito/perplexity-ai/pplx-decider-v1.2@eu` | Native |
| `nebius/openai/gpt-oss-120b` | Emulated |
| `nebius/zai-org/GLM-5.3-Flash` | Emulated |
| `openai/eu.gpt-5.6-luna` | Emulated |
| `openai/eu.gpt-6-luna` | Native |
| `openai/gpt-5.6-luna` | Emulated |
| `openai/gpt-6-luna` | Native |
| `orq/decider-4b` | Native |
| `perplexity/perplexity/glm-5.3-flash` | Emulated |
| `perplexity/pplx-decider-v1-27b` | Native |
| `tencent/glm-5.3-flash` | Emulated |
| `tencent/glm-5.3-flashx` | Emulated |
| `tensorix/openai/gpt-oss-120b` | Emulated |
| `tensorix/openai/gpt-oss-20b` | Emulated |
| `tensorix/z-ai/glm-5.3-flash` | Emulated |
| `typesafe/jev-latest` | Native |
| `typesafe/jev-preview` | Native |
| `wafer/GLM-5.3-Flash` | Emulated |
| `wafer/Qwen3.8-27B` | Emulated |
| `zai/glm-5.3-flash` | Emulated |
| `zai/glm-5.3-flashx` | Emulated |

This list includes active offerings and versions from the model catalog. Native models use the provider's decision API; emulated models answer through structured outputs or a forced tool call. See [Pricing and limits](#pricing-and-limits) for billing details.

Chat models return the same fields with their own probabilities, which express the model's estimate and are not calibrated, so thresholds tuned on one model do not transfer to another.

For emulated classification, reasoning is set to the minimum the model allows.

OpenAI Decisions accepts text, message arrays and function or custom-tool history in `state`. User messages support `input_text`, `input_image` and `input_json` parts. Images can use inline base64 data URLs or public HTTP(S) URLs. System and developer messages accept text; assistant messages accept output text and refusals. File IDs, audio, item references and bare content parts are unsupported. Other objects and arrays are serialized as text. OpenAI can return `{"type": "refusal"}` under an individual question key; handle refusals before reading scored fields.

Any other model returns `400`.

## Quick start

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -X POST https://router.toqan.ai/eu/v3/router/decisions \
  -H "Authorization: Bearer $TOQAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "typesafe/jev-latest",
    "state": "The package arrived two days late and the box was damaged, but support refunded me right away.",
    "questions": {
      "is_complaint": {
        "type": "noul",
        "instructions": "Does the customer report a problem with their order?",
        "criteria": {
          "true": "A problem with the order is described",
          "false": "No problem is described"
        }
      },
      "topic": {
        "type": "choice",
        "instructions": "What is the main topic of the message?",
        "criteria": {
          "shipping": "Delivery timing or packaging",
          "billing": "Charges, refunds or invoices",
          "other": null
        }
      },
      "satisfaction": {
        "type": "score",
        "instructions": "How satisfied is the customer overall?",
        "criteria": ["Unhappy", "Neutral", "Satisfied"]
      }
    }
  }'
```

## Request

| Field | Type | Required | Description |
| - | - | - | - |
| `model` | string | Yes | One of the [supported models](#supported-models). |
| `state` | string, object or array | Yes | The content to classify. Objects and ordinary arrays are serialized as JSON. Native OpenAI also accepts message and tool-history arrays with text, JSON and inline or hosted images. |
| `questions` | object | Yes | Maps each question key to its definition. The response uses the same keys in `answers`. Native OpenAI accepts 1 to 400 questions. |
| `name` | string | No | Name shown on the trace. A default name is used when omitted. |
| `metadata` | object | No | Key-value pairs attached to the trace, for filtering later. |
| `fallbacks` | array | No | Up to 10 ordered fallback models, each with a `model` ID. Every model must support classification and pass access checks. Requests with images in state or question instructions require native OpenAI fallbacks. |
| `retry` | object | No | Retry configuration for this request: `count` (0 to 5) and `on_codes`, the status codes that trigger a retry. |
| `timeout` | object | No | Set `call_timeout` to a positive number of milliseconds for each model call, including every retry and fallback. Use `2000` for two seconds. |
| `prompt_cache_key` | string | No | Native OpenAI only. Partition reuse of shared input with a key of at most 64 characters. |
| `prompt_cache_options` | object | No | Native OpenAI only. Set `mode` to `implicit` or `explicit`; `ttl` supports only `30m`. |
| `cache` | object | No | Exact-match response cache for this request: `type` must be `exact_match`, and `ttl` is the lifetime in seconds from 1 to 259200 (3 days), 3600 by default. See [Caching](#caching). |
| `identity` | object | No | Attributes the request to an end user, like the other gateway endpoints: the body identity takes precedence over `X-TOQAN-IDENTITY-ID` and the API-key owner. See [Request Metadata](/docs/ai-gateway/request-metadata.md). |

Each question has a `type` and `instructions`, which can be a string, object, or array. Native OpenAI preserves arrays of `input_text`, `input_image` and `input_json` parts, so each question can include its own images. The 128-image limit covers shared input and question instructions together. The shape of `criteria` depends on the question type.

### Question types

#### `noul`

Returns the probability that the statement in `instructions` is true for `state`. The probability is calibrated on `typesafe/jev-latest` and model-reported on chat models.

The optional `criteria` object can define what counts as `true` and `false`.

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "type": "noul",
  "instructions": "Does the customer report a problem with their order?",
  "criteria": {
    "true": "A problem with the order is described",
    "false": "No problem is described"
  }
}
```

#### `choice`

Selects one option from a rubric. The required `criteria` object maps each option name to a description. Set a description to `null` when the option name is enough for the model to interpret it.

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "type": "choice",
  "instructions": "What is the main topic of the message?",
  "criteria": {
    "shipping": "Delivery timing or packaging",
    "billing": "Charges, refunds or invoices",
    "other": null
  }
}
```

#### `score`

Returns a position on an ordered scale. Set `criteria` to an array with at least two level descriptions, ordered from lowest to highest. The answer index refers to this array.

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "type": "score",
  "instructions": "How satisfied is the customer overall?",
  "criteria": ["Unhappy", "Neutral", "Satisfied"]
}
```

## Response

Native providers return their own `confidence` and `score`. Emulated classification uses one structured-output call. OpenAI Decisions returns a probability-weighted score and preserves its own confidence. The fields below describe these provider differences.

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "model": "typesafe/jev-latest",
  "answers": {
    "is_complaint": { "type": "noul", "noul": 0.97 },
    "topic": {
      "type": "choice",
      "choice": "shipping",
      "probabilities": { "shipping": 0.79, "billing": 0.2, "other": 0.01 },
      "confidence": 0.68
    },
    "satisfaction": {
      "type": "score",
      "score": 1.41,
      "legend": { "0": "Unhappy", "1": "Neutral", "2": "Satisfied" },
      "probabilities": { "0": 0.03, "1": 0.52, "2": 0.45 },
      "confidence": 0.28
    }
  },
  "usage": {
    "input_tokens": 412,
    "output_tokens": 0,
    "input_cost": 0.000017304,
    "output_cost": 0,
    "total_cost": 0.000017304
  },
  "telemetry": {
    "trace_id": "4f2294c4bd55372a354be496871aa917",
    "span_id": "a354be496871aa91"
  }
}
```

| Field | Description |
| - | - |
| `model` | The requested ID of the model that answered, including a fallback model when one was used. |
| `answers` | One entry for each question key. Each entry repeats the question `type`. |
| `answers.<key>.noul` | `noul` only. Probability in the range `[0, 1]` that the statement holds. |
| `answers.<key>.choice` | `choice` only. The selected option name. |
| `answers.<key>.score` | `score` only. Position on the scale, not an index. On emulated chat models and OpenAI Decisions it is the weighted index: each level index multiplied by that level's probability, summed, so a three-level answer lands on a fraction such as `1.42` for the distribution above. `typesafe/jev-latest` returns its own score instead, which is why that answer shows `1.41` beside a distribution whose weighted index is `1.42`. |
| `answers.<key>.type` | `noul`, `choice`, `score`, or `refusal`. Refusals contain no scored fields. |
| `answers.<key>.legend` | `score` only. Maps each level index to its description in the `criteria` array. |
| `answers.<key>.probabilities` | `choice` and `score`. Probability for each option name or level index. |
| `answers.<key>.confidence` | `choice` and `score`. How sure the model is of the answer it selected. On emulated chat models it is the largest value in `probabilities`, so it carries no information beyond them. `typesafe/jev-latest` returns its own calibrated value instead, which can sit below the largest probability: in the example above, `0.68` against `0.79` for the `choice` answer and `0.28` against `0.52` for the `score` answer. |
| `usage.input_tokens` | Total input tokens, including cached tokens. |
| `usage.input_tokens_details` | Optional provider-reported `cached_tokens` and `cache_write_tokens`. These tokens are included in `input_tokens`. OpenAI Decisions charges cache reads and writes at the model cache rates. |
| `usage.output_tokens` | Output tokens. Free for native classify providers, billed at the model rate for emulated chat models. |
| `usage.input_cost` | Cost in USD of the input tokens. Present when billing was computed. |
| `usage.output_cost` | Cost in USD of the output tokens. `0` for native classify providers. Present when billing was computed. |
| `usage.total_cost` | Total cost in USD. This follows the same format as the Responses API `usage` fields. |
| `telemetry.trace_id` / `telemetry.span_id` | Identifiers of the trace and root span recorded for the request. The same values are returned in the `x-orq-trace-id` and `x-orq-trace-span-id` headers. |

The response includes optional `telemetry`. Read `x-orq-trace-id` and `x-orq-trace-span-id` from the response headers when older SDKs or deployments omit it. The handler sets those once the request is traced, so a request rejected before that point carries neither: authentication and authorization (`401`, and `403` for a missing `classify` permission or a model that is not enabled for the workspace or shared with the project), plan and budget limits (`429`), and request-body validation (the `400` for a malformed body, the `422` for an invalid request). Everything rejected after tracing carries them, including the `400` for a model that does not support classify. The `telemetry` object itself is optional and returned by builds newer than some deployments run, so fall back to the headers when it is absent.

Emulated chat models return the same scored fields, but the numbers do not mean the same thing: their probabilities are the model's own estimates and are not calibrated, and their `confidence` is simply the largest of them. Compare `choice` and `score` across models, not `confidence`.

See the [Create decisions](/docs/reference/decisions/create-decisions.md) API reference for the full request and response schema, including the fields accepted alongside the ones above.

## Fallbacks, retries, timeouts, caching and identity

Use these options on `/decisions`:

```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "model": "openai/gpt-6-luna",
  "state": "The customer loves the product.",
  "questions": {
    "positive": { "type": "noul", "instructions": "Is the sentiment positive?" }
  },
  "fallbacks": [{ "model": "openai/gpt-5.6-luna" }],
  "timeout": { "call_timeout": 2000 },
  "retry": { "count": 2, "on_codes": [429, 502, 503, 504] },
  "cache": { "type": "exact_match", "ttl": 3600 },
  "identity": { "id": "customer-demo", "display_name": "Sample customer" }
}
```

`retry.count` sets the retries after the first attempt, from 0 to 5. Set `0` to make exactly one call per model.

* Omitting `retry` leaves the gateway default of two retries on the default status codes.
* Retries fire for the codes in `on_codes`, with `408` always included. The gateway honors `Retry-After` and waits one second when the header is absent.
* After retries are exhausted, the gateway tries the next fallback, up to 10 per request.
* The response reports the successful model and its costs. A chat fallback pays the chat model's input and output rates.

`timeout.call_timeout` applies separately to every model call, including native and emulated models. Each retry and fallback receives a fresh timeout. If every model times out, the request returns `408`. These options also apply to `/classify`.

For inline-image requests, every fallback must support native OpenAI image classification; incompatible chains are rejected with `422` before a provider call.

An inaccessible fallback stops the remaining fallback chain. Session keys must allow every requested model under their factory policy.

### Caching

Native OpenAI prompt caching reuses matching shared `state` across requests, even when questions change. It is enabled implicitly by default for prefixes of at least 1,024 tokens. Set `prompt_cache_key` to partition reuse. Add `prompt_cache_breakpoint: {"mode": "explicit"}` to shared text or image parts to mark boundaries; set `prompt_cache_options.mode` to `explicit` to use only marked boundaries. Question instructions cannot contain breakpoints and are outside the reusable prefix. Decisions supports a 30-minute minimum lifetime; the 12-hour option applies to the Responses API.

Set `cache` to serve a repeated request from the stored response instead of calling the provider again, as on the other gateway endpoints. The cache key is the full request body together with the workspace and project, so any change to `state`, `questions`, `model`, or another body field is a different entry. A cached answer returns the stored `model`, `answers`, and `usage`; the request is still traced, but no provider call is made and no cost is added. The `X-Cache` response header reports `HIT` or `MISS`, and `telemetry` identifies the trace of the current request. A routing rule with a cache configuration applies the same cache to matching Decisions requests.

## Batch

Send large offline workloads as a Decisions batch instead of one request per decision. Set the batch `endpoint` to `/v3/router/decisions` and give each line a Decisions request as `body`. Each line supports the fields in [Request](#request), and the API key needs both the `batches` and `classify` permissions.

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{"custom_id": "ticket-1", "method": "POST", "url": "/v3/router/decisions", "body": {"model": "typesafe/jev-latest", "state": "My card was charged twice for the same order.", "questions": {"refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"}}}}
```

See [Batches](/docs/ai-gateway/features/batches.md) for the input file format, the create, poll, and download flow, the results format, and the batch limits.

## Enforcement during the Beta

Enforced on every request:

| What | Effect |
| - | - |
| Authentication | A missing or invalid key is rejected with `401`. |
| The `classify` permission | A key without it is rejected with `403`. The standard gateway key preset grants it. |
| Model access | The model must be enabled for the workspace or shared with the project, otherwise the request is rejected with `403`; it must also support classify, otherwise the request is rejected with `400`. |
| Plan rate limits and budgets | The request counts toward both, and a rejection returns `429` exactly as on other endpoints. See [Rate limits and quotas](/docs/ai-gateway/features/rate-limits.md). |
| Tracing, usage, and cost | The request is traced and billed, and the response reports `usage`; `telemetry` is optional. |
| Trace masking | Masks configured through workspace plugins apply to the stored trace. On chat models, the instructions that drive the structured answer are always scrubbed from the model span. |
| Request-level `retry` | Retries each model for matching `on_codes` and timeout errors before moving to its fallback. |
| Request-level `timeout` | Limits each model call, including retries and fallbacks, to `call_timeout` milliseconds. |
| Request-level `fallbacks` | Tries models in the supplied order when the previous model fails. Every model must support classify and pass access checks. A refusal is a successful answer and does not trigger fallback. |
| Request-level `cache` | Serves an identical request from the stored response for the configured `ttl`. The `X-Cache` header reports `HIT` or `MISS`. |
| Identity | Body `identity.id`, then `x-toqan-identity-id`, then the API-key owner identity determine trace attribution. Body identity details are published for upsert. |

Not applied during the Beta:

* **PII plugins.** `state` and `instructions` reach the provider exactly as sent, even when a redaction plugin is configured for traffic that would otherwise match.
* **Guardrails and evaluators.** They do not run, so a matching guardrail rule cannot block a Decisions request.
* **Routing rules.** A rule does not rewrite the model or add fallbacks; only its cache configuration applies.
* **Request-level `load_balancer`, `plugins`, and `guardrails`.** They are not part of this contract. Unknown body fields are ignored rather than rejected, so sending them has no effect and returns no error.

## Pricing and limits

| Model | Input | Output | Limits |
| - | - | - | - |
| `typesafe/jev-latest` | \$0.042 per 1M tokens | Free | 64k context window, and `state` plus the longest question must fit within 32k tokens |
| OpenAI Decisions | $0.10 per million uncached input tokens; $0.20 above 272K input tokens. Cache reads cost 0.1× input; writes cost 1.25× input. EU rates include a 10% premium | Free | Only `gpt-6-luna`; 1 to 400 questions, at most 128 images |
| Emulated chat models | The model's own rate | The model's own rate | The model's limits from the Model Garden. The structured answer adds a few hundred output tokens per request |

## Errors

| Status | Cause |
| - | - |
| `400` | Malformed JSON body, an invalid retry, timeout or cache configuration, an empty fallback model, more than 10 fallbacks, or a model is not a [supported model](#supported-models). |
| `408` | The model call exceeded `timeout.call_timeout` and no retry or fallback succeeded. |
| `401` | The API key is missing, invalid, or revoked. |
| `403` | The key is valid but lacks the `classify` permission, or the model is not enabled for the workspace or shared with the project. |
| `422` | The JSON is valid, but the classification structure is not. For example, a `choice` question has no `criteria`, or a `score` question has fewer than two levels, or an image request has an incompatible fallback. |
| `429` | A plan rate limit or a budget limit was reached before the provider was called. See [Rate limits and quotas](/docs/ai-gateway/features/rate-limits.md). |
| `429` / `529` | The upstream provider reached a rate limit or is overloaded. The response keeps the upstream status and `Retry-After` header. Retry after the specified delay. |
| `500` | An internal model-resolution error occurred. Internal infrastructure details are logged without being returned in the response. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.