> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Annotation Queues

> Organize production traces into annotation queues in AI Studio to review LLM outputs and apply human feedback annotations in bulk.

See [HTTP URLs and authentication](/docs/reference/base-urls.md) and the API reference for complete request and response formats.

Annotation Queues organize Traces into review workflows so [Annotations](/docs/ai-studio/observability/annotations.md) can be applied in bulk in **AI Studio**. To define annotations and apply them individually, through the API and SDK, or the CLI, see [Annotations](/docs/ai-studio/observability/annotations.md).

## Create Annotation Queues

Annotation Queues help you organize and apply annotations effectively to relevant incoming Traces.

> [tab] AI Studio
>
> To create an Annotation Queue, head to **AI Studio > Annotation Queue**.
>
> Choose **Create Annotation Queue**.
>
> The following fields are configurable:
>
> * The **Name** of the queue
> * The **Description** of the Annotation Queue
> * The **Project** that owns the queue, chosen from **Select a project**. The queue shows the [**Annotations**](/docs/ai-studio/observability/annotations.md#create-annotations) defined on that project.

## Fill Annotation Queues

Once a queue exists, fill it with the Traces to review. Traces can be added automatically or manually.

> [tab] Automatically
>
> Use [Trace Automations](/docs/ai-studio/observability/automations.md) to route Traces into a queue based on configured rules. Add an **Add to Annotation Queue** action to an automation and select the target queue. As matching Traces arrive, they are added to the queue without manual effort, which keeps a steady stream of relevant Traces ready for review.

> [tab] Manually
>
> Open a Trace, select a span, and choose **Add to annotation queue** to send it to a queue one at a time. This is useful for ad hoc review of specific Traces that are worth a closer look.

## Use Annotation Queues

Open an Annotation Queue to step through its Traces one at a time in the review screen.

The screen is divided into three panels:

* **Left**: details for the selected Trace.

  * **Inputs**: the variables mapped to inputs, when configured.
  * **Metrics**: latency, cost, and token usage.
  * **Task**: the model, provider, and other configuration parameters.

  The header shows the current position, the total number of items in the queue, and how many have already been reviewed.
* **Center**: the full interaction for the selected Trace.
* **Right**: the **Annotations** panel with the [**Annotations**](/docs/ai-studio/observability/annotations.md#create-annotations) defined on the queue's project, such as a rating with categorical buttons or an open comment field. Selecting a value saves immediately and marks the Trace as reviewed.

Navigate between items with **`K`** (previous) and **`J`** (next), or use the up and down buttons at the top left.

When a data point is worth reusing, select **Add to dataset** to send the Trace to a Dataset for use in a future Experiment.

> [!NOTE]
>
> Adding a Trace to a Dataset copies annotations only when **Copy annotations** is selected. The annotation values stay on the originating Trace as well, where they remain queryable via the **Toqan Router MCP**.

Evaluator results shown in the review screen can also be corrected, not just annotated.

See [Correct an Evaluator Result](/docs/ai-studio/observability/traces.md#correct-an-evaluator-result) on the Traces page for the full flow.

## Delete or Clear Annotation Queues

Select the <kbd /> menu on a queue row and choose **Delete** to remove the queue together with its items. The same menu offers **Edit** for the queue's name, description, and project.

To empty a queue while keeping it and its configuration, clear it:

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -X DELETE "https://router.toqan.ai/eu/v2/annotation-queues/$ANNOTATION_QUEUE_ID/clear" \
  -H "Authorization: Bearer $TOQAN_API_KEY"
```

Clearing is irreversible: every item is removed from the queue and its item count resets to zero. The Annotations already applied to those Traces stay on the Traces.

See [Delete All Items](/docs/reference/annotation-queues/delete-all-items.md) and [Delete an Annotation Queue](/docs/reference/annotation-queue/delete-an-annotation-queue.md) for the request and response details.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.