> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LangGraph framework integration

> Connect LangGraph to Toqan Router's AI Gateway for complete observability, built-in reliability, and access to 500+ LLMs across 30+ providers.

> [card] [AI Gateway](#ai-gateway)
>
> Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

> [card] [Observability](#observability)
>
> Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

## AI Gateway

### Overview

LangGraph is a framework for building stateful, multi-actor AI applications with LLMs. It extends LangChain with graph-based agent orchestration, cycles, and controllability. By connecting LangGraph to Toqan Router's AI Gateway, you get production-ready agentic workflows with access to 500+ models.

### Key Benefits

Toqan Router's AI Gateway enhances your LangGraph applications with:

> [card] Complete Observability
>
> Track every agent step, tool use, and graph transition with detailed traces

> [card] Built-in Reliability
>
> Automatic fallbacks, retries, and load balancing for production resilience

> [card] Cost Optimization
>
> Real-time cost tracking and spend management across all your AI operations

> [card] Multi-Provider Access
>
> Access 500+ LLMs and 30+ providers through a single, unified integration

### Prerequisites

Before integrating LangGraph with Toqan Router, ensure you have:

* A Toqan Router account and API Key
* Python 3.8 or higher

> [!NOTE]
>
> To setup your API key, see API keys & Endpoints.

### Installation

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install langgraph langchain langchain-openai langchain-core
```

### Configuration

Configure LangGraph to use Toqan Router's AI Gateway by passing a `ChatOpenAI` instance with a custom `base_url`:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)
```

> **base\_url**: `https://router.toqan.ai/eu/v3/router`

### Basic Agent Example

Here's a complete example using `create_agent` with a tool:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

result = agent.invoke({"messages": [("user", "What's the weather in San Francisco?")]})
print(result["messages"][-1].content)
```

### Agent with Multiple Tools

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

@tool
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

agent = create_agent(
    llm,
    tools=[get_weather, add, multiply],
    system_prompt="You are a helpful assistant with access to weather and math tools.",
)

result = agent.invoke({
    "messages": [("user", "What is 15 * 4? Also check the weather in Tokyo.")]
})
print(result["messages"][-1].content)
```

### Streaming

Stream agent steps as they happen:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

llm = ChatOpenAI(
    model="openai/gpt-5.6-sol",
    api_key=os.getenv("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

agent = create_agent(llm, tools=[get_weather])

for chunk in agent.stream(
    {"messages": [("user", "What's the weather in Paris?")]},
    stream_mode="updates",
):
    print(chunk)
```

### Model Selection

With Toqan Router, you can use any supported model from 30+ providers:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
import os

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 72°F"

# Use Claude
claude_agent = create_agent(
    ChatOpenAI(
        model="anthropic/claude-sonnet-5",
        api_key=os.getenv("TOQAN_API_KEY"),
        base_url="https://router.toqan.ai/eu/v3/router",
    ),
    tools=[get_weather],
)

# Use Gemini
gemini_agent = create_agent(
    ChatOpenAI(
        model="google-ai/gemini-2.5-flash",
        api_key=os.getenv("TOQAN_API_KEY"),
        base_url="https://router.toqan.ai/eu/v3/router",
    ),
    tools=[get_weather],
)

result = claude_agent.invoke({"messages": [("user", "What's the weather in London?")]})
print(result["messages"][-1].content)
```

## Observability

Model calls through the AI Gateway appear in [Traces](/docs/ai-studio/observability/traces.md). To capture the surrounding application workflow, use OpenTelemetry with a framework instrumentor and the regional OTLP endpoint.

For Python, install `openinference-instrumentation-langchain` and follow the [OpenInference LangChain setup](https://arize-ai.github.io/openinference/python/instrumentation/openinference-instrumentation-langchain/). It supports LangChain and LangGraph. For Node.js, use [OpenInference LangChain.js](https://arize-ai.github.io/openinference/js/packages/openinference-instrumentation-langchain/).

Configure the exporter using the [Observability quickstart](/docs/ai-studio/observability/quickstart.md):

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export OTEL_EXPORTER_OTLP_ENDPOINT="https://router.toqan.ai/eu/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $TOQAN_API_KEY"
```

Use the provider API directly when exporting framework spans for the same model calls, or choose which layer to trace so one call is not recorded twice.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.