# Call LLM providers through Catalyst

Your app can reach a model through Catalyst in two ways:

| Route | Use it when |
|---|---|
| **LLM proxy** | You already use a provider SDK (OpenAI, Anthropic) or an agent framework built on one, and want to keep it. |
| **[Conversation API](https://docs.diagrid.io/concepts/dapr-apis)** | You want provider-neutral code, with the provider and key held in a Catalyst [conversation component](https://docs.diagrid.io/references/catalyst/components-reference/conversation/openai). |

This page covers the LLM proxy. It is transparent: the request and response bodies, your provider key and your SDK stay as they are. You change two things on the client: the base URL and one header.

## How it works

The proxy is a path on your project's HTTP endpoint:

```
<DAPR_HTTP_ENDPOINT>/v1.0/diagrid/llm/<provider-host>/<provider-path>
```

For each call, Catalyst:

1. Authenticates your workload by the `dapr-api-token` header, then removes the header so it never reaches the provider.
2. Checks that `<provider-host>` is a permitted provider.
3. Checks the call against your [token budgets](https://docs.diagrid.io/concepts/token-budgets) and rejects it if one is exhausted, before the provider sees it.
4. Forwards the call to `https://<provider-host>/<provider-path>` with your provider key unchanged.
5. Counts the tokens in the response and records the call in [API logs](https://docs.diagrid.io/operate/project-operations/observability/api-logs).

`DAPR_HTTP_ENDPOINT` and `DAPR_API_TOKEN` are the same values your workload uses for every other Catalyst API. `diagrid dev run` sets both for you. See [Connect to Catalyst](https://docs.diagrid.io/develop/connect) for where to find them for a deployed workload.

## Point your SDK at Catalyst

Set the SDK's base URL to the proxy path for your provider, and add the `dapr-api-token` header.

**OpenAI**

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['DAPR_HTTP_ENDPOINT']}/v1.0/diagrid/llm/api.openai.com/v1",
    api_key=os.environ["OPENAI_API_KEY"],
    default_headers={"dapr-api-token": os.environ["DAPR_API_TOKEN"]},
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarise this incident report."}],
)
```

**Anthropic**

```python
import os
from anthropic import Anthropic

client = Anthropic(
    # The SDK appends /v1/messages itself, so the base URL ends at the host.
    base_url=f"{os.environ['DAPR_HTTP_ENDPOINT']}/v1.0/diagrid/llm/api.anthropic.com",
    api_key=os.environ["ANTHROPIC_API_KEY"],
    default_headers={"dapr-api-token": os.environ["DAPR_API_TOKEN"]},
)

message = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this incident report."}],
)
```

**curl**

```bash
curl "$DAPR_HTTP_ENDPOINT/v1.0/diagrid/llm/api.openai.com/v1/chat/completions" \
  -H "dapr-api-token: $DAPR_API_TOKEN" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'
```

The same two changes work in any language and in any agent framework that lets you set the model client's base URL and headers.

Streaming works as usual. For OpenAI streaming calls, Catalyst asks the provider to include token usage in the stream so the call can be counted.

:::note Permitted providers
On Diagrid Cloud the proxy reaches `api.openai.com` and `api.anthropic.com`. A call to any other host is rejected with `403`, so the proxy can't be used as general-purpose egress. Ask Diagrid if you need another provider added.
:::

## Errors

These come from Catalyst, not the provider. Anything else is the provider's own response, passed through unchanged.

| Status | Cause |
|---|---|
| `400` | The path has no provider host, or the host is not a bare hostname. |
| `401` | The `dapr-api-token` header is missing or wrong. |
| `403` | The provider host is not permitted. |
| `413` | The request body is larger than 8 MiB. |
| `429` | An `enforce`-mode token budget is exhausted. Most SDKs treat this as a rate limit and back off. |

## Limits

- **HTTP only.** The proxy is served on the HTTP endpoint, not gRPC.
- **No component scope.** A proxied call uses no Catalyst component, so a token budget scoped to a component never matches it. Scope by workload identity or model instead. See [Scope across the two routes](https://docs.diagrid.io/concepts/token-budgets#scope-across-the-two-routes).
- **Total tokens only.** API logs show proxied calls under `API = conversation` with the model and total tokens, but no prompt and completion split.

## See also

- [Token budgets](https://docs.diagrid.io/concepts/token-budgets): how budgets cap LLM spend on both routes.
- [Apply token budgets](https://docs.diagrid.io/operate/project-operations/token-budgets): create and monitor budgets.
- [MCP on Catalyst](https://docs.diagrid.io/develop/mcp): the same pattern for MCP tool calls.
- [Agent frameworks](https://docs.diagrid.io/develop/agents): frameworks you can point at the proxy.