Call LLM providers through Catalyst

Your app can reach a model through Catalyst in two ways:

Route	Use it when
LLM proxy	You already use a provider SDK (OpenAI, Anthropic) or an agent framework built on one, and want to keep it.
Conversation API	You want provider-neutral code, with the provider and key held in a Catalyst conversation component.

This page covers the LLM proxy. It is transparent: the request and response bodies, your provider key and your SDK stay as they are. You change two things on the client: the base URL and one header.

How it works

The proxy is a path on your project's HTTP endpoint:

<DAPR_HTTP_ENDPOINT>/v1.0/diagrid/llm/<provider-host>/<provider-path>

For each call, Catalyst:

1. Authenticates your workload by the dapr-api-token header, then removes the header so it never reaches the provider.
2. Checks that <provider-host> is a permitted provider.
3. Checks the call against your token budgets and rejects it if one is exhausted, before the provider sees it.
4. Forwards the call to https://<provider-host>/<provider-path> with your provider key unchanged.
5. Counts the tokens in the response and records the call in API logs.

DAPR_HTTP_ENDPOINT and DAPR_API_TOKEN are the same values your workload uses for every other Catalyst API. diagrid dev run sets both for you. See Connect to Catalyst for where to find them for a deployed workload.

Point your SDK at Catalyst

Set the SDK's base URL to the proxy path for your provider, and add the dapr-api-token header.

OpenAI

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['DAPR_HTTP_ENDPOINT']}/v1.0/diagrid/llm/api.openai.com/v1",
    api_key=os.environ["OPENAI_API_KEY"],
    default_headers={"dapr-api-token": os.environ["DAPR_API_TOKEN"]},
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarise this incident report."}],
)

Anthropic

import os
from anthropic import Anthropic

client = Anthropic(
    # The SDK appends /v1/messages itself, so the base URL ends at the host.
    base_url=f"{os.environ['DAPR_HTTP_ENDPOINT']}/v1.0/diagrid/llm/api.anthropic.com",
    api_key=os.environ["ANTHROPIC_API_KEY"],
    default_headers={"dapr-api-token": os.environ["DAPR_API_TOKEN"]},
)

message = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this incident report."}],
)

curl

curl "$DAPR_HTTP_ENDPOINT/v1.0/diagrid/llm/api.openai.com/v1/chat/completions" \
  -H "dapr-api-token: $DAPR_API_TOKEN" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

The same two changes work in any language and in any agent framework that lets you set the model client's base URL and headers.

Streaming works as usual. For OpenAI streaming calls, Catalyst asks the provider to include token usage in the stream so the call can be counted.

Permitted providers

On Diagrid Cloud the proxy reaches api.openai.com and api.anthropic.com. A call to any other host is rejected with 403, so the proxy can't be used as general-purpose egress. Ask Diagrid if you need another provider added.

Errors

These come from Catalyst, not the provider. Anything else is the provider's own response, passed through unchanged.

Status	Cause
400	The path has no provider host, or the host is not a bare hostname.
401	The dapr-api-token header is missing or wrong.
403	The provider host is not permitted.
413	The request body is larger than 8 MiB.
429	An enforce-mode token budget is exhausted. Most SDKs treat this as a rate limit and back off.

Limits

• HTTP only. The proxy is served on the HTTP endpoint, not gRPC.
• No component scope. A proxied call uses no Catalyst component, so a token budget scoped to a component never matches it. Scope by workload identity or model instead. See Scope across the two routes.
• Total tokens only. API logs show proxied calls under API = conversation with the model and total tokens, but no prompt and completion split.

See also

• Token budgets: how budgets cap LLM spend on both routes.
• Apply token budgets: create and monitor budgets.
• MCP on Catalyst: the same pattern for MCP tool calls.
• Agent frameworks: frameworks you can point at the proxy.