Carraggon CarraRouter
One API for every model you'll ever run.
CarraRouter puts 400+ models from every major provider — plus your own fine-tunes — behind a single OpenAI-compatible endpoint, and routes each request to the cheapest provider that still clears your latency and quality bar.
Models
400+ across 60 providers
Integration
One OpenAI-compatible API
Failover
Sub-second, automatic
Overhead
Zero markup with BYOK
The best model for your workload changes every few weeks. Your architecture shouldn't have to. CarraRouter turns model choice into a runtime decision instead of a rewrite.
The gateway
Six things a router must do.
Access, routing, resilience, key custody, observability and spend control — in one layer that sits between your apps and every model you use.
01
One API, every model
Change a single string to switch model, provider or region.
CarraRouter speaks the OpenAI chat, embeddings and responses schemas, so any SDK you already use works unchanged. Frontier models, open weights, your own CarraZero and CarraWorks deployments — all addressed as vendor/model.
- —OpenAI-compatible /v1
- —Streaming + tool calling
- —Vision, audio and long context
- —Drop-in for Vercel AI SDK, LangChain, LlamaIndex
02
Smart routing
Route on price, latency, quality or policy — per request.
Declare an intent instead of a model. CarraRouter scores live provider health, throughput and token price, then picks the cheapest endpoint that meets your latency and quality floor. Pin a model when you must; let the router optimise when you can.
- —Cheapest-that-qualifies
- —Lowest-latency routing
- —Quality floor per route
- —Region and residency rules
03
Failover & uptime
Provider outages stop being your outages.
Every route carries an ordered fallback chain. Rate limits, 5xx responses, timeouts and degraded time-to-first-token trigger an instant retry on the next healthy provider — mid-stream, without the client noticing.
- —Ordered fallback chains
- —Mid-stream retry
- —Rate-limit shedding
- —Live provider health board
04
BYOK & private routes
Bring your own provider keys, keep your own contracts.
Attach your existing OpenAI, Anthropic, Google, Bedrock or Azure keys and CarraRouter bills you nothing on the tokens — you keep negotiated rates and committed spend while gaining routing, logging and governance.
- —Bring your own keys
- —Azure & Bedrock endpoints
- —VPC and on-prem targets
- —Zero token markup
05
Observability
Every call, every token, every dollar — searchable.
Traces for each request with prompt, completion, model, provider, latency percentiles, cache hits and cost. Slice by team, app, customer or route, export to your warehouse, and alert on regressions.
- —Per-request traces
- —P50/P95/P99 latency
- —Cost per team & per customer
- —Warehouse + webhook export
06
Spend governance
Budgets, keys and guardrails that finance can sign off.
Issue scoped virtual keys per app or customer with hard monthly caps, rate limits and allowed-model lists. Prompt and PII filters run before the request leaves your perimeter.
- —Virtual keys with hard caps
- —Allowed-model policy
- —PII redaction
- —SSO, audit log, SOC-ready trails
Same prompts. Lower bill.
31–54%
Typical spend reduction
Moving non-frontier traffic to the cheapest qualifying model.
up to 90%
Cache hit savings
Prompt-prefix caching across repeated system prompts.
99.99%
Effective uptime
Composite across fallback chains, versus single-provider SLAs.
< 1 hour
Integration time
One base URL and one key; the rest of your code is unchanged.
Live gateway
Run any model, right here.
This is the real CarraRouter gateway, not a mock. Pick any model in the catalog, send the same prompt, and switch providers mid-conversation — the API contract never changes.
Try a prompt, then switch models and re-run it:
Catalog
Browse every model on the router.
Search the full catalog — frontier chat, open weights like GLM-5.3 Flash, reasoning, multimodal, embeddings and Carraggon's own specialists — with live context windows and token pricing, exactly as you'd address them in the API.
21 models — Live models run instantly on the shared gateway; BYOK models route through your own provider key at zero markup.
| Model | Context | In / 1M | Out / 1M | Access |
|---|---|---|---|---|
Nova Pro 2 amazon/nova-pro-2 Bedrock-native multimodal model. | 300K | $0.80 | $3.20 | BYOK |
Claude Haiku 4.5 anthropic/claude-haiku-4.5 Low-latency Claude for high-volume routing. | 200K | $1.00 | $5.00 | BYOK |
Claude Opus 4.8 anthropic/claude-opus-4.8 Frontier reasoning, long-horizon agentic work. | 500K | $15.00 | $75.00 | BYOK |
Claude Sonnet 4.8 anthropic/claude-sonnet-4.8 Balanced frontier model for production agents. | 1M | $3.00 | $15.00 | BYOK |
CARRA27 Orchestrator carraggon/carra27-orchestrator Carraggon's agent-crew planner model. | 400K | $0.90 | $3.60 | BYOK |
CarraEmbed v2 carraggon/carraembed-v2 Domain-tuned embeddings for enterprise retrieval. | 32K | $0.02 | — | BYOK |
CarraRank v2 carraggon/carrarank-v2 Specialized reranker, ~80ms p50. | 32K | $0.05 | $0.05 | BYOK |
Command A cohere/command-a Enterprise RAG and tool-calling workhorse. | 256K | $2.50 | $10.00 | BYOK |
DeepSeek R2 deepseek/deepseek-r2 Open reasoning model with visible chain summaries. | 164K | $0.55 | $2.19 | BYOK |
DeepSeek V4 deepseek/deepseek-v4 Open-weight MoE with strong code and math. | 164K | $0.27 | $1.10 | BYOK |
Llama 4.1 Maverick meta/llama-4.1-maverick Open-weight multimodal MoE for self-hosting. | 1M | $0.22 | $0.88 | BYOK |
Llama 4.1 Scout meta/llama-4.1-scout Small, fast Llama for edge and bulk workloads. | 10M | $0.11 | $0.34 | BYOK |
Codestral 2 mistral/codestral-2 Fill-in-the-middle code completion at low latency. | 256K | $0.30 | $0.90 | BYOK |
Mistral Large 3 mistral/mistral-large-3 EU-hosted frontier model with strong tool use. | 256K | $2.00 | $6.00 | BYOK |
Kimi K2.5 moonshot/kimi-k2.5 Agentic open-weight model with long context. | 256K | $0.60 | $2.50 | BYOK |
Qwen3 Coder qwen/qwen3-coder Repo-scale coding and agentic tool use. | 1M | $0.30 | $1.20 | BYOK |
Qwen3 Max qwen/qwen3-max Alibaba's flagship multilingual model. | 262K | $1.20 | $6.00 | BYOK |
Grok 4 Fast xai/grok-4-fast Cheap, fast Grok tier for bulk traffic. | 2M | $0.20 | $0.50 | BYOK |
Grok 4.1 xai/grok-4.1 Real-time reasoning with live search grounding. | 256K | $3.00 | $15.00 | BYOK |
GLM-5.3 zai/glm-5.3 Flagship GLM reasoning and coding model. | 400K | $0.60 | $2.20 | BYOK |
GLM-5.3 Flash zai/glm-5.3-flash Opus-adjacent quality, 1M-token context, native multimodal. 18B active params. | 1M | $0.15 | $0.50 | BYOK |
Frontier chat
- —GPT-class
- —Claude-class
- —Gemini-class
- —Grok-class
Open weights
- —Llama
- —Qwen
- —Mistral
- —DeepSeek
- —Kimi
- —GLM
Reasoning
- —Long-horizon planners
- —Tool-use specialists
- —Code reasoners
Embeddings & rerank
- —CarraEmbed
- —CarraRank
- —Third-party encoders
Multimodal
- —Vision
- —Speech-to-text
- —Text-to-speech
- —Document parsing
Carraggon models
- —CARRA27 stack
- —CarraZero specialists
- —CarraWorks deployments
The API
Two lines to adopt. Zero to leave.
Point your existing OpenAI client at CarraRouter and keep every SDK, prompt and evaluation you already have. Routing rules travel in an optional body field, so standard requests keep working exactly as written.
- —No proprietary SDK
- —No prompt rewrites
- —No lock-in — point back any time
from openai import OpenAI
client = OpenAI(
base_url="https://router.carraggon.ai/v1",
api_key=CARRAGGON_API_KEY,
)
resp = client.chat.completions.create(
model="carraggon/auto", # or "vendor/model" to pin
messages=[{"role": "user", "content": "Summarise this lease."}],
extra_body={
"route": {
"optimize": "cost", # cost | latency | quality
"quality_floor": 0.86,
"fallbacks": ["vendor/model-a", "vendor/model-b"],
"max_cost_per_1k": 0.90,
}
},
)How it works
From first call to governed spend.
01
Point at the gateway
Swap your base URL to the CarraRouter endpoint and use one Carraggon key. Nothing else in your code changes.
02
Declare routes
Name a model, or name an intent — cheapest, fastest, or highest quality — with a fallback chain behind it.
03
Attach your keys
Optionally bring existing provider contracts so tokens bill to you and CarraRouter charges no margin.
04
Watch and tighten
Read the traces, set budget caps per team, and let the router keep moving traffic to the best endpoint.
Pricing
Pay for tokens, not for the router.
Build
Free
Pass-through provider pricing on credits, community rate limits, full observability on the last 7 days.
- —One workspace key
- —400+ models
- —7-day trace retention
Scale
Usage-based
Team workspaces, virtual keys with budget caps, prompt caching and priority capacity across providers.
- —Virtual keys & budgets
- —Prompt-prefix caching
- —90-day traces
- —Slack & webhook alerts
Enterprise
Contracted
BYOK at zero markup, private VPC targets, residency rules, SSO, audit export and a named architect.
- —BYOK, zero markup
- —VPC / on-prem routes
- —SSO + audit export
- —99.99% composite uptime target
Buy credits
Buy routing credit. Spend it on any model.
Credit is prepaid and never tied to a single provider — spend it across every model in the CarraRouter catalog at pass-through token pricing. No subscription, no minimum commitment.
Not just a price aggregator.
Generic routers stop at cheapest-token. CarraRouter also carries your own models, your own keys and your own governance — and connects straight into the Carraggon agent stacks already running your workflows.
| Capability | CarraRouter | Generic router |
|---|---|---|
| Single OpenAI-compatible endpoint | Yes | Yes |
| Automatic mid-stream failover | Yes | — |
| Bring your own provider keys at zero markup | Yes | — |
| Route on quality floor, not just price | Yes | — |
| Per-customer virtual keys and hard budget caps | Yes | — |
| Private VPC / on-prem model targets | Yes | — |
| Your own fine-tunes as first-class routes | Yes | — |
| Vertical agent stacks on the same gateway | Yes | — |
Bring one workload. We'll show you the bill.
Send us a week of representative prompts and we'll model what CarraRouter would have cost, at equal or better quality and latency.