Provider routing

Choose who serves a request, and what that choice costs.

Most models are served by more than one provider, and they do not agree. The same Llama 3.3 70B is offered at thirteen endpoints whose prompt prices span ten times, whose context windows range from 12K to 131K, and whose speeds differ by an order of magnitude. By default we let the router pick. Send a provider block and you decide.

The providers directory lists everyone who serves a request here, and every model page shows exactly which of them serve that model, at what price and how fast. The slugs on those pages are the values you use below.

The provider block

Add provider alongside model and messages. It is passed to the router untouched.

curl https://api.tiny-router.dev/v1/chat/completions \
  -H "Authorization: Bearer $TR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/llama-3.3-70b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "provider": { "only": ["groq"] }
  }'

Fields

Field	Type	Description
`only`	string[]	Use nothing but these providers. If none of them serves the model, the request fails rather than falling back.
`ignore`	string[]	Never use these. Everything else stays eligible.
`order`	string[]	Try these first, in this order, then fall back to the rest. A preference, not a restriction.
`sort`	"price" \| "throughput" \| "latency"	Rank the eligible providers by one property instead of the router’s default balance of price and availability.
`allow_fallbacks`	boolean	Default `true`. Set it to `false` to fail rather than route to a provider you did not name.

It changes what you pay

Providers are priced independently, and the choice lands on your bill. Two identical requests for the same model, differing only in the provider:

{"provider": {"only": ["deepinfra"]}} — billed 3,105 nano-USD.
{"provider": {"only": ["groq"]}} — billed 28,968 nano-USD, roughly nine times more, for a faster answer.

Pinning can cost more than the catalog quotes

The price shown on a model page is that model’s headline figure. A model page also lists what each provider charges, and pinning an expensive one means you pay that instead. You are always charged what the request actually cost, plus our margin — never the headline — so check the provider table before pinning.

Knowing which provider served you

Every response carries a provider field naming who answered, whether or not you asked for one.

{
  "id": "gen-...",
  "model": "meta-llama/llama-3.3-70b-instruct",
  "provider": "Groq",
  "choices": [ ... ]
}

When nothing matches

If your constraints leave no eligible provider, the request fails with 404 and the router names the providers that do serve the model, so the fix is usually visible in the error itself.

{
  "error": {
    "code": 404,
    "message": "No allowed providers are available for the selected model.
Providers serving meta-llama/llama-3.3-70b-instruct: deepinfra, nebius,
novita, akashml, parasail, crusoe, cloudflare, sambanova, groq, ..."
  }
}

A pin is a single point of failure

only with one provider, or allow_fallbacks: false, turns that provider’s outage into your outage. The routing default exists because it survives one provider going down; narrow it deliberately, and prefer order when a preference is all you need.

Direct and bring-your-own-key routes

provider steers the router. On a direct provider route, where we call OpenAI, Anthropic or Google with your own key, there is no router to steer and the block does not apply.