English · 简体中文

API reference

← Back to README · Documentation index

Any OpenAI-compatible client works (Anthropic / Claude clients too — see Anthropic / Claude clients). Base URL http://localhost:3001/v1, unified key from the dashboard's Keys page. An interactive OpenAPI viewer covering every proxy endpoint is served at GET /v1/docs; the spec itself lives at GET /v1/openapi.json.

• Chat completions
• Routing strategies (auto:*)
• Model listing
• Streaming
• Tool calling
• Gemini Google Search grounding
• Native Gemini API
• Ollama emulation
• Revocable URL tokens
• Vision / image input
• Document attachments
• Images, video & text-to-speech
• Fusion (multi-model synthesis)
• Response headers
• Embeddings
• Anthropic / Claude clients
• Free-tier budget API
• Backups API

Chat completions

Python

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="f3router-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",  # let the router pick; or specify e.g. "gemini-2.5-flash"
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)
print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

curl

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer f3router-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "hi"}]
  }'

Routing strategies (auto:*)

Plain auto follows your active fallback chain. Add a suffix to steer a single request instead — no dashboard changes needed:

• auto:smart — favor the highest-intelligence models
• auto:fast — favor measured speed (throughput and time-to-first-byte)
• auto:cheap — budget-leaning; currently the same blend as balanced (everything in the pool is already free)
• auto:reliable — favor recent success rate
• auto:balanced — the default blend (reliability first, speed and intelligence split the rest)

These rank every enabled model, ignoring your chain order. Common synonyms resolve too (auto:fastest, auto:speed, auto:smartest, auto:cheapest, auto:budget, …), and the whole model string is case-insensitive.

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer f3router-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto:fast",
    "messages": [{"role": "user", "content": "hi"}]
  }'

auto:<profile-name> routes through a named profile's chain instead of the active one, so different tools can use different chains through the same key:

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer f3router-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto:coding",
    "messages": [{"role": "user", "content": "Write a binary search in Rust."}]
  }'

An unknown profile name returns a clear 400 rather than silently falling back. Profiles are named fallback chains (see Features) — create and switch them from the dashboard; whichever is active is what plain auto uses.

Model listing

GET /v1/models lists the whole catalog, including models you cannot call yet, so a client can show what exists as well as what is connected. ?available=true (aliases ?connected=true, ?ready=true) narrows to models that can serve a request now.

Each catalog entry also carries execution_status, a live answer to "would a request work right now":

• ready — at least one key can actually serve it: enabled, healthy or not yet probed, not scoped away from this model, not on a rate-limit cooldown, and inside its rate and token windows. A key whose status is still unknown counts as ready, because probing is lazy and a fresh key is usable until something says otherwise.
• needsKey — the model is disabled, or no enabled key matches it at all.
• exhausted — keys match the model but every one of them is blocked at this moment, so a request would fail immediately. This is the state a burst of 429s produces, and it clears on its own when the cooldowns expire.

?execution_status=ready (also needsKey, exhausted; matched case-insensitively) narrows the catalog rows to one status, so an agent can ask for models it can call without reading the whole list. An unrecognised value filters nothing. Like ?available=, the filter applies only to catalog rows: auto, fusion and the auto:<profile> chains are router entries rather than models with keys of their own, and are always listed.

curl "http://localhost:3001/v1/models?execution_status=ready" \
  -H "Authorization: Bearer f3router-your-unified-key"

Streaming

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Stream me a haiku about SQLite."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Tool calling

Pass OpenAI-style tools and tool_choice; the assistant response round-trips back through the proxy exactly like the OpenAI API. Multi-step flows (assistant tool_calls → tool role follow-up → final answer) work across every provider the router can reach.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

# 1. Model asks for a tool call
first = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What's the weather in Karachi?"}],
    tools=tools,
    tool_choice="required",
)
call = first.choices[0].message.tool_calls[0]

# 2. You execute the tool, feed the result back
final = client.chat.completions.create(
    model="auto",
    messages=[
        {"role": "user", "content": "What's the weather in Karachi?"},
        first.choices[0].message,
        {"role": "tool", "tool_call_id": call.id, "content": '{"temp_c": 32, "cond": "sunny"}'},
    ],
    tools=tools,
)
print(final.choices[0].message.content)

Works with stream=True as well — you'll get delta.tool_calls chunks followed by a finish_reason: "tool_calls" close. Under the hood, OpenAI-compatible providers (Groq, Cerebras, Mistral, OpenRouter, GitHub Models, HuggingFace, Cloudflare, Cohere compat) get the request passed through; Gemini requests get translated into Google's functionDeclarations / functionResponse shape and the response is translated back.

Gemini Google Search grounding

Google's models can ground their answers in live Google Search results. Since the OpenAI wire format has no way to express that, request a tool named google_search and the Google provider translates it into Gemini's native grounding tool. It can be sent on its own or alongside your normal function tools.

resp = client.chat.completions.create(
    model="gemini-2.5-flash",  # pin a Google model so the request routes there
    messages=[{"role": "user", "content": "Who won the F1 race this weekend?"}],
    tools=[{"type": "function", "function": {"name": "google_search", "parameters": {}}}],
)
print(resp.choices[0].message.content)

Native Gemini API

Gemini SDKs and Gemini CLI can use the native /v1beta surface:

• GET /v1beta/models
• GET /v1beta/models/{model}
• POST /v1beta/models/{model}:generateContent
• POST /v1beta/models/{model}:streamGenerateContent (?alt=sse for Gemini CLI)
• POST /v1beta/models/{model}:countTokens

curl "http://localhost:3001/v1beta/models/gemini-2.5-flash:generateContent" \
  -H "x-goog-api-key: f3router-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{"contents":[{"role":"user","parts":[{"text":"hello"}]}]}'

Gemini model names like gemini-2.5-flash resolve through the Gemini family map (Keys → Gemini model mapping) rather than the catalog directly, so they route to Auto or whichever catalog model each family is pinned to. Catalog ids work verbatim too.

contents text, inline data, function calls/responses, system instructions, function declarations, structured JSON output, generation controls, and thinking budgets translate into the same internal chat/fallback pipeline. Bearer auth works too. Gemini's ?key= query parameter is accepted only below /v1beta; prefer a header because URL credentials leak into history and logs.

Ollama emulation

The opt-in Ollama surface implements tags, chat, generate, show, version, embed, and legacy embeddings under /api/*. Streaming is Ollama-compatible NDJSON. It defaults to off; choose open-loopback or key-required on Keys → Agents. Open-loopback checks the direct socket peer, so desktop LAN access cannot silently turn it into an unauthenticated LAN endpoint.

The dashboard also owns /api/embeddings. A request with a valid dashboard session continues to the dashboard handler; all other requests at that exact path are treated as Ollama legacy embeddings and follow the emulation policy.

Revocable URL tokens

Headerless clients can mirror models, chat completions, Responses, and Ollama-style chat/tags under /v1/t/{token}/…. These tokens are random, stored only as hashes, separately revocable, and are not the unified key. Create/revoke them on Keys → Agents.

Treat them as sensitive anyway: URLs are routinely retained by shell history, reverse proxies, browser history, and telemetry. Revocation is immediate.

Vision / image input

Send images with the standard OpenAI image_url content blocks (base64 data: URLs or http(s) URLs). When a request contains an image, the router restricts itself to vision-capable models and ignores text-only ones. Vision models are tagged with a Vision badge on the Fallback Chain page; the current set includes Gemini (2.5 / 3.x), Llama 4 Scout/Maverick (Groq, NVIDIA), GLM-4.6V Flash (Z.ai), Nemotron Nano 12B VL (OpenRouter), and GitHub's GPT-4o / GPT-4.1.

resp = client.chat.completions.create(
    model="auto",  # auto-routes to a vision model
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "data:image/png;base64,<...>"}},
        ],
    }],
)
print(resp.choices[0].message.content)

If no vision-capable model is enabled in your Fallback Chain, an image request returns a clear 422 (code: "no_vision_model") rather than silently dropping the image. (Image input on /v1/responses isn't supported yet — use /v1/chat/completions.)

Document attachments

Anthropic's document content blocks are accepted on /v1/messages when their source is already text:

{"type": "document", "title": "contract.txt",
 "source": {"type": "text", "media_type": "text/plain", "data": "PAYMENT TERMS: net 30."}}

text and content sources are inlined into the prompt, wrapped in a fence tagged with a hash of the document body so the model can tell quoted material from your instructions. The tag is derived from the content rather than randomized, so a repeated attachment keeps the prompt prefix stable and stays cacheable.

Binary sources — base64 (PDF, DOCX, XLSX…) and url — return a 400. No provider in the pool accepts them, and there is no local converter, so the honest answer is to refuse: forwarding the request would drop the attachment and produce a confident answer about a document the model never saw. The rejection happens before routing, so it spends no provider quota. Send the extracted text instead.

url sources are refused rather than fetched on your behalf; making the proxy retrieve arbitrary URLs would turn it into a request forwarder for whatever a client names.

Images, video & text-to-speech

POST /v1/images/generations, POST /v1/videos/generations, and POST /v1/audio/speech route across the providers that serve media models. Images and speech also accept custom OpenAI-compatible media endpoints; video does not. Browse and toggle them on the dashboard's Models → Image / Video / Audio tabs.

Video generation is served by two platforms — pollinations and huggingface (VIDEO_PLATFORMS). It accepts a JSON body with prompt, optional model (auto by default), and provider-dependent duration, aspect_ratio, image, seed, and audio options. A video job runs for minutes, so the whole surface is bounded to a 5-minute timeout (VIDEO_FETCH_TIMEOUT_MS); if the provider hasn't finished by then the request returns 504. A client that hangs up mid-generation is detected via the socket close event, which aborts the work so the gateway does not keep polling (and fail over to a second provider) for a response nobody is waiting for.

HuggingFace models route through the fal queue (router.huggingface.co/fal-ai/…): the request is submitted as a queued job, polled every 500 ms until COMPLETED, and the resulting video URL is fetched and streamed back. Pollinations models are requested directly. Both return the generated video as binary MP4 data with X-Provider and X-Model response headers:

curl http://localhost:3001/v1/videos/generations \
  -H "Authorization: Bearer f3router-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","prompt":"a sunrise over a quiet lake","duration":6}' \
  --output video.mp4

Fusion (multi-model synthesis)

Request the virtual fusion model and the router fans your prompt out to a panel of diverse free models in parallel, then a judge model synthesizes one answer from the drafts. Panel, judge, and strategy are configurable on the dashboard's Fusion page or per request via the fusion field; each sub-call goes through normal routing, quotas, and analytics.

Response headers

Every response carries an X-Routed-Via: <platform>/<model> header so you can see which provider actually served each call. If a request fell over between providers, you'll also see X-Fallback-Attempts: N and X-Fallback-Trail, which names each hop that failed and why:

X-Fallback-Attempts: 2
X-Fallback-Trail: groq/llama-3.3-70b key1=rate_limited; google/gemini-2.5-flash key2=timeout

X-Fallback-Detail adds what each of those hops cost, which is the part you cannot reconstruct from the trail — a request answered in 40s reads identically whether one provider stalled for 39 seconds or four failed fast:

X-Fallback-Detail: groq/llama-3.3-70b key1=rate_limited t=0+39000ms msg=Groq API error 429: rate limit; google/gemini-2.5-flash key2=timeout t=39000+12ms

t=<start>+<duration>ms is the offset from the start of the failover chain and how long that hop ran. msg= is the provider's error, redacted and truncated; it is last in each record, and any semicolon inside it becomes a comma so ; stays an unambiguous record separator.

The detail header is off by default — it puts hop timings and provider error text on the response. Turn it on with FALLBACK_DETAIL_HEADER=1 or the expose_fallback_detail_header settings key, which takes precedence. Both headers list at most ten hops and then a ; +N more marker.

Only hops that already failed can appear. The hop actually serving your request is recorded after its response finishes — after the JSON is sent, or after the stream closes — so its duration does not exist while the headers are still open. Use X-Routed-Via to see which provider that was.

HTTP headers only carry printable ASCII, so a model id with characters outside that range (a Chinese name from a relay catalog, for example) is percent-encoded in the header — run the value through decodeURIComponent (or urllib.parse.unquote) to read it back.

The opt-in response cache can be toggled per request with X-FreeLLM-Cache: on|off — an exact-match in-memory LRU for identical requests, streaming included (a streamed hit replays the stored SSE verbatim) (canonical SHA-256 keys over the full request, TTL and temperature gates, saved-token stats on the dashboard). Off by default; cache hits consume zero provider quota.

When prompt compression is enabled, X-FreeLLM-Compress: off|on|lossless|standard|aggressive can disable or lower the configured mode for one request. It cannot raise the operator's configured mode. The response reports the effective mode and estimated savings, for example X-FreeLLM-Compress: standard; saved~=1840.

X-FreeLLM-Task-Type: code|chat|auto tells the router what kind of turn this is, so it can bias the routing weights: code trades some of the speed weight for intelligence (a wrong answer costs more than a slow one), chat does the reverse. It is a soft preference only — capability gates, the failover chain and the model catalog are untouched, and the shift is 30% of one axis by default. The default is auto: tool-bearing requests and messages carrying unambiguous code markers (a fenced block, a declaration, a stack frame, a file.ts:214 reference) are treated as code, and everything else keeps the preset weights exactly as configured. The fastest, reliable and custom routing strategies ignore the header — those are an explicit operator choice about where on the axis to sit.

How far the shift goes is operator-tunable: GET /api/settings/task-weight-share reads the current fraction and PUT /api/settings/task-weight-share with {"share": 0..1} sets it (default 0.3). 0 disables the bias while leaving the header itself accepted, and null clears the setting back to the default. It takes effect on the next request, with no restart.

Embeddings

/v1/embeddings is OpenAI-compatible, with one deliberate difference from chat routing: failover never crosses models. Vectors from different models live in incompatible spaces — silently switching models would corrupt any vector store built on top of the proxy. So embeddings route by family (one model identity + dimension), and failover only walks the providers serving that same family.

resp = client.embeddings.create(
    model="auto",          # default family; or a family name like "bge-m3"
    input=["the quick brown fox", "pack my box with five dozen liquor jugs"],
)
print(len(resp.data), "vectors of", len(resp.data[0].embedding), "dims")

curl http://localhost:3001/v1/embeddings \
  -H "Authorization: Bearer f3router-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "input": "hello world"}'

model accepts auto (the configured default family), a family name, or a provider-specific model id (which resolves to its family). Available families:

The default family, per-provider toggles, and priorities live on the dashboard's Models → Embeddings page. Pick your family once and stick with it for a given vector store — that's the whole point of the family model.

Anthropic / Claude clients

f3router also speaks Anthropic's Messages API, so anything built for Claude — including Claude Code and the official Anthropic SDKs — can run against your free pool. Point the client at your server's origin (Anthropic clients append /v1/messages themselves) and authenticate with your unified key. Both x-api-key and Authorization: Bearer are accepted.

curl http://localhost:3001/v1/messages \
  -H "x-api-key: f3router-your-unified-key" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "hi"}]
  }'

Claude model names map to your free pool on the Keys → Anthropic tab: each family (default, opus, sonnet, haiku) routes to auto (the router picks a free model) or a model you pin. POST /v1/messages/count_tokens and a content-negotiated GET /v1/models (Anthropic shape when anthropic-version is sent) are implemented too. Streaming, system prompts, tool use, and image input all translate across the same router as the OpenAI endpoints.

Claude Code — point it at your server and start it:

On macOS / Linux (Bash):

export ANTHROPIC_BASE_URL=http://localhost:3001
export ANTHROPIC_AUTH_TOKEN=f3router-your-unified-key   # NOT ANTHROPIC_API_KEY
claude

On Windows (PowerShell):

$env:ANTHROPIC_BASE_URL="http://localhost:3001"
$env:ANTHROPIC_AUTH_TOKEN="f3router-your-unified-key"
claude

Use ANTHROPIC_AUTH_TOKEN (sent as a Bearer token), not ANTHROPIC_API_KEY — Claude Code treats a set ANTHROPIC_API_KEY as a conflicting first-party credential and refuses to start.

Free-tier budget API

GET /api/free-tier returns the operator's free-tier budget as a pool-deduped monthly overview. Many models on one platform share the same free pool, so summing every model's label would double-count; the endpoint collapses models into quota pools (inferQuotaPoolKey) and reports one documented budget per pool (the largest parseBudget value seen in it), scaled by the number of usable (enabled + healthy/unknown) keys for the platform. Chain-disabled rows still contribute to a pool — they share the same provider allowance — and are only marked, never dropped, so the totals match the dashboard's stacked per-model bar.

{
  "generatedAt": "2026-08-25T12:00:00.000Z",
  "summary": {
    "poolCount": 18,
    "documentedMonthlyTokens": 7400000000,
    "creditsBasedPools": 2,
    "unpublishedPools": 3
  },
  "pools": [
    {
      "poolKey": "google",
      "platform": "google",
      "memberModelIds": ["gemini-3.0-flash", "gemini-2.5-flash"],
      "modelCount": 2,
      "disabledModelCount": 0,
      "keyCount": 1,
      "documentedBudget": 120000000,
      "bestLabel": "~120M",
      "kind": "documented",
      "quota": { "limit": 150000000, "remaining": 98000000, "resetAt": "2026-08-26T00:00:00.000Z", "metric": "tokens", "keyCount": 1 }
    }
  ]
}

• kind ∈ documented (a parseable monthly budget), credits (credit-based), or unpublished (no public figure).
• quota carries live observations (limit / remaining / reset / metric) when the provider reports them; metric prefers tokens → credits → neurons → requests.
• Mounted behind the dashboard session (requireAuth); not the unified /v1 key.

Backups API

GET /api/backups lists the encrypted database snapshots the gateway has taken (scheduled or manual via POST /api/backups). Each backup is an SQL dump with a header carrying its sha256 key fingerprint — restore refuses a dump written under a different ENCRYPTION_KEY (reports backup: <fingerprint>; server: <fingerprint>). The on-disk layout is versioned with DUMP_FORMAT = 1; a dump of a different format is rejected with 409.

{
  "items": [
    {
      "id": 14,
      "filename": "f3router-20260825-040000-enc.sql",
      "filesize": 2418688,
      "isFull": true,
      "source": "scheduled",
      "createdAt": "2026-08-25T04:00:00.000Z",
      "tables": ["api_keys", "models", "fallback_config", "settings", "..."]
    }
  ],
  "total": 14
}

Endpoints (all behind requireAuth):

Monthly provider-key budgets

Authenticated dashboard clients can set monthlyRequestCap and monthlyTokenCap through PATCH /api/keys/:id. Both are nonnegative integers; 0 means unlimited. These limits apply to an upstream provider key across chat, embeddings, and keyed media requests, and reset at 00:00 UTC on the first of each month. They do not apply separately to downstream client profiles.

Successful request counts and reported tokens are stored in a durable monthly ledger. Request-log cleanup does not clear this ledger. Existing retained request history is backfilled during upgrade; usage already pruned before the upgrade cannot be reconstructed. In-flight requests reserve capacity until completion, including long-running streams. Failed attempts release their reservation.

Token reservations use routing estimates. Actual provider token usage can differ, so the final response can take recorded usage above the configured token cap; further requests are then rejected. This is a usage guard, not an exact billing limit. Media requests count toward the request cap; the current media adapters do not report token usage. Reservations coordinate requests within one gateway process, not across multiple gateway replicas.

When all otherwise eligible keys have exhausted their monthly budget, standard chat, embeddings, and media routing returns HTTP 429 with a Retry-After header pointing to the next UTC month. OpenAI-compatible endpoints also return error.code: "quota_exceeded"; other protocol adapters preserve their native error format. Fusion subcalls obey the caps but retain Fusion's aggregate error format. If another key has capacity, normal fallback can use it.

Embeddings additionally relay the upstream providers' own back-off. A provider that answers 429 is simply skipped for the next one in the family; only when every provider in the family was rate limited and each stated a Retry-After does the gateway's 429 carry one: the soonest of them, in whole seconds (clamped to 24h). A chain that also hit a non-rate-limit failure returns 502 without Retry-After, since waiting would not be a promise.