TrueFoundry vs OpenRouter: Rate Limits, Budgets, Caching, and LLM Cost Control Compared (2026)

If you have hit OpenRouter rate limits, the reason is usually not traffic. OpenRouter limits requests per key and per account, and the headroom you get is indexed to how much credit you have purchased, so a low balance is a low rate limit. That is a billing property doing the job of an availability control, and it is the first clue that OpenRouter's cost model and a platform team's cost model are built for different people.

This post covers the cost control and optimization heading of our TrueFoundry vs OpenRouter series: rate limits, budgets and whether they warn or block, quotas and reset windows, cost-aware routing and how the price table is maintained, caching, and attribution for chargeback. It is written for the engineer who is being asked to reduce LLM API costs and the platform lead who has to report them, and every claim is drawn from the two vendors' public documentation. For where an LLM gateway sits in that picture, start with our primer.

At a glance

How OpenRouter rate limits work, and why you hit them

OpenRouter rate-limits by requests, and the limits are tied to your balance. Free-model variants are capped at 20 requests per minute; accounts with under $10 in lifetime credit purchases get 50 free-model requests per day, rising permanently to 1,000 per day once $10 has been spent, and failed requests may still count against the cap. Beyond the free tier, free-model rate limits depend on purchased credits, so throughput headroom is a function of account balance rather than a published per-tenant quota. Token-based limits and concurrency limits are not documented. Rate limits are rejected immediately with no queuing.

Two consequences follow. First, an OpenRouter credit limit is both a spending control and an availability control: let the balance run low and the rate limit tightens with it; let it hit zero and traffic stops. Second, because credits sit at the account level while workspaces sit below, one workspace's traffic plausibly consumes headroom shared with every other workspace in the organisation, and nothing in the product fences that off.

TrueFoundry rate-limits by requests and by tokens, per minute, hour or day, with rules applied per user, team, virtual account, model or request metadata and first-matching-rule-wins semantics (rate limiting). Limits are independent of what you have paid us and of other teams' traffic, because the gateway holds the rules in memory and never consults a balance to serve a request. Concurrency limits are not documented on our side either; that is the one row where both products should be tested rather than read about. Our guide to rate limiting in an LLM gateway covers how to size the rules.

Budgets: hard stop vs warn-then-block

An OpenRouter budget lives at three levels. Per key, a credit limit with an optional daily, weekly or monthly reset at midnight UTC and auto-disable when the key exceeds it. Per member and per workspace, guardrails that enforce budgets, assignable to keys or members, with the account-level policy as a ceiling a workspace can only narrow. Workspace Budgets is a first-class feature with its own API, with intervals that nest (lifetime, monthly, weekly, daily), limits that must strictly decrease as the interval narrows, and an include_byok_in_budgets switch that decides whether bring-your-own-key inference counts.

Three design choices matter. Budgets never pool: the same daily budget guardrail applied to three members gives each of them that budget separately, and a request counts toward both the key's budget and the owning member's. Per model and per application have no direct mechanism; an app budget works only where the app is the key. And budgets are hard-stop, not alert: a warn-only mode is not documented, and a request over the limit returns HTTP 403, which is how a team usually discovers its budget is gone.

TrueFoundry Budget Limiting V2 defines rules scoped at tenant or team and filtered by users, teams, virtual accounts, models (concrete or virtual), provider accounts and request metadata via the X-TFY-METADATA header, so a per-app budget is a virtual account or a metadata tag away (budget limiting v2). Rules run in enforce mode (request blocked) or warn-only mode; multiple rules AND together; limits apply in aggregate or per user, per virtual account, per model or per metadata value; and periods are daily, weekly, monthly, quarterly or lifetime, stackable within one rule ($0.50 per day and $2 per quarter, any breach blocks). Milestone alerts fire at 75, 90, 95 and 100 per cent to email, Slack webhook or Slack bot. One gotcha we state plainly: tracking starts when the rule is created, not at the start of the period, so prior spend is not counted retroactively.

The practical difference: on TrueFoundry a team is warned at 75 per cent and can keep working past 100 per cent if the rule is warn-only; on OpenRouter the team finds out when requests start returning 403.

OpenRouter prompt caching vs semantic caching

Two different things are called caching here, and conflating them is the most common mistake we see in cost reviews.

Provider prompt caching is the model provider charging less for cached input tokens. OpenRouter passes it through and routes around it: a cache read costs roughly 0.1x to 0.5x of normal input pricing depending on provider, and session_id keeps a session sticky to the provider holding the warm cache. TrueFoundry passes provider prompt caching through as well, and our documentation explicitly frames combining it with gateway caching (caching). Neither vendor invented it; both let you keep it.

Gateway response caching is the gateway answering the request itself, with no provider call. OpenRouter Response Caching, currently in beta, is exact-match: two requests are identical only when they share the same API key, model, endpoint type, streaming mode and full request body. It is opt-in per request via the X-OpenRouter-Cache: true header, model-agnostic, covers chat completions, Responses, Messages and embeddings but not legacy completions, TTS, STT, rerank or video, and on a hit returns the response with no billing and no token consumption. Two properties deserve credit: cache hits do not count toward provider rate limits, and hits are visible in the Activity log. One property limits it: the OpenRouter cache is scoped to a single API key, so different keys under the same account do not share it, and a fleet of services each holding its own key gains nothing from each other's traffic. Semantic caching is not offered. In our own testing we measured a 100 per cent hit rate on identical re-requests, 58.5 ms on a hit against 2,866.8 ms on a miss, and a time-to-live between two and five minutes.

TrueFoundry offers both exact-match and semantic caching at the gateway, backed by Redis. A hit returns the full response with zero provider invocation and zero provider cost, across all providers, and the cache is shared across every caller rather than fenced per key, so the second team to ask a question benefits from the first. A semantic cache LLM strategy matters for the workloads where exact matching never hits: customer-support assistants, internal search, and any prompt template with a variable slot. We do not publish measured hit rates, because they depend entirely on your traffic; the right test is a week of your own prompts.

Cost-aware routing: cheapest provider vs cheapest model

With no routing directives set, OpenRouter already optimises for price in three steps: deprioritise providers that have seen significant outages in the last 30 seconds, pick among the stable ones from the lowest-cost candidates weighted by the inverse square of price, and hold the rest as fallbacks. Explicit controls layer on top: provider.sort: "price" or the :floor model-slug shortcut routes cheapest-first, :nitro routes for throughput, provider.max_price sets a hard ceiling in dollars per million tokens that composes with a sort, and provider.only, ignore and quantization preferences narrow the candidate set. The price table cannot go stale because OpenRouter is the seller: its price list is the price. Two honest limits: this is provider arbitrage for a given model, not model substitution, and price-sorted routing trades quality consistency, since cheaper providers may serve different quantizations of the same model.

TrueFoundry does not do price-table-driven provider arbitrage. What we do is complexity-based routing: the gateway classifies each request as simple, medium or complex and routes it to the model you configured for that tier, with sticky sessions, so you pick the cheap model per tier and the gateway picks the tier (auto routing). Around that sit weight-, latency- and priority-based load balancing across targets, and a virtual-model fallback chain that skips any target whose budget is exceeded, which works as a cost circuit breaker. Price data comes from an open-source catalogue plus private rates for negotiated contracts and fine-tuned models (cost tracking). So the honest sentence runs both ways: cheapest acceptable model, yes; cheapest provider for this model, no. How the two approaches fit into a routing design is covered in our LLM router explainer.

Attribution and chargeback

Governing spend starts with knowing who spent it. TrueFoundry attributes cost and tokens per user, team, virtual account, model, provider account and arbitrary metadata keys; the X-TFY-METADATA header is the chargeback hook, so a request tagged with a cost-centre or application ID is reported under it, dashboards break spend down on every one of those dimensions, and the budget APIs expose usage and a top-spenders leaderboard (cost tracking, budgets API). The same gateway plane that enforces these rules in memory keeps overhead at roughly 3–4 ms and 350+ requests per second per vCPU across 1,000+ LLMs, so attribution is not something you pay for in latency. For the finance-facing side of this, see our guide to an LLM cost tracking solution.

OpenRouter attributes by key (with descriptive labels), workspace, model and provider; Activity and Logs are an account-level view with optional workspace filtering, usage: {include: true} returns per-request usage in the response body, Activity Export produces CSV or PDF grouped by model, key or member, and a User Tracking feature plus Custom Classifiers add per-user and per-category dimensions. Billing is one bill across all workspaces. Per-application or per-cost-centre attribution is achievable, natively through custom user IDs and classifiers or downstream through Broadcast metadata into your observability platform.

That is the shape of LLM cost optimization on each side. If the goal is to reduce LLM API costs for one application, OpenRouter's routing does the heavy lifting automatically. If the goal is to reduce them for an organisation, you need to know which team is spending, warn them before they are cut off, and route the easy requests to the cheap model, which is the TrueFoundry shape.

TrueFoundry vs OpenRouter: cost control head-to-head

TrueFoundry capabilities are from our documentation; OpenRouter capabilities reflect its publicly documented functionality at the time of writing.

When to choose OpenRouter

• Your main cost lever is the price of one model across providers, and you want that arbitrage done automatically with a hard price ceiling you set once.
• Keys map one-to-one to applications, so a per-key budget is a per-app budget and the missing per-model scope does not matter to you.
• A hard cap equal to your prepaid balance is the exposure model you prefer, and a 403 at the limit is acceptable because the limit is the point.

When to choose TrueFoundry

• Finance needs chargeback by team and cost centre, and the metadata header plus dashboards is how you get it without a data pipeline.
• Teams need to be warned before they are blocked, and some rules should never block at all.
• Budgets must apply per model or per application across many callers, and rate limits must count tokens as well as requests.
• Your cost strategy is model substitution (simple requests to the cheap model) rather than provider shopping, and you want a semantic cache shared across a fleet of services.

Neither vendor documents spend forecasting or anomaly detection; TrueFoundry lists anomaly alerts as “coming soon” on its agent-governance roadmap and OpenRouter has a Notifications feature whose alert types are not documented. Budget for a FinOps tool on either side, and see our guide to OpenRouter alternatives for the wider field.

Related reading

• Rate Limiting in an LLM Gateway — sizing request and token limits per team
• LLM Cost Tracking Solution — the finance-facing side of attribution and chargeback
• LLM Load Balancing — weight, latency and priority routing across targets
• What is an LLM Router? — model-level vs provider-level routing
• OpenRouter Alternatives — the wider field of enterprise options

Conclusion

OpenRouter rate limits, budgets and caching are built for a developer watching one key: limits that follow the balance, budgets that stop rather than warn, a cache that belongs to the key, and in exchange the best automatic provider arbitrage in the market. TrueFoundry's cost controls are built for a platform team answering to finance: limits by requests and tokens, budgets that warn before they block and scope to teams, models and applications, a semantic cache shared across every caller, and chargeback by cost centre out of the box. Neither forecasts or detects anomalies, so plan for that on either side. The practical dividing line is whether the person reading the budget report is a developer or a finance partner.

To see budgets, limits and caching configured for a multi-team rollout, explore the TrueFoundry AI Gateway, or book a demo if you are sizing cost controls for production now.