If you ship an AI feature inside a SaaS, the bill is not “your” bill. It is the sum of every customer who pastes a novel into the chat box, every agent loop that forgets to stop, and every leaked key that keeps calling gpt-4o at 3 a.m.
OpenAI will email you after the money is gone. Anthropic does not give you a per-user budget at all.
LLM0 is a self-hosted spend firewall that sits in front of OpenAI, Anthropic, Gemini, and local Ollama.
Your app keeps using the OpenAI SDK. You change baseURL to the gateway, send two headers that identify the end user and their plan, and LLM0 refuses the request before the provider call when that user is over their cap.
This post is the Next.js version of that setup: one project, two plans (free and pro), and a route handler that never trusts the browser to name its own tier.
What You Are Wiring Up
Three limits stack, and they answer different questions:
- Per API key, requests per minute: Stops a single key from hammering the gateway. Returns
429withretry_after. - Per project, dollars per month: Your whole product’s ceiling. Estimated before the provider call. Returns
402witherror: "spend_cap_exceeded". - Per end user, dollars and request counts: The one your pricing page is about. Returns
429witherror: "customer_rate_limit_exceeded", or silently swaps in a cheaper model if that plan is set to downgrade.
You do not insert a database row per signup.
On the first request that carries X-Customer-ID, the gateway creates that user’s spend row itself. You define the plans once. Each request says which plan this user is on.
How Tier Resolution Works
On every chat call, the gateway looks up a cap in this order and stops at the first match:
X-Customer-Tiermatches a tier you defined for the project (free,pro, or whatever slug you want).- Otherwise, the project’s default limits apply to every user.
- Otherwise, the user is unlimited.
An unknown tier slug does not error. It falls through to the project default.
That is handy when a typo should not take production down, and dangerous if you forget to create the tier and assume Pro users are capped.
Create the tiers before you start sending the header.
X-Customer-Tier is a server-to-server header. Read the plan from your own users table (or from Stripe) inside the route handler.
Never forward a tier header that came from the browser.
1. Run the Gateway
Docker Compose from the llm0ai/llm0 repository brings up Postgres, Redis, and the gateway.
Copy .env.example to .env, set at least one provider key, and create a gateway API key:
docker compose up -d
./scripts/create_api_key.shThe key looks like llm0_live_… and is shown once.
Put it in the Next.js server environment, not in NEXT_PUBLIC_*.
LLM0_BASE_URL=http://localhost:8080/v1
LLM0_API_KEY=llm0_live_...A second listener, off by default, is how a SaaS creates tiers without SQL.
Set a long random ADMIN_TOKEN in the gateway .env and keep port 8081 on a private network.
Leave the token unset and that listener never starts.
ADMIN_TOKEN=replace-with-a-long-random-string
ADMIN_LISTEN_ADDR=:80812. Define Free and Pro Plans
Tiers are yours. LLM0 does not ship a free or pro plan.
A POST to the admin API is a full replace: omitted cap fields mean “no limit on that axis,” not “leave the old value.”
When you update a tier later, resend every cap you still want.
Here are example numbers for a chat product. Tune them to your model and your price.
Free Plan: Small Daily Budget, Hard Stop
export PROJECT_ID=<project-uuid>
export ADMIN=http://localhost:8081
curl -X POST "$ADMIN/v1/admin/projects/$PROJECT_ID/tiers" \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"slug": "free",
"daily_spend_limit_usd": 0.05,
"monthly_spend_limit_usd": 1.00,
"per_request_max_usd": 0.02,
"requests_per_minute": 10,
"on_limit_behavior": "block"
}'Pro Plan: Larger Budget with Model Downgrade
Instead of blocking users entirely, the Pro plan can fall back to a cheaper model when the configured limit is reached.
curl -X POST "$ADMIN/v1/admin/projects/$PROJECT_ID/tiers" \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"slug": "pro",
"daily_spend_limit_usd": 5.00,
"monthly_spend_limit_usd": 80.00,
"per_request_max_usd": 0.50,
"requests_per_minute": 60,
"on_limit_behavior": "downgrade",
"downgrade_model": "gpt-4o-mini"
}'Available Limit Behaviors
on_limit_behavior supports three options:
- block: Returns
429, and the provider is never called. - downgrade: The request still runs using
downgrade_model. You must send that model in the same body. - warn: The request proceeds. Use the spend headers if you want to notify the user.
Configure a Default Limit
Anyone who hits the gateway with no tier, or with a slug you have not created, gets the project default.
Set that once so a missing plan cannot mean unlimited access.
curl -X PATCH "$ADMIN/v1/admin/projects/$PROJECT_ID/defaults" \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"default_daily_spend_limit_usd": 0.05,
"default_monthly_spend_limit_usd": 1.00,
"default_on_limit_behavior": "block"
}'Unlike tiers, this PATCH is partial.
Omitted fields stay as they are. DELETE …/defaults clears them back to unlimited.
Set a Project-Wide Spending Cap
Also cap the project itself, so one busy day cannot exceed what you are willing to float on the provider invoice.
curl -X PATCH "$ADMIN/v1/admin/projects/$PROJECT_ID" \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{ "monthly_cap_usd": 500 }'3. Integrate LLM0 with a Next.js Route
The browser talks to your application. Your application talks to LLM0.
Install the official OpenAI SDK:
npm install openaiThen point the SDK at the gateway.
Create or update app/api/chat/route.ts:
import OpenAI from "openai";
import { auth } from "@/lib/session"; // your session helper
const client = new OpenAI({
baseURL: process.env.LLM0_BASE_URL!,
apiKey: process.env.LLM0_API_KEY!,
});
export async function POST(req: Request) {
const session = await auth();
if (!session?.user?.id) {
return Response.json(
{ error: "unauthorized" },
{ status: 401 }
);
}
const body = await req.json();
const messages = body.messages;
if (!Array.isArray(messages) || messages.length === 0) {
return Response.json(
{ error: "messages required" },
{ status: 400 }
);
}
// Read the plan from your database after verifying the session.
const tier = session.user.plan === "pro" ? "pro" : "free";
try {
const { data, response } = await client.chat.completions
.create(
{
model: "gpt-4o",
messages,
max_tokens: 800,
},
{
headers: {
"X-Customer-ID": session.user.id,
"X-Customer-Tier": tier,
},
}
)
.withResponse();
return Response.json({
message: data.choices[0]?.message,
model: data.model,
remainingUsd: response.headers.get(
"x-customer-remaining-usd"
),
spentTodayUsd: response.headers.get(
"x-customer-spend-today"
),
downgraded:
response.headers.get("x-downgraded") === "true",
});
} catch (err) {
if (err instanceof OpenAI.APIError && err.status === 429) {
const code = (
err.error as { error?: string } | undefined
)?.error;
if (code === "customer_rate_limit_exceeded") {
return Response.json(
{
error: "user_budget",
message:
"You have used today's AI allowance. Upgrade to Pro or try again tomorrow.",
},
{ status: 429 }
);
}
return Response.json(
{
error: "rate_limited",
message: "Slow down and retry shortly.",
},
{ status: 429 }
);
}
if (err instanceof OpenAI.APIError && err.status === 402) {
return Response.json(
{
error: "workspace_budget",
message: "This workspace has hit its monthly AI budget.",
},
{ status: 402 }
);
}
console.error(err);
return Response.json(
{ error: "upstream" },
{ status: 502 }
);
}
}Production Details That Matter
A few things are worth understanding before deploying this.
1. Set an explicit maximum output token count.
max_tokens tightens the pre-call cost estimate.
Without max_tokens, the gateway assumes an output of about twice the prompt, clamped between 100 and 2,000 tokens.
Near a cap, an explicit max_tokens keeps the estimate honest.
2. Spend headers show a pre-request snapshot.
The following headers are calculated before the current call:
X-Customer-Spend-TodayX-Customer-Limit-DailyX-Customer-Remaining-Usd
You can use them to display something like “$0.04 left today” in the interface.
They appear only when a daily cap actually resolves for that user.
3. Model downgrades still return successful responses.
On a Pro downgrade, the call still returns HTTP 200.
X-Downgraded: true tells you the gateway swapped models, while data.model identifies the model that actually ran.
Tell the user their answer came from the fallback model instead of pretending it was gpt-4o.
4. Streaming is supported.
The same headers work with the Vercel AI SDK.
Set baseURL to the LLM0 gateway and pass X-Customer-ID and X-Customer-Tier on the provider call.
Streaming limits are enforced too. A user who is already over their cap never opens an SSE stream to the provider.
5. Analytics headers are separate from enforcement.
X-LLM0-Team and similar X-LLM0-* headers are labels for your analytics.
They do not select a tier.
The enforcement header is X-Customer-Tier.
4. Handle Spending Limits in Your Application
Map the different responses to different user experiences. They represent different problems.
HTTP 429: User Budget Exceeded
Error: customer_rate_limit_exceeded
This user has used their plan allowance.
Offer an upgrade, or explain that their allowance resets at 00:00 UTC.
Daily and monthly counters roll onto new Redis keys at UTC midnight and on the first day of each month.
You do not need a cron job.
HTTP 402: Project Budget Exceeded
Error: spend_cap_exceeded
The project itself has reached its spending ceiling.
This is an operator-level issue, not an individual user’s problem.
Do not tell a Free user to upgrade when the whole workspace is blocked.
HTTP 200: Model Downgraded
Header: X-Downgraded: true
The Pro user has exceeded the expensive-model budget, and you configured a fallback.
Show the response normally, but inform the user that later messages may use a smaller model until the allowance resets.
5. What Happens When a User Upgrades?
Suppose someone moves from Free to Pro.
Stripe, or whatever billing provider you use, updates users.plan to pro.
The next AI request sends:
X-Customer-Tier: proYou do not need to migrate spending records.
The same X-Customer-ID keeps its daily and monthly totals. Only the cap used to evaluate the user changes.
If you later raise the Pro allowance from $5 per day to $8 per day, send another POST request containing the complete tier configuration.
Tier lookups are cached in-process for about 60 seconds, so configuration changes may take roughly a minute to propagate.
Recommended Production Setup
For a typical multi-tenant Next.js SaaS application, I would use the following configuration:
- One LLM0 project per environment. Preview and production should have separate projects, API keys, and spending limits.
- One gateway API key on the server. End users should never see it.
- Project defaults equal to the Free plan. If a bug drops the tier header, the user still has a spending cap.
- Free blocks, Pro downgrades. Free users hit a hard limit, while Pro users can fall back to a cheaper model.
- A project-wide monthly cap. This protects the overall provider budget regardless of individual allowances.
- Keep the Admin API private. Expose only the necessary gateway endpoints publicly.
Final Thoughts
AI spending is easy to underestimate when you are building a SaaS application.
A few heavy users, unexpectedly long prompts, or agent workflows that repeatedly call a model can quickly increase your provider bill.
The important part is enforcing limits before requests reach the LLM provider, instead of discovering the cost after the fact.
With LLM0, your existing OpenAI SDK integration stays mostly unchanged.
You point the SDK at a different base URL, identify the customer, and send the subscription tier you already store in your application.
From there, the gateway handles spending limits, request caps, and model fallback behavior.
You can self-host the MIT-licensed LLM0 gateway.
Either way, the integration follows the same basic pattern: one base URL, one customer ID, and one plan identifier.
Leave a Reply