Budgets and policy
Every request runs under one set of numbers — a monthly budget, rate limits, and the points where people are warned and stopped — taken from the most specific level that has them.
Policy sets how much someone may spend and how fast: a budgetBudgetNgân sáchThe monthly amount a level may spend on models — organisation default, department or person. The most specific level that is set wins.Open in glossary, rate limits and a monthly token quota. Those numbers come from one of three levels, and the most specific one wins. Which models someone may pick is decided separately.
Key rules
- The most specific level wins every number: person, then department, then organisation.
- A personal limit wins both ways — higher or lower than the department’s.
- A department budget is split equally between the active people billed to it.
- Alert at 80 %, stop at 100 % — seeded defaults you can change per level.
- Only a department blocks models. A personal limit carries money and rate only.
Three levels, one winner
The organisation, a department and a single person can each hold the same set of numbers. For any request the portal takes the most specific set that exists and uses all of its numbers — it never mixes a budget from one level with a limit from another. A level with no numbers of its own falls through to the one above.
Tran Mai has a limit of her own, so it wins — all of its numbers, even though it's far above the department's. Her spend is still billed to Sales, and the models Sales blocks stay blocked for her.
| Level | Reaches |
|---|---|
| Organisation | Every department without numbers of its own |
| Department | Everybody billed there, except people with their own limit |
| Person | That one person, wherever they spend |
A personal limit is written two ways: someone holding policy.manage sets it on the person in Users, or approves the person’s Ask for a higher budget request. An approved request writes the same kind of limit, carrying the asked budget plus the other numbers the person already had.
If no level has any numbers, the request is refused rather than run without a cap.
Your share of a department budget
A department’s budget is divided equally between its live seats — active at the identity provider and not suspended in the portal — rounded down to the cent. 300 USD and three seats holds each person to 100.
The person with their own limit still counts in the headcount, but spends from their own record — so they take nothing from the shared pool, and nobody else's share moves when the limit is granted.
- Nobody types a share in: it is always worked out from the budget and headcount, so it updates the moment someone joins or leaves.
- A department with no live seat sends the whole pool. A department no policy reaches sends no member limit at all.
- A share that rounds below a cent is sent as 0.01, because the gateway reads zero as “no cap”.
- Someone with a personal limit still counts in the headcount, but spends from their own record — they take nothing from the pool, and granting a limit never moves everyone else’s share.
Which models someone can pick
Model access is resolved separately from money, and the two are never merged. Every catalogued model is open to everyone; a department narrows the list by blocking the models it gives up. A personal limit never changes which models someone can use.
| Rule | Detail |
|---|---|
| Only text (chat) models can be blocked | Embedding and background work are organisation-wide assignments |
| Something must be left | Blocking the last served chat model is refused |
| Switched-off model | Disappears from every picker. A block on it is kept and still applies when it is switched back on |
| Where it is checked | When a turn starts, when an agent’s effective access is resolved, and when an agent is created or updated |
| Where it is set | A department’s Models tab only; the catalogue screen shows which departments block a model but doesn’t edit them |
Rate limits
Each level also holds two per-minute ceilings: requests per minute (RPM) and tokens per minute (TPM). They come from the same winning set as the budget.
A per-minute limit clears by itself, so a turn absorbs three refusals before giving up. A spent budget stays spent for the rest of the month, so the turn gives up on the first.
Alert and stop
Two thresholds on each budget decide what happens as spend grows. Both come from the same level as the budget they are measured against, so a percentage is never applied to a figure another level wrote.
A budget alert is raised. Nothing is refused — people keep working.
New model calls are refused until the budget resets next month. A reply already running ends at its next call.
| Threshold | Seeded at | Effect |
|---|---|---|
| Soft alertAlert (soft limit)Cảnh báo (ngưỡng mềm)The point, 80% by default, where a budget alert is raised. Nothing is refused.Open in glossary | 80 % | A budget alert is raised. Nothing is refused |
| Hard stop | 100 % | A budget alert is raised and new model calls are refused until the budget resets next month |
A department told to stop at 90 % is refused at 90 %. Thresholds are checked after every model call, twice: once for the department’s month-to-date spend, and once for the person alone against the number that will actually stop them — their personal limit, or else their share. That second check means nobody is stopped at a line nothing warned them about.
Guardrail charges count toward the department’s spend and its budget alerts. The gateway’s hard stop counts model spend only.
Everyone can check their share in Settings › Usage & budget and ask for more there.
Keys you pay for yourself (BYOK)
A department or a person can paste their own model key (how BYOK works). A call on a model served by such a key is paid by that key’s owner, not by any portal budget:
- It beats a personal limit, which only says whose portal budget pays.
- Its cap is the key’s own monthly figure; its rate ceilings come from the policy of the key’s owner.
- That spend is left out of both department alert sums (pool and share).
- Reports still name the department, and the model breakdown marks the spend as someone’s own key.
Why it works this way
Why does a personal limit replace everything instead of adding to it?
Mixing numbers from different levels makes “what is this person held to?” impossible to answer in one sentence. Taking one whole set means an admin always sees exactly one answer, and an approved request writes the same kind of limit an admin would type.
Why split a budget equally instead of first come, first served?
With one shared pot, the first heavy user could spend the whole month and stop everyone else. An equal share means reaching your limit pauses only you.
Do document uploads and searches count?
Yes. Reading and indexing a document runs on the uploader’s department, and turning a search into a query runs on the searcher’s, so both are bounded by that department’s budget and blocked models. Keyword search itself costs nothing.
Under the hood
Policy storage and errors
One table stores all three levels; a row names its department, its person, or neither (organisation). Each row holds six numbers: budget, soft and hard thresholds, RPM, TPM and monthly token quota.
| Limit | Column |
|---|---|
| Budget | Fixed point, 8 digits + 6 decimals (max 99999999.999999 USD) |
| RPM, TPM | 32-bit integer (max 2147483647) |
| Monthly token quota | 64-bit integer |
| Code | When |
|---|---|
409 no_policy_resolved | No row at any level |
409 department_needs_chat_model | Blocking the last served chat model |
400 policy_budget_invalid / policy_limits_invalid | A figure exceeds its column — refused before anything reaches the gateway |
Model access resolves in two queries however many users are asked about: the catalogue, and the blocks of the departments in play. Admission outcomes are recorded as BLOCKED_MODEL, BLOCKED_BUDGET, BLOCKED_RATE_LIMIT and so on.
LiteLLM teams and virtual keys
All model access goes through LiteLLMLiteLLM (model gateway)LiteLLM (cổng mô hình)The gateway every model call passes through. It routes to Bedrock, applies each department's key and budget, and records the cost of every call.Open in glossary, which targets Bedrock; the app never calls Bedrock directly. A department maps to a LiteLLM team carrying its monthly pool, two rate ceilings and the member share. A person with a personal limit maps to a gateway record of their own.
Each call runs on a short-lived virtual keyVirtual keyKhoá ảoA gateway key issued per department, so each model call is limited and billed to the right department. People never handle it.Open in glossary resolved from the caller:
- No personal limit: a key naming the department’s team and the person, cached per pair. The member name is what makes the share enforceable.
- Personal limit: a key with their own id and no team, drawing on their gateway record.
- Pasted key (BYOK): a key naming that credential, joining no team.
Because keys are short-lived, a saved limit reaches the next key, not one in flight. Clearing a personal limit deletes the policy row only; the gateway record stays idle until a limit is written again.
Admin port: when the gateway is written
The admin port creates teams, sets pools and rate ceilings, issues keys and writes personal limits. It runs when the organisation is created (provisioning General’s team at seeded numbers first), when a department is created, when a policy is written at any level, and when a turn needs a key. A department and its virtual-key row are written in one transaction; without one, its first message is refused.
What travels is the winning row’s numbers. A department’s budget re-caps its team; the organisation’s re-caps only teams of departments without their own. A new department starts on the organisation’s numbers — never zero, which would refuse its second request. A database failure after a successful gateway call is logged, not rolled back.
Model port: keeping the gateway in step
A reconciler: LiteLLM serves exactly what the catalogue says, and an already-correct gateway gets no call. A changed model is updated in place, so a turn on it never hits an unknown-model error between delete and create. Resolved credentials are only passed to the call, never logged or stored. Each model records whether its serving endpoint answers for it. A write that changes what is served reconciles immediately; a timed sweep retries pushes that failed while the gateway was down. There is no publish step.
Budget alert idempotency
After each model call — right after the audit and rollup write — the check sums month-to-date spend from daily rollups and fires once per crossed threshold. It swallows its own errors, so it can never fail a turn that already succeeded.
| Scope | Idempotency |
|---|---|
| Department | Unique constraint; a conflict returns the existing alert |
| Person | Unique constraint on seat, month and threshold |
| Organisation | Advisory lock (its partial predicate can’t be a unique constraint) |
