For Admins
Vocareum enforces rate limits on every request that passes through the GenAI Gateway, protecting your provider quota from a single runaway lab or a burst of simultaneous student activity. Each limit ships with a default, and you can set your own values on the services in a Policy.
This article explains what each limit controls, how to set custom values, and how to confirm a limit is working.
The three rate limits
Every request is checked against three independent limits before it is forwarded to your provider. If any one of them is exceeded, the request is rejected and the caller receives an error naming the limit that was hit.
Limit | What it caps | Window | Default |
Requests per minute | How many calls a user key may make | 60 seconds | 10,000 |
Tokens per minute | How many tokens a user key may consume | 60 seconds | 50,000,000 |
Parallel requests | How many calls a user key may have in flight at once | Concurrent | 50 |
Two points are worth noting before you change anything:
Limits are counted per user key, per model. Each learner's key gets its own allowance, and that allowance is tracked separately for each model they call. Two students working at the same time do not consume each other's allowance, and a student's GPT-4o usage does not count against their Claude usage.
Rate limits are not the same as budgets. A rate limit controls how fast a key may be used and resets every minute. A budget controls how much total spend a key is allowed and does not reset. Use budgets to cap cost; use rate limits to prevent bursts and runaway loops.
Because limits are counted per model, a rule you write once on a service applies separately to each model that service grants. A limit of 10 requests per minute on a service with five models lets one learner make up to 50 calls per minute across them. Pick the number you want a learner to have on a single model.
Set a custom limit on a Policy
Limits are set per service within a Policy. Because each Policy carries its own values, two Policies can apply different limits to the same models — a strict Policy for a large introductory course, and a looser one for a research group, for example.
From the Vocareum Courses landing page, click Control Center, then select GenAI from the sidebar and open the Policies page.
Find the Policy you want to change, click the ⋮ button in its Actions column, and choose Edit.
Under Edit Mode, switch the editor from Form to JSON.
Find the service you want to limit and fill in its
Limitsarray. Each service has its ownLimits, so you can limit one service and leave the others alone. Each entry in the array is one rule:
{ "ServiceType": "OpenAI", "ServiceID": "0", "Limits": [ { "Type": "2", "Grouping": "2", "RateLimit": "10" } ]}
Click Save policy.
The rule above allows 10 requests per minute per user. Type selects which limit the rule sets, and Grouping selects how usage is counted:
Field | Value | Meaning |
|
| Tokens per minute |
|
| Requests per minute |
|
| Parallel requests |
|
| Count per user (the only supported value) |
| A number | The ceiling for that limit |
Add one entry per limit you want to set. To cap requests and parallel calls at the same time, include two rules:
"Limits": [ { "Type": "2", "Grouping": "2", "RateLimit": "60" }, { "Type": "3", "Grouping": "2", "RateLimit": "5" }]Any limit you do not specify keeps its default. A service with an empty Limits array uses the defaults for all three.
Your change applies to everyone already using this Policy. It takes about five minutes to take effect.
Choosing values
The defaults are deliberately high, so they act as a backstop rather than a working limit. Set custom values when you have a specific reason:
Your provider quota is lower than the default. Set the request and token limits below your provider's own per-minute quota. Vocareum then rejects the excess request with a clear message instead of letting your provider return a 429.
You want to stop runaway loops. A student script that calls the model in a tight loop can burn a budget in minutes. A request limit of 30–60 per minute stops this while staying well clear of normal interactive use.
You are sharing one provider key across a large cohort. Lowering the parallel limit to 5–10 smooths out the load when a whole class starts an assignment at the same moment.
Start conservative and loosen if learners report being blocked. Because limits are per user key rather than per organization, a value that feels low is usually still generous for one person working interactively. If a limit turns out to be too tight, edit the Policy and the new value takes effect within a few minutes.
What learners see
When a limit is reached, the request fails with an HTTP 429 and the response names the limit and the window, for example: "GenAI Gateway Limit (60 calls/60 secs) has reached 100%."
The limit clears on its own when the window passes, so a learner who waits a minute can continue without any action from you. Parallel-request limits clear as soon as the in-flight requests finish.
Vocareum also emails an alert when a key reaches 75% and 100% of a limit, at most once per hour per key, so a persistent problem surfaces without flooding your inbox.
Verify a limit is active
Save the Policy, then wait about five minutes so the cached configuration turns over.
Make calls with a key on that Policy until you pass the limit you set. Any existing key works — there is no need to issue a new one.
Confirm the error message names the value you configured. If it names a larger number instead, the limit is not reaching the request — check that the
Limitsarray sits inside the right service and that the field names match exactly, as they are case-sensitive.