For the complete documentation index, see llms.txt. This page is also available as Markdown.

Rate Limiting

Limit how often one client can call your MCP server, by IP, API key, or your own key.

Limit how often one client can call an MCP server you build with mcpServer(). It is off by default.

mcpServer( "myApp" )
    .withRateLimit( 100 )   // 100 requests per 60 seconds, per IP

A client over the limit gets 429 Too Many Requests and has to wait. The check runs before authentication, so floods of failed logins are limited too. A CORS preflight is never counted.

Configure it

mcpServer( "myApp" ).withRateLimit()          // the defaults from the module settings
mcpServer( "myApp" ).withRateLimit( 100 )     // 100 per default window
mcpServer( "myApp" ).withRateLimit( {
    limit    : 100,
    window   : 60,           // seconds
    key      : "ip",         // "ip" | "apiKey" | "global" | lambda
    store    : "cache",      // "cache" | "memory" | class path | IRateLimitStore instance
    cacheName: "mcpLimits",  // which cache, "default" if empty
    failOpen : true          // store fails: allow (true) or answer 503 (false)
} )
mcpServer( "myApp" ).withRateLimit( false )   // remove the limit from this server

An explicit struct overrides the module settings. A bad value throws InvalidArgument.

Several limits

Give a server a limit per minute and a limit per hour, or any other windows:

A request has to pass all the limits. When one refuses, the other limits get the request back, so a refused call never uses up an allowance. Retry-After is the longest of the waits, and the X-RateLimit-* headers describe the limit the client is closest to. Each limit needs its own window. limits is used instead of limit and window.

Turn it on and off

While it is off, requests are not counted or limited and no history is added. withRateLimit( false ) removes the limit for good. BxLens shows the same switch to an admin.

Who is limited: key

Value
Meaning

"ip" (default)

The client IP from getClientIP(). Set trusted proxies in production.

"apiKey"

The key from X-API-Key or a Bearer token, only when your API key provider accepts it. A key the server has not seen yet is counted on the client IP first, and only then checked by your provider, so a flood of fake keys is refused before your provider runs. A missing or rejected key stays on the IP, so fake keys cannot dodge the limit or fill the cache. Needs withApiKeyProvider() first.

"global"

One shared bucket for every client of this server

a lambda

( requestData ) => string. An empty result means this request is not limited, which is how you exempt clients.

With the apiKey key, the provider is asked once per request: the answer is reused by the authentication step.

stdio has no IP. With the ip key every stdio request shares one bucket, so use global or a lambda there.

How it counts

One sliding window counter. Time is cut in windows, each client has a counter per window, and the count that matters is previous * (1 - elapsed) + current. There is no burst at the window edge, and the Retry-After value is exact. A refused request is not counted.

The response

Case
Result

Allowed

The normal response plus X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset (epoch seconds)

Blocked

HTTP 429, JSON-RPC error -32002, the same headers plus Retry-After (seconds), and the request's X-Request-ID and CORS headers

Store fails

failOpen: true (default) lets the request through and logs an error. false answers 503.

Where the counters live

By default in a BoxLang cache. Use a dedicated cache (cacheName), so counters are not evicted early by other entries: an evicted counter starts again at 0.

store

Use

"cache"

The default. Counters are changed under a lock, so counts are exact inside one JVM. Across several nodes, counts are approximate.

"memory"

In this JVM only, lost on restart. Single node or tests.

class path or instance

Your own store, for Redis or a database

A custom store implements IRateLimitStore:

A class path is created with cacheName as its only argument. Use an instance when you need more.

Counter keys look like bxai:mcp:ratelimit:<server>:<hash>:<window number>. The client key is hashed, and counters expire after two windows.

Logs

The rate limit never logs on the request path, so a flood cannot fill your logs or slow the server down with log writes.

  • Allowed requests, blocked requests, store and key lambda errors, and history failures are not logged.

  • They only change the stats and the history, which keep the exact totals.

  • Use the onMCPRateLimitHit event if you want to act on blocks.

  • Turning the limit on or off (enableRateLimit(), disableRateLimit()) writes one info line. That is configuration, not traffic.

Stats

getStats() includes security.rateLimitFailures and a rateLimit section with exact totals. rateLimit.storeErrors counts store failures, since they are not logged.

allowed, blocked and storeErrors count since the server started. topBlocked is today's top 20, so it recycles every UTC day. resetStats() clears the numbers, and resetRateLimitStats() also clears the history.

History and reports

The server remembers what each client did, per client and per UTC day: requests, blocked requests, the busiest minute, and the last block. It keeps this in the BoxLang cache for retention days, so it survives a restart and nodes that share the cache see each other's numbers.

  • days counts today as the first day. top is the number of clients, the busiest by requests first.

  • pctOfLimit is the busiest minute of the client against the limit of the shortest window, scaled to a minute. Over 100 means the client hit the limit.

  • IPs show plain. API keys and custom keys show as a short hash, like in the logs.

  • retention is in days: the module setting retentionDays (default 7, 0 for none, at most 365) unless the server sets its own. With 0 only today is kept, in memory.

How it stays small and fast. A request only changes numbers in memory. About every historyFlushSeconds (default 30, at least 5) the next request writes the day to the cache, with a time to live of the retention, so old days disappear by themselves. A day keeps at most historyMaxClients clients (default 500, from 50 to 5,000). When it is full, the smallest clients are folded into one (other) row: the totals stay exact and only the detail of the smallest is dropped. There is one document per server, node and day, so nodes never overwrite each other. A restarted node continues its day.

If the process stops, up to historyFlushSeconds of history is lost. The counters that enforce the limit are written on every request and are not affected.

Reset

resetRateLimit() deletes the counters from the store when the store implements the optional IClearableRateLimitStore (clear( prefix )). The cache and memory stores do. It returns false when it cannot, and then the counters expire by themselves within two windows. limiter.reset( key ) still resets one client.

Event

onMCPRateLimitHit fires on a client's first block in a window, at most 10 times per minute per server. See events.

Look at or reset a client

Bucket keys are prefixed with their type: ip:, apiKey:, custom:, and global.

Module settings

Defaults for every MCP server, in boxlang.json:

retentionDays, historyFlushSeconds and historyMaxClients are the defaults for history, and a server can override them with retention, historyFlushSeconds and historyMaxClients. With enabled: true every MCP server is limited with no code change. With key: "apiKey" a server is limited once it has an API key provider. withRateLimit( false ) opts one server out.

Next Steps

Last updated