Rate Limiting

The Things Stack supports rate limiting for all outward facing services. Access to each resource is limited by a unique identifier key, and it is possible to define rate limiting classes for fine-grained control.

For more details about the algorithm used, read here.

Example configuration

The rate limiting configuration is split into multiple profiles. For each profile, we provide a name, the maximum allowed rate per minute, and optionally a maximum burst rate. Finally, we associate the profile with a number of rate limiting classes. Refer to the Rate Limited Entities section below for a list of all available rate limiting classes and what they mean.

Enable rate limiting by adding the following configuration to your ttn-lw-stack.yml.

The values shown below are only meant as an example. Make sure to adjust them accordingly, depending on the actual traffic of your deployment.

rate-limiting:
  profiles:
    - name: HTTP servers
      max-per-min: 30
      associations:
        - http
    - name: User login
      max-per-min: 10
      associations:
        - http:account
    - name: Create new users
      max-per-min: 10
      associations:
        - grpc:method:/ttn.lorawan.v3.UserRegistry/Create
    - name: Application downlink traffic
      max-per-min: 10
      associations:
        - as:down:web
        - as:down:mqtt
        - grpc:method:/ttn.v3.lorawan.v3.AppAs/DownlinkQueuePush
        - grpc:method:/ttn.v3.lorawan.v3.AppAs/DownlinkQueueReplace
    - name: Gateway connections
      max-per-min: 5
      associations:
        - gs:accept:mqtt
        - gs:accept:semtechws/lbslns
        - grpc:stream:accept:/ttn.lorawan.v3.GtwGs/LinkGateway
    - name: gRPC API
      max-per-min: 60
      associations:
        - grpc:method
        - grpc:stream:accept
    - name: Override rate for uplink simulation
      max-per-min: 5
      associations:
        - grpc:method:/ttn.lorawan.v3.AppAs/SimulateUplink
    - name: Gateway uplink traffic
      max-per-min: 1000
      max-burst: 1500
      associations:
        - gs:up

Rate Limited Entities

This section lists resources of The Things Stack on which a maximum rate limit can be enforced.

Application Server MQTT Client Connections

Description	Limit number of new MQTT client connections to Application Server by remote IP.
Classes	`as:accept:mqtt`
Key	`as:accept:mqtt:ip:{{ RemoteIP }}`
Example	`as:accept:mqtt:ip:10.10.10.10`

Application Server MQTT Downlink Traffic

Description	Limit number of downlink messages from MQTT clients by application ID and authentication token ID.
Classes	`as:down:mqtt`
Key	`as:down:mqtt:app:{{ AppUID }}:token:{{ AuthTokenID }}`
Example	`as:down:mqtt:app:my-application:token:F69AA8D76F5A78S6FDAASADF`

Application Server Downlink Traffic Via Webhooks

Description	Limit number of downlink messages via webhooks by application ID, device ID and authentication token ID.
Classes	`as:down:web`
Key	`as:down:web:dev:{{ DevUID }}:token:{{ AuthTokenID }}`
Example	`as:down:web:dev:my-application.my-device:token:F69AA8D76F5A78S6FDAASADF`

gRPC Requests

Description	Limit number of gRPC API requests by full gRPC method name, entity ID and authentication token ID.
Classes	`grpc:method:{{ FullMethod }}</code> <code>grpc:method`
Key	`grpc:method:{{ FullMethod }}:{{ EntityUID }}:token:{{ AuthTokenID }}`
Example	`grpc:method:/ttn.lorawan.v3.AppAs/SimulateUplink:end device:my-application.my-device:token:F69AA8D76F5A78S6FDAASADF`

HTTP API Requests

Description	Limit number of HTTP API requests by gRPC method name, entity ID and authentication token ID.
Classes	`grpc:method:{{ FullMethod }}</code> <code>grpc:method`
Key	`grpc:method:{{ FullMethod }}:{{ EntityUID }}:token:{{ AuthTokenID }}`
Example	`grpc:method:/ttn.lorawan.v3.AppAs/SimulateUplink:dev:my-application.my-device:token:F69AA8D76F5A78S6FDAASADF`

gRPC Server Streams

Description	Limit number of new gRPC server streams by gRPC server stream and authentication token ID.
Classes	`grpc:stream:accept:{{ FullMethod }}</code> <code>grpc:stream:accept`
Key	`grpc:stream:accept:{{ FullMethod }}:token:{{ AuthTokenID }}`
Example	`grpc:stream:accept:/ttn.lorawan.v3.GtwGs/LinkGateway:token:F69AA8D76F5A78S6FDAASADF`

Incoming Client Traffic From gRPC Server Streams

Description	Limit number of incoming message for gRPC server streams for individual server streams.
Classes	`grpc:stream:up:{{ FullMethod }}</code> <code>grpc:stream:up`
Key	`grpc:stream:up:{{ FullMethod }}:streamID:{{ CorrelationID }}`
Example	`grpc:stream:up:/ttn.lorawan.v3.GtwGs/LinkGateway:streamID:01F26VW4B39PAPHZKXFV6YM1TP`

Gateway Server MQTT Gateway Connections

Description	Limit number of new MQTT gateway connections to Gateway Server by remote IP.
Classes	`gs:accept:mqtt`
Key	`gs:accept:mqtt:ip:{{ RemoteIP }}`
Example	`gs:accept:mqtt:ip:10.10.10.10`

Gateway Server BasicStation Gateway Connections

Description	Limit number of new BasicStation gateway connections to Gateway Server by remote IP and request URL.
Classes	`gs:accept:semtechws/lbslns</code> <code>http`
Key	`gs:accept:semtechws/lbslns:ip:{{ RemoteIP }}:url:{{ URL }}`
Example	`gs:accept:semtechws/lbslns:ip:10.10.10.10:url:/traffic/eui-0102030405060708`

Gateway Server Uplink Traffic (UDP)

Description	Limit number of UDP packets processed by the Gateway Server by remote IP.
Classes	`gs:up:udp</code> <code>gs:up`
Key	`gs:up:udp:ip:{{ RemoteIP }}`
Example	`gs:up:udp:ip:10.10.10.10`

Gateway Server Uplink Traffic (BasicStation, MQTT and gRPC)

Description	Limit number of uplink messages received by the Gateway Server (for the BasicStation, gRPC and MQTT frontends) by gateway ID.
Classes	`gs:up`
Key	`gs:up:gtw:{{ GtwUID }}`
Example	`gs:up:gtw:my-gateway`

BasicStation CUPS

Description	Limit number of HTTP requests to the BasicStation CUPS by remote IP and request URL.
Classes	`http:gcs:cups</code> <code>http`
Key	`http:gcs:cups:ip:{{ RemoteIP }}:url:{{ URL }}`
Example	`http:gcs:cups:ip:10.10.10.10:url:/update-info`

OAuth Server, Account App, Gateway Configuration Server, Interop Server, Metrics, Health, pprof, Azure IoT Central integration

Description	Limit number of HTTP requests by remote IP and request URL. `{{ Server }}` is `oauth`, `account`, `gcs`, `interop`, `metrics`, `health`, `pprof` or `as:io:packages:azure:iotcentral` respectively. `{{ AuthInfo }}` is either `ip:{{ RemoteIP }}` or `token:{{ AuthTokenID }}` if the caller presents authentication information.
Classes	`http:{{ Server }}:{{ URLTemplate }}</code> <code>http:{{ Server }}</code> <code>http`
Key	`http:{{ Server }}:{{ AuthInfo }}:url:{{ URL }}`
Example	`http:oauth:ip:10.10.10.10:url:/authorize`

Note:

Both gRPC methods and HTTP endpoints support multiple classes. This enables overriding the generic rate limits for specific methods and endpoints.

Warning:

When The Things Stack HTTP and gRPC endpoints are served behind a reverse proxy, the X-Forwarded-For header is respected for the remote IP. The IP address of the reverse proxy must be trusted by The Things Stack for this to work, see HTTP options and gRPC Options.

Warning:

When The Things Stack MQTT and UDP endpoints are served by a reverse proxy, the remote IP address seen by The Things Stack may not be correct. In this case, rate limiting for new MQTT connections (and UDP traffic) should be handled by the reverse proxy and disabled in The Things Stack.

Rate Limiting Actions

The following table describes how The Things Stack reacts when the maximum rate limit for a resource is exceeded.

Resource	Rate Limit Action	Expected Client Action
gRPC Requests	Return an error response of type `ResourceExhausted`, and set rate limiting headers.	The client should wait and then retry the gRPC request. The `X-Rate-Limit-Retry` header from the response can help in this case.
HTTP Requests	Return an `HTTP 429 (Too Many Requests)` error response, and set rate limiting headers.	The client should wait and then retry the HTTP request. The `X-Rate-Limit-Retry` header from the response can help in this case.
New MQTT Connections	Drop (reject) new connection.	The client should reconnect with a backoff.
UDP packets	Drop packet.	The client has no way of knowing whether the sent packet was received and/or processed, which is a limitation of the Legacy UDP packet forwarder.
Application Downlinks	Reject downlink message and terminate connection.	The client should reconnect, and reduce the downlink traffic load.
Gateway Uplinks	Reject uplink message and terminate connection.	The client should reconnect, and reduce the downlink traffic load.

Rate Limiting Headers

When rate limiting is enabled, the following headers are added to all HTTP and gRPC requests:

X-Rate-Limit-Limit: The maximum number of allowed requests for the current time period.
X-Rate-Limit-Available: Number of available requests for the current time period.
X-Rate-Limit-Reset: Seconds until the rate limiter resets.
X-Rate-Limit-Retry: Seconds the client should wait before retrying the request.

Rate limiting providers

The default rate limiting provider for The Things Stack is memory. When using this provider the state of rate limiting is stored locally on each instance. The drawback of using it is inconsistent rate limiting state. The redis provider eliminates this problem by storing a shared rate limiting state for all instances.

Provider	State
`memory`	per instance
`redis`	shared

To enable redis rate limiting provider you can set it in rate limiting configuration.

rate-limiting:
  provider: `redis`

← Packet Broker Agent Options Resource Limiting →