Introducing Throttling

Throttling allows you to limit the number of successful hits to an API during a given period of time, typically in cases such as the following:

To protect your APIs from common types of security attacks such as denial of service (DOS)
To regulate traffic according to infrastructure availability
To make an API, application or a resource available to a consumer at different levels of service, usually for monetization purpose

You can define throttling in the API, application and resource levels. The final request limit granted to a given user on a given API is ultimately defined by the consolidated output of all throttling tiers together.

Example: Let's say two users subscribed to an API using the Gold subscription, which allows 20 requests per minute. They both use the application App1 for this subscription, which again has a throttling tier set to 20 requests per minute. All resource level throttling tiers are unlimited. In this scenario, although both users are eligible for 20 requests per minute access to the API, each ideally has a limit of only 10 requests per minute. This is due to the application-level limitation of 20 requests per minute.

Different levels of throttling

Let's take a look at the different levels of throttling:

For information on adding a new throttling tier or changing the existing, see Adding a new Throttling Tier.

Application-level throttling

Application-level throttling tiers are defined at the time an application is created in the API Store as shown below:

An application is a logical collection of one or more APIs and is required to subscribe to an API. Applications allow you to use a single access token to invoke a collection of APIs and to subscribe to one API multiple times with different SLA levels.

An application is available to a consumer at different levels of service. For example, if you have infrastructure limitations in facilitating more than a certain number of requests to an application at a time, the throttling tiers can be set accordingly so that the application can have a maximum number of requests within a defined time. The default application-level throttling levels are as follows:

Small: 1 request per minute
Medium: 5 requests per minute
Large: 20 requests per minute
Unauthenticated: 60 requests per minute.
Unauthenticated Tier

This level is automatically applied when the authentication type of your resources is 'None'. That is, when you can invoke APIs without tokens. And this tier is not visiblie in the Throttling tier list of the application.
Unlimited: Unlimited access. The Default Application, which is provided out of the box has Unlimited tier set. You can set it to a restricted limit.

Resource-level throttling

An API is made up of one or more resources. Each resource handles a particular type of request and is analogous to a method (function) in a larger API. Resource-level throttling tiers are set to HTTP verbs of an API's resources when Managing APIs using the API Publisher portal as shown below:

The default throttling levels are Gold, bronze, silver and unlimited, as explained in the previous sections.

When a subscriber views an API using the API Store, s/he can see the resource-level throttling tiers using the Throttle Info tab as follows:

Subscribers are not allowed to change these throttling tiers. They are simply notified of the limitations. The default, resource-level throttling levels are as follows:

Basic: 1 request per minute
Plus: 5 requests per minute
Ultimate: 20 requests per minute
Unauthenticated: 60 requests per minute.
Unauthenticated Tier

This level is automatically applied when the authentication type of your resources is 'None'. That is, when you can invoke APIs without tokens. And this tier is not visiblie in the Throttling tier list of the application.
Unlimited: Unlimited access.

API-level throttling

API-level throttling tiers are defined when managing APIs using the API Publisher portal. The UI looks as follows:

After API-level throttling tiers are set and the API is published, at subscription time, the consumers of the API can log in to the API Store and select which tier they are interested in as follows:

According to the tiers the subscriber selects, s/he is granted a maximum number of requests to the API. The default tiers are as follows:

Bronze: 1 request per minute
Silver: 5 requests per minute
Gold: 20 requests per minute
Unlimited: Allows unlimited access (you can disable the Unlimited tier by editing the <TierManagement> node of the <APIM_HOME>/repository/conf/api-manager.xml file)

Setting tier permissions: Users with Manage Tiers permission can set role-based permissions to API-level access throttling tiers. This is done using the Tier Permissions menu in the API Publisher as shown below. For each tier, you can specify a comma-separated list of roles and either Allow or Deny access to the list.

A subscriber logged into the API Store can consume APIs using a specific tier only if s/he is assigned to a role that is allowed access. In the API Store, the subscriber sees a list of tiers that is filtered based on the subscriber's role. Only the ALLOWED roles appear here. By default, all tiers are allowed to everyone.

IP-level throttling

In IP-based throttling, you can limit the number of requests sent by a client IP (e.g., 10 calls from a single client). For example, the throttling policy shown below allows only 1 API call per minute for a client from 10.1.1.1 and 2 calls per minute for a client from any other IP address.

<wsp:Policy xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy" xmlns:throttle="http://www.wso2.org/products/wso2commons/throttle">   
   <throttle:MediatorThrottleAssertion>    
   <wsp:Policy>            
       <throttle:ID throttle:type="IP">10.1.1.1</throttle:ID>            
       <wsp:Policy>                
         <throttle:Control>                    
           <wsp:Policy>                        
              <throttle:MaximumCount>1</throttle:MaximumCount>                        
              <throttle:UnitTime>60000</throttle:UnitTime>                    
           </wsp:Policy>                
        </throttle:Control>           
       </wsp:Policy>        
    </wsp:Policy>
     
<wsp:Policy>            
<throttle:ID throttle:type="IP">other</throttle:ID>            
<wsp:Policy>                
<throttle:Control>                    
<wsp:Policy>                        
<throttle:MaximumCount>2</throttle:MaximumCount>                        
<throttle:UnitTime>60000</throttle:UnitTime>                   
 </wsp:Policy>                
</throttle:Control>            
</wsp:Policy>        
</wsp:Policy>    
</throttle:MediatorThrottleAssertion></wsp:Policy>

If you need to add a list of IPs / subnets for throttling you can derive the IP range from CIDR (Classless Inter-Domain Routing) notation and add it to the throttling policy configuration as follows.

Example :

If IP / subnet is "10.1.1.1/27" in CIDR notation, the IP address range is between "10.1.1.1 - 10.1.1.30" (if we keep 31 for broadcast). Therefore you can add throttling for the IP range as follows.

<throttle:ID throttle:type="IP">10.1.1.1 - 10.1.1.30</throttle:ID>

Hard throttling

Hard throttling limits the total number of calls the API Manager is allowed to make to the backend. While the other throttling levels define the quota the API invoker gets, they do not ensure that the backend is protected from overuse. Hard throttling limits the quota the backend can handle.

You set a Hard throttling limit using the Manage tab of the API Publisher when creating or editing an API. Click the option to enable Hard throttling to the API and then specify limits of the Production and Sandbox endpoints separately as the two endpoints can come from two servers with different capacities.

To view the error codes for hard throttling, see Error Handling.

Alternatively, you can go to the synapse configuration of the API in the <APIM_HOME>/repository/deployment/server/synapse-configs directory and specify the hard throttling limit there. Hard throttling limits are usually counted over a duration of 1 second, but you can increase the duration using the productionUnitTime and sandboxUnitTime properties in the API's synapse configuration. For instance, to allow 600 requests within a duration of 1 minute, see the example below:

<handlers>
     <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.CORSRequestHandler">
        <property name="apiImplementationType" value="ENDPOINT"/>
     </handler>
     <handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>
     <handler class="org.wso2.carbon.apimgt.gateway.handlers.throttling.APIThrottleHandler">
        <property name="id" value="A"/>
        <property name="productionMaxCount" value="600"/>
        <property name="productionUnitTime" value="60000"/>
        <property name="policyKey" value="gov:/apimgt/applicationdata/tiers.xml"/>
     </handler>
     <handler class="org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageHandler"/>
     ...
</handlers>

How throttling works with authentication type 'None'

1. Unauthenticated throttling tier will be applied based on the client IP. For example, if two clients are using the same IP, the request count (for throttling) is the aggregate of requests sent by both clients.

2. The resource level throttling tier will be applied based on the particular resource of the API. For example, if a throttling policy is engaged for /test resource, the request count (for throttling) is the total number of requests sent to that particular resource.

3. For 'Unauthenticated' API resource (authentication type set as 'None'), throttling happens as follows.

Subscription tier is set as unauthenticated (Eg : 60 requests per minute). The /test resource is associated with a tier named 'Plus' (Eg : 5 requests per minute). In that case, it applies the whichever is lower of the aforementioned tiers. A client invoking the /test resource (from a single source IP) will be throttled out at 5 requests per minute.