API Manager-level settings

Timeout configurations for an API call

The following diagram shows the communication/network paths that occur when an API is called. The timeout configurations for each network call are explained below.

Key validation
Key validation occurs via a Servlet HTTP call and the connection timeout can be configured by changing the following configuration details in the <WSO2_OB_APIM_HOME>/repository/conf/axis2/axis2_client.xml file. All timeout values are in milliseconds.
```
<transportSender name="https" class="org.apache.axis2.transport.http.CommonsHTTPTransportSender">
<parameter name="SO_TIMEOUT">60000</parameter>
<parameter name="CONNECTION_TIMEOUT">60000</parameter>
</transportSender>
```
If the Key Manager caching is enabled, the calls between the API Gateway and Key Manager are cached. As a result, the Key Manager is not invoked for each API call.
Client call API Gateway + API Gateway call Backend
For backend communication, the API Manager uses PassThrough transport. This is configured in the <WSO2_OB_APIM_HOME>/repository/conf/passthru-http.properties file. For more information, see Configuring passthru-http.properties in the EI documentation.

Note that the default value for http.socket.timeout differs between WSO2 products. In WSO2 API-M, the default value for http.socket.timeout is 180000ms.

General APIM-level recommendations

Some general APIM-level recommendations are listed below:

Improvement Area Performance Recommendations

API Gateway nodes

Increase memory allocated by modifying the /bin/wso2server. sh file with the following setting:

-Xms2048m -Xmx2048m -XX:MaxPermSize=1024m

Set the following in the <WSO2_OB_APIM_HOME>/repository/conf/axis2/axis2_client.xml file:

The following Axis2 client configurations are only applicable when Web Services key validation (WS key validation) is enabled.
The default values mentioned are the values identified at the time of releasing API-M. However, if you want high concurrency, use the values mentioned below:

<parameter name="defaultMaxConnPerHost">1000</parameter> 
<parameter name="maxTotalConnections">30000</parameter>

The above configurations are only applicable when WS key validation is enabled.

NHTTP transport of API Gateway

Recommended values for the <<WSO2_OB_APIM_HOME>/repository/conf/nhttp.properties file are given below. Note that the commented out values in this file are the default values that will be applied if you do not change anything.

Property descriptions:

snd_t_core	Transport sender worker pool's initial thread count
snd_t_max	Transport sender worker pool's maximum thread count
snd_io_threads	Sender-side IO workers, which is recommended to be equal to the number of CPU cores. I/O reactors usually employ a small number of dispatch threads (often as few as one) to dispatch I/O event notifications to a greater number (often as many as several thousands) of I/O sessions or connections. Generally, one dispatch thread is maintained per CPU core.
snd_alive_sec	Sender-side keep-alive seconds
snd_qlen	Sender queue length, which is infinite by default

Recommended values:

HTTP Sender thread pool parameters
- snd_t_core=200
- snd_t_max=250
- snd_alive_sec=5
- snd_qlen=-1
- snd_io_threads=16
HTTP Listener thread pool parameters
- lst_t_core=200
- lst_t_max=250
- lst_alive_sec=5
- lst_qlen=-1
- lst_io_threads=16
timeout parameters
- http.socket.timeout.receiver: Recommended socket timeout for listener is 180000 ms.
- http.socket.timeout.sender: Recommended socket timeout for sender is 180000 ms.

PassThrough transport of API Gateway

Recommended values for the <WSO2_OB_APIM_HOME>/repository/conf/passthru-http.properties file are given below. Note that the commented out values in this file are the default values that will be applied if you do not change anything.

Property descriptions

`worker_thread_keepalive_sec`	Defines the keep-alive time for extra threads in the worker pool
`worker_pool_queue_length`	Defines the length of the queue that is used to hold runnable tasks to be executed by the worker pool
`io_threads_per_reactor`	Defines the number of IO dispatcher threads used per reactor
`http.max.connection.per.host.port`	Defines the maximum number of connections per host port
`worker_pool_queue_length`	Determines the length of the queue used by the PassThrough transport thread pool to store pending jobs.
`http.connection.timeout`	Defines a maximum time period to establish a connection with the remote host. The `http.connection.timeout` and the `http.socket.timeout`, which is explained below, are two different configuration definitions used to handle connection time out and read timeout for sockets respectively.
`http.socket.timeout`	Defines the waiting time for data after establishing the connection, which refers to the maximum time of inactivity between two data packets.

Recommended values

worker_thread_keepalive_sec: Default value is 60s. This should be less than the socket timeout.
worker_pool_queue_length: Set to -1 to use an unbounded queue. If a bound queue is used and the queue gets filled to its capacity, any further attempts to submit jobs will fail, causing some messages to be dropped by Synapse. The thread pool starts queuing jobs when all the existing threads are busy and the pool has reached the maximum number of threads. So, the recommended queue length is -1.
io_threads_per_reactor: Value is based on the number of processor cores in the system. (Runtime.getRuntime().availableProcessors())
http.max.connection.per.host.port : Default value is 32767, which works for most systems but you can tune it based on your operating system (for example, Linux supports 65K connections).
worker_pool_size_core: 400
worker_pool_size_max: 500
io_buffer_size: 16384
http.socket.timeout: 180000

Make the number of threads equal to the number of processor cores.

Timeout configurations

The API Gateway routes the requests from your client to an appropriate endpoint. The most common reason for your client getting a timeout is when the Gateway's timeout is larger than the client's timeout values. You can resolve this by either increasing the timeout on the client's side or by decreasing it on the API Gateway's side.

Here are a few parameters, in addition to the timeout parameters discussed in the previous sections.

synapse.global_timeout_interval

Defines the maximum time that a callback waits in the Gateway for a response from the backend. If no response is received within this time, the Gateway drops the message and clears out the callback. This is a global level parameter that affects all the endpoints configured in the Gateway.

The global timeout is defined in the <WSO2_OB_APIM_HOME>/repository/conf/synapse.properties file. The recommended value is 120000 ms.

Endpoint-level timeout

You can define timeouts per endpoint for different backend services, along with the action to be taken in case of a timeout.

The example below sets the endpoint to 50 seconds (50000 ms) and executes the fault handler in case of a timeout.

<timeout>
   <duration>50000</duration>
   <responseAction>fault</responseAction>
</timeout>

Alternatively, you can set this through the Publisher UI as well, by following the steps below:

Log in to the API Publisher (https://<HostName>:9443/publisher). Select your API and click Edit API.
Click the Implement tab and click the cogwheel icon next to the endpoint you want to re-configure.
In the Advanced Settings dialog box that appears, increase the duration by modifying the default property set as 30000 ms.

Note that when the endpoint is suspended, the default action is defined here as invoking the fault sequence.
Click Save and re-publish the API.

The http.socket.timeout parameter needs to be adjusted based on the endpoint-level timeout so that it's value is equal or higher than the highest endpoint-level timeout.

If your API is marked as the default version, it has a different template (without the version number) that comes with a pre-defined timeout for the endpoint. This timeout does not change with the changes you do to the API by editing the Advanced Endpoint Configuration. Therefore, if this predefined timeout (60 seconds) is less than the actual API timeout, it triggers the timeout before the actual configured API timeout.

To overcome this, update the default_api_template.xml residing in the <WSO2_OB_APIM_HOME>/ repository/resources/api_templates directory by removing the endpoint timeout configuration from the default API. Then, the APIs marked as the default version also trigger the timeout when the actual API timeout is met.

Follow the steps below to update the default_api_template.xml to remove the endpoint configuration for the default APIs.

If you are using a distributed (clustered) setup, follow these steps in the Publisher node as it is the API Publisher that creates the API definition and pushes it to the Gateway.

Open the <WSO2_OB_APIM_HOME>/repository/resources/api_templates/default_api_template.xml file and remove the following configuration:

<timeout>
	<duration>60000</duration>
	<responseAction>fault</responseAction>
</timeout>
<suspendOnFailure>
	<progressionFactor>1.0</progressionFactor>
</suspendOnFailure>
<markForSuspension>
	<retriesBeforeSuspension>0</retriesBeforeSuspension>
	<retryDelay>0</retryDelay>
</markForSuspension>

Add the following configuration to the same place in the default_api_template.xml file.

By adding this configuration, you ensure that the APIs marked as the default version never timeout or are suspended using the endpoint configuration defined in the synapse file of the API.
```
<suspendOnFailure>
	<errorCodes>-1</errorCodes>
	<initialDuration>0</initialDuration>
	<progressionFactor>1.0</progressionFactor>
	<maximumDuration>0</maximumDuration>
</suspendOnFailure>
<markForSuspension>
	<errorCodes>-1</errorCodes>
</markForSuspension>
```
Go to the API Publisher and republish the default API by clicking Save and Publish.

Key Manager nodes

Set the MySQL maximum connections:

mysql> show variables like "max_connections"; 
 max_connections was 151 
 set to global max_connections = 250;

Set the open files limit to 200000 by editing the /etc/sysctl.conf file:

sudo sysctl -p

Set the following in the <WSO2_OB_APIM_HOME>/repository/conf/tomcat/catalina-server.xml file.

If you use WSO2 Identity Server (WSO2 IS) as the Key Manager, then the root location of the above path and the subsequent path needs to change from <WSO2_OB_APIM_HOME> to <WSO2_OB_KM_HOME>.

maxThreads="750" 
minSpareThreads="150" 
disableUploadTimeout="false" 
enableLookups="false" 
connectionUploadTimeout="120000" 
maxKeepAliveRequests="600" 
acceptCount="600"

Set the following connection pool elements in the <WSO2_OB_APIM_HOME>/repository/conf/datasources/master-datasources.xml file. Time values are defined in milliseconds.

<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>

Note that you set the <testOnBorrow> element to true and provide a validation query (e.g., in Oracle, SELECT 1 FROM DUAL), which is run to refresh any stale connections in the connection pool. Set a suitable value for the <validationInterval> element, which defaults to 30000 milliseconds. It determines the time period after which the next iteration of the validation query will be run on a particular connection. It avoids excess validations and ensures better performance.

Registry indexing configurations

The registry indexing process is only required to be run on the API Publisher and API Store nodes. To disable the indexing process from running on the other nodes (Gateways and Key Managers), you need to set the <wso2registry><indexingConfiguration><startIndexing> element to false in the <WSO2_OB_APIM_HOME>/repository/conf/registry.xml file of the relevant nodes.

Registry Caching based configurations

When Registry caching is enabled, it requires a unique ID in order to cache the resources. The getConnectionId method gets called in the latter mentioned situation order to retrieve the username of the connection. In the case of some database vendors, the getConnectionId method makes a database call to fetch the latter mentioned information. Such a database call is a massive overhead because the getConnectionId method frequently gets called. Therefore, it is recommended to avoid those database calls in order to improve the performance.

You need to get the latest product updates for your product to use the following instructions in the current version of WSO2 API-M. This feature is available as a product update from February 20, 2020 onwards.

Note that you can deploy updates in a production environment only if you have a valid subscription with WSO2. Read more about WSO2 Updates.

Follow the instructions below to avoid those database calls and improve the performance during Registry Caching.

Add the following JVM parameter in the <WSO2_OB_APIM_HOME>/bin/wso2server.sh (for Linux/Mac OS) or <WSO2_OB_APIM_HOME>/bin/wso2server.bat (for Windows OS) file based on your OS and save the file.
```
-Dcarbon.registry.database.connectionid.avoiddbcalls=true
```
Restart the WSO2 API Manager server.

Throttle data and Analytics-related settings

This section describes the parameters you need to configure to tune the performance of API-M Analytics and Throttling when it is affected by high load, network traffic etc. You need to tune these parameters based on the deployment environment.

Tuning carbon.xml parameters

The following parameter is configured in the <<WSO2_OB_APIMAPI-M_HOME>/repository/conf/carbon.xml file:

Parameter Description Default Value Tuning Recommendation


          AnalyticsServerPublishingInterval

The number of milliseconds that should elapse after a batch of statistical data is processed to be published in the Analytics Dashboard before sending another batch.

2000

The default value of 2000 milliseconds (i.e. 5 seconds) is recommended.

When WSO2 API-M is handling a high load of requests, this value can be reduced to increase the frequency with which the resulting statistics published in the Analytics Dashboard. This helps to avoid storing too much data in WSO2 API-M causing an overconsumption of memory.

When the load of requests handled by WSO2 API-M is comparatively low, this time interval can be increased to reduce the system overhead incurred by frequent processing.

Tuning data-agent parameters

The following parameters should be configured in the <WSO2_OB_AM_ANALYTICS_HOME>/repository/conf/data-bridge/data-agent-config.xml file. Note that there are two sub-sections in this file, named Thrift and Binary.

<DataAgentsConfiguration>
    <Agent>
        <Name>Thrift</Name>
        ...
    </Agent>

    <Agent>
        <Name>Binary</Name>
        ...
    </Agent>
</DataAgentsConfiguration>

The Thrift section is related to Analytics and the Binary section is related to Throttling. Same set of parameters mentioned below can be found in both sections. The parameter descriptions and recommendations are intended towards the for performance tuning of Analytics, but the same recommendations are relevant for Throttling data related tuning in the Binary section. Note that the section for Thrift is relevant only if Analytics is enabled.

Parameter	Description	Default Value	Tuning Recommendation
`QueueSize`	The number of messages that can be stored in WSO2 API-M at a given time before they are published to the Analytics Server.	32768	This value should be increased when the Analytics Server is busy due to a request overload or if there is high network traffic. This prevents the generation of the queue `full,` `dropping message` error. When the Analytics server is not very busy and when the network traffic is relatively low, the queue size can be reduced to avoid an overconsumption of memory. The number specified for this parameter should be a power of 2.
`BatchSize`	The WSO2 API-M statistical data sent to the Analytics Server to be published in the Analytics Dashboard are grouped into batches. This parameter specifies the number of requests to be included in a batch.	200	This value should be tuned in proportion to the volume of requests sent from WSO2 API-M to the Analytics Server. This value should be reduced if you want to reduce the system overhead of the Analytics Server. This value should be increased if WSO2 API-M is generating a high amount of statistics and if the `QueueSize` cannot be further increased without causing an overconsumption of memory.
`CorePoolSize`	The number of threads allocated to publish WSO2 API-M statistical data to the Analytics Server via Thrift at the time WSO2 API-M is started. This value increases when the throughput of statistics generated increases. However, the number of threads will not exceed the number specified for the `MaxPoolSize` parameter.	1	The number of available CPU cores should be taken into account when specifying this value. Increasing the core pool size may improve the throughput of statistical data published in the Analytics Dashboard, but latency will also be increased due to context switching.
`MaxPoolSize`	The maximum number of threads that should be allocated at any given time to publish WSO2 API-M statistical data to the Analytics Server.	1	The number of available CPU cores should be taken into account when specifying this value. Increasing the maximum core pool size may improve the throughput of statistical data published in the Analytics Dashboard, since more threads can be spawned to handle an increased number of events. However, latency will also increase since a higher number of threads would cause context switching to take place more frequently.
`MaxTransportPoolSize`	The maximum number of transport threads that should be allocated at any given time to publish WSO2 API-M statistical data to the Analytics Server.	250	This value must be increased when there is an increase in the throughput of events handled by WSO2 API-M Analytics. The value of the `tcpMaxWorkerThreads` parameter in the `<WSO2_OB_AM_ANALYTICS_HOME>/conf/worker/deployement.yaml` must change based on the value specified for this parameter and the number of data publishers publishing statistics. e.g., When the value for this parameter is `250` and the number of data publishers is 7, the value for the `tcpMaxWorkerThreads` parameter must be `1750` (i.e., 7 * 250). This is because you need to ensure that there are enough receiver threads to handle the number of messages published by the data publishers. Example dataReceivers: - # Data receiver configuration dataReceiver: # Data receiver type # THIS IS A MANDATORY FIELD type: Thrift # Data receiver properties properties: tcpPort: '7611' sslPort: '7711' tcpMaxWorkerThreads: '1750' sslMaxWorkerThreads: '1750'
`SecureMaxTransportPoolSize`	The maximum number of secure transport threads that should be allocated at any given time to publish WSO2 API-M statistical data to the Analytics Server.	250	This value must be increased when there is an increase in the throughput of events handled by WSO2 API-M Analytics. The value of the `sslMaxWorkerThreads` parameter in the `<WSO2_OB_AM_ANALYTICS_HOME>/conf/worker/deployement.yaml` must change based on the value specified for this parameter and the number of data publishers publishing statistics. e.g., When the value for this parameter is `250` and the number of data publishers is 7, the value for the `sslMaxWorkerThreads` parameter must be `1750` (i.e., 7 * 250). This is because you need to ensure that there are enough receiver threads to handle the number of messages published by the data publishers. Example dataReceivers: - # Data receiver configuration dataReceiver: # Data receiver type # THIS IS A MANDATORY FIELD type: Thrift # Data receiver properties properties: tcpPort: '7611' sslPort: '7711' tcpMaxWorkerThreads: '1750' sslMaxWorkerThreads: '1750'