...
| Note |
|---|
Important: - Performance tuning requires you to modify important system files, which affect all programs running on the server. We recommend you to familiarize yourself with these files using Unix/Linux documentation before editing them.
- The values we discuss here are general recommendations. They might not be the optimal values for the specific hardware configurations in your environment. We recommend you to carry out load tests on your environment to tune the API Manager accordingly.
|
OS-level settings
When it comes to performance, the OS that the server runs plays an important role.
| Info |
|---|
If you are running MacOS Sierra and experience long startup times for WSO2 products, try mapping your Mac hostname to 127.0.0.1 and ::1 in the /etc/hosts file as described in this blog post. |
Following are the configurations you can apply to optimize OS-level performance:
To optimize network and OS performance, configure the following settings in the /etc/sysctl.conf file of Linux. These settings specify a larger port range, a more effective TCP connection timeout value, and a number of other important parameters at the OS-level.
| Info |
|---|
It is not recommended to use net.ipv4.tcp_tw_recycle = 1 when working with network address translation (NAT), such as if you are deploying products in EC2 or any other environment configured with NAT. |
| Code Block |
|---|
net.ipv4.tcp_fin_timeout = 30
fs.file-max = 2097152
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.core.rmem_default = 524288
net.core.wmem_default = 524288
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.ip_local_port_range = 1024 65535 |
To alter the number of allowed open files for system users, configure the following settings in the /etc/security/limits.conf file of Linux (be sure to include the leading * character).
| Code Block |
|---|
* soft nofile 4096
* hard nofile 65535 |
Optimal values for these parameters depend on the environment.
To alter the maximum number of processes your user is allowed to run at a given time, configure the following settings in the /etc/security/limits.conf file of Linux (be sure to include the leading * character). Each carbon server instance you run would require upto 1024 threads (with default thread pool configuration). Therefore, you need to increase the nproc value by 1024 per each carbon server (both hard and soft).
| Code Block |
|---|
* soft nproc 20000
* hard nproc 20000 |
...
Some general APIM-level recommendations are listed below:
| Improvement Area | Performance Recommendations |
|---|
| API Gateway nodes | Increase memory allocated by modifying the /bin/wso2server. sh file with the following setting: - -Xms2048m -Xmx2048m -XX:MaxPermSize=1024m
Set the following in the <APIM_HOME>/repository/conf/axis2/axis2_client.xml file: | Note |
|---|
- The following Axis2 client configurations are only applicable when Web Services key validation (WS key validation) is enabled.
- The default values mentioned in the APIM 1.10.0 pack are the values that we identified at the time of releasing APIM 1.10.0. However, if you want high concurrency, please use the values mentioned below:
|
| Code Block |
|---|
<parameter name="defaultMaxConnPerHost">1000</parameter>
<parameter name="maxTotalConnections">30000</parameter> |
|
| NHTTP transport of API Gateway | Recommended values for the <AM_HOME>/repository/conf/nhttp.properties file are given below. Note that the commented out values in this file are the default values that will be applied if you do not change anything. Property descriptions: | snd_t_core | Transport sender worker pool's initial thread count | | snd_t_max | Transport sender worker pool's maximum thread count | | snd_io_threads | Sender-side IO workers, which is recommended to be equal to the number of CPU cores. I/O reactors usually employ a small number of dispatch threads (often as few as one) to dispatch I/O event notifications to a greater number (often as many as several thousands) of I/O sessions or connections. Generally, one dispatch thread is maintained per CPU core. | | snd_alive_sec | Sender-side keep-alive seconds | | snd_qlen | Sender queue length, which is infinite by default |
Recommended values: # HTTP Sender thread pool parameters - snd_t_core=200
- snd_t_max=250
- snd_alive_sec=5
- snd_qlen=-1
- snd_io_threads=16
# HTTP Listener thread pool parameters - lst_t_core=200
- lst_t_max=250
- lst_alive_sec=5
- lst_qlen=-1
- lst_io_threads=16
#timeout parameters |
| PassThrough transport of API Gateway | Recommended values for the <AM_HOME>/repository/conf/passthru-http.properties file are given below. Note that the commented out values in this file are the default values that will be applied if you do not change anything. Property descriptions worker_thread_keepalive_sec | Defines the keep-alive time for extra threads in the worker pool | | worker_pool_queue_length | Defines the length of the queue that is used to hold runnable tasks to be executed by the worker pool | | io_threads_per_reactor | Defines the number of IO dispatcher threads used per reactor | http.max.connection.per.host.port | Defines the maximum number of connections per host port | | worker_pool_queue_length | Determines the length of the queue used by the PassThrough transport thread pool to store pending jobs. |
Recommended values worker_thread_keepalive_sec: Default value is 60s. This should be less than the socket timeout. worker_pool_queue_length: Set to -1 to use an unbounded queue. If a bound queue is used and the queue gets filled to its capacity, any further attempts to submit jobs will fail, causing some messages to be dropped by Synapse. The thread pool starts queuing jobs when all the existing threads are busy and the pool has reached the maximum number of threads. So, the recommended queue length is -1. io_threads_per_reactor: Value is based on the number of processor cores in the system. (Runtime.getRuntime().availableProcessors()) http.max.connection.per.host.port : Default value is 32767, which works for most systems but you can tune it based on your operating system (for example, Linux supports 65K connections). - worker_pool_size_core: 400
- worker_pool_size_max: 500
- io_buffer_size: 16384
- http.socket.timeout: 60000
- snd_t_core: 200
- snd_t_max: 250
- snd_io_threads: 16
- lst_t_core: 200
- lst_t_max: 250
- lst_io_threads: 16
Make the number of threads equal to the number of processor cores. |
| Time-out configurations | The API Gateway routes the requests from your client to an appropriate endpoint. The most common reason for your client getting a timeout is when the Gateway's timeout is larger than client's timeout values. You can resolve this by either increasing the timeout on the client's side or by decreasing it on the API Gateway's side. Here are few parameters, in addition to the timeout parameters discussed in the previous sections. | synapse.global_timeout_interval | Defines the maximum time that a callback is waiting in the Gateway for a response from the backend. If no response is received within this time, the Gateway drops the message and clears out the callback. This is a global level parameter that affects all the endpoints configured in Gateway. Global timeout is defined in the <APIM_HOMe>/repository/conf/synapse.properties file. Recommended value is 120000. | | Endpoint-level timeout | You can define timeouts per endpoint for different backend services, along with the action to be taken in case of a timeout. The example below sets the endpoint to 30 seconds and executes the fault handler in case of a timeout. | Code Block |
|---|
| <timeout>
<duration>10000</duration>
<responseAction>fault</responseAction>
</timeout> |
|
|
| Key Manager nodes | Set the MySQL maximum connections: | Code Block |
|---|
| mysql> show variables like "max_connections";
max_connections was 151
set to global max_connections = 250;
|
Set the open files limit to 200000 by editing the /etc/sysctl.conf file: Set the following in the <APIM_HOME>/repository/conf/tomcat/ catalina-server.xml file. | Code Block |
|---|
maxThreads="750"
minSpareThreads="150"
disableUploadTimeout="false"
enableLookups="false"
connectionUploadTimeout="120000"
maxKeepAliveRequests="600"
acceptCount="600" |
Set the following connection pool elements in the <APIM_HOME>/repository/conf/datasources/master-datasources.xml file: | Code Block |
|---|
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval> |
Note that you set the <testOnBorrow> element to true and provide a validation query (e.g., in Oracle, SELECT 1 FROM DUAL), which is run to refresh any stale connections in the connection pool. Set a suitable value for the <validationInterval> element, which defaults to 30000 milliseconds. It determines the time period after which the next iteration of the validation query will be run on a particular connection. It avoids excess validations and ensures better performance. |
Database-level settings
Set up the following database indexes in the API Manager database to improve performance:
...