Tuning Performance

This section describes some recommended performance tuning configurations to optimize the API Manager. It assumes that you have set up the API Manager on Unix/Linux, which is recommended for a production deployment. We also recommend a distributed API Manager setup for most production systems. Out of all components of an API Manager distributed setup, the API Gateway is the most critical, because it handles all inbound calls to APIs. Therefore, we recommend you to have at least a 2-node cluster of API Gateways in a distributed setup.

OS-level settings

When it comes to performance, the OS that the server runs plays an important role.

127.0.0.1 localhost john-mbpro.local

Following are the configurations you can apply to optimize OS-level performance:

1. To optimize network and OS performance, configure the following settings in the /etc/sysctl.conf file of Linux. These settings specify a larger port range, a more effective TCP connection timeout value, and a number of other important parameters at the OS-level.

   It is not recommended to use net.ipv4.tcp_tw_recycle = 1 when working with network address translation (NAT), such as if you are deploying products in EC2 or any other environment configured with NAT.

   net.ipv4.tcp_fin_timeout = 30
   fs.file-max = 2097152
   net.ipv4.tcp_tw_recycle = 1
   net.ipv4.tcp_tw_reuse = 1
   net.core.rmem_default = 524288
   net.core.wmem_default = 524288
   net.core.rmem_max = 67108864
   net.core.wmem_max = 67108864
   net.ipv4.tcp_rmem = 4096 87380 16777216
   net.ipv4.tcp_wmem = 4096 65536 16777216
   net.ipv4.ip_local_port_range = 1024 65535      

2. To alter the number of allowed open files for system users, configure the following settings in the /etc/security/limits.conf file of Linux (be sure to include the leading * character).

   * soft nofile 4096
   * hard nofile 65535

   Optimal values for these parameters depend on the environment.

3. To alter the maximum number of processes your user is allowed to run at a given time, configure the following settings in the /etc/security/limits.conf file of Linux (be sure to include the leading * character). Each carbon server instance you run would require upto 1024 threads (with default thread pool configuration). Therefore, you need to increase the nproc value by 1024 per each carbon server (both hard and soft).

   * soft nproc 20000
   * hard nproc 20000

JVM-level settings

When an XML element has a large number of sub elements and the system tries to process all the sub elements, the system can become unstable due to a memory overhead. This is a security risk.

To avoid this issue, you can define a maximum level of entity substitutions that the XML parser allows in the system. You do this using the entity expansion limit as follows in the <API-M_HOME>/bin/wso2server.bat file (for Windows) or the <API-M_HOME>/bin/wso2server.sh file (for Linux/Solaris). The default entity expansion limit is 64000.

-DentityExpansionLimit=10000

In a clustered environment, the entity expansion limit has no dependency on the number of worker nodes.

WSO2 Carbon platform-level settings

In multitenant mode, the WSO2 Carbon runtime limits the thread execution time. That is, if a thread is stuck or taking a long time to process, Carbon detects such threads, interrupts and stops them. Note that Carbon prints the current stack trace before interrupting the thread. This mechanism is implemented as an Apache Tomcat valve. Therefore, it should be configured in the <PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml file as shown below.

<Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve" threshold="600"/>

• The className is the Java class used for the implementation. Set it to org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.
• The threshold gives the minimum duration in seconds after which a thread is considered stuck. The default value is 600 seconds.

APIM-level settings

Timeout configurations for an API call

The following diagram shows the communication/network paths that occur when an API is called. The timeout configurations for each network call are explained below.
 

• Key validation
   Key validation occurs via a Servlet HTTP call and the connection timeout can be configured by changing the following configuration details in the <API-M_HOME>/repository/conf/axis2/axis2_client.xml file. All timeout values are in milliseconds.

  <transportSender name="https" class="org.apache.axis2.transport.http.CommonsHTTPTransportSender">
  <parameter name="SO_TIMEOUT">60000</parameter>
  <parameter name="CONNECTION_TIMEOUT">60000</parameter>
  </transportSender>

  If the Key Manager caching is enabled, the calls between the API Gateway and Key Manager are cached. As a result, the Key Manager is not invoked for each API call.
   

• Client call API Gateway + API Gateway call Backend
  For backend communication, the API Manager uses PassThrough transport. This is configured in the <API-M_HOME>/repository/conf/passthru-http.properties file. For more information, see Configuring passthru-http.properties in the EI documentation.

  Note that the default value for http.socket.timeout differs between WSO2 products. In WSO2 API-M, the default value for http.socket.timeout is 180000ms.

General APIM-level recommendations

Some general APIM-level recommendations are listed below:

<parameter name="defaultMaxConnPerHost">1000</parameter> 
<parameter name="maxTotalConnections">30000</parameter> 

<timeout>
   <duration>50000</duration>
   <responseAction>fault</responseAction>
</timeout>

mysql> show variables like "max_connections"; 
 max_connections was 151 
 set to global max_connections = 250; 

sudo sysctl -p

maxThreads="750" 
minSpareThreads="150" 
disableUploadTimeout="false" 
enableLookups="false" 
connectionUploadTimeout="120000" 
maxKeepAliveRequests="600" 
acceptCount="600" 

<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>

Registry indexing configurations

The registry indexing process is only required to be run on the API Publisher and API Store nodes. To disable the indexing process from running on the other nodes (Gateways and Key Managers), you need to set the <wso2registry><indexingConfiguration><startIndexing> element to false in the <API-M_HOME>/repository/conf/registry.xml file of the relevant nodes.

Registry Caching based configurations

When Registry caching is enabled, it requires a unique ID in order to cache the resources. The getConnectionId method gets called in the latter mentioned situation order to retrieve the username of the connection. In the case of some database vendors, the getConnectionId method makes a database call to fetch the latter mentioned information. Such a database call is a massive overhead because the getConnectionId method frequently gets called. Therefore, it is recommended to avoid those database calls in order to improve the performance.

Follow the instructions below to avoid those database calls and improve the performance during Registry Caching.

1. Add the following JVM parameter in the <API-M_HOME>/bin/wso2server.sh (for Linux/Mac OS) or <API-M_HOME>/bin/wso2server.bat (for Windows OS) file based on your OS and save the file. 

   -Dcarbon.registry.database.connectionid.avoiddbcalls=true

2. Restart the WSO2 API Manager server.

Throttle data and Analytics-related settings

This section describes the parameters you need to configure to tune the performance of API-M Analytics and Throttling when it is affected by high load, network traffic etc. You need to tune these parameters based on the deployment environment.

Tuning carbon.xml parameters

The following parameter is configured in the <API-M_HOME>/repository/conf/carbon.xml file:

Tuning data-agent parameters

The following parameters should be configured in the <APIM_ANALYTICS_HOME>/repository/conf/data-bridge/data-agent-config.xml file. Note that there are two sub-sections in this file, named Thrift and Binary.

<DataAgentsConfiguration>
    <Agent>
        <Name>Thrift</Name>
        ...
    </Agent>

    <Agent>
        <Name>Binary</Name>
        ...
    </Agent>
</DataAgentsConfiguration>

The Thrift section is related to Analytics and the Binary section is related to Throttling. Same set of parameters mentioned below can be found in both sections. The parameter descriptions and recommendations are intended towards the for performance tuning of Analytics, but the same recommendations are relevant for Throttling data related tuning in the Binary section. Note that the section for Thrift is relevant only if Analytics is enabled. 

dataReceivers:
  -
      # Data receiver configuration
    dataReceiver:
        # Data receiver type
        # THIS IS A MANDATORY FIELD
      type: Thrift
        # Data receiver properties
      properties:
        tcpPort: '7611'
        sslPort: '7711'
        tcpMaxWorkerThreads: '1750'
        sslMaxWorkerThreads: '1750'

dataReceivers:
  -
      # Data receiver configuration
    dataReceiver:
        # Data receiver type
        # THIS IS A MANDATORY FIELD
      type: Thrift
        # Data receiver properties
      properties:
        tcpPort: '7611'
        sslPort: '7711'
        tcpMaxWorkerThreads: '1750'
        sslMaxWorkerThreads: '1750'