Tuning the JMS Transport

The Java Message Service (JMS) transport of the WSO2 Enterprise Integrator (WSO2 EI) allows you to easily send and receive messages to queues and topics of any JMS service that implements the JMS specification. The following sections describe how you can tune the JMS transport of WSO2 EI for better performance.

Increase the maximum number of JMS proxies

If you create several JMS proxy services in WSO2 EI, you will see a message similar to the following:

WARN - JMSListener Polling tasks on destination : JMStoHTTPStockQuoteProxy18 of type queue for service JMStoHTTPStockQuoteProxy18 have not yet started after 3 seconds ..

This issue occurs when you do not have enough threads available to consume messages from JMS queues. The maximum number of concurrent consumers (that is, the number of JMS proxies) that can be deployed is limited by the base transport worker pool that is used by the JMS transport. You can configure the size of this worker pool using the system properties lst_t_core and lst_t_max. Note that increasing these values will also increase the memory consumption, because the worker pool will allocate more resources.

Similarly, you can configure the current number and the anticipated number of outbound JMS proxies using the system properties snd_t_core and snd_t_max.

To adjust the values of these properties, you can modify the server startup script if you want to increase the available threads for NHTTP and JMS transports (requires more memory), or create a jms.properties file if you want to increase the available threads just for the JMS transport. Both approaches are described below.

To increase the threads for NHTTP and JMS transports

Open the integrator.sh or integrator.bat file in your <EI_HOME>/bin directory for editing.
Change the values of the properties as follows:

If you do not have the following properties in the integrator.sh or integrator.bat files, add them with the given values.
- -Dlst_t_core=200
  
  The defined values is applied as a System Property.
- -Dlst_t_max=250
- -Dsnd_t_core=200
- -Dsnd_t_max=250

To increase the threads for just the JMS transport

Create a file named jms.properties with the following properties:
- lst_t_core=200
- lst_t_max=250
- snd_t_core=200
- snd_t_max=250
Create a directory called conf under your <EI_HOME> directory and save the file in this directory.

Configuring JMS Listener

You can increase the JMS listener performance by

Using concurrent consumers
Enabling caching

Using concurrent consumers

Concurrent consumers is the minimum number of threads for message consuming. If there are more messages to be consumed while the running threads are busy, then additional threads are started until the total number of threads reaches the value of the maximum number of concurrent consumers (ie., MaxConcurrentConsumers). The maximum number of concurrent consumers (or the number of JMS proxies) that can be deployed is limited by the base transport worker pool that is used by the JMS transport. The size of this worker pool can be configured via the system property 'lst_t_core' and 'lst_t_max' as described above. The number of concurrent producers are generally limited by the Synapse core worker pool.

Note

Concurrent consumers are only applicable to JMS queues not for JMS topics.

To increase the JMS listener performance

Add the following parameters to the JMS listener configuration of the <EI_HOME>/conf/axis2/axis2.xml file:

<parameter name="transport.jms.ConcurrentConsumers" locked="false">50</parameter> 
<parameter name="transport.jms.MaxConcurrentConsumers" locked="false">50</parameter>

Parameters

transport.jms.ConcurrentConsumers: the concurrent threads that need to be started to consume messages when polling.
transport.jms.MaxConcurrentConsumers: the maximum number of concurrent threads to use during polling.

If you set the locked property to true, the JMS proxy creates only one listener thread at a given time. If you set it to false, then it creates multiple listener threads from a single proxy to consume messages concurrently.

Enable caching

Add the following parameter to the JMS listener configuration of the <EI_HOME>/conf/axis2/axis2.xml file to enable caching:

<parameter name="transport.jms.CacheLevel">consumer</parameter>

The possible values for the cache level are none, auto, connection, session and consumer. Out of the possible values, consumer is the highest level that provides maximum performance.

After adding concurrency consumers and cache level, your complete configuration would be as follows:

Sample JMS listener configuration with concurrent consumers and caching

<transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener"> 
....
<parameter name="myQueueConnectionFactory" locked="false"> 
<parameter name="java.naming.factory.initial" locked="false">org.apache.activemq.jndi.ActiveMQInitialContextFactory</parameter> 
<parameter name="java.naming.provider.url" locked="false">tcp://localhost:61616</parameter> 
<parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">QueueConnectionFactory</parameter> 
<parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter> 
<parameter name="transport.jms.ConcurrentConsumers" locked="false">50</parameter> 
<parameter name="transport.jms.MaxConcurrentConsumers" locked="false">50</parameter> 
<parameter name="transport.jms.CacheLevel">consumer</parameter> 
</parameter>
....
</transportReceiver>

Configuring proxy-level throttling

The following feature is supported in WSO2 EI 6.1.1 as a product update. This update is availble through WSO2 Update Manager from the 17th of September, 2019 onwards.

Throttling can be configured in a proxy service (JMS consumer) to control the number of JMS messages that are processed by the service. To enable throttling, the 'jms.proxy.throttle.enabled' parameter should be added to the proxy service as shown below. By default, throttling is disabled for a JMS consumer.

<?xml version="1.0" encoding="UTF-8"?><proxy xmlns="http://ws.apache.org/ns/synapse" name="StockQuoteProxy" startOnLoad="true" statistics="disable" trace="disable" transports="jms">
    <target>
        <inSequence>
            ..........
        </inSequence>
    </target>
    <parameter name="jms.proxy.throttle.enabled">true</parameter>
    .........
    <description/>
</proxy>

When this parameter is enabled, the JMS consumer can consume a maximum of 60 messages during a 60 second (one minute) window at a fixed rate of one message per second. If required, you can configure the maximum number of messages that can be consumed (throttle limit) and the rate at which messages are consumed (throttle mode) during the 60 second window.

Throttle Limit and Message Mediation Time

Note that the actual throttle limit will be equal or less than the value specified by the throttle limit parameter depending on the time taken for message mediation.

For example, if the throttle limit is 60 and if each message takes more than one second for mediation, the total number of messages that can be consumed during the one minute window will always be less than 60.

In the following example, a maximum of 30 messages can be consumed during 60 seconds at a default fixed rate of one message every two seconds.

<?xml version="1.0" encoding="UTF-8"?><proxy xmlns="http://ws.apache.org/ns/synapse" name="StockQuoteProxy" startOnLoad="true" statistics="disable" trace="disable" transports="jms">
    <target>
        <inSequence>
            ..........
        </inSequence>
    </target>
    <parameter name="jms.proxy.throttle.enabled">true</parameter>
    <parameter name="jms.proxy.throttle.limitPerMinute">30</parameter>
    .........
    <description/>
</proxy>

In the following example, the throttle mode is set to 'batch'. Therefore, messages received will be consumed as quickly as possible until the throttle limit is reached or until the window expires.

<?xml version="1.0" encoding="UTF-8"?><proxy xmlns="http://ws.apache.org/ns/synapse" name="StockQuoteProxy" startOnLoad="true" statistics="disable" trace="disable" transports="jms">
    <target>
        <inSequence>
            ..........
        </inSequence>
    </target>
    <parameter name="jms.proxy.throttle.enabled">true</parameter>
    <parameter name="jms.proxy.throttle.mode">batch</parameter>
    <parameter name="jms.proxy.throttle.limitPerMinute">30</parameter>
    .........
    <description/>
</proxy>

Configuring JMS Sender

Enabling caching

Add the following parameter to the JMS sender configuration of the <EI_HOME>/conf/axis2/axis2.xml file:

<parameter name="transport.jms.CacheLevel">producer</parameter>

The possible values for the cache level are none, auto, connection,session and producer. Out of the possible values, producer is the highest level that provides maximum performance.

When using producer as the cache level, ensure to add the JMS destination parameter to avoid the following error:

INFO - AxisEngine [MessageContext: logID=2eabe85aeeb3bb62c26bb46d21b11b087ebf1e5e0b350839] JMSCC0029: A destination must be specified when sending from this producer.

Remove `ClientApiNonBlocking` when sending messages via JMS

By default, Axis2 spawns a new thread to handle each outgoing message. To change this behavior, you need to remove the ClientApiNonBlocking property from the message.