Tuning Flow Control

Flow control is typically employed for controlling fast producers from overloading slow consumers in producer-consumer scenarios. There may be several reasons for a fast producer-slow consumer scenario. For example, the consumer can be on a low resource footprint; or the message broker, which lies in the middle of the producer and the consumer, may get overloaded at a particular moment due to message accumulation within the broker itself. This can cause message broker instances to run out of resources, such as memory.

The Message Broker profile primarily supports buffer limit-based flow control. This involves blocking message acceptance when the rate at which messages are transmitted reaches a high limit, and unblocking it when this rate reaches a low limit.

This is further illustrated in the following matrix.

	Global	Per-publisher
High Limit	When the total number of message content chunks reach the global high limit, flow control is enabled and message acceptance is blocked for all the publishers.	When the total number of message content chunks by an individual publisher reach the per-publisher high limit, flow control is enabled for that publisher. As a result, message acceptance will be blocked for the publisher.
Low Limit	If flow control is currently enabled, it will be disabled only when the total number of message content chunks reach the global low limit. Once this limit is reached, all the publishers are notified so that they can resume sending messages.	If flow control is currently enabled for an individual publisher, it will be disabled only when the total number of message content chunks reach the per-publisher low limit. Once this limit is reached, the publisher will be notified so that he/she can resume sending messages.

This section explains how to tune the performance of the Message Broker profile in relation to flow control. The parameters described below can be configured in the <EI_HOME>/wso2/broker/conf/broker.xml file.

Improvement Area Description Performance Recommendations

Enabling/disabling flow control based on message content chunk limits

You can set the global limits and channel-specific (i.e., per publisher) limits for message content chunks in the broker.xml file as shown below.

To define the global message content limits, set the <lowLimit> and <highLimit> parameters as shown below. These limits are applicable to all channels collectively.

<flowControl>
        <!-- This is the global buffer limits which enable/disable the flow control  globally -->
        <global>
            <lowLimit>800</lowLimit>
            <highLimit>8000</highLimit>
        </global>
        .................
</flowControl>

To define the channel specific buffer limits, set the <lowLimit> and <highLimit> parameters as shown below. These limits will be applicable to each channel.

<flowControl>
        ................
        <!-- This is the channel specific buffer limits which enable/disable the flow control locally.-->
        <bufferBased>
            <lowLimit>100</lowLimit>
            <highLimit>1000</highLimit>
        </bufferBased>
</flowControl>

Having a large number as the higher limit would increase the number of messages stored in memory before they are stored in databases. This would result in a higher overall message publishing rate, but with reduced reliability.

If the difference between the higher limit and the lower limit is too small, it would cause frequent enabling and disabling of flow control. This would reduce the overall message publishing rate.

Default or recommended values are as follows.

Global limits:

Low limit: 800
High limit: 8000

Buffer based limits:

High limit: 100
Low limit: 1000

Enabling/disabling flow control based on the memory

The globalMemoryRecoveryThresholdRatio parameter allows you to specify the memory consumption ratio at which flow control should be enabled (if it is currently disabled). This ratio is calculated using the following formula.

Used Memory/Allocated Memory

You can also specify the time interval at which the server should check whether the above ratio is reached via the memoryCheckInterval parameter.

If the publisher throughput is very high compared to the consumer throughput, this ratio should be reduced (to a maximum value of 1) to avoid out of memory scenarios. At the same time, the value for the memoryCheckInterval parameter should be reduced to perform more frequent checks on memory availability.