Configuring the Message Delivery Strategy

When you work with topics, messages that are not acknowledged by subscriber clients can cause your broker server to go out of memory. The message delivery strategy configuration in the EI Message Broker controls this issue. This configuration allows you to control how messages that accumulate in memory should be handled by the server. See the following topics for more details:

Understanding message delivery patterns in EI Message Broker

All messages published to the broker are stored in a database, which makes them inherently persistent. The broker will then read the messages from the database and deliver them to the relevant subscribers. The way messages flow through EI Message Broker to subscribers works differently for Queues and Topics.

Queues are typically used for storing messages. A JMS client can then subscribe to the queue at any time and consume the messages stored for the queue. Unlike with Queues, Topics in EI Message Broker are typically used for realtime message brokering. That is, messages published to a topic are required to be delivered to all the active subscribers instantly (in realtime). Messages that are published to a topic while a subscriber is inactive will be handled according to the durability of the subscription as explained below.

Message delivery for non-durable topic subscriptions

When there are non-durable topic subscriptions for a topic, the messages are temporarily stored in memory of the broker server until all the subscribers acknowledge that the message is received. However, sometimes a subscriber can be late to acknowledge, which will cause messages to accumulate in memory. The following steps describe how this message flow works:

1. Messages published to the EI Message Broker are first stored in the database.
2. The broker fetches the messages from the DB and dispatches them to all the subscriber clients instantly.
3. The broker maintains a list of the dispatched messages in memory as meta data until the messages are successfully received by all the clients.
4. The messages that are successfully received will send an acknowledgment back to the broker.
5. Messages that are acknowledged by all the subscribers are removed from the meta data in memory.
6. The messages that are not acknowledged by the subscriber will be retained in memory until such acknowledgment is received.

It is important to note here that the messages are not dispatched to subscriber clients from the broker according to the rate at which the messages are consumed by the subscriber. For example, consider that the broker dispatches 1000 messages per second to each subscriber: Subscriber A may be consuming messages at the rate of 500 messages per second, whereas subscriber B will consume all 1000 messages per second. In this scenario, 500 messages that are not acknowledged by subscriber A will always be accumulating in the broker server's memory. If the number of unacknowledged messages increases beyond a certain point, the broker server could run out of memory. Therefore, when you work with topics, you can change the message delivery strategy according to the requirement.

Changing the default message delivery strategy

Follow the steps given below.

1. Open the broker.xml file from the <EI_HOME>/wso2/broker/conf directory.

2. Locate the parameters defining the topic message delivery strategy:

   <topicMessageDeliveryStrategy>
   	<strategyName>DISCARD_NONE </strategyName>
   	<!-- If you choose DISCARD_ALLOWED topic message delivery strategy, we keep messages in memory
         	 until ack is done until this timeout. If an ack is not received under this timeout, ack will
   	 be simulated internally and real acknowledgement is discarded.-->
   	<deliveryTimeout>60</deliveryTimeout>
   </topicMessageDeliveryStrategy>

3. Update the <StategyName> parameter using one of the following values:

   • DISCARD_NONE: This is the default setting, which ensures that none of the messages are discarded from memory in the broker server. All the unacknowledged messages will be retained in memory of the broker.
   • DISCARD_ALLOWED: If you set this value, messages dispatched from the broker will be removed from memory without waiting for an acknowledgment from the subscriber clients.
   • SLOWEST_SUB_RATE: If you set this value, messages will be dispatched to subscriber clients at the rate of the slowest subscriber. For example, if subscriber A consumes messages at the rate of 500 messages per second and subscriber B consumes messages at a rate of 1000 messages per second, MB will always dispatch messages at the rate of 500 messages per second to both the subscribers.

4. Update the <deliveryTimeout> parameter with time period in seconds that is allowed before the broker removes any unacknowledged messages.