Implementing Message Expiration

According to the default setting in WSO2 Message Broker, messages published to a queue do not expire. Therefore, if required, a publisher can set an expiration period when publishing messages. This is done by setting the timeToLive (TTL) value at the publisher level.

If the specified timeToLive value is 0, the message never expires. When a message is published to the broker, the message expiration time is calculated by adding the TTL value sent by the publisher to the current time. Messages that are not delivered before the specified expiration time are deleted. The destruction of obsolete messages conserves storage and computing resources. See the section below on deleting expired messages to understand how the deletion process works.

Setting the expiration time for messages

The TTL value for messages can be specified by the publisher in two ways:

Setting a default expiration time for all messages

You can use the setTimeToLive method of the MessageProducer interface to set a default expiration time for all messages sent by that producer as shown below.

MessageProducer messageProducer = session.createProducer(requestDestination);
//time to live value in milliseconds
messageProducer.setTimeToLive(1000);

Setting an expiration time for a specific message

You can use the long form of the send or the publish method to set an expiration time for a specific message. The fourth argument sets the expiration time in milliseconds. In the following example, the TTL value of a message is set to 10 seconds: 

QueuePublisher.publish(message, DeliveryMode.NON_PERSISTENT, 3, 10000);

Deleting expired messages

Messages will not be delivered to consumers after the TTL expires. These expired messages will be deleted in batches periodically. The following values set in the broker.xml (stored in the <MB_HOME>/repository/conf directory) file controls the process of deleting expired messages.

1. When messages are received by the broker, they are first stored in a database. The system periodically checks the database for received messages that have expired the TTL and deletes them. Note that at this stage, the system only checks for messages that are not assigned to a slot delivery worker. The following property in the broker.xml file specifies the time gap (in seconds) between periodic message deletion from the database.

   <periodicMessageDeletionInterval>900</periodicMessageDeletionInterval> 

2. When checking the database for expired messages, the messages that are about to be assigned to a slot delivery worker should not be deleted (even if they have expired) since that will interfere with the slot delivery functionality. Therefore, the following property in the broker.xml file specifies a safety buffer to identify the messages (number of slot allocations waiting to be assigned) that will not be affected by the periodic message deletion. In the following example, the first three slot allocations that are waiting to be assigned to the slot delivery worker from the database should not be affected by periodic message deletion even if the TTL of the allocated messages have expired. These expired messages will later be deleted at the message delivery path.

   <safetySlotCount>3</safetySlotCount> 

3. Once messages are assigned to the slot delivery worker, the delivery process is started. If any expired messages are identified in this delivery path, they are added to a queue for batch deletion. The following property in the broker.xml file specifies the time gap (in seconds) between batch deletes in the delivery path.

   <preDeliveryExpiryDeletionInterval>10</preDeliveryExpiryDeletionInterval>