Message Brokering

WSO2 Enterprise Integrator (WSO2 EI) is shipped with a separate runtime for message brokering (EI Message Broker profile). This EI Message Broker profile is an instance of the WSO2 Message Broker product; thereby, it contains all the functionality that was available with WSO2 MB.When you use WSO2 EI for integration, you can easily configure the Message Broker runtime along with the Integrator runtime to achieve guaranteed delivery and reliable messaging. Further, you can also run the Message Broker profile in WSO2 EI to facilitate message brokering using Topics and Queues.

Key concepts of message brokering

Explained below are some of the key elements and capabilities of the EI Message Broker profile in WSO2 EI.

Queues

Queues are essentially message stores, which can store messages from external JMS clients (publishers) and maintain them in an intermediate state until they are consumed by other JMS clients (subscribers). See Managing Queues for more information.

Topics

Topics are used for realtime message brokering. As depicted by the following diagram, you can maintain a hierarchy of topics or subjects in the broker for which messages can be published by external JMS clients (publishers). Other JMS clients (subscribers) can subscribe to the topics of interest and receive the messages published to the topic. See Managing Topics and Sub Topics for more information.

Subscribers and Publishers

A JMS client that is configured to send messages to a queue or topic is known as a publisher. A JMS client that is configured to receive messages published to a particular queue or topic is known as a subscriber. See JMS Subscribers and Publishers for more information.

Role-based permissions

The possibility to work with queues and topics in the broker is strictly secured by role-based permissions. That is, users created in the broker runtime should be granted the required permissions in order to publish messages to queues/topics and to consume the messages that are published. External clients will then be able to use the credentials of these authorized users in order to send messages and consume messages.

Metrics monitoring

You can effectively monitor the performance and statistics of your broker runtime using Java Metrics.

Management Console

The management console of EI Message Broker is the user interface of the product, which can be used to conveniently create topics and queues and to manage subscriptions. External clients can then connect to the broker for publishing messages and consuming already published messages.

The slot-based architecture of the EI Message Broker

The slot-based message delivery system is designed to enable global queues to be shared among the nodes in a broker cluster. A queue is mapped to a row in a message store and it can be divided into many slots. A slot is a chunk of messages in the row that can be owned by one broker node at a given time. 

The slot manager communicates with both publishers and subscribers and acts as the coordinator for distributing slots among the nodes. A slot assignment map is maintained to track the slots that are assigned to nodes at a given time. The activities of a slot manager can be illustrated as follows.

Communicating with publishers

Each publisher belongs to a queue/topic and its messages are published in the row of the message store mapped to this queue. A publisher returns its last message ID to the slot manager node in the broker cluster after every 1000 messages or after a timeout. The slot manager updates the Hazelcast Distributed Map with these IDs and uses them to generate slots when it receives requests for slots from slot delivery worker nodes.

Assigning slots

When a client subscribes to a queue/topic, a slot delivery worker requests for a slot. Then the slot manager first looks for returned slots (i.e. slots that were previously assigned to another subscriber node which has left the cluster) and assigns one of them if any are available. If there are no such slots, an empty slot (i.e. a slot with no messages currently published in it) is generated and assigned to the slot delivery worker.

Deleting a slot

Once a subscriber node has sent all the messages it has read from a slot and received acknowledgments, it sends a request to the slot manager to delete the slot. The slot manager removes the relevant entry from the slot assignment map once it receives the request to delete.

Reassigning slot when the last subscriber leaves

If a subscriber node to which a slot is assigned leaves the cluster, the slot manager reassigns the slot to another subscriber node to free the slot pool. 

Delivering messages to subscribers

A slot delivery worker reads all the messages published in a slot assigned to it and passes them to the message flusher, which delivers them to subscribers who have subscribed to the relevant queue/topic in a Round Robin manner. Messages that were not delivered due to a delivery failure and messages rejected by the subscriber are buffered queue-wise in the Message Flusher.