Setting up Multi Receiver and Load Balancing Data Agent

You can send events to multiple BAM/CEP receivers, either by sending the same event to many BAM/CEP receivers or by load balancing events among a set of servers. This handles the fail over problem. When events are load balanced within a set of servers, if one receiver becomes unreachable, events will be automatically sent to the other available and active BAM receivers.

The following scenarios are covered in this section.

All the scenarios described below are different ways to use Data Agents with multiple receivers using the load balancing functionality. They have their own advantages and you need to select the most appropriate scenario depending on your requirements.

Load balancing configurations

The load balancing configurations that can be used when sending events to multiple BAM/CEP receivers are as follows.

Load balancing events to a set of servers

This setup shows load balancing the event to publish it to all three BAM receivers. The load balanced publishing is done in a Round Robin manner, sending each event to each receiver in a circular order without any priority. It also handles fail over cases such as, if BAM Receiver-1 is marked as down, then the Data Agent will send the data only to BAM Receiver-2 and BAM Receiver-3 in a round robin manner. When BAM Receiver-1 becomes active after some time, the Data Agent will automatically detect it, add it to the operation and will again start to load balance between all three receivers. This functionality significantly reduces the data loss, and provides more concurrency.

For this functionality, include the server URL in the Data Agent, as a general BAM receiver URL in comma separated value format as follows.

Receiver URL = tcp://<BAM Receiver -1>:<port>,tcp://<BAM Receiver -2>:<port>,tcp://<BAM Receiver -3>:<port>

In the above format, <BAM Receiver – 1, 2, 3> can be either host names or IP addresses, and <port> is the port of the corresponding BAM receiver.

For example: tcp://10.100.2.32:7611, tcp://10.100.2.33:7611, tcp://10.100.2.34:7611

Load balancing events to sets of servers

In this setup there are two sets of servers as set-A and set-B. You can send events to both the sets, and you can do load balancing as mentioned in load balancing between a set of servers. This scenario is a combination of load balancing between a set of servers and sending an event to several receivers. An event will be sent to both set-A and set-B. Within set-A, it will be sent either to BAM ReceiverA-1 or BAM ReceiverA-2. Similarly within set-B, it will be sent either to BAM ReceiverB-1 or BAM ReceiverB-2. In the setup, you can have any number of sets and any number of servers as required, mentioning them accurately in the server URL.

Similar to the other scenarios, you can describe this as a Receiver URL. The sets should be mentioned within curly braces separated by commas. Further more, each receiver that belongs to the set, should be within the curly braces and with the receiver URLs in comma separated format. The receiver URL format is given below.

 Receiver URL = {tcp://<BAM Receiver -A-1>:<port>, tcp://<BAM Receiver -A-2>:<port>},{tcp://<BAM Receiver -B-1>:<port>, tcp://<BAM Receiver -B-2>:<port>}

<BAM Receiver– (A-1, 2)> and <BAM Receiver–B-(1, 2)> can be host name or IP addresses, and <port> is the port of the corresponding BAM receiver.

For example: {tcp://10.100.2.32:7611, tcp://10.100.2.33:7611}, {tcp://10.100.2.34:7611, tcp://10.100.2.35:7611}

Sending all the events to several receivers

This setup shows sending all the events to more than one BAM receiver. That is mainly used, where you use other servers to analyze events together with BAM servers. For example, you can use the same Data Agents to publish the events to WSO2 CEP also. You can use this functionality to publish the same event to both BAM and CEP servers at the same time. This will be useful to perform real time analytics with CEP and to persist the data, and also to perform complex analysis with BAM in nearly real time with the same data.

Similar to load balancing between a set of servers, in this scenario you need to modify the Data Agent URL. You should include all BAM/CEP receiver URLs within curly braces ({}) separated with commas according to the following pattern.

Receiver URL = {tcp://<BAM Receiver -1>:<port>}, {tcp://<BAM Receiver -2>:<port>}, {tcp://<BAM Receiver -3>:<port>}

<BAM Receiver – 1, 2, 3> can be either host name or IP addresses, and <port> is the port of the corresponding BAM receiver.

For example: {tcp://10.100.2.32:7611},{ tcp://10.100.2.33:7611}, {tcp://10.100.2.34:7611}

Failover configuration

As per now this is only supported by BAM mediator, and other data agents doesn't include this feature.

When using the failover configuration in publishing events to BAM/CEP, events will be sent to multiple BAM/CEP receivers in a prioritized manner. You can specify multiple BAM/CEP receivers so that if sending to the the first server failed, then the events will be sent to the server, which is specified next in the sequence. In the scenario depicted in the above image, first events are sent to BAM Receiver-1. If it is unavailable, then events will be sent to BAM Receiver-2. Further, if that is also available, then events will be sent to BAM Receiver-3.

For this functionality, include the server URLs in the Data Agent, separated by the vertical bar (|) symbols as follows.

Receiver URL = tcp://<BAM Receiver -1>:<port>|tcp://<BAM Receiver -2>:<port>|tcp://<BAM Receiver -3>:<port>
tcp://localhost:7611|tcp://localhost:7612

In the above format, <BAM Receiver – 1, 2, 3> can be either host names or IP addresses, and <port> is the port of the corresponding BAM receiver.

For example: tcp://10.100.2.32:7611| tcp://10.100.2.33:7611| tcp://10.100.2.34:7611