Setting up Multi Receiver and Load Balancing Data Agent

You can send events to multiple DAS/CEP receivers, either by sending the same event to many DAS/CEP receivers or by load balancing events among a set of servers. This handles the fail-over problem. When events are load balanced within a set of servers and if one receiver cannot be reached, events are automatically sent to the other available and active DAS/CEP receivers.

The following scenarios are covered in this section.

All the scenarios described below are different ways to use Data Agents with multiple receivers using the load balancing functionality. Each approach has its own advantages. Select the most appropriate scenario depending on your requirements.

Load balancing configurations

The load balancing configurations that can be used when sending events to multiple DAS/CEP receivers are as follows.

Load balancing events to a set of servers

This setup shows load balancing the event to publish it to all three DAS/CEP receivers. The load balanced publishing is done in a Round Robin manner where each event is sent to each receiver in a circular order without any priority.

It also handles fail-over cases. e.g., If DAS/CEP Receiver-1 is marked as inactive, then the Data Agent sends the data only to DAS/CEP Receiver-2 and DAS/CEP Receiver-3 in a Round Robin manner. When DAS/CEP Receiver-1 becomes active again after some time, the Data Agent automatically detects it, adds it to the operation, and starts to load balance between all three receivers again. This functionality significantly reduces the loss of data and provides more concurrency.

For this functionality, include the server URL in the Data Agent as a general DAS/CEP receiver URL. The URL should be entered in a comma separated format as shown below.

Receiver URL = tcp://<DAS/CEP Receiver -1>:<port>,tcp://<DAS/CEP Receiver -2>:<port>,tcp://<DAS/CEP Receiver -3>:<port>

In the above format, <DAS/CEP Receiver – 1, 2, 3> can be either host names or IP addresses, and <port> is the port of the corresponding DAS/CEP receiver.

e.g., tcp://10.100.2.32:7611, tcp://10.100.2.33:7611, tcp://10.100.2.34:7611

Load balancing events to sets of servers

In this setup there are two sets of servers that are referred to as set-A and set-B. You can send events to both the sets. You can also carry out load balancing for both sets as mentioned in load balancing between a set of servers. This scenario is a combination of load balancing between a set of servers and sending an event to several receivers. An event is sent to both set-A and set-B. Within set-A, it will be sent either to DAS/CEP ReceiverA-1 or DAS/CEP ReceiverA-2. Similarly within set-B, it will be sent either to DAS/CEP ReceiverB-1 or DAS/CEP ReceiverB-2. In the setup, you can have any number of sets and any number of servers as required by mentioning them accurately in the server URL.

Similar to the other scenarios, you can describe this as a receiver URL. The sets should be mentioned within curly braces separated by commas. Further more, each receiver that belongs to the set, should be within the curly braces and with the receiver URLs in a comma separated format. The receiver URL format is given below.

 Receiver URL = {tcp://<DAS/CEP Receiver -A-1>:<port>, tcp://<DAS/CEP Receiver -A-2>:<port>},{tcp://<DAS/CEP Receiver -B-1>:<port>, tcp://<DAS/CEP Receiver -B-2>:<port>}

<DAS/CEP Receiver– (A-1, 2)> and <DAS/CEP Receiver–B-(1, 2)> can be host name or IP addresses, and <port> is the port of the corresponding DAS/CEP receiver.

e.g., {tcp://10.100.2.32:7611, tcp://10.100.2.33:7611}, {tcp://10.100.2.34:7611, tcp://10.100.2.35:7611}

Sending all the events to several receivers

This setup involves sending all the events to more than one DAS/CEP receiver. This approach is mainly followed when you use other servers to analyze events together with DAS/CEP servers. For example, you can use the same Data Agents to publish the events to WSO2 CEP. You can use this functionality to publish the same event to both DAS and CEP servers at the same time. This is useful to perform real time analytics with CEP, to persist the data, and also to perform complex analysis with DAS in nearly real time with the same data.

Similar to load balancing between a set of servers, in this scenario you need to modify the Data Agent URL. You should include all DAS/CEP receiver URLs within curly braces ({}) separated with commas as shown below.

Receiver URL = {tcp://<DAS/CEP Receiver -1>:<port>}, {tcp://<DAS/CEP Receiver -2>:<port>}, {tcp://<DAS/CEP Receiver -3>:<port>}

<DAS/CEP Receiver – 1, 2, 3> can be either host name or IP addresses, and <port> is the port of the corresponding DAS/CEP receiver.

e.g., {tcp://10.100.2.32:7611},{ tcp://10.100.2.33:7611}, {tcp://10.100.2.34:7611}

Failover configuration

When using the failover configuration in publishing events to DAS/CEP, events are sent to multiple DAS/CEP receivers in a sequential order based on priority. You can specify multiple DAS/CEP receivers so that events can be sent to the next server in the sequence in a situation where they were not successfully sent to the first server. In the scenario depicted in the above image, the events are first sent to DAS/CEP Receiver-1. If it is unavailable, then events will be sent to DAS/CEP Receiver-2. If DAS/CEP Receiver-2 is also unavailable, then the events will be sent to DAS/CEP Receiver-3.

For this functionality, include the server URLs in the Data Agent, separated by the vertical bar (|) symbols as follows.

Receiver URL = tcp://<DAS/CEP Receiver -1>:<port>|tcp://<DAS/CEP Receiver -2>:<port>|tcp://<DAS/CEP Receiver -3>:<port>
tcp://localhost:7611|tcp://localhost:7612

In the above format, <DAS/CEP Receiver – 1, 2, 3> can be either host names or IP addresses, and <port> is the port of the corresponding DAS/CEP receiver.

e.g., tcp://10.100.2.32:7611| tcp://10.100.2.33:7611| tcp://10.100.2.34:7611