Load Balancing the ESB Data Published to Analytics
- Former user (Deleted)
You can send events to multiple Analytcis receivers, either by sending the same event to many Analytcis receivers or by load balancing events among a set of servers. This handles the fail-over problem. When events are load balanced within a set of servers and if one receiver cannot be reached, events are automatically sent to the other available and active Analytcis receivers.
The following scenarios are covered in this section.
All the scenarios described below are different ways to use Data Agents with multiple receivers using the load balancing functionality. Each approach has its own advantages. Select the most appropriate scenario depending on your requirements.
Load balancing configurations
The load balancing configurations that can be used when sending events to multiple Analytcis receivers are as follows.
Load balancing events to a set of serversÂ
This setup shows load balancing the event to publish it to all three Analytcis receivers. The load balanced publishing is done in a Round Robin manner, sending each event to each receiver in a circular order without any priority. It also handles fail-over cases such as, if Analytcis Receiver-1 is marked as down, then the Data Agent will send the data only to Analytcis Receiver-2 and Analytcis Receiver-3 in a round robin manner. When Analytcis Receiver-1 becomes active after some time, the Data Agent automatically detects it, adds it to the operation, and again starts to load balance between all three receivers. This functionality significantly reduces the loss of data and provides more concurrency.
For this functionality, include the server URL in the Data Agent as a general Analytcis receiver URL. The URL should be entered in a comma-separated format as shown below.
Receiver URL = tcp://<Analytics Receiver -1>:<port>,tcp://<Analytics Receiver -2>:<port>,tcp://<Analytics Receiver -3>:<port>
In the above format, <Analytics Receiver – 1, 2, 3> can be either host names or IP addresses, and <port> is the port of the corresponding Analytcis receiver.
E.g., tcp://10.100.2.32:7611, tcp://10.100.2.33:7611, tcp://10.100.2.34:7611
Load balancing events to sets of servers Â
In this setup there are two sets of servers that are referred to as set-A and set-B. You can send events to both the sets. You can also carry out load balancing for both sets as mentioned in load balancing between a set of servers. This scenario is a combination of load balancing between a set of servers and sending an event to several receivers. An event is sent to both set-A and set-B. Within set-A, it will be sent either to Analytcis ReceiverA-1 or Analytcis ReceiverA-2. Similarly within set-B, it will be sent either to Analytcis ReceiverB-1 or Analytcis ReceiverB-2. In the setup, you can have any number of sets and any number of servers as required by mentioning them accurately in the server URL.
Similar to the other scenarios, you can describe this as a receiver URL. The sets should be mentioned within curly braces separated by commas. Further more, each receiver that belongs to the set, should be within the curly braces and with the receiver URLs in a comma separated format. The receiver URL format is given below.
Receiver URL = {tcp://<Analytics Receiver -A-1>:<port>, tcp://<Analytics Receiver -A-2>:<port>},{tcp://<Analytics Receiver -B-1>:<port>, tcp://<Analytics Receiver -B-2>:<port>}
<Analytics Receiver– (A-1, 2)> and <Analytics Receiver–B-(1, 2)> can be host name or IP addresses, and <port> is the port of the corresponding Analytcis receiver.
E.g., {tcp://10.100.2.32:7611, tcp://10.100.2.33:7611}, {tcp://10.100.2.34:7611, tcp://10.100.2.35:7611}
Sending all the events to several receiversÂ
This setup involves sending all the events to more than one Analytcis receiver. This approach is mainly followed when you use other servers to analyze events together with Analytcis servers. For example, you can use the same Data Agents to publish the events to WSO2 EI. You can use this functionality to publish the same event to multiple EI servers at the same time. This is useful to perform real time analytics with EI Analytics, to persist the data, and also to perform complex analysis with Analytcis in nearly real time with the same data.
Similar to load balancing between a set of servers, in this scenario you need to modify the Data Agent URL. You should include all Analytcis receiver URLs within curly braces ({}) separated with commas as shown below.
Receiver URL = {tcp://<Analytics Receiver -1>:<port>}, {tcp://<Analytics Receiver -2>:<port>}, {tcp://<Analytics Receiver -3>:<port>}
<Analytics Receiver – 1, 2, 3> can be either host name or IP addresses, and <port> is the port of the corresponding Analytcis receiver.
E.g., Â {tcp://10.100.2.32:7611},{Â tcp://10.100.2.33:7611}, {tcp://10.100.2.34:7611}
Failover configuration
When using the failover configuration in publishing events to Analytcis, events are sent to multiple Analytcis receivers in a sequential order based on priority. You can specify multiple Analytcis receivers so that events can be sent to the next server in the sequence in a situation where they were not successfully sent to the first server. In the scenario depicted in the above image, the events are first sent to Analytcis Receiver-1. If it is unavailable, then events will be sent to Analytcis Receiver-2. If Analytcis Receiver-2 is also unavailable, then the events will be sent to Analytcis Receiver-3.
For this functionality, include the server URLs in the Data Agent, separated by the vertical bar (|) symbols as follows.
Receiver URL = tcp://<Analytics Receiver -1>:<port>|tcp://<Analytics Receiver -2>:<port>|tcp://<Analytics Receiver -3>:<port> tcp://localhost:7611|tcp://localhost:7612
In the above format, <Analytics Receiver – 1, 2, 3> can be either host names or IP addresses, and <port> is the port of the corresponding Analytcis receiver.
E.g., tcp://10.100.2.32:7611|Â tcp://10.100.2.33:7611|Â tcp://10.100.2.34:7611