The WSO2 Elastic Load Balancer is built using Apache Tribes, which is the group management framework used by Apache Tomcat as well as Apache Axis2, Apache Axis2 clustering module, Apache Synapse - one of the best performant mediation frameworks, and the autoscaling component from the award-winning WSO2 Carbon framework. The diagram below depicts its high-level architectural view.
Figure: The WSO2 ELB Architectural Components
WSO2 Carbon Auto-Scaling
Keeps track of the traffic to each Cloud Service cluster, and makes decisions related to scaling the system up and down depending on the load fluctuation. It is also responsible for keeping a sanity check on the system. This sanity check ensures that the minimum system configuration, such as the minimum number of running instances of each Service and load balancer cluster is maintained at all times.
Synapse Mediation Framework
A Synapse endpoint by the name "ServiceDynamicLoadBalanceEndpoint" routes messages to the appropriate node.
First, the cloud service cluster to which the message has to be sent is identified using the Host HTTP header. Then a member from that service cluster to which the message has to be routed to, is selected according to the load balancing algorithm specified, and the message is sent to that member. The response from that member is sent back to the client, which originated the request. There is the possibility of a member failing while the LB is trying to forward a request to it. In that case, "ServiceLoadBalanceEndpoint" will try to fail-over to another available member if possible. A failed member will be suspended for a specified period.
This endpoint is also responsible for handling sticky sessions. When a request comes in, it first checks whether there is sticky session created for that client. If found, the request will be forwarded to the relevant member. Sticky sessions are identified using the value of the JSESSIONID cookie.
Binary Relay
Binary relay or message relay is the message pass-through mechanism used in the WSO2 Elastic Load Balancer, which enables the LB to pass messages without building or processing the them, ensuring minimum overhead.
From WSO2 ELB 2.0.1 onwards, Pass-Through transport is used, which has introduced significant performance efficiency. The pass-through transport supports extremely high-speed message relaying at the transport level. It uses the high-performing HTTPCore-NIO library and Java NIO concepts to achieve a high throughput specially in direct streaming scenarios.
Apache Axis2 Clustering
Apache Axis2 Clustering module handles membership discovery/management and service membership. There are multiple membership handlers, one for each clustering domain, responsible for managing membership. The "ServiceDynamicLoadBalanceEndpoint" obtains the relevant members of the cluster from the Axis2 membership handlers, and then applys the load balance algorithm to determine the next member to which the request has to be sent.
The Axis2 clustering implementation uses the underlying Apache Tribes Group Management Framework for group membership management.