ELB Architecture

The WSO2 Elastic Load Balancer is built using Apache Tribes, which is the group management framework used by Apache Tomcat as well as Apache Axis2, Apache Axis2 clustering module, Apache Synapse - one of the best performant mediation frameworks, and the autoscaling component from the award-winning WSO2 Carbon framework. The diagram below depicts its high-level architectural view.

Figure: The WSO2 ELB Architectural Components

WSO2 Carbon Auto-Scaling

Keeps track of the traffic to each Cloud Service cluster, and makes decisions related to scaling the system up and down depending on the load fluctuation.

It is also responsible for keeping a sanity check on the system. This sanity check ensures that the minimum system configuration, such as the minimum number of service instances specified in the loadbalancer.conf file, is maintained at all times.  If not, it automatically calls auto-scaler service and spawns instances to match the minimum count of each service domain.

Synapse Mediation Framework

A Synapse endpoint by the name "ServiceDynamicLoadBalanceEndpoint" routes messages to the appropriate node.

First, the cloud service cluster to which the message has to be sent is identified using the Host HTTP header. Then a member from that service cluster to which the message has to be routed to, is selected according to the load balancing algorithm specified, and the message is sent to that member. The response from that member is sent back to the client, which originated the request. There is the possibility of a member failing while the LB is trying to forward a request to it. In that case, "ServiceLoadBalanceEndpoint" will try to fail-over to another available member if possible. A failed member will be suspended for a specified period.

This endpoint is also responsible for handling sticky sessions. When a request comes in, it first checks whether there is sticky session created for that client. If found, the request will be forwarded to the relevant member. Sticky sessions are identified using the value of the JSESSIONID cookie.

Binary Relay

Binary-Relay (or Message-Relay) is a term used to describe a type of behavior which enables Apache Synapse to "relay" messages to a different party, efficiently at byte level, while making decisions using transport headers. Often, load balancing scenarios operate solely on transport headers. In that case, binary relay allows Apache Synapse to process those messages without processing content of the messages, which could yield significant performance gains. Binary relay is the message pass-through mechanism used in the WSO2 Elastic Load Balancer, which enables the LB to pass messages without building or processing the them, ensuring minimum overhead on the load balancer.

Binary-Relay can be selectively enabled for different content types. For instance, you may enable Relay only for text/html and let the other content types ( e.g. text/ xml) to be processed as usual.

Also, from WSO2 ELB 2.0.1 onwards, Pass-Through transport is used, for better efficiency. The pass-through transport supports extremely high-speed message relaying at the transport level. It uses the high-performing HTTPCore-NIO library and Java NIO concepts to achieve a high throughput specially in direct streaming scenarios.

Apache Axis2 Clustering

Apache Axis2 Clustering module handles membership discovery/management and service membership. There are multiple membership handlers, one for each clustering domain, responsible for managing membership. The "ServiceDynamicLoadBalanceEndpoint" obtains the relevant members of the cluster from the Axis2 membership handlers, and then applys the load balance algorithm to determine the next member to which the request has to be sent.

The Axis2 clustering implementation uses the underlying Apache Tribes Group Management Framework for group membership management.