Page Comparison

A WSO2 ESB cluster should contain two or more ESB instances that are configured to run within the same domain. To make an instance a member of the cluster, you must configure it to either of the available membership schemes:

Well Known Address (WKA) membership scheme
Multicast membership scheme

In this example, we will be using the WKA membership scheme, and the ELB will act as the well-known member in the cluster. It will accept all the service requests on behalf of the ESBs and divide the load among worker nodes in the ESB cluster.

This page describes how to create an ESB cluster with an ELB front end in the following sections:a cluster by walking through the example in the Overview. Although this example uses WSO2 ESB, these steps apply to other WSO2 products as well. For details on further configuration required for the WSO2 product you are clustering, see the links in the table of contents.

Table of Contents

Installing the products

Before you begin, download and extract WSO2 ESB and WSO2 ELB to a local directory on the sever. For this example, we have extracted one copy of the ELB and two copies of the ESB on the server with IP xxx.xxx.xxx.206 (the x's represent your actual IP prefix), and we extracted two copies of the ESB on the server with the IP xxx.xxx.xxx.132:

...

2 ESB instances (worker nodes)

Configuring the load balancer

You configure the ELB with the overall definition of the cluster and how it should distribute the load. You can achieve this by adding a few lines to a configuration file called loadbalancer.conf. You specify the detailed clustering configurations in the axis2.xml file. This section describes how to perform these steps.

Setting up load-balancing configurations

Open the <ELB_HOME>/repository/conf/loadbalancer.conf file.

Locate the ESB configuration and edit it as follows:

Code Block

language	html/xml

esb {
        domains   {
            wso2.esb.domain {
                hosts esb.cloud-test.wso2.com;
                sub_domain worker;
                tenant_range    *;
            }
        }
    }

...

In summary, we have configured the load balancer to handle requests sent to esb.cloud-test.wso2.com and to distribute the load among the worker nodes in the worker sub-domain of the wso2.esb.domain cluster. We are now ready to set up the cluster configurations.

Setting up cluster configurations on the ELB

Previously, we configured several properties of the cluster such as domain name and sub-domain, but we didn’t define them there. We now define these properties as we build the cluster.

...

We have now completed the clustering-related configuration for the ELB. In the next section, we will make one last change to the ELB that will increase usability.

Configuring the ELB to listen on default ports

We will now change the ELB configuration to listen to the default HTTP and HTTPS ports.

...

In the next section, we will map the host names we specified to real IPs.

Mapping the host name to the IP

In the ELB, we configured a host name in loadbalancer.conf to front the worker service requests. We must now map this host name ( esb.cloud-test.wso2.com ) to the actual IP address. Open the server's /etc/hosts file and add the following line, where <ELP-IP> is the actual IP address:

...

We have now finished configuring the ELB and ready to start the ELB server.

Starting the ELB server

Start the ELB server by typing the following command in the terminal:

...

Info
If you skipped the step of configuring the ELB to listen on the default ports, you do not need to use the `sudo` command and can just start the ELB with the following command: `sh <ELB_HOME>/bin/wso2server.sh`

...

Now that the ELB is configured and running, you create a central database for all the nodes to use.

Setting up the central database

Each Carbon-based product uses a database to store information such as user management details and registry data. All nodes in the cluster must use one central database.

...

We have now created a central database called carbondb with host carbondb.mysql-wso2.com, and with permission for user username with password password. The next step is to configure the manager node.

Configuring the manager node

In this section, we will configure data sources to allow the manager node to point to the central database, enable the manager node for clustering, change the port offset, and map the host names to IPs.

Configuring the data source

Make sure that you have copied the MySQL JDBC driver JAR to the manager node as described in Creating an ESB Cluster Setting up the central database.
Open the master-datasources.xml file located in the <ESB_MANAGER_HOME>/repository/conf/datasources/ directory.
Locate the WSO2_CARBON_DB data source configurations and change them as follows:

...

Info
In most WSO2 products, only one data source is used. If there is more than one data source, make sure they reference the central databases accordingly. For example, the API Manager deployment setup requires more specific data source configurations.

Setting up cluster configurations for the manager node

Configuring clustering for the manager node is similar to the way you configured it for the ELB node, but the localMemberPort is 4001 instead of 4000, and you define the ELB node instead of the ESB manager node as the well-known member.

...

- Enable clustering for this node: <clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
- Set the membership scheme to wka to enable the Well Known Address registration method (this node will send cluster initiation messages to WKA members that we will define later): <parameter name="membershipScheme">wka</parameter>
- Specify the name of the cluster this node will join: <parameter name="domain">wso2.esb.domain</parameter>
- Specify the port used to communicate cluster messages: <parameter name="localMemberPort">4001</parameter>
  Note: This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server.
- Clear the members from the <members> element so that it is now empty:
  Code Block
  language html/xml
  <members> </members>

Configuring the port offset and host name

Because we are running two Carbon-based products on the same server, we must change the port offset to avoid port conflicts. Additionally, we will add the cluster host name so that any requests sent to the manager host are redirected to the cluster, where the ELB will pick them up and manage them.

...

In the next section, we will map the host names we specified to real IPs.

Mapping host names to IPs

In the manager node we have specified two host names: carbondb.mysql-wso2.com for the MySQL server and esb.cloud-test.wso2.com for the cluster. We will now map them to the actual IPs. Note that if you created the database on the same server as the manager node, you will have already added the first line, and if you created it on the same server as the ELB, you will have already added the second line.

...

We have now finished configuring the manager node and are ready to start the ESB server.

Starting the ESB server

Start the ESB server by typing the following command in the terminal:

...

We have now finished configuring the manager node. Next, we will configure the ESB worker nodes.

Configuring the worker nodes

You configure worker nodes in very much the same way as you configured the manager node. Be sure to follow these steps for each worker node in the cluster.

Configuring the data source

You configure the data source to connect to the central database. Because the data source changes are the same for all worker nodes, you can configure this file on one node and then copy it to the other worker nodes.

Make sure that you have copied the MySQL JDBC driver JAR to each worker node as described in Creating an ESB Cluster Setting up the central database.
Open the master-datasources.xml file located in the <ESB_WORKER_HOME>/repository/conf/datasources/ directory.
Locate the WSO2_CARBON_DB data source configurations and change them as follows:

...

After you have finished configuring the data source, be sure to copy this configuration to the other worker nodes in the cluster.

Setting up cluster configurations for the worker nodes

Configuring clustering for the worker nodes is similar to the way you configured it for the manager node, but the localMemberPort will vary for each worker node, you add the subDomain property, and you add the ELB and ESB manager node to the well-known members, as described in the following steps.

...

Enable clustering for this node: <clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
Set the membership scheme to wka to enable the Well Known Address registration method (this node will send cluster initiation messages to WKA members that we will define later): <parameter name="membershipScheme">wka</parameter>
Specify the name of the cluster this node will join: <parameter name="domain">wso2.esb.domain</parameter>
Specify the port used to communicate cluster messages (if this node is on the same server as the ELB, manager node, or another worker node, be sure to set this to a unique value, such as 4000 and 4001 for worker nodes 1 and 3 and 4002 for worker node 2, which is on the same server as the ELB and manager node): <parameter name="localMemberPort">4000</parameter>
Note: This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server.
Define the sub-domain as worker by adding the following property under the <parameter name="properties"> element: <property name="subDomain" value="worker"/>

Define the ELB and manager nodes as well-known members of the cluster by providing their host name and localMemberPort values. The manager node is defined here because it is required for the Deployment Synchronizer to function.

Code Block

language	html/xml

<members>
    <member>
        <hostName>elb.wso2.com</hostName>
        <port>4000</port>
    </member>
    <member>
        <hostName>mgt.esb.wso2.com</hostName>
        <port>4001</port>
    </member>
</members>

Adjusting the port offset

Because we are running two Carbon-based products on the same server, we must change the port offset to avoid port conflicts.

...

In the next section, we will map the host names we specified to real IPs.

Mapping host names to IPs

In the worker nodes, we have used three hosts names: carbondb.mysql-wso2.com for the MySQL server, elb.wso2.com for the ELB, and mgr.esb.wso2.com for the ESB manager node. We will now map them to the actual IPs.

...

We have now finished configuring the worker nodes and are ready to start them.

Starting the ESB server

Start the ESB server by typing the following command in the terminal:

...

If you have similar messages in your consoles, you have finished configuring the worker nodes and the cluster is running. When you terminate one node, all nodes identify that the node has left the cluster. The same applies when a new node joins the cluster. If you want to add another new worker node, you can simply copy worker1 without any changes if you are running it on a new server (such as xxx.xxx.xxx.184). If you intend to use the new node on a server where another WSO2 product is running, you can use a copy of worker1 and change the port offset accordingly in the carbon.xml file. You may also have to change localMemberPort in axis2.xml if that product has clustering enabled. Be sure to map all host names to the relevant IP addresses when creating a new node.

Testing the cluster

To test the cluster, open the ESB management console on the manager node (use the management console URL displayed in the terminal when you started the node), add a sample proxy service with the log mediator in the inSequence so that logs will be displayed in the worker terminals, and then observe the cluster messages sent from the manager node to the workers.

...

Versions Compared

Old Version 9

New Version 10

Key

Installing the products

Configuring the load balancer

Setting up load-balancing configurations

Setting up cluster configurations on the ELB

Configuring the ELB to listen on default ports

Mapping the host name to the IP

Starting the ELB server

Setting up the central database

Configuring the manager node

Configuring the data source

Setting up cluster configurations for the manager node

Configuring the port offset and host name

Mapping host names to IPs

Starting the ESB server

Configuring the worker nodes

Configuring the data source

Setting up cluster configurations for the worker nodes

Adjusting the port offset

Mapping host names to IPs

Starting the ESB server

Testing the cluster