Page Comparison

This topic is based on the worker/manager clustering pattern you choose. Note that this page describes using WSO2 Elastic Load Balancer (ELB), but you can use a third-party load balancer in its place (for configuration details, see your load balancer's documentation).

For details on further configuration required for the WSO2 product you are clustering, see the links in the table of contents.

Table of Contents

maxLevel	3
minLevel	3

Installing the products

Before you begin, download and extract WSO2 ESB and WSO2 ELB to a local directory on the sever. For this example, we have extracted one copy of the ELB and two copies of the ESB on the server with IP xxx.xxx.xxx.206 (the x's represent your actual IP prefix), and we extracted two copies of the ESB on the server with the IP xxx.xxx.xxx.132:

Server xxx.xxx.xxx.206:

1 ELB instance (Well Known Member)
1 ESB instance (worker node)
1 ESB instance (Dep-sync management / manager node )

Server xxx.xxx.xxx.132:

2 ESB instances (worker nodes)

Configuring the load balancer

In this scenario, we are using WSO2 ELB for the load balancer (you can also use a third-party load balancer; for details on configuration, see your load balancer's documentation). You configure the ELB with the overall definition of the cluster and how it should distribute the load. You can achieve this by adding a few lines to a configuration file called loadbalancer.conf. You specify the detailed clustering configurations in the axis2.xml file. This section describes how to perform these steps.

Info
The system should have at least two Well-known Address (WKA) members in order to work correctly and to recover if a single WKA member fails. The member can either be another ELB or another manager or worker node.

Setting up load-balancing configurations

Open the <ELB_HOME>/repository/conf/loadbalancer.conf file.

Locate the ESB configuration and edit it as follows:

Code Block

language	html/xml

esb {
  domains{
     wso2.esb.domain {
        tenant_range *;
        group_mgt_port 5000;
        worker {
               hosts esb.cloud-test.wso2.com;
        }
     }
   }
}

In this file, we specified the domain name (wso2.esb.domain), which is used to identify the cluster. On startup, node that are configured with this domain name can join this cluster.

All the service requests need to be routed to the worker nodes through the ELB, which is the front end to the entire cluster. Therefore, we specify the worker sub-domain and use the hosts attribute to configure the publicly accessible host name (esb.cloud-test.wso2.com) that clients can use to send their requests to the cluster. We will map the host name to the ELB server IP address later.

For service-based products (such as WSO2 ESB, Data Services Server, and Application Server), the ELB creates one group management agent per cluster to manage the service groups. Because we are configuring an ESB cluster, we used the group_mgt_port attribute to specify the port for this cluster's group management agent. The port should be a unique value between 4000 and 5000. In scenarios where you are not using the ELB in front of the cluster, you configure the group management agent in the node's axis2.xml file instead as described in Group Management.

Finally, the tenant_range attribute is used to handle tenant-aware load-balancing, which is another very powerful feature of the ELB. This attribute allows us to partition tenants into several clusters, so that when there is a large number of tenants to work with, we can instruct each cluster to work only with a particular tenant or a few selected tenants. This approach is also useful if you need a particular cluster to be dedicated to work for a single special tenant ("Private Jet Mode"). Following are examples of the values you can specify for tenant_range:

1,6,3,4: Tenant IDs 1, 6, 3 and 4
1-3: Tenant IDs 1 to 3 (inclusive of both 1 and 3)
43: Tenant ID 43
*: All tenants

In this example, we are not enabling tenant partitioning, so we have used an asterisk ( * ) in front of tenant_range to represent all possible tenants.

When there is more than one load balancer fronting the cluster, these configurations change as follows:

Code Block
esb { domains{ wso2.esb.domain { tenant_range *; group_mgt_port 5000; members 127.0.0.1:6000; mgt { hosts mgt.esb.wso2.com; } worker { hosts esb.wso2.com; } } } }

Info
Note that 5000 is the `group_mgt_port` value used only for ESB. This value needs to be different for different products.

In summary, we have configured the load balancer to handle requests sent to esb.cloud-test.wso2.com and to distribute the load among the worker nodes in the worker sub-domain of the wso2.esb.domain cluster. We are now ready to set up the cluster configurations.

Setting up cluster configurations on the ELB

Previously, we configured several properties of the cluster such as domain name and sub-domain, but we didn’t define them there. We now define these properties as we build the cluster.

Open the <ELB_HOME>/repository/conf/axis2/axis2.xml file.
Locate the Clustering section and verify or configure the properties as follows (some of these properties are already set correctly by default):

- Enable clustering for this node: <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
- Set the membership scheme to wka to enable the Well Known Address registration method (this node will send cluster initiation messages to WKA members that we will define later): <parameter name="membershipScheme">wka</parameter>
- Specify a domain name for the ELB node (note that this domain it for potentially creating a cluster of ELB nodes and is not the cluster of ESB nodes that the ELB will load balance):
  <parameter name="domain">wso2.carbon.lb.domain</parameter>
- Specify the port used to communicate with this ELB node: <parameter name="localMemberPort">4000</parameter>
  Note: This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server.

Info

In the case of multiple load balancers fronting the cluster, you need to add members to the axis2.xml file.

Code Block

language	html/xml

<members> 
      <member> 
            <hostName>127.0.0.1</hostName> 
           <port>4200</port> 
      </member> 
</members>

Here, the second ELB runs in localhost and port 4200 is the localMemberPort (found in the axis2.xml file) of the second ELB. Note that 4200 is the port value used only for ESB. The port value needs to be different for different products.

Do this configuration for both the ELBs. This configuration is done primarily to make both ELBs aware of each other. This is exceptionally helpful in scenarios when one ELB restarts and it would need to get the existing cluster information to serve requests again. To do that, it would need to communicate with the other ELB.

We have now completed the clustering-related configuration for the ELB. In the next section, we will make one last change to the ELB that will increase usability.

Configuring the ELB to listen on default ports

We will now change the ELB configuration to listen to the default HTTP and HTTPS ports.

Open the <ELB_HOME>/repository/conf/axis2/axis2.xml file.
Locate the Transport Receiver section and configure the properties as follows:

- In the <transportReceiver name="http" class="org.apache.synapse.transport.passthru.PassThroughHttpListener"> transport, enable service requests to be sent to the ELB's default HTTP port instead of having to specify port 8280:
  <parameter name="port">80</parameter>
- In the <transportReceiver name="https" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLListener"> transport, enable service requests to be sent to the ELB's default HTTPS port instead of having to specify port 8243: <parameter name="port">443</parameter>

In the next section, we will map the host names we specified to real IPs.

Mapping the host name to the IP

In the ELB, we configured a host name in loadbalancer.conf to front the worker service requests. We must now map this host name (esb.cloud-test.wso2.com) to the actual IP address. Open the server's /etc/hosts file and add the following line, where <ELP-IP> is the actual IP address:

Code Block

language	none

<ELB-IP> esb.cloud-test.wso2.com

In this example, it would look like this:

Code Block

language	none

xxx.xxx.xxx.206 esb.cloud-test.wso2.com

We have now finished configuring the ELB and ready to start the ELB server.

Starting the ELB server

Start the ELB server by typing the following command in the terminal:

sudo -E sh <ELB_HOME>/bin/wso2server.sh

Info
If you skipped the step of configuring the ELB to listen on the default ports, you do not need to use the `sudo` command and can just start the ELB with the following command: `sh <ELB_HOME>/bin/wso2server.sh`

The ELB should print logs to the server console indicating that the cluster initialization is complete.

Now that the ELB is configured and running, you create a central database for all the nodes to use.

Setting up the database

Each Carbon-based product uses a database to store information such as user management details and registry data. All nodes in the cluster must use one central database.

Download and install MySQL server.
Download the MySQL jdbc driver.
Define the host name for configuring permissions for the new database by opening the /etc/hosts file and adding the following line:
Code Block
<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
Open a terminal/command window and log in to MySQL with the following command:
Code Block
mysql -u username -p
When prompted, specify the password, and then create the database with the following command:
Code Block
mysql> create database wso2conum_db;
Create a database schema and populate tables as follows. Make sure to replace <CARBON_HOME> with the absolute path of the WSO2 ELB directory.
Code Block
mysql> use wso2conum_db; mysql> source CARBON_HOME/dbscripts/mysql.sql;
Create another database which will be used as the shared governance and configuration registry database.
Code Block
mysql> create database wso2conreg_db;
Create registry database schema and populate tables as follows. Make sure to replace CARBON_HOME with the absolute path of wso2elb-2.1.0 directory
Code Block
mysql> use wso2conreg_db; mysql> source CARBON_HOME/dbscripts/mysql.sql;

Grant permission to access the created database with the following command:

Code Block
mysql> grant all on carbondb.* TO username@carbondb.mysql-wso2.com identified by "password";

Unzip the downloaded MySQL driver zipped archive and copy the MySQL JDBC driver JAR (mysql-connector-java-x.x.xx-bin.jar) to the <ESB_HOME>/repository/component/lib directory for each worker and manager node.

We have now created a central database called carbondb with host carbondb.mysql-wso2.com, and with permission for user username with password password.

To configure a user management database and shared registry database, edit <ESB_MGR_HOME>/repository/conf/datasoruces/master-datasources.xml as shown below:

Code Block

language	html/xml

<datasource>
	<name>WSO2_SHARED_REG_DB</name>
	<description>The datasource used for shared config and governance registry</description>
	<jndiConfig>
		<name>jdbc/WSO2SharedDB</name>
	</jndiConfig>
	<definition type="RDBMS">
		<configuration>
			<url>jdbc:mysql://localhost:3306/wso2conreg_db</url>
			<username>root</username>
			<password>root</password>
			<driverClassName>com.mysql.jdbc.Driver</driverClassName>
			<maxActive>50</maxActive>
			<maxWait>60000</maxWait>
			<testOnBorrow>true</testOnBorrow>
			<validationQuery>SELECT 1</validationQuery>
			<validationInterval>30000</validationInterval>
		</configuration>
	</definition>
</datasource>
<datasource>
	<name>WSO2_UM_DB</name>
	<description>The datasource used for registry and user manager</description>
	<jndiConfig>
		<name>jdbc/WSO2UmDB</name>
	</jndiConfig>
	<definition type="RDBMS">
		<configuration>
			<url>jdbc:mysql://localhost:3306/wso2conum_db</url>
			<username>root</username>
			<password>root</password>
			<driverClassName>com.mysql.jdbc.Driver</driverClassName>
			<maxActive>50</maxActive>
			<maxWait>60000</maxWait>
			<testOnBorrow>true</testOnBorrow>
			<validationQuery>SELECT 1</validationQuery>
			<validationInterval>30000</validationInterval>
		</configuration>
	</definition>
</datasource>

Info
Make sure to replace `username` and `password` with your MySQL database username and password.

To configure the datasource, update the dataSource property found in <ESB_MGR_HOME>/repository/conf/user-mgt.xml as shown below:

Code Block

language	html/xml

<Property name="dataSource">jdbc/WSO2UmDB</Property>

Configure the shared registry database and mounting details in <ESB_MGR_HOME>/repository/conf/registry.xml as follows:

Code Block

language	html/xml

<dbConfig name="sharedregistry">
	<dataSource>jdbc/WSO2SharedDB</dataSource>
</dbConfig>
<remoteInstance url="https://localhost:9443/registry">
	<id>instanceid</id>
	<dbConfig>sharedregistry</dbConfig>
	<readOnly>false</readOnly>
	<enableCache>true</enableCache>
	<registryRoot>/</registryRoot>
</remoteInstance>
<mount path="/_system/config" overwrite="true">
	<instanceId>instanceid</instanceId>
	<targetPath>/_system/esbnodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
	<instanceId>instanceid</instanceId>
	<targetPath>/_system/governance</targetPath>
</mount>

Now your database is set up. The next step is to configure the manager node.

Configuring the manager node

In this section, we will configure data sources to allow the manager node to point to the central database, enable the manager node for clustering, change the port offset, and map the host names to IPs.

Configuring the data source

Make sure that you have copied the MySQL JDBC driver JAR to the manager node as described in Setting up the central database.
Open the master-datasources.xml file located in the <ESB_MANAGER_HOME>/repository/conf/datasources/ directory.
Locate the WSO2_CARBON_DB data source configurations and change them as follows:

- Define the location of the central database:
  <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/carbondb?DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000</url>
- Give user username access to the database: <username>username</username> <password>password</password>
- Specify the driver to use for connecting to the central database (the driver we copied in the previous section): <driverClassName>com.mysql.jdbc.Driver</driverClassName>

When you are finished, the data source configuration should look like this:

Code Block

language	html/xml

<datasource>
   <name>WSO2_CARBON_DB</name>
   <description>The datasource used for registry and user manager</description>
   <jndiConfig>
       <name>jdbc/WSO2CarbonDB</name>
   </jndiConfig>
   <definition type="RDBMS">
       <configuration>
           <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/carbondb?DB_CLOSE_ON_EXIT=FALS E;LOCK_TIMEOUT=60000</url>
           <username>username</username>
           <password>password</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
       </configuration>
   </definition>
</datasource>

Info
In most WSO2 products, only one data source is used. If there is more than one data source, make sure they reference the central databases accordingly. For example, the API Manager deployment setup requires more specific data source configurations.

Setting up cluster configurations for the manager node

Configuring clustering for the manager node is similar to the way you configured it for the ELB node, but the localMemberPort is 4001 instead of 4000, and you define the ELB node instead of the ESB , see Creating a Worker/Manager Cluster.manager node as the well-known member.

Open the <ESB_HOME>/repository/conf/axis2/axis2.xml file.
Locate the Clustering section and verify or configure the properties as follows (some of these properties are already set correctly by default):

Enable clustering for this node:
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
Set the membership scheme to wka to enable the Well Known Address registration method (this node will send cluster initiation messages to WKA members that we will define later): <parameter name="membershipScheme">wka</parameter>
Specify the name of the cluster this node will join:
<parameter name="domain">wso2.esb.domain</parameter>
Specify the port used to communicate cluster messages:
<parameter name="localMemberPort">4001</parameter>
Note: This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server.

Add a new property "subDomain" and set it to "mgt" to denote that this node belongs to mgt subdomain of the cluster as defined in loadbalancer.conf.

Code Block

language	html/xml

<parameter name="properties">
            <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/>
            <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/>
            <property name="subDomain" value="mgt"/>
            <property name="port.mapping.8290" value="9763"/>
</parameter>

The receiver's http/https port values are without the portOffset addition; they get auto-incremented by portOffset. The 'WSDLEPRPrefix' parameter should point to the worker node's host name (esb.cloud-test.wso2.com) and ELB's http (8280)/https (8243) transport ports.

Change the members listed in the <members> element so that it is applicable for your worker/manager clustering deployment pattern:

Localtabgroup

Localtab

active	true
title	Worker/Manager Clustering Pattern 1

Clear the members from the <members> element so that it is now empty. This is done as members and load balancers are not necessary in this pattern.

Code Block

language	html/xml

<members>
   
</members>

Localtab

title	Worker/Manager Clustering Pattern 2 and 3

All other worker/manager clustering deployment patterns require members. Use the following as the configurations for the <members> element.

Code Block

language	html/xml

<members>
	<member>
		<hostName>127.0.0.1</hostName>
		<port>5000</port>
	</member>
</members>

Configuring the port offset and host name

Because we are running two Carbon-based products on the same server, we must change the port offset to avoid port conflicts. Additionally, we will add the cluster host name so that any requests sent to the manager host are redirected to the cluster, where the ELB will pick them up and manage them.

Open <ESB_MANAGER_HOME>/repository/conf/carbon.xml.
Locate the <Ports> tag and change the value of its sub-tag to:
<Offset>1</Offset>
Locate the <HOSTNAME> tag and add the cluster host name: <HostName>esb.cloud-test.wso2.com</HostName>
Locate the <MgtHostName> tag and uncomment it. Make sure that the management host name is defined as follows:<MgtHostName>mgt.wso2.org</MgtHostName>

In the next section, we will map the host names we specified to real IPs.

Mapping host names to IPs

In the manager node we have specified two host names: carbondb.mysql-wso2.com for the MySQL server and esb.cloud-test.wso2.com for the cluster. We will now map them to the actual IPs. Note that if you created the database on the same server as the manager node, you will have already added the first line, and if you created it on the same server as the ELB, you will have already added the second line.

Open the server's /etc/hosts file and add the following lines, where <MYSQL-DB-SERVER-IP> and <ELB-IP> are the actual IP addresses (in this example, xxx.xxx.xxx.206):

Code Block

language	none

<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
<ELB-IP> esb.cloud-test.wso2.com

In this example, it would look like this:

Code Block

language	none

xxx.xxx.xxx.206 carbondb.mysql-wso2.com
xxx.xxx.xxx.206 esb.cloud-test.wso2.com

We have now finished configuring the manager node and are ready to start the ESB server.

Starting the ESB server

Start the ESB server by typing the following command in the terminal:

Code Block
sh <ESB_MANAGER_HOME>/bin/wso2server.sh -Dsetup

The additional -Dsetup argument will clean the configurations, recreate the central database, and create the required tables in the database.

The ESB should print logs to the server console indicating that the cluster initialization is complete.

We have now finished configuring the manager node. Next, we will configure the ESB worker nodes.

Configuring the worker nodes

You configure worker nodes in very much the same way as you configured the manager node. Be sure to follow these steps for each worker node in the cluster.

Configuring the data source

You configure the data source to connect to the central database. Because the data source changes are the same for all worker nodes, you can configure this file on one node and then copy it to the other worker nodes.

Make sure that you have copied the MySQL JDBC driver JAR to each worker node as described in Setting up the central database.
Open the master-datasources.xml file located in the <ESB_WORKER_HOME>/repository/conf/datasources/ directory.
Locate the WSO2_CARBON_DB data source configurations and change them as follows:

- Define the location of the central database:
  <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/carbondb?DB_CLOSE_ON_EXIT=FALS E;LOCK_TIMEOUT=60000</url>
- Give user username access to the database: <username>username</username> <password>password</password>
- Specify the driver to use for connecting to the central database: <driverClassName>com.mysql.jdbc.Driver</driverClassName>

When you are finished, the data source configuration on each worker node should look like this:

Code Block

language	html/xml

<datasource>
   <name>WSO2_CARBON_DB</name>
   <description>The datasource used for registry and user manager</description>
   <jndiConfig>
       <name>jdbc/WSO2CarbonDB</name>
   </jndiConfig>
   <definition type="RDBMS">
       <configuration>
           <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/carbondb?DB_CLOSE_ON_EXIT=FALS E;LOCK_TIMEOUT=60000</url>
           <username>username</username>
           <password>password</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
       </configuration>
   </definition>
</datasource>

As mentioned previously, if there is more than one data source, configure them to reference the central database as well.

After you have finished configuring the data source, be sure to copy this configuration to the other worker nodes in the cluster.

Setting up cluster configurations for the worker nodes

Configuring clustering for the worker nodes is similar to the way you configured it for the manager node, but the localMemberPort will vary for each worker node, you add the subDomain property, and you add the ELB and ESB manager node to the well-known members, as described in the following steps.

Open the <ESB_HOME>/repository/conf/axis2/axis2.xml file.
Locate the Clustering section and verify or configure the properties as follows (some of these properties are already set correctly by default):

Enable clustering for this node: <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
Set the membership scheme to wka to enable the Well Known Address registration method (this node will send cluster initiation messages to WKA members that we will define later):
<parameter name="membershipScheme">wka</parameter>
Specify the name of the cluster this node will join: <parameter name="domain">wso2.esb.domain</parameter>
Specify the port used to communicate cluster messages (if this node is on the same server as the ELB, manager node, or another worker node, be sure to set this to a unique value, such as 4000 and 4001 for worker nodes 1 and 3 and 4002 for worker node 2, which is on the same server as the ELB and manager node):
<parameter name="localMemberPort">4000</parameter>
Note: This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server.

Add a new property "subDomain" and set it to "worker" to denote that this node belongs to worker subdomain of the cluster as defined in loadbalancer.conf.

Code Block

language	html/xml

<parameter name="properties">
            <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/>
            <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/>
            <property name="subDomain" value="mgt"/>
            <property name="port.mapping.8290" value="9763"/>
</parameter>

Define the ELB and manager nodes as well-known members of the cluster by providing their host name and localMemberPort values. The manager node is defined here because it is required for the Deployment Synchronizer to function.

Code Block

language	html/xml

<members>
    <member>
        <hostName>elb.wso2.com</hostName>
        <port>5000</port>
    </member>
    <member>
        <hostName>mgt.esb.wso2.com</hostName>
        <port>4001</port>
    </member>
</members>

Adjusting the port offset

Because we are running two Carbon-based products on the same server, we must change the port offset to avoid port conflicts.

Open <ESB_WORKER_HOME>/repository/conf/carbon.xml on each worker node.
Locate the <Ports> tag and change the value of its sub-tag as follows on each worker node:

- Worker1: <Offset>0</Offset> - No changes needed, because this will be the first node on this (xxx.xxx.xxx.132) server.
- Worker2: <Offset>2</Offset> - Set the offset to 2, because there are already two more Carbon products (ELB and ESB manager node) running on this (xxx.xxx.xxx.206) server.
- Worker3: <Offset>1</Offset> - Set the offset of 1, because Worker1 occupies the default ports on this (xxx.xxx.xxx.132) server.

In the next section, we will map the host names we specified to real IPs.

Mapping host names to IPs

In the worker nodes, we have used three hosts names: carbondb.mysql-wso2.com for the MySQL server, elb.wso2.com for the ELB, and mgt.esb.wso2.com for the ESB manager node. We will now map them to the actual IPs.

Open the server's /etc/hosts file and add the following lines, where <MYSQL-DB-SERVER-IP>, <ELB-IP>, and <ESB-Manager-IP> are the actual IP addresses (in this example, xxx.xxx.xxx.206):

Code Block

language	none

<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
<ELB-IP> elb.wso2.com
<ESB-Manager-IP> mgt.esb.wso2.com

In this example, it would look like this:

Code Block

language	none

xxx.xxx.xxx.206 carbondb.mysql-wso2.com
xxx.xxx.xxx.206 elb.wso2.com
xxx.xxx.xxx.206 mgt.esb.wso2.com

We have now finished configuring the worker nodes and are ready to start them.

Info
If you want to remove all UI components from the worker nodes you need to run the `ant createWorker` task before you start the worker nodes. Note that this will remove management console capability from worker nodes.

Starting the ESB server

Start the ESB server by typing the following command in the terminal:

Code Block
sh <ESB_WORKER_HOME>/bin/wso2server.sh -DworkerNode=true

The additional -DworkerNode=true argument indicates that this is a worker node.

When starting the Worker1, it should display logs in the console indicating that the cluster initialization is complete.

The ELB console should have these new messages:

Code Block

language	none

INFO - RpcMembershipRequestHandler Received JOIN message from xxx.xxx.xxx.132:4000(wso2.esb.domain)
INFO - MembershipManager Application member xxx.xxx.xxx.132:4000(wso2.esb.domain) joined group wso2.esb.domain
INFO - DefaultGroupManagementAgent Application member Host:xxx.xxx.xxx.132, Port: 4000, HTTP:8280, HTTPS:8243, Domain: wso2.esb.domain, Sub-domain:worker, Active:true joined application clustert

The manager node console should have these new messages:

Code Block

language	none

INFO - RpcMembershipRequestHandler Received JOIN message from xxx.xxx.xxx.132:4000(wso2.esb.domain)
INFO - RpcInitializationRequestHandler Received GetConfigurationCommand initialization request message from xxx.xxx.xxx.132:4000(wso2.esb.domain)

If you have similar messages in your consoles, you have finished configuring the worker nodes and the cluster is running. When you terminate one node, all nodes identify that the node has left the cluster. The same applies when a new node joins the cluster. If you want to add another new worker node, you can simply copy worker1 without any changes if you are running it on a new server (such as xxx.xxx.xxx.184). If you intend to use the new node on a server where another WSO2 product is running, you can use a copy of worker1 and change the port offset accordingly in the carbon.xml file. You may also have to change localMemberPort in axis2.xml if that product has clustering enabled. Be sure to map all host names to the relevant IP addresses when creating a new node.

Testing the cluster

To test the cluster, open the ESB management console on the manager node (use the management console URL displayed in the terminal when you started the node), add a sample proxy service with the log mediator in the inSequence so that logs will be displayed in the worker terminals, and then observe the cluster messages sent from the manager node to the workers.

The load balancer manages the active and passive states of the worker nodes, activating nodes as needed and leaving the rest in passive mode. To test this, send a request to the end point through the load balancer to verify that the proxy service is activated only on the active worker node(s) while the remaining worker nodes remain passive. For example, you would send the request to the following URL:

http://{Load_Balancer_Mapped_URL_for_worker}/services/{Sample_Proxy_Name}

Additional configuration

For information on additional configuration required for clustering other WSO2 products, click that product's link in the table of contents of this document.
WSO2 provides Hazelcast Community Edition as its default clustering engine. Advanced users can fine-tune Hazelcast by creating a <CARBON_HOME>/repository/conf/hazelcast.properties file and adding the relevant Hazelcast properties as described in the Hazelcast Advanced Configuration Properties documentation. If you use Hazelcast Enterprise Edition or Hazelcast Management Center, see the Hazelcast documentation for details on configuring those products.

If you need to provide access to the management node from outside your network so external clients can upload applications and perform other management tasks, you configure the mgt sub-domain in loadbalancer.conf and map the host to the IP address of the ELB. For example, you would add the mgt sub-domain to loadbalancer.conf as follows:

Code Block

language	html/xml
title	loadbalancer.conf

esb {
  domains{
     wso2.esb.domain {
        tenant_range *;
        group_mgt_port 5000;
        mgt {
                hosts management.esb.cloud-test.wso2.com;
        }
        worker {
               hosts esb.cloud-test.wso2.com;
        }
     }
   }
}

You would then add the management.esb.cloud-test.wso2.com port mapping in the /etc/hosts file as follows:

Code Block

language	none
title	/etc/hosts file

xxx.xxx.xxx.206 management.esb.cloud-test.wso2.com 
xxx.xxx.xxx.206 esb.cloud-test.wso2.com

Versions Compared

Old Version 3

New Version 4

Key

Installing the products

Configuring the load balancer

Setting up load-balancing configurations

Setting up cluster configurations on the ELB

Configuring the ELB to listen on default ports

Mapping the host name to the IP

Starting the ELB server

Setting up the database

Configuring the manager node

Configuring the data source

Setting up cluster configurations for the manager node

Configuring the port offset and host name

Mapping host names to IPs

Starting the ESB server

Configuring the worker nodes

Configuring the data source

Setting up cluster configurations for the worker nodes

Adjusting the port offset

Mapping host names to IPs

Starting the ESB server

Testing the cluster

Additional configuration