Page Comparison

The following sections provide information and instructions on how to cluster the ESB profile of WSO2 Enterprise Integrator (WSO2 EI) with a third-party load balancer.

...

All profiles of WSO2 EI uses a database to store information such as user management details and registry data. All nodes in the cluster must use one central database for config and governance registry mounts. You can create the following databases and associated datasources.

Database Name	Description
WSO2_USER_DB	JDBC user store and authorization manager
`REGISTRY_DB`	Shared database for config and governance registry mounts in the product's nodes
`REGISTRY_LOCAL1`	Local registry space in Node 1
`REGISTRY_LOCAL2`	Local registry space in Node 2

Warning
It is recommended to use an industry-standard RDBMS such as Oracle, PostgreSQL, MySQL, MS SQL, etc. for most enterprise testing and production environments. However, you can also use the embedded H2 database only for the `REGISTRY_LOCAL1` and `REGISTRY_LOCAL2`.

...

Open the <EI_HOME>/conf/datasources/master-datasources.xml file, and configure the datasources to point to the relevant databases for each ESB node.

Tip
Replace the username, password, and database URL of your MySQL environment accordingly. If you have not enabled SSL, append the `useSSL=false` property to the value of the `<url>` property.

Localtabgroup

Localtab

title	ESB Node 1

For ESB node 1, configure the datasources to point to the REGISTRY_LOCAL1, WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows:

Code Block

language	xml

<datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> 
     <providers> 
        <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> 
    </providers> 
    <datasources> 
        <datasource> 
            <name>REGISTRY_LOCAL1</name> 
            <description>The datasource used for registry- local</description> 
            <jndiConfig> 
                <name>jdbc/WSO2CarbonDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL1?autoReconnect=true</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
        <datasource> 
            <name>REGISTRY_DB</name> 
            <description>The datasource used for registry- config/governance</description> 
            <jndiConfig> 
                <name>jdbc/WSO2RegistryDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
         <datasource> 
            <name>WSO2_USER_DB</name> 
            <description>The datasource used for registry and user manager</description> 
            <jndiConfig> 
                <name>jdbc/WSO2UMDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
   </datasources> 
</datasources-configuration>

Localtab

title	ESB Node 2

For node 2, configure the datasources to point to the REGISTRY_LOCAL2, WSO2_REGISTRY_DB, and WSO2_USER_DB databases as shown below. Change the username, password, and database URL as needed for your environment.

Code Block

language	xml

<datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration">     <providers>
        <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider>
    </providers>
    <datasources>
        <datasource>
            <name>REGISTRY_LOCAL2</name>
            <description>The datasource used for registry- local</description>
            <jndiConfig>
                <name>jdbc/WSO2CarbonDB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL2?autoReconnect=true</url>
                    <username>regadmin</username>
                    <password>regadmin</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource>
        <datasource>
            <name>REGISTRY_DB</name>
            <description>The datasource used for registry- config/governance</description>
            <jndiConfig>
                <name>jdbc/WSO2RegistryDB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url>
                    <username>regadmin</username>
                    <password>regadmin</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource>
         <datasource>
            <name>WSO2_USER_DB</name>
            <description>The datasource used for registry and user manager</description>
            <jndiConfig>
                <name>jdbc/WSO2UMDB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url>
                    <username>regadmin</username>
                    <password>regadmin</password>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource>
   </datasources>
</datasources-configuration>

Add the following configuration in the <EI_HOME>/conf/user-mgt.xml file to configure the user stores. Update the dataSource property in all nodes in the <EI_HOME>/conf/user-mgt.xml file as shown below to configure the datasource:
Code Block
<Property name="dataSource">jdbc/WSO2UMDB</Property>

...

Open the <EI_HOME>/conf/axis2/axis2.xml file for each of the two ESB nodes, and appply the following cluster configurations:

Enable clustering for each node as follows:

Code Block
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

Set the membership scheme to "wka" to enable the well-known address registration method as shown below. Each node sends cluster initiation messages to the WKA members.
Code Block
<parameter name="membershipScheme">wka</parameter>
Specify the name of the cluster to which the node joins:
Code Block
<parameter name="domain">wso2.ei.domain</parameter>

Specify the host to communicate cluster messages. For example, if the IP addresses of the two ESB nodes are xxx.xxx.xxx.xx1 and xxx.xxx.xxx.xx2, they should be specified in the configuration as shown below.

Localtabgroup

Localtab

title	For ESB Node 1

Code Block

language	xml

<parameter name="localMemberHost">xxx.xxx.xxx.xx1</parameter>

Localtab

title	For ESB Node 2

Code Block

language	xml

<parameter name="localMemberHost">xxx.xxx.xxx.xx2</parameter>

Specify the port to communicate cluster messages as follows:

Code Block
<parameter name="localMemberPort">4100</parameter>

Info
This port number is not affected by the port offset value specified in the `<EI_HOME>/conf/carbon.`xml file. If this port number is already assigned to another server, the clustering framework automatically increments this port number. However, if there are two servers running on the same machine, ensure that a unique port is set for each server.

Specify the well-known members in the cluster as shown below. For example, when you configure one ESB node, you need to specify the other nodes in the cluster as well-known members as shown below. The port value for the WKA node must be the same value as it's localMemberPort (in this case it is 4100).

Info
You can also use IP address ranges for the hostname (e.g., 192.168.1.2-10). However, you can define a range only for the last portion of the IP address. Smaller the range, the faster the time it takes to discover members since each node has to scan a lesser number of potential members.

Localtabgroup

Localtab

title	For ESB Node 1

Code Block

language	xml

<members>
	<member>
		<hostName>xxx.xxx.xxx.xx2</hostName>
		<port>4100</port> 
	</member>
</members>

Localtab

title	For ESB Node 2

Code Block

language	xml

<members>
	<member>
		<hostName>xxx.xxx.xxx.xx1</hostName>
		<port>4100</port> 
	</member>
</members>

Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener in the transportReceiver section.
Code Block
language xml
<parameter name="WSDLEPRPrefix" locked="false">http://ei.wso2.com:80</parameter>
Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver section.
Code Block
language xml
<parameter name="WSDLEPRPrefix" locked="false">https://ei.wso2.com:443</parameter>

Edit the <EI_HOME>/conf/carbon.xml file as follows to configure the hostname: <HostName>ei.wso2.com</HostName>

Configuring Hazelcast properties

WSO2 products use Hazelcast as its default clustering engine. You can configure the hazelcast properties for the product nodes by following the steps given below.

Create the hazelcast.properties file with the following property configurations, and copy the file to the <EI_HOME>/conf/ directory.
Code Block
#Disabling the hazelcast shutdown hook hazelcast.shutdownhook.enabled=false #Setting the hazelcast logging type to log4j hazelcast.logging.type=log4j
The above configurations are explained below.
- Hazelcast shutdown hook: This configuration disables the shutdown hook in hazelcast, which ensures that the hazelcast instance shuts down gracefully whenever the product node shuts down. If the hazelcast shutdown hook is enabled (which is the default behavior of a product), you will see errors such as "Hazelcast instance is not active!" at the time of shutting down the product node: This is because the hazelcast instance shuts down too early when the shutdown hook is enabled.
- Hazelcast logging type: This configuration sets the hazelcast logging type to log4j, which allows hazelcast logs to be written to the wso2carbon.log file.
If you have enabled log4j for hazelcast logging as shown above, be sure to enter the configuration shown below in the log4j.properties file (stored in the <EI_HOME>/conf/). This can be used to configure the log level for hazelcast logging. For a clustered production environment, it is recommended to use INFO as the log level as shown below.
Code Block
log4j.logger.com.hazelcast=INFO

General configurations

Specify the port offset value in the <EI_HOME>/conf/carbon.xml file.

Warning
This step is optional and only required if all server instances are running on the same machine. This is not recommended for production environments. Change all ports used in your configurations based on the offset value if you are setting a port offset.

Expand

title	Click here for more information on configuring the port offset.

When you run multiple products/clusters or multiple instances of the same product on the same server or virtual machines (VMs), change their default ports with an offset value to avoid port conflicts. An offset defines the number by which all ports in the runtime (e.g., HTTP(S) ports) are increased. For example, if the default HTTP port is 9763 and the offset is 1, the effective HTTP port will change to 9764. For each additional product instance, set the port offset to a unique value. The offset of the default ports is zero.

The port value will automatically increase as shown in the Port Value column in the following table, allowing all five WSO2 product instances or servers to run on the same machine.

WSO2 product instance	Port Offset	Port Value
WSO2 server 1	0	9443
WSO2 server 2	1	9444
WSO2 server 3	2	9445
WSO2 server 4	3	9446
WSO2 server 5	4	9447

Code Block

language	xml

<Ports>
	...
	<Offset>0</Offset>
	...
</Ports>

Add the host entries to your DNS, or “/etc/hosts” file (in Linux) in all the nodes of the cluster to map the hostnames to the IP addresses. For an example, you can map the IP address of the database server. In this example, MySQL is used as the database server, so <MYSQL-DB-SERVER-IP> is the actual IP address of the database server and the host entry is as follows: <IP-of-MYSQL-DB-SERVER> carbondb.mysql-wso2.com
Edit the <EI_HOME>/conf/tomcat/catalina-server.xml file as follows:

...

Versions Compared

Old Version 15

New Version Current

Key

Configuring Hazelcast properties

General configurations