Page Comparison

...

Each Carbon-based product uses a database to store information such as user management details and registry data. All nodes in the cluster must use one central database for config and governance registry mounts.

...

These instructions assume you are installing MySQL as your relational database management system (RDBMS), but you can install another supported RDBMS as needed. You can create the following databases and associated data sources:

Database Name	Description
WSO2_USER_DB	JDBC user store and authorization manager
`REGISTRY_DB`	Shared database for config and governance registry mounts in the product's nodes
`REGISTRY_LOCAL1`	Local registry space in the manager node
`REGISTRY_LOCAL2`	Local registry space in the worker node

The following diagram illustrates how these databases are connected to the manager and worker nodes.

Image Added

...

Download and install MySQL Server.
Download the MySQL JDBC driver.
Unzip the downloaded MySQL driver zipped archive, and copy the MySQL JDBC driver JAR (mysql-connector-java-x.x.xx-bin.jar)
to the <ESB
into the <PRODUCT_HOME>/repository/
component
components/lib
directory for each worker and manager node.

...

directory of both the manager and worker nodes.
Define the host name for configuring permissions for the new database by opening the /etc/hosts file and adding the following line:
```
<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
```

...

Configuring the manager node

In this section, we will configure data sources to allow the manager node to point to the central database, enable the manager node for clustering, change the port offset, and map the host names to IPs.

Configuring the data source

Make sure that you have copied the MySQL JDBC driver JAR to the manager node as described in Setting up the central database.
Open the master-datasources.xml file located in the <ESB_MANAGER_HOME>/repository/conf/datasources/ directory.
Locate the WSO2_CARBON_DB data source configurations and change them as follows:

- Define the location of the central database: <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/carbondb?DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000</url>
- Give user username access to the database: <username>username</username> <password>password</password>
- Specify the driver to use for connecting to the central database (the driver we copied in the previous section): <driverClassName>com.mysql.jdbc.Driver</driverClassName>

When you are finished, the data source configuration should look like this:

...

language	html/xml

...

 Info
You would do this step only if your database is not on your local machine and on a separate server.

Enter the following command in a terminal/command window, where username is the username you want to use to access the databases:
mysql -u username -p
When prompted, specify the password that will be used to access the databases with the username you specified.

Create the databases using the following commands, where <PRODUCT_HOME> is the path to any of the product instances you installed, and username and password are the same as those you specified in the previous steps:

Code Block

language	none

mysql> create database WSO2_USER_DB;
mysql> use WSO2_USER_DB;
mysql> source <PRODUCT_HOME>/dbscripts/mysql.sql;
mysql> grant all on WSO2_USER_DB.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin";

mysql> create database REGISTRY_DB;
mysql> use REGISTRY_DB;
mysql> source <PRODUCT_HOME>/dbscripts/mysql.sql;
mysql> grant all on REGISTRY_DB.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin";

mysql> create database REGISTRY_LOCAL1;
mysql> use REGISTRY_LOCAL1;
mysql> source <PRODUCT_HOME>/dbscripts/mysql.sql;
mysql> grant all on REGISTRY_LOCAL1.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin";
 
mysql> create database REGISTRY_LOCAL2;
mysql> use REGISTRY_LOCAL2;
mysql> source <PRODUCT_HOME>/dbscripts/mysql.sql;
mysql> grant all on REGISTRY_LOCAL2.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin";

Configuring the database on the manager node

On the manager node, open <PRODUCT_HOME>/repository/conf/datasources/master-datasource.xml, and configure the data sources to point to the REGISTRY_LOCAL1, WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows (change the username, password, and database URL as needed for your environment):

Code Block

language	html/xml

<datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> 
     <providers> 
        <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> 
    </providers> 
    <datasources> 
        <datasource>

...

<name>REGISTRY_LOCAL1</name> 
            <description>The datasource used for registry-

...

local</description>

...

Info
In most WSO2 products, only one data source is used. If there is more than one data source, make sure they reference the central databases accordingly. For example, the API Manager deployment setup requires more specific data source configurations.

Setting up cluster configurations for the manager node

Configuring clustering for the manager node is similar to the way you configured it for the ELB node, but the localMemberPort is 4001 instead of 4000, and you define the ELB node instead of the ESB manager node as the well-known member.

Open the <ESB_HOME>/repository/conf/axis2/axis2.xml file.
Locate the Clustering section and verify or configure the properties as follows (some of these properties are already set correctly by default):

- Enable clustering for this node: <clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
- Set the membership scheme to wka to enable the Well Known Address registration method (this node will send cluster initiation messages to WKA members that we will define later): <parameter name="membershipScheme">wka</parameter>
- Specify the name of the cluster this node will join: <parameter name="domain">wso2.esb.domain</parameter>
- Specify the port used to communicate cluster messages: <parameter name="localMemberPort">4001</parameter>
  Note: This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server.
- The receiver's http/https port values are without the portOffset addition; they get auto-incremented by portOffset. The 'WSDLEPRPrefix' parameter should point to the worker node's host name (esb.cloud-test.wso2.com) and ELB's http (8280)/https (8243) transport ports.
- Clear the members from the <members> element so that it is now empty:
  Code Block
  language html/xml
  <members> </members>

Configuring the port offset and host name

Because we are running two Carbon-based products on the same server, we must change the port offset to avoid port conflicts. Additionally, we will add the cluster host name so that any requests sent to the manager host are redirected to the cluster, where the ELB will pick them up and manage them.

Open <ESB_MANAGER_HOME>/repository/conf/carbon.xml.
Locate the <Ports> tag and change the value of its sub-tag to: <Offset>1</Offset>
Located the <HOSTNAME> tag and add the cluster host name: <HostName>esb.cloud-test.wso2.com</HostName>

In the next section, we will map the host names we specified to real IPs.

Mapping host names to IPs

In the manager node we have specified two host names: carbondb.mysql-wso2.com for the MySQL server and esb.cloud-test.wso2.com for the cluster. We will now map them to the actual IPs. Note that if you created the database on the same server as the manager node, you will have already added the first line, and if you created it on the same server as the ELB, you will have already added the second line.

Open the server's /etc/hosts file and add the following lines, where <MYSQL-DB-SERVER-IP> and <ELB-IP> are the actual IP addresses (in this example, xxx.xxx.xxx.206):

Code Block

language	none

<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
<ELB-IP> esb.cloud-test.wso2.com

In this example, it would look like this:

Code Block

language	none

xxx.xxx.xxx.206 carbondb.mysql-wso2.com
xxx.xxx.xxx.206 esb.cloud-test.wso2.com

We have now finished configuring the manager node and are ready to start the ESB server.

Starting the ESB server

Start the ESB server by typing the following command in the terminal:

sh <ESB_MANAGER_HOME>/bin/wso2server.sh -Dsetup

The additional -Dsetup argument will clean the configurations, recreate the central database, and create the required tables in the database.

The ESB should print logs to the server console similar to the following:

Code Block

language	none

INFO - TribesClusteringAgent Initializing cluster...
INFO - TribesClusteringAgent Cluster domain: wso2.esb.domain
INFO - TribesClusteringAgent Using wka based membership management scheme
INFO - WkaBasedMembershipScheme Receiver Server Socket bound to:/xxx.xxx.xxx.206:4001
INFO - WkaBasedMembershipScheme Receiver Server Socket bound to:/xxx.xxx.xxx.206:4001
INFO - TribesClusteringAgent Local Member 10.200.0.102:4001(wso2.esb.domain)
INFO - TribesUtil No members in current cluster
INFO - TribesClusteringAgent Cluster initialization completed

We have now finished configuring the manager node. Next, we will configure the ESB worker nodes.

Configuring the worker nodes

You configure worker nodes in very much the same way as you configured the manager node. Be sure to follow these steps for each worker node in the cluster.

Configuring the data source

You configure the data source to connect to the central database. Because the data source changes are the same for all worker nodes, you can configure this file on one node and then copy it to the other worker nodes.

Make sure that you have copied the MySQL JDBC driver JAR to each worker node as described in Setting up the central database.
Open the master-datasources.xml file located in the <ESB_WORKER_HOME>/repository/conf/datasources/ directory.
Locate the WSO2_CARBON_DB data source configurations and change them as follows:

- Define the location of the central database:
  <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/carbondb?DB_CLOSE_ON_EXIT=FALS E;LOCK_TIMEOUT=60000</url>
- Give user username access to the database: <username>username</username> <password>password</password>
- Specify the driver to use for connecting to the central database: <driverClassName>com.mysql.jdbc.Driver</driverClassName>

When you are finished, the data source configuration on each worker node should look like this:

Code Block

language	html/xml

<datasource>
   <name>WSO2_CARBON_DB</name>
   <description>The datasource used for registry and user manager</description>
   <jndiConfig>
       <name>jdbc/WSO2CarbonDB</name>
   </jndiConfig>
   <definition type="RDBMS">
       <configuration>
           <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/carbondb?DB_CLOSE_ON_EXIT=FALS E;LOCK_TIMEOUT=60000</url>
           <username>username</username>
           <password>password</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
       </configuration>
   </definition>
</datasource>

...

      <jndiConfig> 
                <name>jdbc/WSO2CarbonDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL1?autoReconnect=true</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
        <datasource> 
            <name>REGISTRY_DB</name> 
            <description>The datasource used for registry- config/governance</description> 
            <jndiConfig> 
                <name>jdbc/WSO2RegistryDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
         <datasource> 
            <name>WSO2_USER_DB</name> 
            <description>The datasource used for registry and user manager</description> 
            <jndiConfig> 
                <name>jdbc/WSO2UMDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
   </datasources> 
</datasources-configuration>

Info
Make sure to replace `username` and `password` with your MySQL database username and password.

To configure the datasource, update the dataSource property found in <PRODUCT_HOME>/repository/conf/user-mgt.xml of the manager node as shown below:
Code Block
language html/xml
<Property name="dataSource">jdbc/WSO2UMDB</Property>

Configure the shared registry database and mounting details in <PRODUCT_HOME>/repository/conf/registry.xml of the manager node as shown below:

Code Block

language	html/xml

<dbConfig name="sharedregistry">
	<dataSource>jdbc/WSO2RegistryDB</dataSource>
</dbConfig>

<remoteInstance url="https://localhost:9443/registry">
	<id>instanceid</id>
	<dbConfig>sharedregistry</dbConfig>
	<readOnly>false</readOnly>
	<enableCache>true</enableCache>
	<registryRoot>/</registryRoot>
	<cacheId>REGISTRY_DB@jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL1?autoReconnect=true</cacheId>
</remoteInstance>

<mount path="/_system/config" overwrite="true">
	<instanceId>instanceid</instanceId>
	<targetPath>/_system/asNodes</targetPath>
</mount>

<mount path="/_system/governance" overwrite="true">
	<instanceId>instanceid</instanceId>
	<targetPath>/_system/governance</targetPath>
</mount>

The following are some key points to note when adding these configurations:

- The registry mount path is used to identify the type of registry. For example, ”/_system/config” refers to configuration registry, and "/_system/governance" refers to the governance registry.
- The dbconfig entry enables you to identify the datasource you configured in the master-datasources.xml file. We use the unique name sharedregistry to refer to that datasource entry.
- The remoteInstance section refers to an external registry mount. We can specify the read-only/read-write nature of this instance as well as caching configurations and the registry root location. In case of a worker node, the readOnly property should be true, and in case of a manager node, this property should be set to false.
- You must define a unique name “id” for each remote instance, which is then referred to from mount configurations. In the above example, the unique ID for the remote instance is instanceId.
- In each of the mounting configurations, we specify the actual mount path and target mount path. The targetPath can be any meaningful name. In this instance, it is /_system/asNodes.

Configuring the database on the worker node

On the worker node, open <PRODUCT_HOME>/repository/conf/datasources/master-datasource.xml and configure the data sources to point to the REGISTRY_LOCAL2, WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows (change the username, password, and database URL as needed for your environment):

Code Block

language	html/xml

<datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> 
     <providers> 
        <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> 
    </providers> 
    <datasources> 
        <datasource> 
            <name>REGISTRY_LOCAL2</name> 
            <description>The datasource used for registry- local</description> 
            <jndiConfig> 
                <name>jdbc/WSO2CarbonDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL2?autoReconnect=true</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
        <datasource> 
            <name>REGISTRY_DB</name> 
            <description>The datasource used for registry- config/governance</description> 
            <jndiConfig> 
                <name>jdbc/WSO2RegistryDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
         <datasource> 
            <name>WSO2_USER_DB</name> 
            <description>The datasource used for registry and user manager</description> 
            <jndiConfig> 
                <name>jdbc/WSO2UMDB</name> 
            </jndiConfig> 
            <definition type="RDBMS"> 
                <configuration> 
                    <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> 
                    <username>regadmin</username> 
                    <password>regadmin</password> 
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName> 
                    <maxActive>50</maxActive> 
                    <maxWait>60000</maxWait> 
                    <testOnBorrow>true</testOnBorrow> 
                    <validationQuery>SELECT 1</validationQuery> 
                    <validationInterval>30000</validationInterval> 
                </configuration> 
            </definition> 
        </datasource> 
   </datasources> 
</datasources-configuration>

Info
Make sure to replace `username` and `password` with your MySQL database username and password.

Now your database is set up. The next step is to configure the manager and worker node.

Configuring the manager node

In this section, we will configure data sources to allow the manager node to point to the central database, enable the manager node for clustering, change the port offset, and map the host names to IPs.

Configuring the data source

You configure datasources to allow the manager node to point to the central database. Make sure that you copy the database driver JAR to the manager node and follow the steps described in Setting up the central database.

Info
In most WSO2 products, only one data source is used. If there is more than one data source, make sure they reference the central databases accordingly. For example, the API Manager deployment setup requires more specific data source configurations.

Setting up cluster configurations for the manager node

Configuring clustering for the manager node is similar to the way you configured it for the ELB node, but the localMemberPort is 4001 instead of 4000, and you define the ELB node instead of the ESB manager node as the well-known member.

Open the <ESB_HOME>/repository/conf/axis2/axis2.xml file.
Locate the Clustering section and verify or configure the properties as follows (some of these properties are already set correctly by default):

- Enable clustering for this node: <clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
- Set the membership scheme to wka to enable the Well Known Address registration method (this node will send cluster initiation messages to WKA members that we will define later): <parameter name="membershipScheme">wka</parameter>
- Specify the name of the cluster this node will join: <parameter name="domain">wso2.esb.domain</parameter>
- Specify the port used to communicate cluster messages: <parameter name="localMemberPort">4001</parameter>
  Note: This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server.
- The receiver's http/https port values are without the portOffset addition; they get auto-incremented by portOffset. The 'WSDLEPRPrefix' parameter should point to the worker node's host name (esb.cloud-test.wso2.com) and ELB's http (8280)/https (8243) transport ports.
- Clear the members from the <members> element so that it is now empty:
  Code Block
  language html/xml
  <members> </members>

Configuring the port offset and host name

Because we are running two Carbon-based products on the same server, we must change the port offset to avoid port conflicts. Additionally, we will add the cluster host name so that any requests sent to the manager host are redirected to the cluster, where the ELB will pick them up and manage them.

Open <ESB_MANAGER_HOME>/repository/conf/carbon.xml.
Locate the <Ports> tag and change the value of its sub-tag to: <Offset>1</Offset>
Located the <HOSTNAME> tag and add the cluster host name: <HostName>esb.cloud-test.wso2.com</HostName>

In the next section, we will map the host names we specified to real IPs.

Mapping host names to IPs

In the manager node we have specified two host names: carbondb.mysql-wso2.com for the MySQL server and esb.cloud-test.wso2.com for the cluster. We will now map them to the actual IPs. Note that if you created the database on the same server as the manager node, you will have already added the first line, and if you created it on the same server as the ELB, you will have already added the second line.

Open the server's /etc/hosts file and add the following lines, where <MYSQL-DB-SERVER-IP> and <ELB-IP> are the actual IP addresses (in this example, xxx.xxx.xxx.206):

Code Block

language	none

<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
<ELB-IP> esb.cloud-test.wso2.com

In this example, it would look like this:

Code Block

language	none

xxx.xxx.xxx.206 carbondb.mysql-wso2.com
xxx.xxx.xxx.206 esb.cloud-test.wso2.com

We have now finished configuring the manager node and are ready to start the ESB server.

Starting the ESB server

Start the ESB server by typing the following command in the terminal:

sh <ESB_MANAGER_HOME>/bin/wso2server.sh -Dsetup

The additional -Dsetup argument will clean the configurations, recreate the central database, and create the required tables in the database.

The ESB should print logs to the server console similar to the following:

Code Block

language	none

INFO - TribesClusteringAgent Initializing cluster...
INFO - TribesClusteringAgent Cluster domain: wso2.esb.domain
INFO - TribesClusteringAgent Using wka based membership management scheme
INFO - WkaBasedMembershipScheme Receiver Server Socket bound to:/xxx.xxx.xxx.206:4001
INFO - WkaBasedMembershipScheme Receiver Server Socket bound to:/xxx.xxx.xxx.206:4001
INFO - TribesClusteringAgent Local Member 10.200.0.102:4001(wso2.esb.domain)
INFO - TribesUtil No members in current cluster
INFO - TribesClusteringAgent Cluster initialization completed

We have now finished configuring the manager node. Next, we will configure the ESB worker nodes.

Configuring the worker nodes

You configure worker nodes in very much the same way as you configured the manager node. Be sure to follow these steps for each worker node in the cluster.

Configuring the data source

You configure the data source to connect to the central database. If there are multiple data sources, configure them to reference the central database as well. Since the data source changes are the same for all worker nodes, you can configure this file on one node and then copy it to the other worker nodes. Make sure you copy the database driver JAR to each worker node and follow the steps described in Setting up the central database.

After you have finished configuring the data source, be sure to copy this configuration to the other worker nodes in the cluster.

...

Versions Compared

Old Version 16

New Version 17

Key

Configuring the manager node

Configuring the data source

Configuring the database on the manager node

Setting up cluster configurations for the manager node

Configuring the port offset and host name

Mapping host names to IPs

Starting the ESB server

Configuring the worker nodes

Configuring the data source

Configuring the database on the worker node

Configuring the manager node

Configuring the data source

Setting up cluster configurations for the manager node

Configuring the port offset and host name

Mapping host names to IPs

Starting the ESB server

Configuring the worker nodes

Configuring the data source