Clustering Configuration

This document describes the steps you need to follow to setup a working WSO2 Data Services Server cluster in your environment.

Clustering has two different aspects:

• Configuration sharing across the cluster nodes.
• Runtime state replication among the cluster nodes.

Configuration Sharing Across the Cluster Nodes

The configuration sharing is done using the WSO2 Governance Registry. All Gadget Server nodes in the cluster are pointed to one instance of WSO2 Governance Registry through which, configuration is shared.

WSO2 Governance Registry consists of three registry spaces. For information, refer to Registry.

• Local Registry - Per running instance of every product.
• Configuration Registry - Shared among all the instance of a cluster of one product.
• Governance Registry - Centrally managed and shared across multiple products and multiple instances of those.

A local registry is used per each instance, mounting the configuration registry of each Data Services Server instance to the same configuration registry. The governance registry is shared among Governance Registry and Data Services Server instances.

Setting Up WSO2 Data Services Server Cluster with Governance Registry

After extraction of the Data Services Server distribution, configure the database.

Both Governance Registry and Data Services Server are shipped with an embedded H2 registry. However, for clustering purposes MySQL is used.

In the following setup, the Governance Registry is running on MySQL database. That means its "local/config/governance" registries are stored in a MySQL database and its user-manager tables are also inside the same database.

In Data Services Server instances, both master and slave are using unique local registries, both configuration registries are mounted to the same directory of Governance Registry.

All the Data Services Server instances in the cluster are sharing a common database for user-manager.

Data Services Server Cluster Database Configuration

If you are having your database remotely, please configure mysql instance to allow remote connections to accept.

Running Multiple Instance of WSO2 Carbon Servers in the Same Machine

If you are running multiple instance same or different WSO2 Product Instances, you need to configure ports for each instance. You can configure $CARBON_HOME/repository/conf/carbon.xml by using portOffset.

<Offset>0</Offset>

For example, Offset=2 and HTTPS port=9443 will set the effective HTTPS port to 9445.

Configuring Governance Registry

1. Change the configuration in registry.xml and user-mgt.xml.

1.1. Open the GREG_HOME/repository/conf/registry.xml file, where GREG_HOME refers to Governance Registry home directory throughout this document. Remove the original dbConfig element and add the database configuration below to the registry.xml file.

<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<registryRoot>/</registryRoot>
<dbConfig name="wso2registry">
     <url>jdbc:mysql://x.x.x.x:3306/amagregdb?autoReconnect=true</url>
     <userName>root</userName>
     <password>root123</password>
     <driverName>com.mysql.jdbc.Driver</driverName>
     <maxActive>50</maxActive>
     <maxWait>60000</maxWait>
     <minIdle>5</minIdle>
</dbConfig>

1.2. Open the GREG_HOME/repository/conf/user-mgt.xml file and change the database configuration as shown below.

<Property name="url">jdbc:mysql://x.x.x.x:3306/amagregdb?autoReconnect=true</Property>
<Property name="userName">root</Property>
<Property name="password">root123</Property>
<Property name="driverName">com.mysql.jdbc.Driver</Property>
<Property name="maxActive">50</Property>
<Property name="maxWait">60000</Property>
<Property name="minIdle">5</Property>

1.3. Copy MySQL jdbc driver library to GREG_HOME/repository/components/lib directory since, by default, we do not ship the MySQL drivers in Governance Registry.

2. In the above configuration, the database is pointed in to same database amagregdb. Now start the Governance Registry instance with -Dsetup.

This will create tables for user manager database and registry database

3. After starting the Governance Registry instance successfully, create the following directory from the Governance Registry UI. This directory will be used by the Data Services Server instances as its mounted registry.

/_system/dssConfig.

4. To configure the Data Services Server master node, change the default database configuration. To point its local registry to amamaster, point the user manager database to amausermgtdb and mount its configuration and governance registries from the Governance Registry database.

4.1. Add local registry configuration and user-manager configuration.

4.1.1. Open the DSS_HOME/repository/conf/registry.xml file.                                       
4.1.2. Remove the original dbConfig element.                                                 
4.1.3. Add the database configuration shown below.

<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<registryRoot>/</registryRoot>
<dbConfig name="wso2registry">
    <url>jdbc:mysql://x.x.x.x:3306/amamaster?autoReconnect=true</url>
    <userName>root</userName>
    <password>root123</password>
    <driverName>com.mysql.jdbc.Driver</driverName>
    <maxActive>50</maxActive>
    <maxWait>60000</maxWait>
    <minIdle>5</minIdle>
</dbConfig>

4.1.4. Open the <DSS_HOME>/repository/conf/user-mgt.xml file and change the database configuration as shown below.

<Property name="url">jdbc:mysql://x.x.x.x:3306/amausermgtdb?autoReconnect=true</Property>
<Property name="userName">root</Property>
<Property name="password">root123</Property>
<Property name="driverName">com.mysql.jdbc.Driver</Property>
<Property name="maxActive">50</Property>
<Property name="maxWait">60000</Property>
<Property name="minIdle">5</Property>

4.2. To configure the mounting configuration, more parameters should be added to registry.xml file.

4.2.1. Open the <DSS_HOME>/repository/conf/registry.xml file.                               
4.2.2. Remove the original dbConfig element.                                               
4.2.3. Add the following database configuration as shown below.

<dbConfig name="dssMountRegistry">
    <url>jdbc:mysql://x.x.x.x:3306/amagregdb?autoReconnect=true</url>
    <userName>root</userName>
    <password>root123</password>
    <driverName>com.mysql.jdbc.Driver</driverName>
    <maxActive>50</maxActive>
    <maxWait>60000</maxWait>
    <minIdle>5</minIdle>
</dbConfig>

<remoteInstance url="https://x.x.x.x:9443/registry">
    <id>Mount1</id>
    <dbConfig>dssMountRegistry</dbConfig>
    <readOnly>false</readOnly>
    <registryRoot>/</registryRoot>
</remoteInstance>

<mount path="/_system/config" overwrite="true">
    <instanceId>Mount1</instanceId>
    <targetPath>/_system/dssConfig</targetPath>
</mount>

<mount path="/_system/governance" overwrite="true">
    <instanceId>Mount1</instanceId>
    <targetPath>/_system/governance</targetPath>
</mount>

4.2.4. Change the remoteInstance URL according to the configuration of the machine running the Governance Registry.

4.2.5. Copy MySQL jdbc driver library to <DSS_HOME>/repository/components/lib directory.

5. To configure DSS slave node, change the default database configuration, to point its local registry to amaslave1, point user manager database to amausermgtdb and mount its configuration and governance registries from Governance Registry database.

5.1. Add local registry configuration and user-manager configuration.

5.1.1. Open the <DSS_SLAVE_HOME>/repository/conf/registry.xml file.                           
5.1.2. Remove the original dbConfig element.                                                 
5.1.3. Add the database configuration shown below.

<currentDBConfig>wso2registry</currentDBConfig>
<readOnly>false</readOnly>
<registryRoot>/</registryRoot>
<dbConfig name="wso2registry">
    <url>jdbc:mysql://x.x.x.x:3306/amaslave1?autoReconnect=true</url>
    <userName>root</userName>
    <password>root123</password>
    <driverName>com.mysql.jdbc.Driver</driverName>
    <maxActive>50</maxActive>
    <maxWait>60000</maxWait>
    <minIdle>5</minIdle>
</dbConfig>

<dbConfig name="dssMountRegistry">
    <url>jdbc:mysql://x.x.x.x:3306/amagregdb?autoReconnect=true</url>
    <userName>root</userName>
    <password>root123</password>
    <driverName>com.mysql.jdbc.Driver</driverName>
    <maxActive>50</maxActive>
    <maxWait>60000</maxWait>
    <minIdle>5</minIdle>
</dbConfig>

<remoteInstance url="https://x.x.x.x:9443/registry">
    <id>Mount1</id>
    <dbConfig>dssMountRegistry</dbConfig>
    <readOnly>true</readOnly>
    <registryRoot>/</registryRoot>
</remoteInstance>

<mount path="/_system/config" overwrite="true">
    <instanceId>Mount1</instanceId>
    <targetPath>/_system/dssConfig</targetPath>
</mount>

<mount path="/_system/governance" overwrite="true">
    <instanceId>Mount1</instanceId>
    <targetPath>/_system/governance</targetPath>
</mount>

5.1.4. Open the <DSS_SLAVE_HOME>/repository/conf/user-mgt.xml file and change the database configuration as shown below.

<Configuration>
    <AdminRole>admin</AdminRole>
    <AdminUser>
        <UserName>admin</UserName>
        <Password>admin</Password>
    </AdminUser>
    <EveryOneRoleName>everyone</EveryOneRoleName> <\!-\- By default users in this role sees the registry root \-->
    <ReadOnly>true</ReadOnly>
    <Property name="url">jdbc:mysql://localhost:3306/amausermgtdb?autoReconnect=true</Property>
    <Property name="userName">root</Property>
    <Property name="password">root</Property>
    <Property name="driverName">com.mysql.jdbc.Driver</Property>
    <Property name="maxActive">50</Property>
    <Property name="maxWait">60000</Property>
    <Property name="minIdle">5</Property>
</Configuration>

5.1.5. Change the remoteInstance URL according to the configuration of the G-Red running machine.

5.2. Copy MySQL jdbc driver library to <DSS_SLAVE_HOME>/repository/components/lib directory.

6. Load registry database schema to amaslave1 database before starting the slave DSS node. You can find the required SQL script at <DSS_SLAVE_HOME>/dbscripts directory.

7. Run the wso2server.sh or wso2server.bat of master node based on your platform with -Dsetup option for the first time start up to create the database tables, and after creating database tables, do not use -Dsetup option in the start.sh CARBON_HOME/bin/wso2server.sh -Dsetup.

8. After setting up the Data Services Server pointing to a single Registry, you can share the configuration. We will only look at manual configuration sharing here. You can also share the configuration using deployment synchronizer, which is not included in the Data Services Server by default. However, you can add that as a feature.

• Manual configuration sharing - In this mode, the Data Services Servers can be configured to load the configuration from the Registry instead of the file system. Configuration loading happens only when the Data Services Server starts up. So if a change is made to the master DSS node, other nodes have to be restarted to get the new configuration. A lot of users prefer this mode because it guarantees a consistent configuration at the start up and configuration is not changed while the Data Services Server is running.

If the configuration has to be loaded from the Registry, following configuration has to be uncommented and changed in the carbon.xml.

<MediationConfig>
    <LoadFromRegistry>false</LoadFromRegistry>
    <SaveToFile>false</SaveToFile>
    <Persistence>enabled</Persistence>
    <RegistryPersistence>enabled</RegistryPersistence>
</MediationConfig>