Config and Governance Partitions in a Remote Registry

In this deployment strategy, the configuration and governance spaces are shared among instances of a group/cluster. For example, two WSO2 Application Server instances that have been configured to operate in a clustered environment can have a single configuration and governance registry which is shared across each node of the cluster. A separate instance of the WSO2 Governance Registry (G-Reg) is used to provide the space used in common.

Figure 2: Config and governance partitions in the remote Governance Registry instance.

Configuration steps are given in the following sections.

• Creating the Database
• Configuring Governance Registry as the Remote Registry Instance
• Configuring Carbon Server Nodes

Creating the Database

In a production setup, it is recommended to use an Oracle or MySQL database for the governance registry. As an example, we use MySQL database named ‘registrydb’. Instructions are as follows:

1. Access MySQL using the command:

mysql -u root -p

2. Enter the password when prompted.

3. Create 'registrydb' database.

create database registrydb;

The MySQL database for G-Reg is now created.

Configuring Governance Registry as the Remote Registry Instance

Database configurations are stored in $CARBON_HOME/repository/conf/datasources/master-datasources.xml file for all carbon servers. By default, all WSO2 products use the in-built H2 database. Since Governance Registry in this example is using a MySQL database named 'registrydb', the master-datasources.xml file needs to be configured so that the datasource used for the registry and user manager in Governance Registry is the said MySQL database.

1. Download and extract WSO2 Governance Registry distribution from http://wso2.com/products/governance-registry. 

2. Navigate to $G-REG_HOME/repository/conf/datasources/master-datasources.xml file where G-REG_HOME is the Governance Registry distribution home. Replace the existing WSO2_CARBON_DB datasource with the following configuration:

<datasource>
   <name>WSO2_CARBON_DB</name>
   <description>The datasource used for registry and user manager</description>
   <jndiConfig>
        <name>jdbc/WSO2CarbonDB</name>
   </jndiConfig>
   <definition type="RDBMS">
       <configuration>
           <url>jdbc:mysql://x.x.x.x:3306/registrydb</url>
           <username>root</username>
           <password>root</password>
           <driverClassName>com.mysql.jdbc.Driver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
        </configuration>
   </definition>
</datasource>

Change the values of the following elements according to your environment. 

• <url> : URL of the MySQL database.
• <username> and <password> : username and password of the mySQL database.
• <validationQuery> : Validate and test the health of the DB connection.
• <validationInterval> : specified time intervals at which the DB connection validations should run.

3. Navigate to $G-REG_HOME/repository/conf/axis2/axis2.xml file in all Carbon-based product instances to be connected with the remote registry, and enable tribes clustering with the following configuration.

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true"/>

The above configuration is required only when caching is enabled for the Carbon server instances and <enableCache> parameter is set to true. This provides cache invalidation at the event of any updates on the registry resources.

4. Copy the 'mySQL JDBC connector jar' (http://dev.mysql.com/downloads/connector/j/5.1.html) to G-REG_HOME/repository/components/lib directory.

5. Start the Governance Registry server with -Dsetup so that all the required tables are created in the database. For example, in Linux

sh wso2server.sh -Dsetup

The Governance Registry server is now running with all required user manager and registry tables for the server also created in ‘registrydb’ database.

Configuring Carbon Server Nodes

Now that the shared registry is configured, let's take a look at the configuration of Carbon server nodes that use the shared, remote registry.

1. Download and extract the relevant WSO2 product distribution from the 'Products' menu of https://wso2.com. In this example, we use two server instances (of any product) by the names CARBON-Node1 and CARBON-Node2. 

2. We use the same datasource used for Governance Registry above as the registry space of Carbon-based product instances.

Configure master-datasources.xml File

3. Configure $CARBON_HOME/repository/conf/datasource/master-datasources.xml where CARBON_HOME is the distribution home of any WSO2 Carbon-based product you downloaded in step 1. Then, add the following datasource for the registry space.

<datasource>
           <name>WSO2_CARBON_DB_GREG</name>
           <description>The datasource used for registry and user manager</description>
           <jndiConfig>
                <name>jdbc/WSO2CarbonDB_GREG</name>
           </jndiConfig>
           <definition type="RDBMS">
            <configuration>
                       <url>jdbc:mysql://x.x.x.x:3306/registrydb</url>
                       <username>root</username>
                       <password>root</password>
                       <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                       <maxActive>50</maxActive>
                       <maxWait>60000</maxWait>
                       <testOnBorrow>true</testOnBorrow>
                       <validationQuery>SELECT 1</validationQuery>
                       <validationInterval>30000</validationInterval>
              </configuration>
              </definition>
</datasource>

Change the values of the relevant elements accordingly. 

Configuring registry.xml File

4. Navigate to $CARBON_HOME/repository/conf/registry.xml file and specify the following configurations for both server instances setup in step 1.

Add a new db config to the datasource configuration done in step 3 above. For example,

<dbConfig name="remote_registry">
      <dataSource>jdbc/WSO2CarbonDB_GREG</dataSource>
</dbConfig>

Specify the remote Governance Registry instance with the following configuration:

<remoteInstance url="https://x.x.x.x:9443/registry">
    <id>instanceid</id>
    <dbConfig>remote_registry</dbConfig>
    <cacheId>root@jdbc:mysql://x.x.x.x:3306/registrydb</cacheId>
    <readOnly>false</readOnly>
    <enableCache>true</enableCache>
    <registryRoot>/</registryRoot>
</remoteInstance>

Change the values of the following elements according to your environment.

• <remoteInstance url> : URL of the remote G-Reg instance.
• <dbConfig> : The dbConfig name specified for the registry database configuration.
• <cacheId> : This provides information on where the cache resource resides.
• <enableCache> : Whether caching is enabled on the Carbon server instance.

Define the registry partitions using the remote Governance Registry instance. In this deployment strategy, we are mounting the config and governance partitions of the Carbon-based product instances to the remote Governance Registry instance. This is graphically represented in Figure 2 at the beginning. 

<mount path="/_system/config" overwrite="true">
    <instanceId>instanceid</instanceId>
    <targetPath>/_system/nodes</targetPath>
</mount>
<mount path="/_system/governance" overwrite="true">
    <instanceId>instanceid</instanceId>
    <targetPath>/_system/governance</targetPath>
</mount>

• mount path : Registry collection of Carbon server instance that needs to be mounted
• mount overwrite : Defines if an existing collection/resource at the given path should be overwritten or not. Possible vales are:
  
  • true - The existing collection/resource in the specified location will always be deleted and overwritten with the resource/s in the remote registry instance. 
  • false - The resource/s will not be overwritten. An error will be logged if a resource exists at the existing location.
  • virtual - If the existing location has a resource/collection, it will be preserved but virtual view of the remote registry resource/s can be viewed. The original resource/collection can be viewed once the remote registry configuration is removed.
• target path : Path to the remote Governance Registry instance where the registry collection is mounted. 

Configuring axis2.xml File

1. Navigate to $CARBON_HOME/repository/conf/axis2/axis2.xml file where CARBON_HOME is the distribution home of any WSO2 Carbon-based products to be connected with the remote registry. Enable carbon clustering by copying the following configuration to all Carbon server instances:

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true"/>

2. Copy 'MySQL JDBC connector jar' (http://dev.mysql.com/downloads/connector/j/5.1.html) to $G-REG_HOME/repository/components/lib in both Carbon server instances.

3. Start both servers and note the log entries that indicate successful mounting to the remote Governance Registry instance. For example,

4. Navigate to the registry browser in the Carbon server's management console and note the config and governance partitions indicating successful mounting to the remote registry instance. For example,