Configuring Clustering for a Selected Pattern

Hiera data sets that match the distributed profiles of WSO2 API Manager (api-store, api-publisher, api-key-manager, gateway-manager, gateway-worker, traffic-manager) are shipped with clustering related configuration that is already enabled. Therefore, you need to only do a few changes to set up a distributed deployment in your preferred deployment pattern, before running the Puppet Agent.

Follow the instructions below to configure clustering for a selected pattern:

1. Add or update the host name mapping list.
   Puppet adds the required host entries explicitly in the /etc/hosts file in the Puppet Agent. For this purpose you have to update the hosts mappings appropriately in one of the following Hiera data files (YAML file) based on the pattern that you are using.

   • /etc/puppet/hieradata/<environment-name>/dev/wso2/wso2am_runtime/<pattern-number>/default.yaml

   • /etc/puppet/hieradata/<environment-name>/dev/wso2/wso2am_runtime/<pattern-name>/common.yaml 

2. Add the Well Known Address (WKA) list for Gateway clusters and Publisher-Store cluster.
   The required configurations for clustering are already added, but you need to update the WKA IP addresses in the respective Hiera data files based on your deployment.

   This is not applicable to patterns 0, 1, 2, and 7 as those patterns do not have Gateway or Publisher-Store clusters.

3. Modify all the MySQL based datasources in the respective Hiera data file (YAML file) that corresponds to the pattern you are using in order to point to the external MySQL servers. 

   This is not applicable for pattern 0 as pattern-0 uses a H2 database.

   Update the following in the /etc/puppet/hieradata/wso2/wso2am_runtime/[pattern-number]/default.yaml file for patterns 1, 2 and 7 and  /etc/puppet/hieradata/wso2/wso2am_runtime/[pattern-number]/common.yaml file for patterns 3 to 6.

   1. You need to only replace the IP address, with the IP address of database server that you are using.

      If you want to use any other database except MySQL, update the data sources appropriately.

      Example:

         wso2_am_db:
            name: WSO2_AM_DB
            description: The datasource used for API Manager database
            driver_class_name: "%{hiera('wso2::datasources::mysql::driver_class_name')}"
            url: jdbc:mysql://192.168.57.210:3306/apimgtdb?autoReconnect=true
            username: "%{hiera('wso2::datasources::mysql::username')}"
            password: "%{hiera('wso2::datasources::mysql::password')}"
            jndi_config: jdbc/WSO2AM_DB
            max_active: "%{hiera('wso2::datasources::common::max_active')}"
            max_wait: "%{hiera('wso2::datasources::common::max_wait')}"
            test_on_borrow: "%{hiera('wso2::datasources::common::test_on_borrow')}"
            default_auto_commit: "%{hiera('wso2::datasources::common::default_auto_commit')}"
            validation_query: "%{hiera('wso2::datasources::mysql::validation_query')}"
            validation_interval: "%{hiera('wso2::datasources::common::validation_interval')}"

   2. Create following database schemas in MySQL server. The databases need to be created in each of the pattern is listed below.

      Names of the databases that need to be created	Description
      apimgtdb	Database used for managing APIs.
      configdb	Database used for the config registry.
      govregdb	Database used for the gov registry.
      userdb	Database used for the user management and user store.
      mbstoredb	Database used for message broker.
      statdb	Database used for getting statistics for API Manager.

      Addition to that if you are using pattern-6, create analyticseventstoredb and analyticprocesseddatastoredb which are used for Analytics recode store.

      You need to run MySQL database scripts for following tables as mentioned below.

      database	script
      apimgtdb	<API-M_HOME>/dbscripts/apimgt/mysql.sql If you are using MySQL 5.7 or later version, select mysql5.7.sql located in same directory without using mysql.sql script.
      configdb	<API-M_HOME>/dbscripts/mysql.sql If you are using MySQL 5.7 or later version, select mysql5.7.sql located in same directory without using mysql.sql script.
      govregdb	<API-M_HOME>/dbscripts/mysql.sql If you are using MySQL 5.7 or later version, select mysql5.7.sql located in same directory without using mysql.sql script.
      userdb	<API-M_HOME>/dbscripts/mysql.sql If you are using MySQL 5.7 or later version, select mysql5.7.sql located in same directory without using mysql.sql script.
      mbstoredb	<API-M_HOME>/dbscripts/mb-store/mysql-mb.sql

      You do not need to run the database scripts against following databases as the database tables of them will be created at run time.

      • statdb
      • analyticseventstoredb
      • analyticprocesseddatastoredb

   3. If you are using a MySQL database, make the following changes:

      1. Download and copy the MySQL driver JAR (mysql-connector-java-5.1.39-bin.jar) to the locations /etc/puppet/environments/production/modules/wso2am_runtime/files/configs/repository/components/lib and /etc/puppet/environments/production/modules/wso2am_analytics/files/configs/repository/components/lib in Puppet Master.

      2. Uncomment the file_list entry for JDBC connector JAR in the relevant Hiera data files.

         wso2::file_list:
         	- "repository/components/lib/%{hiera('wso2::datasources::mysql::connector_jar')}"

      3. Update the JAR file name appropriately if your file name is not mysql-connector-java-5.1.39-bin.jar, which is set as the default value.

         wso2::datasources::mysql::connector_jar: mysql-connector-java-5.1.39-bin.jar

4. Configure deployment synchronization in each of the Gateway related nodes. 
   Use one of the following methods to carry out this configuration. 

   • Rsync

     WSO2 recommends rsync instead of SVN, for deployment synchronization as it is an efficient, easy to use and lightweight solution compared to SVN.

     
     

     Why can't I use SVN based deployment synchronization (Dep Sync)?

     WSO2 has identified some inconsistencies when using Hazelcast clustering. As a result, from API-M 2.1.0 onwards WSO2 API-M has been designed so that it is possible to deploy API-M in a clustered setup without using Hazelcast clustering, so that users can use Hazelcast clustering only when absolutely necessary. However, if you use deployment synchronization as a content synchronization mechanism, you are compelled to use Hazelcast clustering. Therefore, WSO2 does not recommend using SVN based deployment synchronization.

     If you prefer to use rsync, see Configuring rsync for Deployment Synchronization.

   • SVN Based
     Make the respective SVN based configurations in the following files based on the pattern that you are using.

     Pattern	Files to Update
     Pattern 3	/etc/puppet/hieradata/wso2/wso2am_runtime/pattern-3/gateway-manager.yaml /etc/puppet/hieradata/wso2/wso2am_runtime/pattern-3/gateway-worker.yaml
     Pattern 4	/etc/puppet/hieradata/wso2/wso2am_runtime/pattern-4/gateway-manager-env1.yaml /etc/puppet/hieradata/wso2/wso2am_runtime/pattern-4/gateway-worker-env1.yaml /etc/puppet/hieradata/wso2/wso2am_runtime/pattern-4/gateway-manager-env2.yaml /etc/puppet/hieradata/wso2/wso2am_runtime/pattern-4/gateway-worker-env1.yaml
     Pattern 5	/etc/puppet/hieradata/wso2/wso2am_runtime/pattern-5/gateway-manager.yaml /etc/puppet/hieradata/wso2/wso2am_runtime/pattern-5/gw-plus-km.yaml.yaml
     Pattern 6	/etc/puppet/hieradata/wso2/wso2am_runtime/pattern-6/gateway-manager.yaml /etc/puppet/hieradata/wso2/wso2am_runtime/pattern-6/gateway-worker.yaml

     Configure the SVN based deployment synchronization changes as follows:

   • 1. Uncomment the SVN based deployment synchronization configurations in the following files based on the pattern and update the configurations where required.
        Patterns 3 to 6 are configured for SVN based deployment synchronization; however, the configurations are commented out by default.

        Example

        wso2::dep_sync:
            enabled: true
            auto_checkout: true
            auto_commit: true
            repository_type: svn
            svn:
               url: http://svnrepo.example.com/repos/
               user: username
               password: password
               append_tenant_id: true

     2. Copy the required JARs for SVN, into the respective locations.
        1. Copy the svnkit-all-1.8.7.wso2v1.jar into the <PUPPET_HOME>/modules/wso2am/files/configs/repository/components/dropins directory.
        2. Copy the  trilead-ssh2-1.0.0-build215.jar into the <PUPPET_HOME>/modules/wso2am/files/configs/repository/components/lib directory.

        3. Uncomment the file_list entries for the latter mentioned two JAR files in the respective Hiera data files related to gateway nodes. 

           wso2::file_list:
              -  "repository/components/dropins/svnkit-all-1.8.7.wso2v1.jar"
              -  "repository/components/lib/trilead-ssh2-1.0.0-build215.jar"