Clustering DS 2.0.0

When clustering the WSO2 Dashboard Server, it is not necessary to separate the worker and manager concerns in the deployment pattern used. Instead, the Dashboard Server consists of two nodes in the cluster. Both the Portal and the Management Console can be accessed from both nodes. The nodes are load balanced in a round robin fashion using a load balancer of your choice. This topic uses Nginx to demonstrate load balancing.

The following table indicates the URLs used to access the Portal and Management Console.

Configuring Nginx as the load balancer

The following are the steps required to configure NGINX Plus version 1.7.11 (nginx-plus-r6-p1).

1. Install Nginx
2. Navigate to the following location
   /etc/nginx/conf.d
3. Create a file named ds.conf (you can also use the default.conf file that is available there by default).

4. Add the following content into the ds.conf file.

   upstream httpdsportal {
           server 192.168.48.96:9763 backup;
           server 192.168.48.77:9763;
   
   
   }
   
   upstream httpsdsportal {
       server 192.168.48.96:9443 backup;
       server 192.168.48.77:9443;
   
               sticky learn create=$upstream_cookie_jsessionid
               lookup=$cookie_jsessionid
               zone=client_sessions_ds:1m;
   
   }
   
   
   map $http_upgrade $connection_upgrade {
       default upgrade;
       '' close;
   }
   
   server {
           listen 80;
           server_name ds.wso2.com;
           location / {
                  proxy_set_header X-Forwarded-Host $host;
                  proxy_set_header X-Forwarded-Server $host;
                  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                  proxy_set_header Host $http_host;
                  proxy_read_timeout 5m;
                  proxy_send_timeout 5m;
                  proxy_pass http://httpdsportal/;
                  proxy_redirect http://httpdsportal/ http://ds.wso2.com/;
   
                  proxy_http_version 1.1;
                  proxy_set_header Upgrade $http_upgrade;
                  proxy_set_header Connection "upgrade";
           }
   }
   
   
   server {
       listen 443;
       server_name ds.wso2.com;
       ssl on;
       ssl_certificate /etc/nginx/ssl/ds/ds_mgt3.crt;
       ssl_certificate_key /etc/nginx/ssl/ds/ds_mgt3.key;
       location / {
                  proxy_set_header X-Forwarded-Host $host;
                  proxy_set_header X-Forwarded-Server $host;
                  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                  proxy_set_header Host $http_host;
                  proxy_read_timeout 5m;
                  proxy_send_timeout 5m;
                  proxy_pass https://httpsdsportal/;
                  proxy_redirect https://httpsdsportal/ https://ds.wso2.com/;
   
           }
   
   }

   Tip: The servers marked as "backup" are used for failover scenarios.

5. Restart the NGinx server using the following command.
   sudo service nginx restart

Create SSL certificates

See Configuring NGINX for more information on how to configure this. The following is a sample set of instructions.

1. Copy the cert file from Nginx to the <DS_HOME>/repository/resources/security directory.
2. Import the certificate
   keytool -import -alias certalias -file ds_mgt3.crt -keystore client-truststore.jks -storepass wso2carbon

Configurations in the cluster nodes

The configurations in this section must be done for both nodes in the cluster.

Configuring the axis2.xml file

Open the <DS_HOME>/repository/conf/axis2/axis2.xml file and do the following configurations.

1. Enable clustering by setting enable to true in the clustering tag.
   <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
2. Set the membership scheme to wka to enable the well-known address registration method
   <parameter name="membershipScheme">wka</parameter>
3. Specify the name of the cluster this node will join
   <parameter name="domain">wso2.ds.domain</parameter>

4. Specify the IP addresses of your server.

5. Specify the port used to communicate cluster messages.

6. Specify the well known members.

Configuring the carbon.xml file

Open the <DS_HOME>/repository/conf/carbon.xml file and do the following configurations.

1. Configure the HostName and MgtHostName.

   <HostName>ds.wso2.com</HostName>
   <MgtHostName>ds.wso2.com</MgtHostName>

2. Enable SVN-based deployment synchronization.

   <DeploymentSynchronizer>
   	<Enabled>true</Enabled>
   	<AutoCommit>true</AutoCommit>
   	<AutoCheckout>true</AutoCheckout>
   	<RepositoryType>svn</RepositoryType>
   	<SvnUrl>http://192.168.48.248/svn/wso2qarepo/ds200repo/</SvnUrl>
   	<SvnUser>wso2qa</SvnUser>
   	<SvnPassword>wso2qa123</SvnPassword>
   	<SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
   </DeploymentSynchronizer>

3. Download http://product-dist.wso2.com/tools/svnkit-all-1.8.7.wso2v1.jar and install it by copying it into the <DS_HOME>/repository/components/dropins folder.
4. Download http://maven.wso2.org/nexus/content/groups/wso2-public/com/trilead/trilead-ssh2/1.0.0-build215/trilead-ssh2-1.0.0-build215.jar and copy it to the <DS_HOME>/repository/components/lib folder.

For more information see Configuring SVN-Based Deployment Synchronizer.

Configuring the catalina-server.xml file

Open the <DS_HOME>/repository/conf/catalina-server.xml file and do the following. 

• Configure the proxy ports by doing the following configurations.

  <Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
  port="9763"
  proxyPort="80"
  ...
  />
  
  <Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
  port="9443"
  proxyPort="443"
  ...
  />

Configuring etc/hosts

Open the etc/hosts file and map the host names to the IP address of the Nginx load balancer. This should be configured in all the nodes used in the cluster (including the Nginx node).

192.168.48.xx ds.wso2.com

Connecting to a Oracle database and mounting the registry

master-datasources.xml Configurations

1. Add the data sources for Registry and User databases

<datasource>
<name>WSO2_DS_USER_DB</name>
<jndiConfig>
<name>jdbc/DS_USER_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<url>jdbc:oracle:thin:@192.168.48.121:1521/pdbora12c</url>
<maxActive>100</maxActive>
<maxWait>60000</maxWait>
<minIdle>5</minIdle>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1 FROM DUAL</validationQuery>
<validationInterval>30000</validationInterval>
<username>ds_xxxx</username>
<password>ds_xxxx</password>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>

<datasource>
<name>WSO2_DS_REGISTRY_DB</name>
<jndiConfig>
<name>jdbc/DS_REGISTRY_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
<url>jdbc:oracle:thin:@192.168.48.121:1521/pdbora12c</url>
<maxActive>100</maxActive>
<maxWait>60000</maxWait>
<minIdle>5</minIdle>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1 FROM DUAL</validationQuery>
<validationInterval>30000</validationInterval>
<username>ds_xxx</username>
<password>ds_xxx</password>
<defaultAutoCommit>false</defaultAutoCommit>
</configuration>
</definition>
</datasource>

2. Copy the ojdbc7.jar driver for the database to the <DS_HOME>/repository/components/lib directory.

registry.xml Configurations

1. Add/Update the following configurations

<dbConfig name="sharedregistry">
<dataSource>jdbc/DS_REGISTRY_DB</dataSource>
</dbConfig>

<remoteInstance url="https://localhost:9443/registry">
<id>instanceid</id>
<dbConfig>sharedregistry</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
<cacheId>ds_long_reg@jdbc:oracle:thin:@192.168.48.121:1521/pdbora12c</cacheId>
</remoteInstance>

<mount path="/_system/config" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/config</targetPath>
</mount>

<mount path="/_system/governance" overwrite="true">
<instanceId>instanceid</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

user-mgt.xml Configurations

1. Point the user store to the new database created

<Configuration>
<AddAdmin>true</AddAdmin>
<AdminRole>admin</AdminRole>
<AdminUser>
<UserName>admin</UserName>
<Password>admin</Password>
</AdminUser>
<EveryOneRoleName>everyone</EveryOneRoleName> <!-- By default users in this role sees the registry root -->
<Property name="isCascadeDeleteEnabled">true</Property>
<Property name="dataSource">jdbc/DS_USER_DB</Property>
</Configuration>

repository/deployment/server/jaggeryapps/portal/configs/designer.json configuration

Add the hostname of the cluster in the below configurations in order to render the gadgets in the dashboards. Also, you can access the dashboard using the ip address without any issue with this configuration.

"host": {
"hostname": "ds.wso2.com",
"port": "443",
"protocol": "https"
}
}

Starting the server
If you have configured a different database instead of the default start as follows.
sh wso2server.sh -Dsetup

2. To start the pack normally
sh wso2server.sh or sh wso2server.sh start