Clustering DS 2.0.0

When clustering the WSO2 Dashboard Server, it is not necessary to separate the worker and manager concerns in the deployment pattern used. Instead, the Dashboard Server consists of two nodes in the cluster. Both the Portal and the Management Console can be accessed from both nodes. The nodes are load balanced in a round robin fashion using a load balancer of your choice. This topic uses Nginx to demonstrate load balancing.

The following table indicates the URLs used to access the Portal and Management Console.

Portal	Management Console
https://ds.wso2.com/portal	https://ds.wso2.com/carbon
http://ds.wso2.com/portal

Configuring Nginx as the load balancer

The following are the steps required to configure NGINX Plus version 1.7.11 (nginx-plus-r6-p1).

Install Nginx
Navigate to the following location
/etc/nginx/conf.d
Create a file named ds.conf (you can also use the default.conf file that is available there by default).

Add the following content into the ds.conf file.

upstream httpdsportal {
        server 192.168.48.96:9763 backup;
        server 192.168.48.77:9763;


}

upstream httpsdsportal {
    server 192.168.48.96:9443 backup;
    server 192.168.48.77:9443;

            sticky learn create=$upstream_cookie_jsessionid
            lookup=$cookie_jsessionid
            zone=client_sessions_ds:1m;

}


map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}

server {
        listen 80;
        server_name ds.wso2.com;
        location / {
               proxy_set_header X-Forwarded-Host $host;
               proxy_set_header X-Forwarded-Server $host;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header Host $http_host;
               proxy_read_timeout 5m;
               proxy_send_timeout 5m;
               proxy_pass http://httpdsportal/;
               proxy_redirect http://httpdsportal/ http://ds.wso2.com/;

               proxy_http_version 1.1;
               proxy_set_header Upgrade $http_upgrade;
               proxy_set_header Connection "upgrade";
        }
}


server {
    listen 443;
    server_name ds.wso2.com;
    ssl on;
    ssl_certificate /etc/nginx/ssl/ds/ds_mgt3.crt;
    ssl_certificate_key /etc/nginx/ssl/ds/ds_mgt3.key;
    location / {
               proxy_set_header X-Forwarded-Host $host;
               proxy_set_header X-Forwarded-Server $host;
               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header Host $http_host;
               proxy_read_timeout 5m;
               proxy_send_timeout 5m;
               proxy_pass https://httpsdsportal/;
               proxy_redirect https://httpsdsportal/ https://ds.wso2.com/;

        }

}

Tip: The servers marked as "backup" are used for failover scenarios.

Restart the NGinx server using the following command.
sudo service nginx restart

Create SSL certificates

See /wiki/spaces/CLUSTER44x/pages/9732195 for more information on how to configure this. The following is a sample set of instructions.

Copy the cert file from Nginx to the <DS_HOME>/repository/resources/security directory.
Import the certificate
keytool -import -alias certalias -file ds_mgt3.crt -keystore client-truststore.jks -storepass wso2carbon

Configurations in the cluster nodes

The configurations in this section must be done for both nodes in the cluster.

Configuring the axis2.xml file

Open the <DS_HOME>/repository/conf/axis2/axis2.xml file and do the following configurations.

Enable clustering by setting enable to true in the clustering tag.
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
Set the membership scheme to wka to enable the well-known address registration method
<parameter name="membershipScheme">wka</parameter>
Specify the name of the cluster this node will join
<parameter name="domain">wso2.ds.domain</parameter>
Specify the IP addresses of your server.
Specify the port used to communicate cluster messages.
Specify the well known members.

Configuring the carbon.xml file

Open the <DS_HOME>/repository/conf/carbon.xml file and do the following configurations.

Configure the HostName and MgtHostName.

<HostName>ds.wso2.com</HostName>
<MgtHostName>ds.wso2.com</MgtHostName>

Enable SVN-based deployment synchronization.

<DeploymentSynchronizer>
	<Enabled>true</Enabled>
	<AutoCommit>true</AutoCommit>
	<AutoCheckout>true</AutoCheckout>
	<RepositoryType>svn</RepositoryType>
	<SvnUrl>http://192.168.48.248/svn/wso2qarepo/ds200repo/</SvnUrl>
	<SvnUser>wso2qa</SvnUser>
	<SvnPassword>wso2qa123</SvnPassword>
	<SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

Download http://product-dist.wso2.com/tools/svnkit-all-1.8.7.wso2v1.jar and install it by copying it into the <DS_HOME>/repository/components/dropins folder.
Download http://maven.wso2.org/nexus/content/groups/wso2-public/com/trilead/trilead-ssh2/1.0.0-build215/trilead-ssh2-1.0.0-build215.jar and copy it to the <DS_HOME>/repository/components/lib folder.

For more information see Configuring SVN-Based Deployment Synchronizer.

Configuring the catalina-server.xml file

Open the <DS_HOME>/repository/conf/catalina-server.xml file and do the following.

Configure the proxy ports by doing the following configurations.

<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="9763"
proxyPort="80"
...
/>

<Connector protocol="org.apache.coyote.http11.Http11NioProtocol"
port="9443"
proxyPort="443"
...
/>

Configuring etc/hosts

Open the etc/hosts file and map the host names to the IP address of the Nginx load balancer. This should be configured in all the nodes used in the cluster (including the Nginx node).

192.168.48.xx ds.wso2.com

Configuring the designer.json file

Open the <DS_HOME>/repository/deployment/server/jaggeryapps/portal/configs/designer.json configuration file and do the following configurations.

Add the hostname of the cluster as indicated in the following configuration snippet in order to render the gadgets in the dashboards. Also, you can access the dashboard using the IP address without any issue with this configuration.
```
"host": {
	"hostname": "ds.wso2.com",
	"port": "443",
	"protocol": "https"
}
```

Connecting to a Oracle database and mounting the registry

The configurations in this section must be done in both nodes.

Configuring the master-datasources.xml file

Do the following configurations in the <DS_HOME>/repository/conf/datasources/master-datasources.xml file.

Add the datasources for the registry and the user databases.

User DB datasource

<datasource>
	<name>WSO2_DS_USER_DB</name>
	<jndiConfig>
		<name>jdbc/DS_USER_DB</name>
	</jndiConfig>
	<definition type="RDBMS">
		<configuration>
			<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
			<url>jdbc:oracle:thin:@192.168.48.121:1521/pdbora12c</url>
			<maxActive>100</maxActive>
			<maxWait>60000</maxWait>
			<minIdle>5</minIdle>
			<testOnBorrow>true</testOnBorrow>
			<validationQuery>SELECT 1 FROM DUAL</validationQuery>
			<validationInterval>30000</validationInterval>
			<username>ds_xxxx</username>
			<password>ds_xxxx</password>
			<defaultAutoCommit>false</defaultAutoCommit>
		</configuration>
	</definition>
</datasource>

Registry DB datasource

<datasource>
	<name>WSO2_DS_REGISTRY_DB</name>
	<jndiConfig>
		<name>jdbc/DS_REGISTRY_DB</name>
	</jndiConfig>
	<definition type="RDBMS">
		<configuration>
			<driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
			<url>jdbc:oracle:thin:@192.168.48.121:1521/pdbora12c</url>
			<maxActive>100</maxActive>
			<maxWait>60000</maxWait>
			<minIdle>5</minIdle>
			<testOnBorrow>true</testOnBorrow>
			<validationQuery>SELECT 1 FROM DUAL</validationQuery>
			<validationInterval>30000</validationInterval>
			<username>ds_xxx</username>
			<password>ds_xxx</password>
			<defaultAutoCommit>false</defaultAutoCommit>
		</configuration>
	</definition>
</datasource>

Copy the ojdbc7.jar driver for the database to the <DS_HOME>/repository/components/lib directory.

Configuring the registry.xml file

Open the <DS_HOME>/repository/conf/registry.xml file and do the following configurations.

Do the following configurations. These configurations enable you to mount the shared registry on both nodes.

<dbConfig name="sharedregistry">
	<dataSource>jdbc/DS_REGISTRY_DB</dataSource>
</dbConfig>


<remoteInstance url="https://localhost:9443/registry">
	<id>instanceid</id>
	<dbConfig>sharedregistry</dbConfig>
	<readOnly>false</readOnly>
	<enableCache>true</enableCache>
	<registryRoot>/</registryRoot>
	<cacheId>ds_long_reg@jdbc:oracle:thin:@192.168.48.121:1521/pdbora12c</cacheId>
</remoteInstance>


<mount path="/_system/config" overwrite="true">
	<instanceId>instanceid</instanceId>
	<targetPath>/_system/config</targetPath>
</mount>


<mount path="/_system/governance" overwrite="true">
	<instanceId>instanceid</instanceId>
	<targetPath>/_system/governance</targetPath>
</mount>

Configuring the user-mgt.xml file

Open the <DS_HOME>/repository/conf/user-mgt.xml file and do the following configurations.

Point the user store to the new database that you create for this purpose.

<Configuration>
	<AddAdmin>true</AddAdmin>
	<AdminRole>admin</AdminRole>
	<AdminUser>
		<UserName>admin</UserName>
		<Password>admin</Password>
	</AdminUser>


	<EveryOneRoleName>everyone</EveryOneRoleName> <!-- By default users in this role sees the registry root -->
	<Property name="isCascadeDeleteEnabled">true</Property>
	<Property name="dataSource">jdbc/DS_USER_DB</Property>
</Configuration>

Starting the server

If you have configured a different database instead of the default, start the server using the following command.
sh wso2server.sh -Dsetup
To start the pack normally, use the following command.
sh wso2server.sh or sh wso2server.sh start