Configuring the Manager Node of a Cluster

Configuring the manager node involves the following tasks:

Configuring the datasource

You configure datasources to allow the manager node to point to the central database. Make sure that you copy the database driver JAR to the manager node and follow the steps described in Setting up the Database.

Setting up cluster configurations for the manager node

Configuring clustering for the manager node is similar to the way you configured it for the load balancer node, but the localMemberPort is 4001 instead of 4000, and you define the load balancer node instead of the AS manager node as the well-known member.

1. Open the <AS_HOME>/repository/conf/axis2/axis2.xml file.
2. Locate the clustering section and verify or configure the properties as follows (some of these properties are already set correctly by default):
   
   1. Enable clustering for this node: 
      <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
   2. Set the membership scheme to wka to enable the well-known address registration method (this node will send cluster initiation messages to WKA members that we will define later): 
      <parameter name="membershipScheme">wka</parameter>
   3. Specify the name of the cluster this node will join:
      <parameter name="domain">wso2.as.domain</parameter>

   4. Specify the host used to communicate cluster messages:

      You may run into issues when using host names in products based on WSO2 Carbon 4.2.0, so it is recommended that you use the IP address directly here.

      <parameter name="localMemberHost">xxx.xxx.xxx.xx3</parameter>

   5. Specify the port used to communicate cluster messages: 
      <parameter name="localMemberPort">4100</parameter>

      This port number will not be affected by the port offset in carbon.xml. If this port number is already assigned to another server, the clustering framework will automatically increment this port number. However, if two servers are running on the same machine, you must ensure that a unique port is set for each server. 

   6. The receiver's HTTP/HTTPS port values are without the portOffset addition; they get auto-incremented by portOffset. In the case of an ESB cluster, the 'WSDLEPRPrefix' parameter should point to the worker node's host name (as.wso2.com) and load balancer's HTTP (80)/HTTPS (443) transport ports.

   7. Change the members listed in the <members> element so that it is applicable for your worker/manager clustering deployment pattern. Note that clustering deployment pattern 1 is not relevant here; hence its exclusion.

   8. Change the following clustering properties. Ensure that you set the value of the subDomain as mgt to specify that this is the manager node, which will ensure that traffic for the manager node is routed to this member.

      <parameter name="properties">
                  <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/>
                  <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/>
      			<property name="subDomain" value="mgt"/>
      </parameter>

Configuring the port offset and host name

If we are running two Carbon-based products on the same server, we must change the port offset to avoid port conflicts. Additionally, we will add the cluster host name so that any requests sent to the manager host are redirected to the cluster, where the load balancer will pick them up and manage them.

1. Open <AS_HOME>/repository/conf/carbon.xml.
2. Locate the <Ports> tag and appropriately change the value of its sub-tag titled Offset. If you run three Carbon-based products on the same server, note that you would have a value of 2 here. This value increments based on the number of products you are running.
   <Offset>0</Offset> 
3. Locate the <HOSTNAME> tag and add the cluster host name: 
<HostName>as.wso2.com</HostName>
4. Locate the <MgtHostName> tag and uncomment it. Make sure that the management host name is defined as follows:
<MgtHostName>mgt.as.wso2.com</MgtHostName> 

If desired, you can also configure the deployment synchronizer in the carbon.xml file.

Configuring the catalina-server.xml file

Specify the following configurations in the catalina-server.xml file located in the <AS_HOME>/repository/conf/tomcat/ directory.

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
--------
/>
<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
--------
/>

The Connector protocol tag sets the protocol to handle incoming traffic. The default value is HTTP/1.1, which uses an auto-switching mechanism to select either a blocking Java-based connector or an APR/native connector. If the PATH (Windows) or LD_LIBRARY_PATH (on most UNIX systems) environment variables contain the Tomcat native library, the APR/native connector will be used. If the native library cannot be found, the blocking Java-based connector will be used. Note that the APR/native connector has different settings from the Java connectors for HTTPS.

The non-blocking Java connector used is an explicit protocol that does not rely on the auto-switching mechanism described above. The following is the value used:
org.apache.coyote.http11.Http11NioProtocol

The TCP port number is the value that this Connector will use to create a server socket and await incoming connections. Your operating system will allow only one server application to listen to a particular port number on a particular IP address. If the special value of 0 (zero) is used, Tomcat will select a free port at random to use for this connector. This is typically only useful in embedded and testing applications.

In the next section, we will map the host names we specified to real IPs.

Mapping host names to IPs

In the manager node we have specified two host names: carbondb.mysql-wso2.com for the MySQL server and as.wso2.com for the cluster. We will now map them to the actual IPs. Note that if you created the database on the same server as the manager node, you would have already added the first line, and if you created it on the same server as the load balancer, you would have already added the second line.

Open the server's /etc/hosts file and add the following lines, where <MYSQL-DB-SERVER-IP> is the actual IP address (in this example, xxx.xxx.xxx.206) of the database server.

<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
<AS MANAGER IP> mgt.as.wso2.com

In this example, it would look like this:

xxx.xxx.xxx.206 carbondb.mysql-wso2.com
xxx.xxx.xxx.xx3 mgt.as.wso2.com

We have now finished configuring the manager node and are ready to start the AS server.

Starting the AS server

Start the AS server by typing the following command in the terminal:

sh <AS_HOME>/bin/wso2server.sh -Dsetup 

The additional -Dsetup argument will create the required tables in the database.

AS should print logs to the server console indicating that the cluster initialization is complete.

We have now finished configuring the manager node. Next, we will configure the AS worker nodes.