Page Comparison

The following sections give you sections provide information and instructions on how to cluster the integration Integration profile of WSO2 Enterprise Integrator (WSO2 EI) with a third-party load balancer. 

The clustering deployment pattern

This pattern has two WSO2 EI nodes to serve service requests with high availability and scalability. It allows access to the Management Console through an external load balancer and directs service requests to nodes through this load balancer. The following image depicts the sample pattern this clustering deployment scenario will follow.

Image RemovedImage Added

...

The load balancer automatically distributes incoming traffic across multiple WSO2 product instances. It enables you to achieve greater levels of fault tolerance in your cluster and provides the required balancing of load needed to distribute traffic.

Follow the steps below to configure NGINX Plus version 1.7.11 or NGINX community version 1.9.2 as the load balancer.

1. Install NGINX Plus or Plus or Nginx community version configured version configured in a server within your cluster network.

2. Create a VHost file (ei.http.conf) in the /etc/nginx/conf.d directory and add the following configurations into it. This configures NGINX Plus to direct the HTTP requests to the two WSO2 EI nodes via the HTTP 80 port using the Integration nodes (xxx.xxx.xxx.xx1 and xxx.xxx.xxx.xx2) via the HTTP 80 port using the http://ei.wso2.com/> URL. 

   Code Block
   title Nginx Community Version and NGINX Plus
   upstream wso2.ei.com { server xxx.xxx.xxx.xx3xx1:8280; server xxx.xxx.xxx.xx4xx2:8280; } server { listen 80; server_name ei.wso2.com; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_read_timeout 5m; proxy_send_timeout 5m; proxy_pass http://wso2.ei.com; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } }

3. Create a VHost file (ei.https.conf) in the /etc/nginx/conf.d directory and add the following configurations into it. This configures NGINX Plus to direct the HTTPS requests to the WSO2 EI nodes via two Integration nodes (xxx.xxx.xxx.xx1 and xxx.xxx.xxx.xx2) via the HTTPS 443 port using the  the  https://ei.wso2.com/> URL URL.

   Localtabgroup

   Localtab active true title Nginx Community Version Code Block upstream ssl.wso2.ei.com { server xxx.xxx.xxx.xx3xx1:8243; server xxx.xxx.xxx.xx4xx2:8243; ip_hash; } server { listen 443; server_name ei.wso2.com; ssl on; ssl_certificate /etc/nginx/ssl/server.crt; ssl_certificate_key /etc/nginx/ssl/server.key; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_read_timeout 5m; proxy_send_timeout 5m; proxy_pass https://ssl.wso2.ei.com; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } } Localtab title NGINX Plus Code Block upstream ssl.wso2.ei.com { server xxx.xxx.xxx.xx3xx1:8243; server xxx.xxx.xxx.xx4xx2:8243;   sticky learn create=$upstream_cookie_jsessionid lookup=$cookie_jsessionid zone=client_sessions:1m; } server { listen 443; server_name ei.wso2.com; ssl on; ssl_certificate /etc/nginx/ssl/server.crt; ssl_certificate_key /etc/nginx/ssl/server.key; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_read_timeout 5m; proxy_send_timeout 5m; proxy_pass https://ssl.wso2.ei.com;   proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } }

4. Configure Nginx to access the Management Console as management console as https://mgtui.asei.wso2.com/carbon via HTTPS  via HTTPS 443 port. To do this, create a VHost file (ui.asei.https.conf ) in the in the /etc/nginx/conf.d/ directory and  directory and add the following configurations into it.

   Code Block
   title Nginx Community Version and NGINX Plus
   server { listen 443; server_name ui.ei.wso2.com; ssl on; ssl_certificate /etc/nginx/ssl/server.crt; ssl_certificate_key /etc/nginx/ssl/server.key; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_read_timeout 5m; proxy_send_timeout 5m; proxy_pass https://xxx.xxx.xxx.xx3xx1:9443/;   proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } error_log /var/log/nginx/ui-error.log ; access_log /var/log/nginx/ui-access.log; }

5. Follow the instructions below to create SSL certificates for both WSO2 EI Integration nodes.

   Tip
   Enter the host name (i.e., ei.wso2.com) as the common name when creating keys.

   1. Execute the following command to create the Server Key:

       $sudo opensslgenrsa -des3 -out server.key

       

      Code Block
      $sudo openssl genrsa -des3 -out server.key 1024

   2. Execute the following command to request to sign the certificate:

       $sudo openssl req 

      Code Block
      $sudo openssl req -new -key server.key -out server.csr

   3. Execute the following commands to remove the passwords:

      Code Block
      $sudo cp server.

      key 

      key server.key.org

       
      $sudo openssl rsa -in 

      $sudo openssl rsa -in server.key.org

       

      -out server.key

   4. Execute the following command to sign your SSL Certificate: $sudo openssl x509

      Code Block
      $sudo openssl x509 -req

       

      -days 365 -in server.csr

       

      -

      signkey server

      signkey server.key -out server.crt

   5. Execute the following command to add the certificate to the the <EI_HOME>/repository/resources/security/client-truststore.jks file: keytool

      Code Block
      keytool -import -trustcacerts -alias server -file server.crt -keystore client-truststore.jks

      Tip
      Give the default password wso2carbon when prompted.

6. Execute the following command to restart the NGINX Plus server: $sudo service nginx  restart

   Code Block
   $sudo service nginx restart

    

   Tip
   Execute the following command if you do not need to restart the server when you are simply making a modification to the VHost file: $sudo service nginx reload  Code Block $sudo service nginx reload

Creating the databases

All profiles of WSO2 EI uses a database to store information such as user management details and registry data. All nodes in the cluster must use one central database for config and governance registry mounts. You can create the following databases and associated datasources.

...

.

WSO2_USER_DB

...

Following the steps below to create the databases necessary.

1. Download and install MySQL Server.

2. Download the MySQL JDBC driver.

3. Download and unzip the WSO2 EI binary distribution. 

   Tip
   Throughout this guide, <EI_HOME> refers to the extracted directory of the WSO2 EI product distribution.

4. Unzip the downloaded MySQL driver, and copy the MySQL JDBC driver JAR (mysql-connector-java-x.x.xx-bin.jar) into the <EI_HOME>/lib/ directory of both WSO2 EI nodes.

5. Add the following line to the /etc/hosts file to define the hostname for configuring permissions for the new database: <MYSQL-DB-SERVER-IP> IP> carbondb.mysql-wso2.com

   Note
   Do this step only if your database is not on your local machine and on a separate server.

6. Execute the following command in a terminal/command window, where username is the username you want to use to access the databases: mysql -u username -p

7. Specify the password to access the databases with the username you specified when prompted.

8. Create the databases using the following commands:

   Code Block
   language sql
   mysql> create database WSO2_USER_DB; mysql> use WSO2_USER_DB; mysql> source <EI_HOME>/dbscripts/mysql.sql; mysql> grant all on WSO2_USER_DB.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin"; mysql> create database REGISTRY_DB; mysql> use REGISTRY_DB; mysql> source <EI_HOME>/dbscripts/mysql.sql; mysql> grant all on REGISTRY_DB.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin"; mysql> create database REGISTRY_LOCAL1; mysql> use REGISTRY_LOCAL1; mysql> source <EI_HOME>/dbscripts/mysql.sql; mysql> grant all on REGISTRY_LOCAL1.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin"; mysql> create database REGISTRY_LOCAL2; mysql> use REGISTRY_LOCAL2; mysql> source <EI_HOME>/dbscripts/mysql.sql; mysql> grant all on REGISTRY_LOCAL2.* TO regadmin@"carbondb.mysql-wso2.com" identified by "regadmin";

   Info
   title About using MySQL in different operating systems
   For users of Microsoft Windows, when creating the database in MySQL, it is important to specify the character set as latin1. Failure to do this may result in an error (error code: 1709) when starting your cluster. This error occurs in certain versions of MySQL (5.6.x) and is related to the UTF-8 encoding. MySQL originally used the latin1 character set by default, which stored characters in a 2-byte sequence. However, in recent versions, MySQL defaults to UTF-8 to be friendlier to international users. Hence, you must use latin1 as the character set as indicated below in the database creation commands to avoid this problem. Note that this may result in issues with non-latin characters (like Hebrew, Japanese, etc.). The following is how your database creation command should look. mysql> create database <DATABASE_NAME> character set latin1; For users of other operating systems, the standard database creation commands will suffice. For these operating systems, the following is how your database creation command should look. mysql> create database <DATABASE_NAME>;

...

Add the following configurations to the the <EI_HOME>/conf/registry.xml file of each WSO2 EI node Integration node to configure the shared registry database and mounting details. This ensures that the shared registry for governance and configurations (i.e., the the REGISTRY_DB database database) mounts on both WSO2 EI Integration nodes. 

<dbConfig name="sharedregistry">
<dataSource>jdbc/WSO2RegistryDB</dataSource>
</dbConfig> 

<remoteInstance url="https://localhost:9443/registry">
    <id>instanceid</id>
    <dbConfig>sharedregistry</dbConfig>
    <readOnly>false</readOnly>
    <enableCache>true</enableCache>
    <registryRoot>/</registryRoot>
<cacheId>regadmin@jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</cacheId>
</remoteInstance>
 
<mount path="/_system/config" overwrite="true">
    <instanceId>instanceid</instanceId>
    <targetPath>/_system/eiconfig</targetPath>
</mount>
 
<mount path="/_system/governance" overwrite="true">
    <instanceId>instanceid</instanceId>
    <targetPath>/_system/governance</targetPath>
</mount>

Configuring the

...

Integration profile node

Do the following configurations for all nodes of your cluster.

1. Configure the datasourcestopoint to the REGISTRY_LOCAL1, WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows in the Open the <EI_HOME>/conf/datasources/master-datasources.xml file, and configure the datasources to point to the relevant databases for each Integration node. 

   Tip
   Replace the username, password, and database URL of your MySQL environment  accordingly. If you have not enabled SSL, append the useSSL=false property to the value of the <url> property.

   Localtabgroup
   Localtab title ESB Node 1 For Integration node 1, configure the datasources to point to the REGISTRY_LOCAL1, WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows: Code Block language xml <datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> <providers> <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> </providers> <datasources> <datasource> <name>REGISTRY_LOCAL1</name> <description>The datasource used for registry- local</description> <jndiConfig> <name>jdbc/WSO2CarbonDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL1?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>REGISTRY_DB</name> <description>The datasource used for registry- config/governance</description> <jndiConfig> <name>jdbc/WSO2RegistryDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>WSO2_USER_DB</name> <description>The datasource used for registry and user manager</description> <jndiConfig> <name>jdbc/WSO2UMDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> </datasources> </datasources-configuration>
   Tip
   Repeat this configuration on the second WSO2 EI node to Localtab title ESB Node 2 For node 2, configure the datasources to point to the REGISTRY_LOCAL2, WSO2_REGISTRY_DB, and WSO2_USER_DB databases as follows: ( shown below. Change the username, password, and database URL as needed for your environment ): . Code Block language xml <datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> <providers> <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> </providers> <datasources> <datasource> <name>REGISTRY_LOCAL2</name> <description>The datasource used for registry- local</description> <jndiConfig> <name>jdbc/WSO2CarbonDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL2?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>REGISTRY_DB</name> <description>The datasource used for registry- config/governance</description> <jndiConfig> <name>jdbc/WSO2RegistryDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>WSO2_USER_DB</name> <description>The datasource used for registry and user manager</description> <jndiConfig> <name>jdbc/WSO2UMDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> </datasources> </datasources-configuration>

2. Add the following configuration in the the <EI_HOME>/conf/user-mgt.xml file to configure the user stores. 

   Tip
   Enter the datasource information for the user store that you configured in the <EI_HOME>/conf/datasources/master-datasources.xml file. You can change the admin username and password as well. However, you should do this before starting the server.

   Code Block
   language xml
   <Configuration> <AddAdmin>true</AddAdmin> <AdminRole>admin</AdminRole> <AdminUser> <UserName>admin</UserName> <Password>admin</Password> </AdminUser> <EveryOneRoleName>everyone</EveryOneRoleName> <Property name="dataSource">jdbc/WSO2UMDB</Property> </Configuration>

3. Update the dataSource property in all nodes in

   the

   the <EI_HOME>/conf/user-mgt.xml

   file

    file as shown below to configure the datasource: 

   Code Block
   <Property name="dataSource">jdbc/WSO2UMDB</Property>

4. Edit Open the <EI_HOME>/conf/axis2/axis2.xml file as follows to set up the cluster configurations.for each of the two Integration nodes, and appply the following cluster configurations:

   • Enable clustering for

     this

     each node as follows:

      

     Code Block
     <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">

      


   • Set the membership scheme to "wka" to enable the well-known address registration method as shown below.

     (This

     Each node sends cluster initiation messages to the WKA members

     ):

     . 

     Code Block
     <parameter name="membershipScheme">wka</parameter>

   • Specify the name of the cluster

     this node will join as folows: <parameter

     to which the node joins:

     Code Block
     <parameter name="domain">wso2.ei.domain</parameter>

   • Specify the host to communicate cluster messages

     as follows: <parameter name="localMemberHost">xxx.xxx.xxx.xx2</parameter>

     . For example, if the IP addresses of the two Integration nodes are xxx.xxx.xxx.xx1 and xxx.xxx.xxx.xx2, they should be specified in the configuration as shown below.

     Localtabgroup
     Localtab title For ESB Node 1 Code Block language xml <parameter name="localMemberHost">xxx.xxx.xxx.xx1</parameter> Localtab title For ESB Node 2 Code Block language xml <parameter name="localMemberHost">xxx.xxx.xxx.xx2</parameter>

   • Specify the port to communicate cluster messages as follows: 

     Code Block
     <parameter name="localMemberPort">4100</parameter>

     Info
     This port number is not affected by the port offset value specified in the <EI_HOME>/conf/carbon.xml file. If this port number is already assigned to another server, the clustering framework automatically increments this port number. However, if there are two servers running on the same machine, ensure that a unique port is set for each server.

   • Specify the well-known members as follows: (The port value in the cluster as shown below. For example, when you configure one Integration node, you need to specify the other nodes in the cluster as well-known members as shown below. The port value for the WKA node must be the same value as it's localMemberPort (in this case it is 4100).

     Info

     You can also use IP address ranges for the hostname (e.g.,   192.168.1.2-10). However, you can define a range only for the last portion of the IP address. Smaller the range, the faster the time it takes to discover members since each node has to scan a lesser number of potential members. The best practice is to add all the members (including itself) in all the nodes to avoid any conflicts in configurations.

     Code Block
     language xml
     <members> <member> <hostName>xxx.xxx.xxx.xx1</hostName> <port>4100</port> </member> <member> <hostName>xxx.xxx.xxx.xx3<xx2</hostName> <port>4100</port> </member> </members>

   • Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpListener and
     org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver transportReceiver section.

     Code Block
     language xml
     <parameter name="WSDLEPRPrefix" locked="false">http://esbei.wso2.com:80</parameter>

   • Uncomment and edit the WSDLEPRPrefix element under org.apache.synapse.transport.passthru.PassThroughHttpSSLListener in the transportReceiver section.

     Code Block
     language xml
     <parameter name="WSDLEPRPrefix" locked="false">https://esbei.wso2.com:443</parameter>

5. Specify the port offset value in the <EI_HOME>/conf/carbon.xml file.

   Warning
   This step is optional and only required if all server instances are running on the same machine. This is not recommended for production environments. Change all ports used in your configurations based on the offset value if you are setting a port offset.

   Expand
   title Click here for more information on configuring the port offset.
   When you run multiple products/clusters or multiple instances of the same product on the same server or virtual machines (VMs), change their default ports with an offset value to avoid port conflicts. An offset defines the number by which all ports in the runtime (e.g., HTTP(S) ports) are increased. For example, if the default HTTP port is 9763 and the offset is 1, the effective HTTP port will change to 9764. For each additional product instance, set the port offset to a unique value. The offset of the default ports is zero. The port value will automatically increase as shown in the Port Value column in the following table, allowing all five WSO2 product instances or servers to run on the same machine. WSO2 product instance Port Offset Port Value WSO2 server 1 0 9443 WSO2 server 2 1 9444 WSO2 server 3 2 9445 WSO2 server 4 3 9446 WSO2 server 5 4 9447

   Code Block
   language xml
   <Ports> ... <Offset>0</Offset> ... </Ports>

6. Edit the <EI_HOME>/conf/carbon.xml file as follows to configure the hostname: <HostName>esbei.wso2.com</HostName>

7. Add the host entries to your DNS, or “/etc/hosts” file (in Linux) in all the nodes of the cluster to map the hostnames to the IP addresses. For an example, you can map the IP address of the database server. In this example, MySQL is used as the database server, so <MYSQL-DB-SERVER-IP> is the actual IP address of the database server and the host entry is as follows: <IP-of-MYSQL-DB-SERVER> carbondb.mysql-wso2.com

8. Edit the <EI_HOME>/conf/tomcat/catalina-server.xml file as follows to configure the HTTP/HTTPS proxy ports to communicate through the load balancer. This allows accessing the Management Console only through the load balancer.  :

   • Add proxyPort="80" to the org.apache.coyote.http11.Http11NioProtocol class with the port defined as 9763:

     Code Block
     language xml
     <Connector protocol="org.apache.coyote.http11.Http11NioProtocol" port="9763" proxyPort="80" ... />

   <Connector protocol
   • Add proxyPort="443" to the org.apache.coyote.http11.Http11NioProtocol

   "

   •  class with the port defined as 9443:

     Code Block
     language xml
     <Connector protocol="org.apache.coyote.http11.Http11NioProtocol" port="9443" proxyPort="443" ... />

   Expand
   title Click here for more information on this configuration.
   The Connector protocol tag sets the protocol to handle incoming traffic. The default value is HTTP/1.1, which uses an auto-switching mechanism to select either a blocking Java-based connector or an APR/native connector. If the PATH (Windows) or LD_LIBRARY_PATH (on most UNIX systems) environment variables contain the Tomcat native library, the APR/native connector will be used. If the native library cannot be found, the blocking Java-based connector will be used. Note that the APR/native connector has different settings from the Java connectors for HTTPS. The non-blocking Java connector used is an explicit protocol that does not rely on the auto-switching mechanism described above. The following is the value used: org.apache.coyote.http11.Http11NioProtocol The TCP port number is the value that this Connector will use to create a server socket and await incoming connections. Your operating system will allow only one server application to listen to a particular port number on a particular IP address. If the special value of 0 (zero) is used, Tomcat will select a free port at random to use for this connector. This is typically only useful in embedded and testing applications.

Setting up scheduled tasks

This section is applicable if you have Scheduled Tasks configured in your WSO2 EI server nodes. For instructions on how to configure task distribution in a clustered deployment, see Configuring a clustered task server in the Common Administration Guide.

Edit the clusteringPattern parameter in the <EI_HOME>/conf/axis2/axis2.xml file as follows, to ensure that tasks are scheduled in all WSO2 EI nodes: 

...

Configuring Hazelcast properties

WSO2 products use Hazelcast as its default clustering engine. You can configure the hazelcast properties for the product nodes by following the steps given below.

1. Create the hazelcast.properties file with the following property configurations, and copy the file to the <EI_HOME>/conf/ directory. 

   Code Block
   #Disabling the hazelcast shutdown hook hazelcast.shutdownhook.enabled=false #Setting the hazelcast logging type to log4j hazelcast.logging.type=log4j

   The above configurations are explained below.

   • Hazelcast shutdown hook: This configuration disables the shutdown hook in hazelcast, which ensures that the hazelcast instance shuts down gracefully whenever the product node shuts down. If the hazelcast shutdown hook is enabled (which is the default behavior of a product), you will see errors such as "Hazelcast instance is not active!" at the time of shutting down the product node: This is because the hazelcast instance shuts down too early when the shutdown hook is enabled.
   • Hazelcast logging type: This configuration sets the hazelcast logging type to log4j, which allows hazelcast logs to be written to the wso2carbon.log file.

2. If you have enabled log4j for hazelcast logging as shown above, be sure to enter the configuration shown below in the log4j.properties file (stored in the <EI_HOME>/conf/). This can be used to configure the log level for hazelcast logging. For a clustered production environment, it is recommended to use INFO as the log level as shown below.

   Code Block
   log4j.logger.com.hazelcast=INFO

Deploying artifacts across the nodes

One common approach for synchronizing artifacts across all cluster nodes is to use use rsync tool, which is a file copying tool. Therefore, you  you can first deploy artifacts in one node of the cluster and then use rsync to then  use rsync to copy those artifacts to other nodes as nodes as described below.

1. Create a file called file called nodes-list.txt that lists  that lists all the nodes in the deployment. The following is a sample of the file for two nodes.

   Tip
   Different nodes are separated by new lines.

   Code Block
   language text title nodes-list.txt
   ubuntu@192.168.1.1:~/setup/192.168.1.1/ei_node/repository/deployment/server ubuntu@192.168.1.2:~/setup/192.168.1.2/ei_node/repository/deployment/server

2. Create a file to synchronize the the  <PRODUCT_HOME>/repository/deployment/server/directory  directory between the nodes.

   Note
   You must create your own SSH key and define it as the pem_file. Alternatively, you can use an existing SSH key. Specify the ei_server_dir depending on the location in your local machine. Change the logs.txt file path and the lock location based on where they are locatedin your machine.

   Tip
   Configure rsync in the <EI_HOME>/repository/tenant/ directory to to share the tenant artifacts across the cluster.

   Code Block
   language applescript title rsync-for-ei-depsync.sh
   #!/bin/sh ei_server_dir=~/wso2ei-6.02.0/repository/deployment/server/ pem_file=~/.ssh/carbon-440-test.pem #delete the lock on exit trap 'rm -rf /var/lock/depsync-lock' EXIT mkdir /tmp/carbon-rsync-logs/ #keep a lock to stop parallel runs if mkdir /var/lock/depsync-lock; then echo "Locking succeeded" >&2 else echo "Lock failed - exit" >&2 exit 1 fi #get the nodes-list.txt pushd `dirname $0` > /dev/null SCRIPTPATH=`pwd` popd > /dev/null echo $SCRIPTPATH for x in `cat ${SCRIPTPATH}/nodes-list.txt` do echo ================================================== >> /tmp/carbon-rsync-logs/logs.txt; echo Syncing $x; rsync --delete -arve "ssh -i $pem_file -o StrictHostKeyChecking=no" $ei_server_dir $x >> /tmp/carbon-rsync-logs/logs.txt echo ================================================== >> /tmp/carbon-rsync-logs/logs.txt; done

3. Execute the following command in your CLI to create a Cron job that executes the above file every minute for deployment synchronization.    

   Code Block
   language actionscript3
   * * * * * /home/ubuntu/setup/rsync-for-depsync/rsync-for-ei-depsync.sh

...

1. Deploy artifacts to each product deployment location. 

   Info
   Use a deployment synchronization mechanism to synchronize the artifacts in the <EI_HOME>/repository/deployment/ directory. Always deploy artifacts first to the WSO2 EI Integration server profile node with the registry configured as read/write. Next, deploy the artifacts to the other nodes.

2. Restart the configured load balancer.

3. Execute the following command and start both WSO2 EI Integration nodes: sh <EI_HOME>/bin/integrator.sh

4. Check for ‘member joined’ log messages in all consoles.

   Info
   title Additional information on logs and new nodes
   When you terminate one node, all nodes identify that the node has left the cluster. The same applies when a new node joins the cluster. If you want to add another new node, copy existing node without any changes if you are running it on a new server (such as xxx.xxx.xxx.184). If you intend to use the new node on a server where another WSO2 product is running, use a copy of node and change the port offset accordingly in the the <EI_HOME>/conf/carbon.xml file. You also have to change localMemberPort in the <EI_HOME>/conf/axis2/axis2.xml file if that product has clustering enabled. Also, map all hostnames to the relevant IP addresses when creating a new node. The log messages indicate if the new node joins the cluster.

5. Access the Management Console management console through the LB using the following URL: https://xxx.xxx.xxx.xx2xx1:443/carbon  
6. Test load distribution via the following URLs: http://xxx.xxx.xxx.xx3xx1:80/ or https://xxx.xxx.xxx.xx3xx1:443/

7. Add a sample proxy service with the log mediator in the inSequence so that it will display logs in the terminals, and then observe the cluster messages sent.

8. Send a request to the endpoint through the load balancer to verify that the proxy service is activated only on the active node(s) while the nodes remain passive. This is to test that the load balancer manages the active and passive states of the nodes, activating nodes as needed and leaving the rest in passive mode. For example, you would send the request to the following URL: http://{Load_Balancer_Mapped_URL_for_worker}/services/{Sample_Proxy_Name 

Tuning performance of the cluster

Follow the steps below to tune performance of the cluster:

...