The following sections give you information and instructions on how to cluster the message broker profile of WSO2 EI with a third-party load balancer.
...
Configure the datasources to point to the
REGISTRY_LOCAL1,WSO2_REGISTRY_DB, andWSO2_USER_DBdatabases as follows in the<EI_HOME>/wso2/broker/conf/datasources/master-datasources.xml file as follows:Tip Replace the username, password, and database URL of your MySQL environment accordingly.
Code Block language xml <datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> <providers> <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> </providers> <datasources> <datasource> <name>REGISTRY_LOCAL1</name> <description>The datasource used for registry- local</description> <jndiConfig> <name>jdbc/WSO2CarbonDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL1?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>REGISTRY_DB</name> <description>The datasource used for registry- config/governance</description> <jndiConfig> <name>jdbc/WSO2RegistryDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>WSO2_USER_DB</name> <description>The datasource used for registry and user manager</description> <jndiConfig> <name>jdbc/WSO2UMDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> </datasources> </datasources-configuration>Tip Repeat this configuration on the second WSO2 EI node to configure the datasources to point to the
REGISTRY_LOCAL2,WSO2_REGISTRY_DB, andWSO2_USER_DBdatabases as follows: (Change the username, password, and database URL as needed for your environment):Code Block language xml <datasources-configuration xmlns:svns="http://org.wso2.securevault/configuration"> <providers> <provider>org.wso2.carbon.ndatasource.rdbms.RDBMSDataSourceReader</provider> </providers> <datasources> <datasource> <name>REGISTRY_LOCAL2</name> <description>The datasource used for registry- local</description> <jndiConfig> <name>jdbc/WSO2CarbonDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_LOCAL2?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>REGISTRY_DB</name> <description>The datasource used for registry- config/governance</description> <jndiConfig> <name>jdbc/WSO2RegistryDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/REGISTRY_DB?autoReconnect=true</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> <datasource> <name>WSO2_USER_DB</name> <description>The datasource used for registry and user manager</description> <jndiConfig> <name>jdbc/WSO2UMDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://carbondb.mysql-wso2.com:3306/WSO2_USER_DB</url> <username>regadmin</username> <password>regadmin</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource> </datasources> </datasources-configuration>Add the following configuration in the
<EI_HOME>/wso2/broker/conf/user-mgt.xml file to configure the user stores.Tip Enter the datasource information for the user store that you configured in the
<EI_HOME>/wso2/broker/conf/datasources/master-datasources.xml file. You can change the admin username and password as well. However, you should do this before starting the server.Code Block language xml <Configuration> <AddAdmin>true</AddAdmin> <AdminRole>admin</AdminRole> <AdminUser> <UserName>admin</UserName> <Password>admin</Password> </AdminUser> <EveryOneRoleName>everyone</EveryOneRoleName> <Property name="dataSource">jdbc/WSO2UMDB</Property> </Configuration>Remove or uncomment the default H2-based WSO2_MB_STORE_DB following configuration in the
<EI_HOME>/wso2/broker/conf/datasources/master-datasources.xmlfile.Code Block language xml <datasource> <name>WSO2_MB_STORE_DB</name> <description>The datasource used for message broker database</description> <jndiConfig> <name>WSO2MBStoreDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:h2:repository/database/WSO2MB_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000</url> <username>wso2carbon</username> <password>wso2carbon</password> <driverClassName>org.h2.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> <defaultAutoCommit>false</defaultAutoCommit> </configuration> </definition> </datasource>Uncomment or add the following
WSO2_MB_STORE_DBconfiguration in the<EI_HOME>/wso2/broker/conf/datasources/master-datasources.xmlfile based on your DBMS type.Tip Update the JDBC URL to correctly point to your database and enter the username and password for your database user with the proper permissions.
Localtabgroup Localtab title Configuring MSSQL Code Block language xml <datasource> <name>WSO2_MB_STORE_DB</name> <jndiConfig> <name>jdbc/WSO2MBStoreDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <defaultAutoCommit>false</defaultAutoCommit> <dataSourceClassName>com.microsoft.sqlserver.jdbc.SQLServerXADataSource</dataSourceClassName> <dataSourceProps> <property name="URL">jdbc:sqlserver://127.0.0.1\SQLExpress</property> <property name="databaseName">wso2mb</property> <property name="user">sa</property> <property name="password">mssql</property> </dataSourceProps> </configuration> </definition> </datasource>Localtab title Configuring MySQL Code Block language xml <datasource> <name>WSO2_MB_STORE_DB</name> <jndiConfig> <name>WSO2MBStoreDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <url>jdbc:mysql://localhost/wso2_mb</url> <username>root</username> <password>root</password> <maxActive>50</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> <defaultAutoCommit>false</defaultAutoCommit> </configuration> </definition> </datasource>Localtab title Configuring Oracle Code Block language xml <datasource> <name>WSO2_MB_STORE_DB</name> <jndiConfig> <name>WSO2MBStoreDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <defaultAutoCommit>false</defaultAutoCommit> <driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName> <url>jdbc:oracle:thin:@localhost:1521/orcl</url> <maxActive>100</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> <username>scott</username> <password>tiger</password> </configuration> </definition> </datasource>Uncomment or add the following configuration in the
<EI_HOME>/wso2/broker/conf/broker.xmlfile. (Enter the message store based on your database type and Andes context store).Localtabgroup Localtab title Configuring MSSQL ...<persistence><messageStoreclass="org.wso2.andes.store.rdbms.RDBMSMessageStoreImpl"><propertyname="dataSource">jdbc/MSSQLWSO2MBStoreDB</property></messageStore><andesContextStoreclass="org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl"><propertyname="dataSource">jdbc/MSSQLWSO2MBStoreDB</property></andesContextStore>...</persistence>Localtab title Configuring MySQL ...<persistence><messageStoreclass="org.wso2.andes.store.rdbms.RDBMSMessageStoreImpl"><propertyname="dataSource">jdbc/MySQLMessageStore</property></messageStore><andesContextStoreclass="org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl"><propertyname="dataSource">jdbc/MySQLMessageStore</property></andesContextStore>...</persistence>Localtab title Configuring Oracle ...<persistence><messageStoreclass="org.wso2.andes.store.rdbms.RDBMSMessageStoreImpl"><propertyname="dataSource">WSO2MBStoreDB</property></messageStore><contextStoreclass="org.wso2.andes.store.rdbms.RDBMSAndesContextStoreImpl"><propertyname="dataSource">WSO2MBStoreDB</property></contextStore>...</persistence>Edit the
<EI_HOME>/wso2/broker/conf/axis2/axis2.xmlfile as follows to set up the cluster configurations.- Enable clustering for this node as follows:
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true"> - Set the membership scheme to "wka" to enable the well-known address registration method. (This node sends cluster initiation messages to the WKA members):
<parameter name="membershipScheme">wka</parameter> - Specify the name of the cluster this node will join as folows:
<parameter name="domain">wso2.ei.domain</parameter>
- Specify the host to communicate cluster messages as follows:
<parameter name="localMemberHost">xxx.xxx.xxx.xx2</parameter> Specify the port to communicate cluster messages as follows:
<parameter name="localMemberPort">4100</parameter>Info This port number is not affected by the port offset value specified in the
<EI_HOME>/wso2/broker/conf/carbon.xmlfile. If this port number is already assigned to another server, the clustering framework automatically increments this port number. However, if there are two servers running on the same machine, ensure that a unique port is set for each server.Specify the well-known members as follows: (The port value for the WKA node must be the same value as it's localMemberPort (in this case it is 4100).
Info You can also use IP address ranges for the hostname (e.g., 192.168.1.2-10). However, you can define a range only for the last portion of the IP address. Smaller the range, the faster the time it takes to discover members since each node has to scan a lesser number of potential members.
- Enable clustering for this node as follows:
Add the following thrift-related configurations in the
<EI_HOME>/wso2/broker/conf/broker.xmlfile.Tip This file is the root configuration file of brokering. Do the changes you do to this file in all the broker profile nodes. Configure the
thriftServerHostvalue to point to the IP address of the message broker profile node.Code Block language xml <coordination> <nodeID>default</nodeID> <thriftServerHost>192.168.0.103</thriftServerHost> <thriftServerPort>7611</thriftServerPort> <thriftSOTimeout>0</thriftSOTimeout> </coordination>Follwingaredetails on these configurations.
Configuration Description coordinationThis configuration is related to brokering thrift communications.
nodeIDIn a clustered deployment, an ID is assigned to each brokering node via the cluster node identifier. This element can be used to override the cluster node identifier for this brokering node. If the value for this element is left as default, the default node ID is generated using the IP and a universally unique identifier (UUID). The node ID of each member in a cluster must be unique.thriftServerHostThis is a sub-element of the
<coordination>tag. WSO2 EI uses Apache Thrift for communications relating to message delivery. Therefore, an Apache Thrift server is started in each brokering node in a clustered deployment. This element should point to the IP address of the Apache Thrift server. This should point to the IP address of the brokering node that hosts the thrift server. The default value for this islocalhost. For example, if you are configuring a brokering node hosted in 192.168.0.102 as the thrift server, this value should be 192.168.0.102.thriftServerPortThis is another sub-element of the
<coordination>tag. This should point to the port of the thrift server in the message broker profile. The default port is 7611. It is recommended to use this port for all broker nodes in your cluster.thriftSOTimeoutThis is used to handle half-open TCP connections between the broker nodes in a cluster. In such situations, the socket may need to have a timeout value to invalidate the connection. (in milliseconds). A timeout of zero is interpreted as an infinite timeout.
Testing the cluster
Follow the steps below to test the cluster.
...