Configuring CQL-based Cassandra

This topic provides instructions on how to configure the WSO2 Message Broker nodes to point to the Apache Cassandra ring.

The following instructions include all steps you must do on the Message Broker side.

1. Open the <MB_HOME>/repository/conf/datasources/master-datasources.xml file. This is where datasources are configured to point to the databases used by the Message Broker. This file consists of commented out configurations for datasources. The datasource configuration for Apache Cassandra is among these. The changes made to this file must be done in both broker nodes.

2. Uncomment the datasource configuration for external Cassandra.
   

   Tip: You can alternatively add the following configuration into the file if you are not familiar with commenting out XML configurations.

    The sample configuration for this is as follows.

   <datasource>
           <name>WSO2_CASSANDRA_DB</name>
           <description>The datasource used for cassandra</description>
           <jndiConfig>
               <name>WSO2MBStoreDB</name>
           </jndiConfig>
           <definition type="CASSANDRA">
              <configuration>
                   <async>false</async>
                   <clusterName>TestCluster</clusterName>
                   <compression>SNAPPY</compression>
                   <concurrency>100</concurrency>
                   <username>admin</username>
                   <password encrypted="true">admin</password>
                   <port>9042</port>
                   <maxConnections>100</maxConnections>
                   <hosts>
                       <host>localhost</host>
                   </hosts>
                   <loadBalancePolicy>
                       <exclusionThreshold>2.5</exclusionThreshold>
                       <latencyAware>true</latencyAware>
                       <minMeasure>100</minMeasure>
                       <policyName>RoundRobinPolicy</policyName>
                       <retryPeriod>10</retryPeriod>
                       <scale>2</scale>
                   </loadBalancePolicy>
                   <poolOptions>
                       <coreConnectionsForLocal>10</coreConnectionsForLocal>
                       <coreConnectionsForRemote>10</coreConnectionsForRemote>
                       <maxConnectionsForLocal>10</maxConnectionsForLocal>
                       <maxConnectionsForRemote>10</maxConnectionsForRemote>
                       <maxSimultaneousRequestsForLocal>10</maxSimultaneousRequestsForLocal>
                       <maxSimultaneousRequestsForRemote>10</maxSimultaneousRequestsForRemote>
                       <minSimultaneousRequestsForLocal>10</minSimultaneousRequestsForLocal>
                       <minSimultaneousRequestsForRemote>10</minSimultaneousRequestsForRemote>
                   </poolOptions>
                   <reconnectPolicy>
                       <baseDelayMs>100</baseDelayMs>
                       <policyName>ConstantReconnectionPolicy</policyName>
                   </reconnectPolicy>
                   <socketOptions>
                       <connectTimeoutMillis>10000</connectTimeoutMillis>
                       <keepAlive>true</keepAlive>
                       <readTimeoutMillis>15000</readTimeoutMillis>
                       <tcpNoDelay>true</tcpNoDelay>
                   </socketOptions>
               </configuration>
           </definition>
   </datasource>

   You must add the correct URL to the database and enter the username and password for a Cassandra database user with the proper permissions.

3. Uncomment the following configuration in the master-datasources.xml file under the providers tag. This ensures that CQL is configured to read and write to Apache Cassandra.
   

   <provider>org.wso2.carbon.cassandra.datareader.cql.CassandraDataSourceReader</provider>

   About data readers

   When connecting to Apache Cassandra, there are different ways to read and write to databases. The two clients used for this are CQL and Hector. These client options are defined as data readers based on this configuration. This is relevant only if you use Cassandra and is not applicable for other databases. So, to read or write to Cassandra, you can either use Hector or CQL.

4. Open the <MB_HOME>/repository/conf/broker.xml file. This is the root configuration file of Message Broker. The changes made to this file must be done in all the WSO2 Message Broker nodes.

5. Uncommenting the following configuration ensures that the messageStore and contextStore are based on CQL. Also, you need to comment out other message store and contex store definitions.

   <messageStore class="org.wso2.andes.store.cassandra.CQLBasedMessageStoreImpl">
       <property name="dataSource">WSO2MBStoreDB</property>
       <property name="keyspace">MB_KEYSPACE</property>
       <property name="gcGraceSeconds">864000</property>
       <property name="strategyClass">org.apache.cassandra.locator.SimpleStrategy</property>
       <property name="replicationFactor">1</property>
       <property name="readConsistencyLevel">ONE</property>
       <property name="writeConsistencyLevel">ONE</property>
   </messageStore>
   <contextStore class="org.wso2.andes.store.cassandra.CQLBasedAndesContextStoreImpl">
       <property name="dataSource">WSO2MBStoreDB</property>
       <property name="keyspace">MB_KEYSPACE</property>
       <property name="strategyClass">org.apache.cassandra.locator.SimpleStrategy</property>
       <property name="replicationFactor">1</property>
       <property name="readConsistencyLevel">ONE</property>
       <property name="writeConsistencyLevel">ONE</property>
   </contextStore>

   Make note of the following when using the above configuration.

   • The replication placement strategy (algorithm) to be used is defined in the strategyClass.

   • Add the following configuration for the GC grace seconds for Cassandra. This is the time to wait before garbage collecting commences in Cassandra.
     <property name="GCGraceSeconds">864000</property>

   • Specify the replicationFactor, readConsistencyLevel and writeConsistencyLevel. For more details on this, please read the Cassandra documentation on data consistency.