Configuring Default Cassandra Environment

WSO2 Storage Server is shipped with a default Cassandra environment, which contains one Cassandra cluster. If required, you can set up additional Cassandra environments for your system, which can consist of multiple clusters. The information given on this page explains how to set up and configure the default Cassandra environment.

Setting up a Cassandra cluster

WSO2 Storage Server consists of an embedded Cassandra instance. Therefore, the default Cassandra environment of Storage Server consists of a Cassandra cluster, which has one Cassandra node (the default embedded Cassandra instance) for provisioning NoSQL data stores. If you use this single node, default Cassandra cluster, tenant users can log in to the management console of SS and manage the NoSQL data stores for that single node Cassandra cluster.

However, in a production environment, you will need to have multiple Cassandra nodes for a Cassandra cluster. You can configure multiple Cassandra nodes for the default cluster by setting up multiple SS deployments as Cassandra nodes.

As depicted above, you will be using one SS deployment as the provisioning SS instance, which will have the UI enabled for users to log in. All the other SS deployments in the system will be used as back-end Cassandra nodes connected to one another, thereby forming the Cassandra cluster.

Configuring the SS nodes in the Cassandra cluster

As explained in the above section, Storage Server comes with configurations suited for standalone Cassandra deployment. However, once you set up a multi-node cluster, you must configure each of the nodes as follows:

Step 1: Changing the default IP's and ports

You must change listening and ports accordingly in <SS_HOME>/repository/conf/etc/cassandra.yaml file for each of the SS nodes.

1. Cassandra listening IP is used for inter-node communications in a clustered environment: listen_address: <Server listening IP or domain name>  

2. Storage port is used to exchange the data and the command between the cluster nodes: storage_port: 7000    

   This port changes according to <Offset> value in <Ports> section in carbon.xml. Changing storage_port value in cassandra.yaml will not affect the server.

3. If encrypted communication is enabled, the cluster uses the port defined in ssl_storage_port for cluster-related commands and data communication: ssl_storage_port: 7001     
   RPC listen address is used for the thrift-based communication between the server and client:   
   

   rpc_address: <IP_ADDRESS>
   # port for Thrift to listen for clients on 
   rpc_port: 9160

   RPC port changes according to <Offset> value in <Ports> section in carbon.xml. Changing rpc_port value in cassandra.yaml will not affect the server.

4. Native Transport Port is the port which is listening to CQL clients. Please note that the address on which the native transport is bound is the same as the rpc_address (to start the native transport server, start_native_transport should be equal to true, which is its default value). This needs to be set as follows:
   

   start_native_transport: true
   native_transport_port: 9042

Step 2: Pointing to remote Cassandra nodes

Each SS node in the cluster should be configured with the details of the other SS nodes in the cluster.

1. Open the hector-config.xml file stored in the <SS_HOME>/repository/conf/etc/hector-config.xml file. 

2. List all the nodes in the cluster. Shown below is how the default Cassnadra node is entered.
   

   <HectorConfiguration>
       <Cluster>
           <Name>ClusterOne</Name>
           <Nodes>localhost:9160</Nodes> 
           <AutoDiscovery disable="false" delay="1000"/>
       </Cluster>
   </HectorConfiguration>

   The XML elements are described below.
   

   Property Name		Description
   <Name/>		Hector reference name for the Cassandra cluster connection.
   <Nodes/>		Comma separated is of Cassandra Cluster nodes.
   <AutoDiscovery disable="" delay=""/>		Enable Hector Auto node discovery service.

Step 3: Cassandra Cluster Configuration for Statistics and Node Operations

To view Cassandra cluster statistics and to perform cluster operations, the cluster-config.xml file stored in the <SS_HOME>/repository/conf/etc/ directory needs to be configured. Here, all the SS nodes and their service URLs need to be configured.

<cluster>
    <configuration>
        <cluster_authentication>
            <username>admin</username>
            <password>admin</password>
        </cluster_authentication>
        <nodes>
            <node>
                <host>127.0.0.1</host>
                <backend_url>local://services/</backend_url>
            </node>
        </nodes>
    </configuration>
</cluster>

The XML elements are described below.

<cluster_authentication/>

<username/>

<password/>

<host/>

<backend_url/>

Step 4: Exposing services to the public

In a IaaS infrastructure, the services, public IP and domain names of the backend Cassandra cluster must be exposed via public addresses. This is done in the cassandra-endpoint.xml file stored in the <SS_HOME>/repository/conf/etc/ directory. Given below is the default configuration, where the <EndPoint> and <HostName> elements represent each Cassandra node by its host name.

<Cassandra>
 <EndPoints>
    <EndPoint>
        <HostName>css0.stratoslive.wso2.com</HostName>
    </EndPoint>
    <EndPoint>
        <HostName>css1.stratoslive.wso2.com</HostName>
    </EndPoint>
    <EndPoint>
        <HostName>css2.stratoslive.wso2.com</HostName>
    </EndPoint>
    <EndPoint>
        <HostName>css3.stratoslive.wso2.com</HostName>
    </EndPoint>
    <EndPoint>
        <HostName>css4.stratoslive.wso2.com</HostName>
    </EndPoint>
 </EndPoints>
</Cassandra>