Performance Tuning Recommendations

This section describes some recommended performance tuning configurations to optimize BAM. It assumes that you have set up the BAM server on Unix/Linux, which is recommended for a production deployment. If you have high volumes of data with high concurrency, it is recommended to use a distributed BAM setup. For instructions, see product deployment and clustering guide.

The values discussed below are general recommendations. They might not be optimal for the specific hardware configurations in your environment. We recommend you to carry out load tests on your environment to tune BAM accordingly.

Improvement Area Performance Recommendations

Data receiver nodes

Xms1024m -Xmx1024m -XX:MaxPermSize=512m
Change the /etc/security/limits.conf with the following values:
```
* soft nofile 4096
* hard nofile 65535
```

Analyzer nodes Xms1024m -Xmx1024m -XX:MaxPermSize=512m

Dashboard nodes Xms1024m -Xmx1024m -XX:MaxPermSize=512m

Hadoop nodes

Recommended OS: Linux
At least 10 GB storage capacity in each node
At least 100 Mbps network bandwidth
Set hadoop.root.logger=ERROR

Cassandra nodes

Have separate disks for your commit log and data dirs (sstables)
Set the Heap memory as follows:
System memory Heap size
Less than 2 GB 1/2 of system memory
2-4 GB 1 GB
More than 4 GB 1/4 system memory, but not more than 8GB
Set following configuration in cassandra.yaml according to your hardware resources:
- concurrent_reads: 4 * no of cores
- concurrent_writes: 8 * no of CPU cores

For more information see Apache Cassandra 1.0 Documentation.

Memory size of Hive

In order to increase the memory size of Hive try the following configurations:

Increase the setting for the Hadoop heapsize. For example increase it to 4096 as shown below:
export HADOOP_HEAPSIZE=4096
Add the below entry in the <BAM_HOME>/repository/conf/advanced/hive-site.xml file:

<property>  
   <name>mapred.child.java.opts</name>  
   <value> -server -Xmx512M</value>  
 </property>

WSO2 Carbon platform-level settings

In multitenant mode, the WSO2 Carbon runtime limits the thread execution time. That is, if a thread is stuck or taking a long time to process, Carbon detects such threads, interrupts and stops them. Note that Carbon prints the current stack trace before interrupting the thread. This mechanism is implemented as an Apache Tomcat valve. Therefore, it should be configured in the <PRODUCT_HOME>/repository/conf/tomcat/catalina-server.xml file as shown below.

<Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve" threshold="600"/>

The className is the Java class used for the implementation. Set it to org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.
The threshold gives the minimum duration in seconds after which a thread is considered stuck. The default value is 600 seconds.