Performance Tuning

This section describes some recommended performance tuning configurations to optimize BAM. It assumes that you have set up the ESB on a server running Unix/Linux, which is recommended for a production deployment.

Hadoop and Cassandra Settings

If you manage a high volume of data with high concurrency, we recommend you to go for a distributed BAM setup.

There is detailed blog post about the Cassandra tuning parameters. Please refer that for more details.

Regarding performance tuning for

depends on their data volume and Hardware configuration etc. Here I have listed some key points.

Tuning Receiver nodes:

-Xms1024m -Xmx1024m -XX:MaxPermSize=512m

-Change the /etc/security/limits.conf

---- soft nofile 4096

---- hard nofile 65535

Tuning Analyzer nodes:

-Xms1024m -Xmx1024m -XX:MaxPermSize=512m

Tuning Dashboard nodes:

-Xms1024m -Xmx1024m -XX:MaxPermSize=512m

There is no much work in Analyzer and Dashboard nodes therefore we don't need much tuning.

Hadoop nodes:

Recommended OS: Linux

Storage capacity of each node should have at-least 10GB

Network bandwidth is recommended to have at-least 100 Mbps.

set hadoop.root.logger=ERROR

Other optimization depends on their data volume and Hardware configuration. More information for performance tuning in Hadoop cluster can be found here [2]

Cassandra nodes:

Make sure your commit log and data dirs (sstables) are on different disks.

Set the Heap memory as below.

System Memory --------- Heap Size

Less than 2GB --------- 1/2 of system memory

2GB to 4GB --------- 1GB

Greater than 4GB --------- 1/4 system memory, but not more than 8GB

Set following configuration in cassandra.yaml according to your hardware resources.

-concurrent_reads

-4 * no of cores.

-concurrent_writes

-8 * no of CPU cores

More advance tuning you can refer cassandra documentations [3]