| Excerpt | ||
|---|---|---|
| ||
CEP Distributed Deployment in wiki format |
Distributed deployment of CEP enables it to achieve High Availability, and Scalability in terms of memory.
There are two modes of distributed deployment for CEP which are as follows,
| Table of Contents |
|---|
Distributed Cache Mode Deployment
WSO2 CEP with Siddhi uses Hazelcast to achieve distributed processing. To enable distributed processing all CEP servers has to be deployed in the same Local Area Network, this enables all the Hazelcast instances of CEP to identify each other using multicast and create an in-memory cluster for supporting distributed processing.
In WSO2 CEP, distributed processing can be enabled at per execution plan level where by only those identified execution plans will use the distributed cache for processing,
Follow below steps to enable distributed processing to an execution plan.
Create distributed CEP execution plans with the following properties
| Warning | ||
|---|---|---|
| ||
| Not following the below conventions will lead to unpredictable results! |
| Info | ||
|---|---|---|
| ||
In order to enable distributed deployments, prior to starting up the server, users need to enable hazelcast clustering by setting <CEP_HOME>/repository/conf/axis2/axis2.xml's clustering to true. <clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true"> |
All execution plans across nodes has to have the same Execution Plan Name
Execution Plans has to have the following Backend Runtime Configuration
No Format Siddhi Configurations Snapshot time interval : 0 Distributed processing : true
These can be added to the configuration file as below.
Code Block language html/xml linenumbers true <executionPlan name="FraudAnalyzer" statistics="enable" trace="enable" xmlns="http://wso2.org/carbon/eventprocessor"> ... <siddhiConfiguration> <property name="siddhi.enable.distributed.processing">true</property> <property name="siddhi.persistence.snapshot.time.interval.minutes">0</property> </siddhiConfiguration> ... </bucket>All execution plans should have identical query expressions (Queries has to use same stream names, same type of events and query expressions in all the execution plans have to be in the same order)
But the input topics need NOT to be the same, by using different input topics users can distribute load when publishing to CEP.
...
Follow the sample Distributed Purchase Analyzer Sample and Fraud Analyzer step by step guide to better understand this functionality
Persistence Mode Deployment
Using persistence mode enables CEP Queries to span lifetimes much greater than server uptime. This is achieved by taking periodic snapshots and storing all state information & windows in a scalable persistence store. In CEP we use Apache Cassandra as the scalable persistence store in order to achieve high scalability and low latency when persisting data. In this mode if server crashes the same server or another CEP server with the same bucket connected to the same Cassandra Data Store can be restored from the last available snapshot.
WSO2 CEP persistence mode processing can be enabled at per bucket level where by only those identified buckets will persist bucket states,
Follow below steps to enable persistence mode processing to a bucket.
Create CEP bucket with the following properties
Bucket has to have the following Backend Runtime Configuration
No Format Backend Runtime Configuration Persistence snapshot time interval in minutes : 5 (Any positive long value) Enable distributed processing : false
These can be added to the configuration file as below.
Code Block language html/xml linenumbers true <bucket name="AllStocksAnalyzer" xmlns="http://wso2.org/carbon/cep"> ... <cep:engineProviderConfiguration engineProvider="SiddhiCEPRuntime"> <cep:property name="siddhi.persistence.snapshot.time.interval.minutes">5</cep:property> <cep:property name="siddhi.enable.distributed.processing">false</cep:property> </cep:engineProviderConfiguration> ... </bucket>
...
The steps are as follows :
...
You can install multiple instances of WSO2 products in a cluster to ensure that if one instance becomes unavailable or is experiencing high traffic, another instance will seamlessly handle the requests. For complete information on clustering WSO2 CEP, go to Clustering WSO2 CEP.