Analyzing Data
- Former user (Deleted)
Data can be processed as batch or real-time or combination of both.
Events can be processed in real-time using WSO2's Siddhi Complex Event Processor (CEP) engine. Before we begin let's understand  the basic components which are used it real time processing. A stream is defined as a sequence of events having the same data type. It can be processed using the Siddhi Query Language (SiddhiQL). Siddhi works as an upside down database by storing all the queries in memory, matching incoming events against the stored queries at runtime and producing events that matched as streams. In order to process events, the execution logic should be written as a script using SiddhiQL and deployed in the server. This execution plan will act as an isolated execution environment and there will be an instance of the Siddhi Execution Runtime created for each execution plan. An execution plan can import one or more streams from the server for processing and push zero or more output streams back to the server.  To import and export streams, the script needs to contain import and export annotations that binding the streams defined in the execution plan to the server.
On the other hand, in order to perform batch analytics data needs to be  retrieved from the data stores and various analysis operations needs to be performed according to the defined analytic queries.
The following sections describe how to work with these components to analyze your stored data:
Follow the WSO2 DAS documentation on how you can perform interactive analytics using the Apache Lucene Query Language.
Batch Analytics Using Spark SQL
Apache Spark is a powerful open-source processing engine built around speed, ease of use, and sophisticated analytics. WSO2 DAS employs Apache Spark as its analytics engine. Follow the WSO2 DAS documentation on how you can perform batch analytics using Apache Spark SQL. Â
Real-time Analytics Using Siddhi
Follow the WSO2 DAS documentation on how you can perform real-time analytics using Siddhi in WSO2 DAS.
Predictive Analytics Using WSO2 ML
Deploy datasets and generates predictive models via the Machine Learner Wizard. Follow WSO2 DAS and WSO2 Machine Learner (ML) documentation for more information on this section.