WSO2 Data Analytics Server 3.0.0 combines real-time, batch, interactive, and predictive (via machine learning) analysis of data into one integrated platform to support the multiple demands of Internet of Things (IoT) solutions, as well as mobile and Web apps as illustrated by the image below.
As a part of WSO2’s analytics platform, WSO2 DAS introduces a single solution with the ability to build systems and applications that collect and analyze both realtime and persisted, data and communicate the results. It is designed to treat millions of events per second, and is therefore capable to handle Big Data volumes and Internet of Things projects.
WSO2 DAS 3.0.0 workflow consists of the following three main phases.
Aggregating data
WSO2 DAS exposes a single API for external data sources to publish data events to it. Further, it provides configurable options either to process the data event stream inflow (in memory) for realtime analytics, to persist (in data storage) for batch analytics and/or to index for interactive analytics as shown in the below diagram.
Creating event streams
The first step in aggregating data is defining an event stream by creating the event stream definition. A stream definition provides the initial structure and identification required for event processing. It includes a set of data types as properties, name, version and other attributes. When an external data publisher sends data events to WSO2 DAS (the receiver), it needs to specify the name and version of the stream intended. Also, data events should be sent according to the structure defined in the stream definition. For more information on event streams, see Event Streams.
Persisting events
WSO2 DAS introduces a pluggable architecture which allows you to persist data events into any relational data storage (i.e. Oracle, MSSQL, MySQL etc.), or NoSQL storage (i.e. Apache HBase, Apache Cassandra etc.). It is also possible for multi data event storage. For an example, the events can be stored in a NoSQL storage while the processed data events can be stored in a relational data storage.
Creating event receivers
Event Receivers are the connectors to different data sources in WSO2 DAS. WSO2 DAS support s event retrieval from many transport protocols and different formats. For information on the supported transport protocols and event formats, see Event Receiver Types.Analyzing data
You can configure any data event stream received by WSO2 DAS or batch and/or real time analytics.
Batch analytics
Spark console
You can obtain faster results by executing adhoc queries on the indexed attributes through an interactive Web console, named as the Spark Console.
Realtime analytics
You can process the event streams inflow through the WSO2 real time analytics engine which is powered by Siddhi. The realtime analytic engine can process multiple event streams in realtime. For this, you need to specify a set of queries or rules using the SQL like Siddhi Query Language in an execution plan. Execution plan is the editor for the event processing logic. An execution plan consists of a set of queries and import and export streams. For more information see, Working with Execution Plans.
Event publishers
Event publishers provide the capability to send event notifications and alerts from WSO2 DAS to external systems. For more information, see Publishing Events.
Event flow
You can use the Event Flow feature of WSO2 DAS to visualize how the components of it are connected with each other. Also, you can use it to validate the flow of the events within the DAS.
Event simulation
Event Simulator is a tool which you can use for monitoring and debugging event streams. You need to create event(s) by assigning values to event stream attributes to simulate them. For more information, see Event Simulator.
Interactive analytics
Interactive analytics are used when you need to get fast results through adhoc querying of a data set. Interactive analytics in WSO2 DAS is possible when you select to index event stream attributes.
Data Explorer
The Data Explorer is the Web console for searching analytical data. Primary key, data range, facet search are some available options for simple analytical record searches. It is also possible to search records by providing Lucene queries for advanced searches. For more information, see Data Explorer.
Presenting data
WSO2 DAS uses several presentation mechanisms to present event notifications and processing results. For more information, see Presenting Data.
Analytical Dashboard
WSO2 DAS provides a feature for creating customizable dashboards for visualization of analytical data. Dashboard creation is wizard driven, where you can use widgets/gadgets such as Line, Bar, and Arc charts to get data from analytical tables and add them on a structured grid layout to provide an overall view. For more information, see Analytics Dashboard.