Info |
---|
To view a screencast of the Quick Start Guide, click here . |
...
Table of Contents | ||||
---|---|---|---|---|
|
Tip |
---|
Before you begin,
|
...
- Log into the DAS Management Console and click on the Main tab. Under Manage, click Streams to open the Available Event Streams page.
- Click Add Event Stream to open the Define New Event Stream page.
Enter information as follows to create a new event stream named
org.wso2.event.sensor.stream
.
Event Stream DetailsParameter Name Value Event Stream Name org.wso2.event.sensor.stream
Event Stream Version 1.0.0 Stream Attributes
Click Add to add the attribute after entering the attribute name and attribute type.
Attribute Category Attribute Attribute Type Meta Data timestamp long isPowerSaverEnabled bool sensorId int sensorName string Correlation Data longitude double latitude double Payload Data humidity float sensorValue double Click Add Event Stream to save the information.
...
- Log into the DAS Management Console and click on the Main tab. Under Manage, click Receivers to open the Available Receivers page.
- Click Add Event Receiver to open the Create a New Event Receiver page.
Enter information as follows to create the new event receiver named
httpReceiver
.
Parameter Name Value Event Receiver Name httpReceiver Input Event Adapter Type http Transports all Event Stream org.wso2.event.sensor.stream Message Format json - Click Add Event Receiver to save the information.
...
- Log into the DAS Management Console and click on the Main tab. Under Manage, click Publishers to open the Available Publishers page.
- Click Add Event Publisher to open the Create a New Event Publisher page.
Enter information as follows to create the new event publisher named
loggerPublisher
.
Parameter Name Description Event Publisher Name loggerPublisher Event Source org.wso2.event.sensor.stream Output Event Adapter Type logger Message Format text Click Add Event Publisher to save the information.
...
This builds the HTTP client and sends the events in the <DAS_HOME>/samples/cep/artifacts/0001/httpReceiver.txt file
to the httpReceiver
endpoint. You can view the details of the events that are sent as shown in the log below. These logs are published by the publisher created in Step 3.
The logs of the JSON events received by the DAS server will be displayed in the CLI as shown in the example below.
...
- Log into the DAS Management Console and click on the Main tab. Under Manage, click Streams to open the Available Event Streams page.
- Click Add Event Stream to open the Define New Event Stream page.
- Enter information as follows to create the event stream named
org.wso2.event.sensor.filtered.stream
.
Event Stream Details
Paramater Name Value Event Stream Name org.wso2.event.sensor.filtered.stream Event Stream Version 1.0.0 Stream Attributes
Attribute Category Attribute Attribute Type Meta Date timestamp long sensorName string Correlation Data longitude double latitude double Payload Data sensorValue double Click Add Event Stream to save the information.
...
- Log into the DAS Management Console and click on the Main tab. Under Manage, click Execution Plans to open the Available Execution Plans page.
- Click Add Execution Plan to open the Create a New Execution Plan page.
Enter information as follows to create the new execution plan
Parameter Name Value Import Stream org.wso2.event.sensor.stream:1.0.0 As sensorStream Value Of filteredStream StreamId org.wso2.event.sensor.filtered.stream:1.0.0 - Click Import and then click Export. The section for query expressions will be updated as shown below.
Add the following query expression.
Anchor filter filter Code Block from sensorStream [sensorValue > 100] select meta_timestamp, meta_sensorName, correlation_longitude, correlation_latitude, sensorValue insert into filteredStream
This query includes the value
sensorValue > 100
. Therefore, when the execution plan forwards events fromorg.wso2.event.sensor.stream:1.0.0
toorg.wso2.event.sensor.filtered.stream:1.0.0
, events in which the value for thesensorValue
attribute is less than 100 will be dropped.- Click Validate Query Expressions. Once you get a message to confirm that the queries are valid, click Add Execution Plan.
...
- Log into the DAS Management Console and click on the Main tab. Under Manage, click Publishers to open the Available Publishers page.
- Click Add Event Publisher to open the Create a New Event Publisher page.
Enter information as follows to create the new event publisher named
UIPublisher
.
Parameter Name Description Event Publisher Name uiPublisher Event Source org.wso2.event.sensor.filtered.stream:1.0.0 Output Event Adapter Type ui Message Format wso2event Click Add Event Publisher to save the information.
Step 9: Create a dashboard and a gadget
WSO2 Analytics Dashboard will be used as the tool to analyse the output of the event flow you created in this guide. This step creates a dashboard and a gadget which analyses events from the org.wso2.event.sensor.filtered.stream
stream published by the uiPublisher
publisher.
- Log into the DAS Management Console. In the Main tab, click Analytics Dashboard.
- Log into the Analytics Dashboard with your username and password.
- Click the menu icon and then click Gadgets to open the Gadgets page as demonstrated below.
- Click GENERATE GADGET, and enter values in the Generate a Gadget wizard as follow.
- In the Select Provider field, select Realtime Data Source . Then click Next .
- In the Event Stream field, select org.wso2.event.sensor.filtered.stream:1.0.0 . Then click Next.
Configure a chart as follows.
Parameter Name Value Gadget Name Sensor Value VS Timestamp Select Chart Type Line Chart X-Axis TIMESTAMP X type time Y-Axis sensorValue Y type default Color domain sensorName Max length 30 - Click Add to Store, and then click Go to Portal. the Dashboards page appears again.
- Click CREATE DASHBOARD to open the Create a Dashboard page. Configure a new dashboard as follows.
Enter a name and a description for the new dashboard as follows, and click Next.
Parameter Name Value Name of your Dashboard Sensor Statistics Description This dashboard indicates the sensor value at different times in a particular location. - Select the Single Column layout. A message appears to indicate that the dashboard is successfully created.
- Click the icon for gadgets. Then select and drag Sensor Value VS Timestamp gadget to the first column as demonstrated above.
Step 10: Send Events to the HTTP Receiver via Curl Command
...
Issue the following curl command.
Code Block curl -X POST -d "{ "event": { "metaData": { "timestamp": 1439468145264 , "isPowerSaverEnabled": false, "sensorId": 701, "sensorName": temperature }, "correlationData": { "longitude": 4.504343, "latitude": 20.44345 }, "payloadData": { "humidity": 2.3, "sensorValue": 96.5 } } }" http://localhost:9763/endpoints/httpReceiver --header "Content-Type:application/json"
The following log will appear in the DAS CLI.
Note that the Sensor the Sensor Statistics dashboard you created does not get updated. This is because the value for the sensorValue attribute is less than 100 in this event. Therefore, it gets dropped by the filter you created in the execution plan, and as a result, it is not forwarded to the
org.wso2.event.sensor.filtered. stream:1.0.0
stream.Issue another command with a value greater than 100 for the
sensorValue
attribute as follows.Code Block curl -X POST -d "{ "event": { "metaData": { "timestamp":1439467524120 , "isPowerSaverEnabled": false, "sensorId": 701, "sensorName": temperature }, "correlationData": { "longitude": 4.504343, "latitude": 20.44345 }, "payloadData": { "humidity": 2.3, "sensorValue": 156 } } }" http://localhost:9763/endpoints/httpReceiver --header "Content-Type:application/json"
The following log will appear in he DAS CLI.
The event is forwarded to the
org.wso2.event.sensor.filtered.
stream:1.0.0
stream since the value for thesensorValue
attribute is greater than 100. Therefore, the Sensor Statistics dashboard will be updated as shown below.
Issue more curl commands as follows with several timestamps and sensor values.
Code Block curl -X POST -d "{ "event": { "metaData": { "timestamp": 1439467524120 , "isPowerSaverEnabled": false, "sensorId": 701, "sensorName": temperature }, "correlationData": { "longitude": 4.504343, "latitude": 20.44345 }, "payloadData": { "humidity": 2.3, "sensorValue": 156 } } }" http://localhost:9763/endpoints/httpReceiver --header "Content-Type:application/json"
Code Block curl -X POST -d "{ "event": { "metaData": { "timestamp": 1439467890957 , "isPowerSaverEnabled": false, "sensorId": 701, "sensorName": temperature }, "correlationData": { "longitude": 4.504343, "latitude": 20.44345 }, "payloadData": { "humidity": 2.3, "sensorValue": 170 } } }" http://localhost:9763/endpoints/httpReceiver --header "Content-Type:application/json"
Code Block curl -X POST -d "{ "event": { "metaData": { "timestamp": 1439467951518 , "isPowerSaverEnabled": false, "sensorId": 701, "sensorName": temperature }, "correlationData": { "longitude": 4.504343, "latitude": 20.44345 }, "payloadData": { "humidity": 2.3, "sensorValue": 131 } } }" http://localhost:9763/endpoints/httpReceiver --header "Content-Type:application/json"
Code Block curl -X POST -d "{ "event": { "metaData": { "timestamp": 1439467992936 , "isPowerSaverEnabled": false, "sensorId": 701, "sensorName": temperature }, "correlationData": { "longitude": 4.504343, "latitude": 20.44345 }, "payloadData": { "humidity": 2.3, "sensorValue": 126 } } }" http://localhost:9763/endpoints/httpReceiver --header "Content-Type:application/json"
Code Block curl -X POST -d "{ "event": { "metaData": { "timestamp": 1439468050928 , "isPowerSaverEnabled": false, "sensorId": 701, "sensorName": temperature }, "correlationData": { "longitude": 4.504343, "latitude": 20.44345 }, "payloadData": { "humidity": 2.3, "sensorValue": 145 } } }" http://localhost:9763/endpoints/httpReceiver --header "Content-Type:application/json"
Since the value for the
sensorValue
attribute is greater than 100 in all these events, the dashboard will be updated as shown below. These events will also be logged in the DAS CLI.Send sensor events with several timestamps and sensorValues, The dashboard will now get effected with the sent event since sensorValue is over 100 and the event gets sent to the dashboard. These events will get logged in the DAS CLI as well.
...
Note |
---|
Before you carry out this step
|
This step involves adding execution plan and stream configurations using the template you created and added in the previous step.
- If the DAS server was running when you created and deployed the template, restart the DAS server.
- Log into the DAS Management Console. Click the Main tab and then click Template Manager. Create a new scenario as demonstrated below.
- Click on SensorStatistics to open the Deployed Scenarios page. Then click Create New Scenario to open the Edit Scenario page.
Enter information as shown in the table below and click Add scenario .
Parameter Name Value Scenario Type AnalyzeSensorStatistics
Scenario Name FilterSensorValues Description Filter events with a sensor value greater than 120 Sensor Value 120 A message appears to inform you that the scenario is successfully created. Close the message. The scenario you configured is displayed in the Deployed Scenarios page
...
The following is a summary of this guide which describes each element in the event flow.
Panel | ||||||
---|---|---|---|---|---|---|
Deploying the sample C-AppYou can deploy artifacts (i.e. event streams, event receivers, Spark scripts, event publishers, and dashboards etc.) as composite Carbon Applications (C-Apps) in WSO2 DAS. This guide uses the
Publishing eventsOnce you develop the complete Event Flow, you can test the flow by publishing the events to the DAS. There are several methods of publishing to DAS. In this section, the events are published via a Navigate to
Viewing the outputFollow the steps below to view the presentation of the output in the Analytics Dashboard.
|
Panel | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Batch and interactive analyticsYou can perform batch analytics when event streams are configured to be persisted for later batch processing scenarios such as data aggregation, summarization etc. WSO2 DAS batch analytics engine is powered by Apache Spark, which accesses the underlying data storage and executes programs to process the event data. The DAS provides an SQL-like query language to create the jobs through scripts that need to be executed. Step 1: Persist event stream
In this step, the
Step 2: Simulate eventsIn this step, multiple events are simulated to be stored in the data base for batch analytics.
Step 3: Create and execute Spark scriptsIn this exercise, a Spark query is written to process the data received by WSO2 DAS and stored in the databases.
You can obtain faster results by executing adhoc queries on the indexed attributes through an interactive Web console named the Interactive Analytics Console. Follow the procedure below to perform a batch analytics operation using the Interactive Analytics Console.
Step 4: Search for dataThis step involves performing interactive analytics for data received by DAS and stored in the configured databases. Searching is supported by the indexing functionality of DAS powered by Apache Lucene. In this step, we search for a specific record by the
|
Panel | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Predictive AnalyticsWSO2 DAS allows you to build predictive models that analyse data and make predictions. In this exercise, a dataset is analyzed and a ML model is trained to predict the possibility of a person suffering with breast cancer when data relating to a set of other bodily characteristics is provided. The dataset used in this scenario contains 10 features to provide data on bodily characteristics, and a response variable with the following labels.
The following is a summary of the procedure to make a prediction via WSO2 DAS. Step 1: Create a dataset Step 2: Create a project Step 3: Create an analysis and train a model Step 4: Predict using the model Step 1: Create a dataset
Follow the procedure below to upload the dataset based on which the training model is created.
|
Anchor | ||||
---|---|---|---|---|
|
Follow the procedure below to create a project for the dataset uploaded.
- Log into the Machine Learner wizard of WSO2 DAS if you are not already logged in. You can access it with the
https://<DAS_HOME>:<DAS_PORT/ml/site/home/login.jag
URL, and log in with your credentials. - Click ADD PROJECT.
In the Create Project page, enter information as shown below.
Parameter Name Description Project Name Breast_Cancer_data_analytics_project Description This project performs predictive analysis on the breast cancer data in Wisconsin. Dataset Breast_Cancer_Dataset - Click Create Project to save the information. The project is displayed in the Projects page as follows.
Step 3: Create an analysis and train a model AnchorCreate Analysis Create Analysis
Create Analysis | |
Create Analysis |
Follow the procedure below to analyse the Breast_Cancer_Dataset
dataset, and then create a training model based on that analysis.
- Log into the Machine Learner wizard if you are not already logged in. You can access it with the
https://<DAS_HOME>:<DAS_PORT/ml/site/home/login.jag
URL, and login with your credentials. - Click the You have X projects link as shown below.
- Click on the Breast_Cancer_data_analytics_project project to expand it.
- Enter breast_cancer_analysis_1 as the analysis name and click CREATE ANALYSIS. The following page appears displaying the summary statistics.
- Click Next without making any changes to the summary statistics.
The Explore view opens. Note that Parallel Sets and Trellis Chart visualizations are enabled, and Scatter Plot and Cluster Diagram visualizations are disabled. This is determined by the feature types of the dataset. Click Next. The Algorithms view is displayed. Enter values as shown below.
Parameter Value Algorithm name LOGISTIC REGRESSION L_BFGS Response variable Class Train data fraction 0.7 - Click Next. The Parameters view appears. Enter L2 as the reg type.
- Click Next. The Model view appears. Select Breast_Cancer_Dataset-1.0.0 as the dataset version.
- Click RUN to train the model.
The training model is created as displayed as shown below.
Step 4: Predict using the model AnchorPredict Predict
Predict | |
Predict |
Follow the procedure below to make a prediction based on the training model you created.
- Log into the Machine Learner wizard if you are not already logged in. You can access it with the
https://<DAS_HOME>:<DAS_PORT/ml/site/home/login.jag
URL, and login with your credentials. - Click the You have X projects link as shown below to open the Projects window.
- Click MODELS for the breast_cancer_analysis_1 analysis.
- Click Predict on the model displayed.
Enter values in the Predict page as shown below.
Parameter Name Value Prediction Source Feature values SampleCodeNumber 1018561 ClumpThickness 2 UniformityOfCellSize 1 UniformityOfCellShape 1 MarginalAdhesion 1 SingleEpithelialCellSize 2 BareNuclei 1 BlandChromati 1 NormalNucleoli 1 Mitoses 5 - Click Predict. The prediction is displayed as follows.
...