Page Comparison

Deploying the sample C-App

You can deploy artifacts (i.e. event streams, event receivers, Spark scripts, event publishers, and dashboards etc.) as composite Carbon Applications (C-Apps) in WSO2 DAS. This guide uses the SMART_HOME.car file as the toolbox which contains all the artifacts required for this guide in a single package. For more information on C-Apps, see Packaging Artifacts as a C-App Archive. Follow the steps below to deploy and use a sample C-App in WSO2 DAS.

1. Log in to the DAS management console using the following URL: https://<DAS_HOST>:<DAS_PORT>/carbon/
2. Click Main, and then click Add in the Carbon Applications menu.
3. Click  Choose File, and upload the <DAS_HOME>/capps/Smart_Home.car file as shown below.
   
4. Click Main , then click Carbon Applications, and then click List view, to see the uploaded Carbon application as shown below.
   

Batch and interactive analytics

You can perform batch analytics when event streams are configured to be persisted for later batch processing scenarios such as data aggregation, summarization etc. WSO2 DAS batch analytics engine is powered by Apache Spark, which accesses the underlying data storage and executes programs to process the event data. The DAS provides an SQL-like query language to create the jobs through scripts that need to be executed.

Step 1: Persist event stream

Anchor
persist persist

In this step, the org.wso2.event.sensor.stream and org.wso2.event.sensor.filtered.stream event streams that you previously created are persisted so that the data received by them are stored in the databases configured for WSO2 DAS by default.
 

1. Log in the the WSO2 DAS Management Console if you are not already logged in.
2. In the Main   tab, click Streams   to open the   Available Event Streams   page.
3. Click Edit for the event stream org.wso2.event.sensor.stream. This opens the Edit Event Stream page.
4. Click Next [Persist Event].
5. Select the Persist Event Stream check box.
6. Select the Persist Attribute check box for all the available attributes.
7. Select the Index Column check box for the sensorid attribute.
8. Click Save Event Stream to save the changes.
9. Similarly, persist the org.wso2.event.sensor.filtered.stream event stream as shown below.
   Image Added 

Step 2: Simulate events

In this step, multiple events are simulated to be stored in the data base for batch analytics.

1. Download and save this file in a preferred location in your machine.
2. Log into the DAS management console using the following URL: https://<DAS_HOST>:<DAS_PORT>/carbon/.
3. Click Tools, and then click Event Simulator.
4. Select org.wso2.event.sensor.stream in the  Event Stream Name  field.
    
5. Click Choose File, and then browse and upload the CSV file you downloaded and saved. Click Upload and refresh the page to view the uploaded file.
6. Click Configure to open the Event Mapping Configuration dialog box. Enter a comma ( , ) in the Field delimiter field, and click Configure.
   
   
7. Click Play to start sending the events in the file. Click OK in the message that appears to confirm that the system has started sending events from the file. The events sent are logged in the CLI as shown below.
    

Step 3: Create and execute Spark scripts

In this exercise, a Spark query is written to process the data received by WSO2 DAS and stored in the databases.

1. Log in to the DAS Management Console, if you are not already logged in.
2. Click Main, and then click Scripts in the Batch Analytics menu to open the Available Analytics Scripts page.
3. Click Add New Analytics Script.

4. Enter BATCH_ANALYTICS_SCRIPT in the  Script Name parameter. Enter the following Spark SQL script in the  Spark SQL Queries parameter as shown below.
    
   

   Code Block
   language sql
   CREATE TEMPORARY TABLE sensorData USING CarbonAnalytics OPTIONS (tableName "ORG_WSO2_EVENT_SENSOR_STREAM", schema "humidity FLOAT, sensorValue DOUBLE"); create temporary table highSensorVal using CarbonAnalytics OPTIONS (tableName "highSensorVal", schema "humidity FLOAT, sensorValue DOUBLE"); insert overwrite table highSensorVal select humidity, sensorValue from sensorData WHERE sensorValue > 100; Select * from highSensorVal

     The above script does the following:   

   • Loads data from the DAS Data Access Layer (DAL), and registers temporary tables in the Spark environment.

   • Performs batch processing by fetching the sensor values greater than 100 from sensorData table.

   • Writes back to a new DAL table named highSensorVal.

   • Executes the following query:
     Select * from highSensorVal

    

5. Click Add to save the script. Click OK in the message that appears to confirm that the script was successfully saved.

6. Click Execute for the script.
   
   The following is displayed to indicate that the queries were successfully executed.
   

You can obtain faster results by executing adhoc queries on the indexed attributes through an interactive Web console named the Interactive Analytics Console. Follow the procedure below to perform a batch analytics operation using the Interactive Analytics Console.

1. Log into the DAS Management Console if you are not already logged in.
2. In the Main tab, click Console to open the Interactive Analytics Console.
3. Enter the following query in the console and press the Enter key.
   Select * from highSensorVal
   
   The output of the query is displayed as follows. 
   

Step 4: Search for data

This step involves performing interactive analytics for data received by DAS and stored in the configured databases. Searching is supported by the indexing functionality of DAS powered by Apache Lucene. In this step, we search for a specific record by the sensorid attribute. This is possible because the sensorid attribute was persisted as an index column when you persisted the event stream.

1. Log into the DAS Management Console if you are not already logged in.
2. In the Main tab, click Data Explorer to open the Data Explorer page.
3. In the Table Name parameter, select ORG_WSO2_EVENT_SENSOR_STREAM.
4. Select the By Query option, and enter meta_sensorid:501 in the data field displayed.
   
    
5. Click Search. The following records are displayed in the Results section.
    

Predictive Analytics

WSO2 DAS allows you to build predictive models that analyse data and make predictions.

In this exercise, a dataset is analyzed and a ML model is trained to predict the possibility of a person suffering with breast cancer when data relating to a set of other bodily characteristics is provided.

The dataset used in this scenario contains 10 features to provide data on bodily characteristics, and a response variable with the following labels.

The following is a summary of the procedure to make a prediction via WSO2 DAS.

Step 1: Create a dataset

Step 2: Create a project

Step 3: Create an analysis and train a model

Step 4: Predict using the model

Step 1: Create a dataset

Anchor
Create Dataset Create Dataset

Follow the procedure below to upload the dataset based on which the training model is created.

1. Access the WSO2 Machine Learner wizard embedded within DAS via the https://<DAS_HOME>:<DAS_PORT/ml/site/home/login.jag URL, and log in with your credentials.
   
2. Click ADD DATASET to open the Create Dataset page.

3. In the Data Source field, click Choose File and browse for the <DAS_HOME>/samples/ml/tuned/naive-bayes/breastCancerWisconsin.csv file. Enter values for the rest of the parameters as shown below.
   

   Parameter Name	Value
   Dataset Name	Breast_Cancer_Dataset
   Version	1.0.0
   Description	Breast cancer data in Wisconsin
   Source Type	File
   Data Format	CSV
   Column Header Available	Yes

4. Click CREATE DATASET to save your changes. The Datasets page opens and the dataset you entered is displayed as follows.
   

 Step 2: Create a project

Anchor
Create Project Create Project

Follow the procedure below to create a project for the dataset uploaded.

1. Log into the Machine Learner wizard of WSO2 DAS if you are not already logged in. You can access it with the https://<DAS_HOME>:<DAS_PORT/ml/site/home/login.jag URL, and log in with your credentials.
2. Click ADD PROJECT.
    
    

3. In the Create Project page, enter information as shown below.
    
   

   Parameter Name	Description
   Project Name	Breast_Cancer_data_analytics_project
   Description	This project performs predictive analysis on the breast cancer data in Wisconsin.
   Dataset	Breast_Cancer_Dataset

    

   
   

    
4. Click Create Project to save the information. The project is displayed in the Projects page as follows.
    

Step 3: Create an analysis and train a model

Anchor
Create Analysis Create Analysis

Follow the procedure below to analyse the Breast_Cancer_Dataset dataset, and then create a training model based on that analysis.

1. Log into the Machine Learner wizard if you are not already logged in. You can access it with the https://<DAS_HOME>:<DAS_PORT/ml/site/home/login.jag URL, and login with your credentials.
2. Click the You have X projects link as shown below.
    
3. Click on the Breast_Cancer_data_analytics_project project to expand it.
4. Enter breast_cancer_analysis_1 as the analysis name and click CREATE ANALYSIS. The following page appears displaying the summary statistics.
    
5. Click Next without making any changes to the summary statistics.
    
   The Explore view opens. Note that Parallel Sets and Trellis Chart visualizations are enabled, and Scatter Plot and Cluster Diagram visualizations are disabled. This is determined by the feature types of the dataset.  

6. Click Next. The Algorithms view is displayed. Enter values as shown below.
    
   

   Parameter	Value
   Algorithm name	LOGISTIC REGRESSION L_BFGS
   Response variable	Class
   Train data fraction	0.7

    

7. Click Next. The Parameters view appears. Enter L2 as the reg type.
    
8. Click Next. The Model view appears. Select Breast_Cancer_Dataset-1.0.0 as the dataset version.
    
9. Click RUN to train the model.
    
   The training model is created as displayed as shown below.
    

Step 4: Predict using the model

Anchor
Predict Predict

Follow the procedure below to make a prediction based on the training model you created.

1. Log into the Machine Learner wizard if you are not already logged in. You can access it with the https://<DAS_HOME>:<DAS_PORT/ml/site/home/login.jag URL, and login with your credentials.
2. Click the You have X projects link as shown below to open the Projects window.
    
3. Click MODELS for the breast_cancer_analysis_1  analysis.
    
4. Click Predict on the model displayed.
    

5. Enter values in the Predict page as shown below.
    

   Parameter Name	Value
   Prediction Source	Feature values
   SampleCodeNumber	1018561
   ClumpThickness	2
   UniformityOfCellSize	1
   UniformityOfCellShape	1
   MarginalAdhesion	1
   SingleEpithelialCellSize	2
   BareNuclei	1
   BlandChromati	1
   NormalNucleoli	1
   Mitoses	5

6. Click Predict. The prediction is displayed as follows.