Analyzing the Data Gathered from Devices

In this tutorial, you will get the average of a device's battery level and create the Battery_Level_Per_Hour table by analyzing the data gathered from all the enrolled devices.MobX creates new mobile applications or sends updates to corporate mobile applications. These applications need to be installed on the user's devices. To ensure that the applications are successfully installed or updated on all the enrolled devices, all the devices need to have a battery level of 50% or above. Using the steps given below, MobX administrators are able to identify the devices that had less battery power when an application installation request was pushed and confirm with those users if the application was installed successfully.

Let's get started!

Configuring WSO2 IoT Server to publish device information

You need to configure WSO2 IoT Server to publish device information as it is disabled by default. Follow the steps given below.

By default, WSO2 IoT Server does not publish device information, such as the device ID and type, which we need to collect to analyze the battery level of devices. To enable IoT Server to publish the device data, do the following:

1. Open the <IOTS_HOME>/conf/cdm-config.xml file.

2. Configure the PublishDeviceInfoResponse parameter as true. 

   <OperationAnalyticsConfiguration>    
       <PublishLocationResponse>false</PublishLocationResponse>
       <PublishDeviceInfoResponse>true</PublishDeviceInfoResponse>
       <PublishOperationResponse>
           <Enabled>false</Enabled>
           <Operations>
               <Operation>*</Operation>
           </Operations>
       </PublishOperationResponse>
   </OperationAnalyticsConfiguration>

   WSO2 IoT Server can now publish the device information to WSO2 Data Analytics Server (WSO2 DAS). The published data can then be stored and analyzed using the steps given below.

Creating a stream and persisting data

You need to create a stream to gather the device data. Follow the steps given below:

1. Start WSO2 IoT Server:

   1. Start the WSO2 IoT Server broker profile.

      cd <IOTS_HOME>/bin
      sh broker.sh

   2. Start the WSO2 IoT Server core profile.

      cd <IOTS_HOME>/bin
      sh iot-server.sh

   3. Next, start the WSO2 IoT Server analytics profile.

      cd <IOTS_HOME>/bin
      sh analytics.sh

2. Access the WSO2 IoT Server's analytics management console.

   1. For access via secured HTTP: https://<IOTS_HTTPS_HOST>:9445/carbon/ 
For example: https://localhost:9445/carbon/

   2. For access via HTTP: http://<IOTS_HTTP_HOST>:9765/carbon/ 

      For example: http://localhost:9765/carbon/

3. On the Main tab, under Events, click Streams.

4. Click + Add Events Streams.

5. Enter the following values:

   Event Stream Name	org.wso2.iot.BatteryStream
   Event Stream Version	1.0.0
   Meta Data Attributes	Add each metadata attribute by entering its name, selecting the type from the drop-down list, and clicking Add. Attribute name Attribute type deviceId String deviceType String timestamp Long
   Payload Data Attributes	Add each payload data attribute by entering the name, selecting the type from the drop-down list, and by clicking Add. Attribute name Attribute type level Double year Int month Int day Int hour Int minute Int

6. Click Persist Events to save the device data into the EVENT_STORE table so that it can be used later on.
7. Select the following attributes:

   • Select Persist Event Stream

   • Under Meta Data Attributes, select Persist Attribute.
     This selects all the sub-attributes too.
   • Under Meta Data Attributes, select Index Column for deviceId, deviceType, and timestamp.
   • Under Payload Data Attributes select Persist Attribute.
     This selects all the sub-attributes too.
     
   
8. Click Save Event Stream.

Creating an execution plan to publish data

Now that you've created the event stream, you will create an execution plan to publish data to that stream. For more information on creating execution plans, see the WSO2 Data Analytics Server (WSO2 DAS) documentation.

Follow the steps given below:

1. On the Main tab, click Execution Plans.
2. Click Add Execution Plan.

3. Copy the execution plan that is given below and replace the sample content that is in the text box.

   @Plan:name('Battery_Level_Average_Summarization')
   @Import('org.wso2.iot.DeviceInfoStream:1.0.0')
   define stream input (meta_deviceId string, meta_deviceType string, timeStamp long, imei string, imsi string, deviceModel string, vendor string, osVersion string, osBuildDate string, batteryLevel double, totalInternalMemory double, availableInternalMemory double, totalExternalMemory double, availableExternalMemory double, operator string, connectionType string, mobileSignalStrength double, ssid string, cpuUsage double, totalRAM double, availableRAM double, pluggedIn bool);
   
   @Export('org.wso2.iot.BatteryStream:1.0.0')
   define stream output (meta_deviceId string, meta_deviceType string, meta_timestamp long, level double, year int, month int, day int, hour int, minute int);
   
   from input
   select meta_deviceId, meta_deviceType, timeStamp as meta_timestamp, batteryLevel as level, time:extract(time:timestampInMilliseconds(), 'year') as year, time:extract(time:timestampInMilliseconds(), 'month') as month, time:extract(time:timestampInMilliseconds(), 'day') as day, time:extract(time:timestampInMilliseconds(), 'hour') as hour, time:extract(time:timestampInMilliseconds(), 'minute') as minute 
   insert into output;

4. Click Add Execution Plan.

You can now use this execution plan to gather the device information that published by the device and pass them on to the BatteryStream stream you created.

Creating an analytics script to summarize the battery data

Follow the steps given below to create a script that summarizes and presents the average battery level of the device every hour. For more information on creating scripts, see the WSO2 Data Analytics Server (WSO2 DAS) documentation.

1. On the Main tab, under Batch Analytics, click Scripts.
2. Click Add New Analytics Scripts.
3. Give the script a name, such as battery_data_analytics.

4. Copy the script given below to the Spark SQL Queries text box.

    CREATE TEMPORARY TABLE BatteryData USING CarbonAnalytics OPTIONS(tableName "ORG_WSO2_IOT_BATTERYSTREAM", incrementalParams "ORG_WSO2_IOT_BATTERYSTREAM, HOUR");
       
    CREATE TEMPORARY TABLE Battery_Level_Per_Hour USING CarbonAnalytics
    OPTIONS (tableName "Battery_Level_Per_Hour",
    schema "deviceId STRING, type STRING, level DOUBLE -i, year INT -i, month INT -i, day INT -i, hour INT -i,
    timestamp LONG", primaryKeys "year, month, day, hour, deviceId, type", mergeSchema "false");
   
    INSERT INTO TABLE Battery_Level_Per_Hour
    SELECT meta_deviceId as deviceId, meta_deviceType as type, avg(level) as level, year, month, day, hour, getHourStartingTime(year, month, day, hour) as   timestamp
    FROM BatteryData
    GROUP BY meta_deviceId, meta_deviceType, year, month, day, hour; 
   
   
    INCREMENTAL_TABLE_COMMIT ORG_WSO2_IOT_BATTERYSTREAM;

   Make sure to add a space when starting a new line in the script. If you copy the script given above, it has the spaces set by default.

   

5. Enter 0 0 0/1 1/1 * ? * as the value for the Cron Expression.
   This field is used to determine how often the analytics script needs to be executed. In this sample, we have configured the script to run every minute.
6. Click Add.

Try it out

The script you wrote above summarizes the battery data of the devices and adds it to the Battery_Level_Per_Hour database table. Now the administrator can check the data in the table and see the devices that did not have a battery level of 50% or above at the time of pushing the applications to the devices, and check with those device owners if the application was installed successfully.

To check out the data that was summarized and added to the table, follow the steps given below:

1. Access the WSO2 IoT Server's analytics management console.

   1. For access via secured HTTP: https://<IOTS_HTTPS_HOST>:9445/carbon/ 
For example: https://localhost:9445/carbon/

   2. For access via HTTP: http://<IOTS_HTTP_HOST>:9765/carbon/ 

      For example: http://localhost:9765/carbon/
2. On the Main tab, click Data Explorer.
3. From the Table Name* drop-down list, select BATTERY_LEVEL_PER_HOUR.
4. Click Search.

The data is displayed in the table format. Now, MobX is able to find out the battery levels of each device on an hourly basis.