Quick Start Guide

WSO2 Stream Processor (SP) is a lightweight and lean, streaming SQL based stream processing platform that allows you to collect events, analyze them in real-time, identify patterns, map their impacts, and communicate the results within milliseconds.  It is powered by Siddhi to be extremely high performing.

First, let's understand the following concepts that are used in this guide:

Stream Processing and Complex Event Processing Overview

Let's understand what an event is through an example. If we consider the transactions carried out via an ATM as a data stream, one withdrawal from it can be considered an event. This event contains data about the amount, time, account number etc. Many such transactions form a stream.

Stream processing engines allow you to create a processing graph and inject events into it. Each operator processes and sends events to next processor.

A complex event is an event that summarizes, represents or denotes a set of other events. Complex Event Processing is a subset of Stream Processing which involves analyzing multiple streams of events in real time, recognizing particular sequences or patterns across streams and inferring a business significant event from correlated events.

Open

The stream processing capabilities of WSO2 SP allow you to capture high volume data flows and process them in real time, and present results in a streaming manner while its complex event processing capabilities detect patterns and trends for decision making via Patterns and Sequences supported for Siddhi.

Siddhi overview

Open

WSO2 SP uses the Siddhi query language to write the processing logic for its Siddhi applications. Siddhi can:

• Accept event inputs from many different types of sources

• Process them to generate insights

• Publish them to many types of sinks.

To use Siddhi, you need to write the processing logic as a Siddhi Application in the Siddhi Streaming SQL language. After writing and starting a Siddhi application, it:

• Takes data one-by-one as events

• Processes the data in each event

• Generates new high level events based on the processing done so far

• Sends newly generated events as the output to streams.

Before you begin:

1. Install Oracle Java SE Development Kit (JDK) version 1.8.

2. Set the JAVA_HOME environment variable.

3. Download the latest WSO2 Stream Processor.

4. Extract the downloaded zip and navigate to the <SP_HOME>/bin directory (<SP_HOME> is the extracted directory).

5. Issue one of the following commands to start the WSO2 Stream Processor Studio.

   • For Windows: editor.bat

   • For Linux: ./editor.sh

Once WSO2 SP server is successfully started, a log similar to the following is printed in the CLI.

Open

The server log prints the Stream Processor Studio URL in the start up logs as shown below.

Let's get started! You can write a simple Siddhi application to calculate the total weight with each cargo box loaded to the ship by following the steps below.

Step 1: Create a Siddhi application

Smith needs to calculate the total weight of the cargo loaded into a ship with every cargo box added. In order to generate this output for him, let's create a Siddhi application as follows:

1. Access the Stream Processor Studio via the http://<HOST_NAME>:<EDITOR_PORT>/editor URL. 

   The Stream Processor Studio opens as shown below.
   

2. Click New to start creating a new Siddhi application.

3. Enter a name for your Siddhi application. In this scenario, let's name the application CargoWeightApp as shown below.

   @App:name("CargoWeightApp")

4. Defining the input stream. The stream needs to have a name and a schema defining the data that each incoming event should contain. The event data attributes are expressed as name and type pairs. In this example:

   • The name of the input stream: CargoStream

   • A name to refer to the data in each event: weight

   • Type of the data received as weight: int

    

   define stream CargoStream (weight int);

5. Define an output stream. This has the same info as the previous definition with an additional totalWeight attribute that contains the total weight calculated so far. Here, we need to add a sink configuration to log the events from the OutputStream so that we can observe the output values.

   @sink(type='log', prefix='LOGGER') define stream OutputStream(weight int, totalWeight long);

6. Enter a Siddhi query that defines the following.

   • A name for the query (i.e., cargoWeightQuery)

   • The input stream from which the events to be processed are taken (i.e., CargoStream)

   • The data that needs to be sent to the output stream (i.e., weight and totalWeight)

   • How the output needs to be calculated (i.e., by calculating the sum of the weight of all the events)

   • The stream to which the output needs to be sent (i.e., OutputStream)

   
   This query is as follows:

   @info(name='CargoWeightQuery') from CargoStream select weight, sum(weight) as totalWeight insert into OutputStream;

   The completed Siddhi file is as follows:

   @App:name("CargoWeightApp") define stream CargoStream (weight int); @sink(type='log', prefix='LOGGER') define stream OutputStream(weight int, totalWeight long); @info(name='CargoWeightQuery') from CargoStream select weight, sum(weight) as totalWeight insert into OutputStream;

7. To save the Siddhi file, click File => Save. This opens the Save to Workspace dialog box. Click Save to save this file in the <SP_HOME>/wso2/editor/deployment/workspace directory (which is the default location where Siddhi applications are saved).