Collecting Events

The first step in stream processing is to collect the data that needs to be analyzed. Collection of data is done through Siddhi source which can be defined via @source() annotation on a steam.

To collect data to be processed a stream should have been defined in the Siddhi Application along with the @source() annotation and deployed in WSO2 SP. Here a single SiddhiApp can contain multiple streams and each of those steams can also contain multiple @source() annotations.

Example Input Source definition as below.

@Source(type = 'http',        
		receiver.url='http://localhost:8006/productionStream',
        basic.auth.enabled='false',
        @map(type='json', @attributes( name='$.name', amount='$.quantity')))
define stream SweetProductionStream (name string, amount double);

The source defines the following :

Input source type: via type = 'http'
Source type configurations (This is optional. In this example http source type configurations are defined via receiver.url='http://localhost:8006/productionStream', basic.auth.enabled='false')
Input message format: with @map() sub-annotation.
Custom attribute mapping of the input message format (optional): with optional @attributes( name='$.name', amount='$.quantity') sub-annotation of @map.

A source could also be defined externally, and referred to from several siddhi applications as described below,Multiple sources can be defined in the <SP HOME>/conf/<PROFILE>/deployment.yaml file. A <PROFILE> could refer to dashboard, editor, manager or worker. The following is a sample configuration of a source.

siddhi:

  refs:
    -
      ref:
        name: 'source1'
        type: '<store.type>'
        properties:
          <property1>: <value1>
          <property2>: <value2>

You can refer to a source configured in the <SP HOME>/conf/<PROFILE>/deployment.yaml file from a Siddhi application as shown in the example below.

@Source(ref='source1', basic.auth.enabled='false',
        @map(type='json', @attributes( name='$.name', amount='$.quantity')))
define stream SweetProductionStream (name string, amount double);

For detailed instructions to configure a source, see Siddhi Guide - Source.

Source types

WSO2 SP supports following source types out of the box, to receive events via corresponding transports. Click on the required source type for instructions to configure a source to receive events via them.

Event format

WSO2 Siddhi allows events to be received in multiple formats. The following formats are currently supported. Once an event is received in a specific format, it is internally converted to a Siddhi event so that it can be processed by applying the WSO2 Siddhi logic. Click on the required format for detailed information on how a source can be configured to receive events in that format.