define table <table-id> (<attribute-name> <type> {, <attribute-name> <type>}*)
{ from ('parameterName'='value')+ }?
Event tables allow you to store, retrieve and process events in a database table-like structure. These are designed for usecases where events need to be extracted from the stream and accumulated over a long period for real-time (or non real-time) batch processing, such as performing comparisons with the incoming event stream or feeding it to BAM.
Unlike predefined tables, event tables can have sophisticated storage and retrieval criteria. A single event table can be used in multiple SiddhiQL expressions. Depending on the requirements, an event table can be defined either in-memory or in a relational database. CEP supports event tables for widely used databases such as MySQL.
In-memory database event tables
These are created in memory and therefore, are faster and easier to define. Following example shows a definition of an in-memory event table.
define table cseEventTable (symbol string, price int, volume float);
Relational database event tables
Two types of relational databases are currently supported by Siddhi as follows:
MySQL
H2
The following example shows how to define a table with table id cseEventTable
. It stores events in a relational database table named cepEventTable
.
define table cseEventTable (symbol string, price int, volume float) from ('datasource.name'='cepDataSource', 'table.name'='cepEventTable')
CEP supports caching for relational database tables with a few different algorithms. To change the algorithm, you use the optional caching.algorithm
parameter when defining the table as follows:
define table cseEventTable (symbol string, price int, volume float) from ('datasource.name'='cepDataSource', 'table.name'='cepEventTable', 'caching.algorithm'='LRU')
Note
At the moment, there are three cache management algorithms supported by CEP as follows:
- Basic: A size-based algorithm. Events are cached in a FIFO manner. The oldest event is dropped when the cache is full.
- LRU (Least Recently Used): The least recently used event is dropped when the cache is full.
- LFU (Least Frequently Used): The least frequently used event is dropped when the cache is full.
Note that specifying an algorithm is optional and when the user doesn't specify an algorithm, caching gets disabled.
Optionally, you can define a cache size using cache.size
parameter when defining the table as follows:
define table cseEventTable (symbol string, price int, volume float) from ('datasource.name'='cepDataSource', 'table.name'='cepEventTable', 'caching.algorithm'='LRU', 'cache.size'='10000')
In addition, you can also define a relational database event table with a user-provided SQL query as follows. This is used when the created table needs to be customized, such as having a primary key etc.
define table cseEventTable (symbol string, price int, volume float) from ('datasource.name'='cepDataSource', 'database.name'='cepdb', 'table.name'='cepEventTable' 'create.query'='CREATE TABLE cepEventTable (symbol VARCHAR(40), price DECIMAL, volume BIGINT )')
Note
The datasource.name
given here is injected to the Siddhi engine by the CEP server. To configure data sources in the CEP, see Datasources in the Admin Guide.
Note
Note that from CEP 3.1.0 onwards, the parameter database.name
is not supported. Instead, you are required to create the datasource pointing to the specific database that is used to create the tables.
Note
The table id can be different from the table name, and Siddhi will always refer to the table id defined here. However, the table id cannot be same as an already existing stream id, since syntactically both are considered in the same manner in the query language.