...
| Excerpt | ||
|---|---|---|
| ||
Buffer logic is not used form SP 4.3.0 |
| Arriving Event | Execution |
|---|---|
| 0 | Let's assume that no buffer is defined (therefore, the buffer size is 0). The system initially processes event 0 at the SECOND executor level. The aggregation is retained in memory until the 58th second elapses. |
| 1 | This event also occurs during the 58th second (same as event 0). Therefore, the system aggregates events 0 and 1 together. |
| 2 | Event 2 arrives during the 59th second. The 58th second has elapsed at the time of this event arrival. Therefore, the system does the following:
|
| 3 | Event 3 arrives during the second 0 of 06.00 minute. With this arrival, the aggregations for the 59th second and 59th minute expire. Therefore, the system does the following:
|
| 4 | At the time event 4 arrives during the 1st second of the 06.00 minute, second 0 of the same minute has elapsed. Therefore, the system does the following:
|
| 5 | Event 5 arrives during the 2nd second. At this time, the 1st second has elapsed. Therefore, the system does the following:
|
...
| In-memory Table | Available Aggregation Records | Processing In-Memory |
|---|---|---|
TradeAggregation_SECONDS |
| Aggregation for |
TradeAggregation_MINUTES | Aggregation for | Aggregation for 2018-01-01 06:00:00 (i.e., minute 0 of the 6th hour. Here, event 3 and 4 are aggregated together.) |
TradeAggregation_HOURS | None | Aggregation for 2018-01-01 05:00:00 (i.e., the 5th hour. Here, events 0,1, and 2 are aggregated together.) |
| Excerpt | ||
|---|---|---|
| ||
Consider that no buffer has been defined (Hence buffer size is 0). When event 0 arrives, it would be initially processed at the second level. This aggregation would be retained in memory until the 58th second elapses. When event 1 arrives, since that too belongs to the 58th second (same as event 0), event 0 and event 1 data would be aggregated together. When event 2 arrives, we would know that the 58th second has now elapsed (since event 2 has the timestamp for the 59th second). Hence the aggregation for the 58th second (that is the total aggregation for event 0 and event 1) would be written to the <Aggregate name>_SECONDS table, and then forwarded to be processed at the minute level. At this phase, 59th second's data (event 2) would be processing in-memory at the second executor level. Aggregation which was forwarded would be processing in-memory for the 59th minute at minute executor level. When event 3 arrives, the 59th seconds' aggregation (event 2) would get expired and get written to <Aggregate name>_SECONDS table. It would also be forwarded to the minute executor. At the minute executor level, this forwarded aggregation (corresponding to event 2) would be aggregated in-memory, for the 59th minute. Effectively, the running in-memory aggregation at the minute level would have an aggregation for events 0, 1 and 2 at this phase. Once event 4 arrives, aggregation for timestamp 2018-01-01 06:00:00 (event 3) would be written to Aggregate name>_SECONDS table and forwarded to the minute executor. With the arrival of event 3 at the minute executor, it would identify that the 59th minute has now elapsed. Hence the aggregation for the 59th minute would be written to <Aggregate name>_MINUTES table and then forwarded to the hour executor. The hour executor would process this forwarded aggregation for the 5th hour. At this stage, event 4 would be processing in-memory at the second level; aggregation for event 3 would be processing in-memory at minute level; aggregation for the 5th hour would be processing in-memory at hour level. When event 5 arrives, aggregation for event 4 would be forwarded to the minute window. Hence event 3 and 4 would be aggregated together since they belong to the same minute (0th minute of the 6th hour). Event 5 would be processed in-memory at the second level. Hence the aggregation status would be as follows after all 5 events have arrived.
|
...