...
| Info | ||
|---|---|---|
| ||
After around 12 million events are published, a sudden drop can be observed in receiver performance that can be considered as the break point of MySQL event store. Another type of event store such as HBase event store should be used when the receiver performance has to be maintained unchanged. |
Batch Analytics
Scenario: Running Spark queries on the 1 billion smart home published events
Spark queries from the Smart Home DAS sample were executed against the published data, and the analyzer node count was kept at 2 and 3 respectively for 2 separate tests. The SPARK JVMs were provided with following during the test.
...
| Query | 2 Analyzers | 3 Analyzers | ||
|---|---|---|---|---|
| Time(s) | Mean TPS | Time(s) | Mean TPS | |
INSERT INTO TABLE wikiAvgArticleLength SELECT AVG(length) as avg_article_length FROM wiki | 222.70 | 75234.03 | 167.27 | 100164.18 |
INSERT INTO TABLE wikiTotalArticleLength SELECT SUM(length) as total_article_chars FROM wiki | 221.74 | 75554.76 | 166.92 | 100373.80 |
INSERT INTO TABLE wikiTotalArticlePages SELECT COUNT(*) as total_pages FROM wiki | 221.80 | 75536.05 | 166.14 | 100842.18 |
INSERT INTO TABLE wikiContributorSummary SELECT contributor_username, COUNT(*) as page_count FROM wiki GROUP BY contributor_username | 236.11 | 70958.52 | 181.42 | 92350.26 |
Scenario: Running Spark queries on 1 million smart home and wikipedia events on MySQL Event Store
The following topics describe the analyzer performance of WSO2 DAS. Spark analyzing performance (time to complete execution) was measured using a 2 node DAS analyzer cluster with MySQL database.
...