...
DAS data persistence was measured by publishing to 2 load balanced receiver nodes with MySQL database.
| Sample | Number of Events | Mean Event Rate |
|---|---|---|
Smart Home sample | 100000000 | 5741 events per second |
Wikipedia sample | 15901127 | 4438 events per second |
Analyzer Performance
The following topics describe the analyzer performance of WSO2 DAS.
...
Time taken for each type of Spark query is given below.
| Data set | Event Count | Query Type | Time Taken (seconds) |
|---|---|---|---|
| Smart Home | 10000000 | INSERT OVERWRITE TABLE cityUsage SELECT metro_area, avg(power_reading) AS avg_usage, min(power_reading) AS min_usage, max(power_reading) AS max_usage FROM smartHomeData GROUP BY metro_area | 26 sec |
| Smart Home | 10000000 | INSERT OVERWRITE TABLE peakDeviceUsageRange SELECT house_id, (max(power_reading) - min(power_reading)) AS usage_range FROM smartHomeData WHERE is_peak = true AND metro_area = "Seattle" GROUP BY house_id | 22 sec |
| Smart Home | 10000000 | INSERT OVERWRITE TABLE stateAvgUsage SELECT state, avg(power_reading) AS state_avg_usage FROM smartHomeData | 21 sec |
| Smart Home | 10000000 | INSERT OVERWRITE TABLE stateUsageDifference SELECT a2.state, (a2.state_avg_usage-a1.overall_avg) AS avg_usage_difference FROM (select avg(state_avg_usage) as overall_avg from stateAvgUsage) as a1 join stateAvgUsage as a2 | 1 sec |
| Wikipedia | 10000000 | INSERT INTO TABLE wikiAvgArticleLength SELECT AVG(length) as avg_article_length FROM wiki | 48 min |
| Wikipedia | 10000000 | INSERT INTO TABLE wikiContributorSummary SELECT contributor_username, COUNT(*) as page_count FROM wiki GROUP BY contributor_username | 1 hour 45 min |
| Wikipedia | 10000000 | INSERT INTO TABLE wikiTotalArticleLength SELECT SUM(length) as total_article_chars FROM wiki | 44 min |
| Wikipedia | 10000000 | INSERT INTO TABLE wikiTotalArticlePages SELECT COUNT(*) as total_pages FROM wiki | 1 hour 17 min |
Single Node Local Clustered Setup Statistics
...
| Mode | Dataset | shardIndexRecordBatchSize | Replication Factor | Event Count | Time Taken (seconds) | Average TPS |
|---|---|---|---|---|---|---|
| Standalone | Wikipedia | 10MB | NA | 15901127 | 7975 | 1993.871724 |
| Standalone | Wikipedia | 20MB | NA | 15901127 | 6765 | 2350.499187 |
| Standalone | Smart Home | 20MB | NA | 20000000 | 1385 | 14440.43321 |
| Minimum Fully Distributed | Wikipedia | 20MB | 1 | 15901127 | 6870 | 2314.574527 |
| Minimum Fully Distributed | Wikipedia | 20MB | 0 | 15901127 | 7280 | 2184.220742 |
REST API Access Performance
This test was conducted on a test setup as shown in the following figure,
Using JMeter the DAS’s search REST API was invoked. 80 JMeter users were used and they sent requests in a tight loop. The request was sent to query a single record from an event table. In this experiment the MySQL server’s event table had data which had been loaded in previous experiments. The experiment was run 45 minutes. Average entire throughput value of 2,695 events/second and an average latency of 29 ms was measured at the JMeter.