Page Comparison

The following sections analyze the results of WSO2 API Manager performance tests done in the Amazon EC2 environment.

...

1. Echo API: This is a secured API, which directly invokes the back-end service.

2. Mediation API: This is also a secured API, which has a “sequence” as a mediation extension to modify the message.

Tests were done using 100, 200, 300, 1000, and 2000 concurrent users. Concurrent Users mean that there are multiple users accessing the API Gateway at the same time. Different Message Sizes (Payload) were used for the tests with different back-end service delays. The message sizes used are 50B, 1KiB, 10KiB, and 100KiB. The back-end delays were 0ms, 30ms, 500ms, and 1s.

...

• # Samples - The number of requests sent with the given number of concurrent users.

• Error Count - How many request errors were recorded.

• Error % - Percent of requests with errors

• Average - The average response time of a set of results

• Min - The shortest time taken for a request

• Max - The longest time taken for a request

• 90th Percentile - 90% of the requests took no more than this time. The remaining samples took at least as long as this

• 95th Percentile - 95% of the requests took no more than this time. The remaining samples took at least as long as this

• 99th Percentile - 99% of the requests took no more than this time. The remaining samples took at least as long as this

• Throughput - The Throughput is measured in requests per second.

• Received KB/sec - The throughput measured in received Kilobytes per second

• Sent KB/sec - The throughput measured in sent Kilobytes per second

In addition, to above details, some additional details were recorded for every test.

• GC Throughput - Time percentage the application was not busy with GC

GC throughput and other GC related details were obtained from the GC logs produced by the WSO2 API Manager.

The following are the GC flags used:

-XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:"$CARBON_HOME/repository/logs/gc.log

Performance Test Scripts

All scripts used to run the performance tests and analyze results are in the following repositories.

...

Table of Content Zone
minLevel 3 location top
Throughput Comparison Image Modified The Echo API has some errors with 100KiB message size for 1000 and 2000 concurrent users. The following charts show what happens to the server throughput when considering all results. Throughput (Requests/sec) vs Concurrent Users Image Modified Throughput (Requests/sec) vs Message Size (Bytes) Image Modified Throughput (Requests/sec) vs Sleep Time (ms) Image Modified Average Response Time Comparison The following charts show what happens to the average response time when considering all results. Average Response Time (ms) vs Concurrent Users Average Response Time (ms) vs Message Size (Bytes) Average Response Time (ms) vs Sleep Time (ms) GC Throughput Comparison The following chart shows the GC throughput behavior when considering all results. API Manager GC Throughput (%) vs Concurrent Users API Manager GC Throughput (%) vs Message Size (Bytes) API Manager GC Throughput (%) vs Sleep Time (ms) Refer Observations from all results for more details on the charts.