The following sections analyze the results of WSO2 API Manager performance tests done in the Amazon EC2 environment.
...
The following figure shows how the Throughput changes for different number of concurrent users.
Key observations:
...
See: JMeter Remote Test. Two JMeter servers are used to simulate high number of concurrent users.
The following are the EC2 instances that the API-M 2.5.0 performance tests were carried out on.
Synapse Gateway
Name | EC2 Instance Type | vCPU | Mem (GiB) |
|---|---|---|---|
Apache JMeter Client | c3.large | 2 | 3.75 |
Apache JMeter Server 01 | c3.xlarge | 4 | 7.5 |
Apache JMeter Server 02 | c3.xlarge | 4 | 7.5 |
WSO2 API Manager (Synapse) | c3.xlarge | 4 | 7.5 |
Netty HTTP Backend | c3.xlarge | 4 | 7.5 |
| MySQL | db.m3.medium (RDS) | 1 | 3.75 |
Ballerina Gateway
Name | EC2 Instance Type | vCPU | Mem (GiB) |
|---|---|---|---|
Apache JMeter Client | c3.large | 2 | 3.75 |
Apache JMeter Server 01 | c3.xlarge | 4 | 7.5 |
Apache JMeter Server 02 | c3.xlarge | 4 | 7.5 |
Microgateway | c3.large | 4 | 7.5 |
Netty HTTP Backend | c3.xlarge | 4 | 7.5 |
| Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Synapse Gateway
Ballerina Gateway
|
Refer to the following links for more details on Amazon Instance Types
...
# Samples - The number of requests sent with the given number of concurrent users.
Error Count - How many request errors were recorded.
Error % - Percent of requests with errors
Average - The average response time of a set of results
Min - The shortest time taken for a request
Max - The longest time taken for a request
90th Percentile - 90% of the requests took no more than this time. The remaining samples took at least as long as this
95th Percentile - 95% of the requests took no more than this time. The remaining samples took at least as long as this
99th Percentile - 99% of the requests took no more than this time. The remaining samples took at least as long as this
Throughput - The Throughput is measured in requests per second.
Received KB/sec - The throughput measured in received Kilobytes per second
Sent KB/sec - The throughput measured in sent Kilobytes per second
In addition, to above details, some additional details were recorded for every test.
GC Throughput - Time percentage the application was not busy with GC
GC throughput and other GC related details were obtained from the GC logs produced by the WSO2 API Manager.
The following are the GC flags used:
-XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:"$CARBON_HOME/repository/logs/gc.log
| Info |
|---|
The process memory was not considered as Java is working on an already reserved heap area. |
Performance Test Scripts
All scripts used to run the performance tests and analyze results are in the following repositories.
...
| Table of Content Zone | ||||
|---|---|---|---|---|
| ||||
Throughput Comparison The following charts show what happens to the server throughput when considering all results.
Average Response Time Comparison The following charts show what happens to the average response time when considering all results.
GC Throughput ComparisonThe following chart shows the GC throughput behavior when considering all results. 
Refer to the Observations from all results section for more details on the charts. |
