Auto-Scaling Configuration
- Former user (Deleted)
- Former user (Deleted)
Enabling Auto-Scaling
By default, autoscaling is disabled in ELB. To enable it, open the <ELB_HOME>/repository/conf/loadbalancer.conf file and navigate to the 'loadbalancer' section, which describes the properties related to the ELB server. At a minimum, set the following properties.
- 'enable_autoscaler' property to 'true'.
- 'autoscaler_task_interval' property, which is the time between two task executions.
- The 'server_startup_delay' property, which is the maximum period of time the auto-scaler task waits before starting up, in order to make time for any instances that may get delayed to join ELB, due to network issues etc.
It is recommended to use a task interval which is greater than the time that a server instance takes to join the ELB. However, when the period defined in 'server_startup_delay' is over and the newly-started instance still fails to join the ELB, a termination request will be issued for that instance assuming it will never join. Therefore, it is strictly recommended to calibrate these values appropriately.
An example configuration is shown below:
loadbalancer {
# minimum number of load balancer instances
instances 1;
# whether autoscaling should be enabled or not.
enable_autoscaler true;
# autoscaling decision making task
#autoscaler_task org.wso2.carbon.mediator.autoscale.lbautoscale.task.ServiceRequestsInFlightAutoscaler;
# whether to use embedded autoscaler or not. By default, we use embedded autoscaler.
#use_embedded_autoscaler true;
#please use this whenever url-mapping is used through LB.
#size_of_cache 100;
# End point reference of the Autoscaler Service. This should be present, if you disabled embedded autoscaling.
#autoscaler_service_epr https://host_address:https_port/services/AutoscalerService/;
# interval between two task executions in milliseconds
autoscaler_task_interval 60000;
# after an instance booted up, task will wait maximum till this much of time and let the server started up
server_startup_delay 180000; #default will be 60000ms
# session time out
session_timeout 90000;
# enable fail over
fail_over true;
}
Described below are the properties in the 'loadbalancer' section of loadbalancer.conf file. Marked with * are vital auto-scaling parameters that require careful consideration when configuring.
| Property | Description |
|---|---|
instances | Minimum number of ELB instances running in the cluster. |
enable_autoscaler | Set to 'false' by default. Set to 'true' to enable auto-scaling. |
autoscaler_task | You can write your own Java implementation which implements 'org.apache.synapse.task.Task' and 'org.apache.synapse.ManagedLifecycle' interfaces. Wrap the implementation class to an OSGi bundle and deploy in WSO2 ELB. Then, point to that class from 'autoscaler-task' property. For example, loadbalancer {
…....
# autoscaling decision making task
autoscaler_task
org.wso2.carbon.mediator.autoscale.lbautoscale.task.ServiceRequestsInFlightAutoscaler;
…...
}
|
use_embedded_autoscaler | Boolean property that defaults to 'true'. If 'true', it triggers ELB's auto-scaling decision making task to call embedded cloud controller service, via OSGi. Cloud Controller service is integrated to ELB to provide capability of an embedded or a remote auto-scaler. |
size_of_cache | |
autoscaler_service_epr | End point reference of the auto-scaler service. |
autoscaler_task_interval* | Time period between two iterations of ‘auto-scaling decision making’ task. When configuring this value, you are advised to consider the time ‘that a service instance takes to join ELB’. This is in milliseconds and the default value is 30000ms. |
server_startup_delay | The maximum period of time the auto-scaler task waits before starting up, in order to make time for any instances that may get delayed to join ELB, due to network issues etc. |
session_timeout | ELB uses this timeout value to provide session affinity. This timeout value needs to be synchronized if you are running another application along with the ELB. |
fail_over |
Auto-Scaling Parameters of a Service Cluster
The 'services' section of the 'loadbalancer.conf' file contains properties used to define service cluster information. Given below is a sample configuration defining an 'appserver' service’s clusters under 'domains' element.
# services' details which are fronted by this WSO2 Elastic Load Balancer
services {
appserver {
domains {
wso2.as1.domain {
hosts as1.cloud.com;
tenant_range *;
min_app_instances 0;
alarming_upper_rate 0.6;
alarming_lower_rate 0.1;
}
wso2.as2.domain {
hosts as2.cloud.com;
tenant_range *;
rounds_to_average 10;
max_requests_per_second 15;
alarming_upper_rate 0.6;
alarming_lower_rate 0.1;
}
}
}
}
Defining Parameters Common to all Service Clusters
Using the 'default' section in 'loadbalancer.conf' file, you can define auto-scaling parameters common to all service clusters in the environment. A sample configuration is given below:
# services' details which are fronted by this WSO2 Elastic Load Balancer
services {
# default parameter values to be used in all services
defaults {
# minimum number of service instances required. WSO2 ELB will make sure that this much of instances are
# maintained in the system all the time, of course only when autoscaling is enabled.
min_app_instances 1;
# maximum number of service instances that will be load balanced by this ELB.
max_app_instances 3;
max_requests_per_second 5;
rounds_to_average 2;
alarming_upper_rate 0.7;
alarming_lower_rate 0.2;
scale_down_factor 0.25;
message_expiry_time 60000;
}
appserver {
domains {
wso2.as1.domain {
hosts as1.cloud.com;
tenant_range *;
min_app_instances 0;
alarming_upper_rate 0.6;
alarming_lower_rate 0.1;
}
wso2.as2.domain {
hosts as2.cloud.com;
tenant_range *;
rounds_to_average 10;
max_requests_per_second 15;
alarming_upper_rate 0.6;
alarming_lower_rate 0.1;
}
}
}
}
Described below are the properties of the 'services' section of 'loadbalancer.conf' file. Marked with * are vital auto-scaling parameters that require careful consideration when configuring.
| Property | Description |
|---|---|
min_app_instances | Used to limit the number of service instances that are maintained in the system at a given time. Set the 'min_app_instances' for any service cluster and the auto-scaler will ensure that the system will not scale down below that, even though there are no considerable service requests in-flight. |
max_app_instances | Controls the number of instances the auto-scaler can start. Set the 'max_app_instances' for any service cluster and the auto-scaler will make sure that the system will not scale up above that limit, even though there is a high load of requests in-flight. This is especially useful when you pay for the instances you start up. |
max_requests_per_second*
| Number of requests a service instance can withstand per second. It is recommended that you calibrate this value for each service instance and also for different scenarios. Ideally, this value can be estimated by load-testing a similar service instance. Default value is 100. |
rounds_to_average* | An auto-scaling decision will be made only after averaging the requests in-flight over this much of iterations of 'auto-scaling decision making' task. Default value is 10. |
alarming_upper_rate* | Without waiting till the service instance reaches its maximum request capacity (alarming_upper_rate = 1), we scale the system up when it reaches the request capacity, corresponds to 'alarming_upper_rate'. This value should be 0<AUR<=1 and default is 0.7. |
alarming_lower_rate* | Lower bound of the alarming rate. Provides guidance when scaling down the system. This value should be 0<ALR<=1 and default is 0.2. |
scale_down_factor* | This factor is needed in order to make the scaling-down slow. ELB scales down slowly to reduce impact of a false, positive event that triggers a scale-down. This value should be 0<SDF<=1 and default is 0.25. |
message_expiry_time |