Page Comparison

This topic provides instructions on how to configure the multiple Gateways in WSO2 API Manager (WSO2 API-M) in a distributed deployment to facilitate high availability (HA). The instructions in this topic are based on the Distributed Deployment of API Manager and these configurations will only work if the configurations in that topic are done correctly. For instance, all datasource configurations are already done for the Gateway in that topic and hence do not need to be repeated here.

The following sections provide specific instructions on configuring the Gateway cluster.

Gateway deployment pattern

The configurations in this topic will be done based on the following pattern. This pattern is used as a basic Gateway cluster where the worker nodes and manager nodes are separated.

Image Removed

Step 1 - Configure the load balancer

NGINX is used for this scenario, but you can use any load balancer that you prefer here. The following are the configurations that need to be done for the load balancer.

Use the following steps to configure Nginx as the load balancer for WSO2 products.

1. Install Nginx using the following command.
   $sudo apt-get install nginx

2. Configure Nginx Plus to direct the HTTP requests to the two worker nodes via the HTTP 80 port using the http://am.wso2.com/<service>. To do this, create a VHost file (am.http.conf) in the /etc/nginx/conf.d/ directory and add the following configurations into it.

   Code Block

   upstream wso2.am.com { sticky cookie JSESSIONID; server xxx.xxx.xxx.xx4:9763; server xxx.xxx.xxx.xx5:9763; } server { listen 80; server_name am.wso2.com; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_read_timeout 5m; proxy_send_timeout 5m; proxy_pass http://wso2.am.com; } }

3. Configure Nginx Plus to direct the HTTPS requests to the two worker nodes via the HTTPS 443 port using https://am.wso2.com/<service>. To do this, create a VHost file (am.https.conf) in the /etc/nginx/conf.d/ directory and add the following configurations into it.

   Code Block

   upstream ssl.wso2.am.com { sticky cookie JSESSIONID; server xxx.xxx.xxx.xx4:9443; server xxx.xxx.xxx.xx5:9443; } server { listen 443; server_name am.wso2.com; ssl on; ssl_certificate /etc/nginx/ssl/wrk.crt; ssl_certificate_key /etc/nginx/ssl/wrk.key; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_read_timeout 5m; proxy_send_timeout 5m; proxy_pass https://ssl.wso2.am.com; } }

4. Configure Nginx Plus to access the Management Console as  https://mgt.am.wso2.com/carbon via HTTPS 443 port. This is to direct requests to the manager node. To do this, create a VHost file (mgt.am.https.conf) in the /etc/nginx/conf.d/ directory and add the following configurations into it.

   Code Block

   server { listen 443; server_name mgt.am.wso2.com; ssl on; ssl_certificate /etc/nginx/ssl/mgt.crt; ssl_certificate_key /etc/nginx/ssl/mgt.key; location / { proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Server $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_read_timeout 5m; proxy_send_timeout 5m; proxy_pass https://xxx.xxx.xxx.xx3:9443/; } error_log /var/log/nginx/mgt-error.log ; access_log /var/log/nginx/mgt-access.log; }

5. Restart the Nginx Plus server.
   $sudo service nginx restart


   Tip
   Tip: You do not need to restart the server if you are simply making a modification to the VHost file. The following command should be sufficient in such cases. sudo service nginx reload

Step 2 - Create SSL certificates

Create SSL certificates for both the manager and worker nodes using the instructions that follow. 

1. Create the Server Key.

   Code Block
   $sudo openssl genrsa -des3 -out <key_name>.key 1024

2. Certificate Signing Request.

   Code Block
   $sudo openssl req -new -key <key_name>.key -out server.csr

3. Remove the password.

   Code Block
   $sudo cp <key_name>.key <key_name>.key.org $sudo openssl rsa -in <key_name>.key.org -out <key_name>.key

4. Sign your SSL Certificate.

   Code Block
   $sudo openssl x509 -req -days 365 -in server.csr -signkey <key_name>.key -out <certificate_name>.crt

5. Copy the key and certificate files generated in above step 4 to /etc/nginx/ssl/ location. 

You have now configured the load balancer to handle requests sent to am.wso2.com and to distribute the load among the worker nodes in the worker sub-domain of the wso2.am.domain cluster.

Step 3 - Configure the Gateway manager

Management nodes specialize in management of the setup. Only management nodes are authorized to add new artifacts into the system or make configuration changes. Management nodes are usually behind an internal firewall and are exposed to clients running within the organization only. This section involves setting up the Gateway node and enabling  

Furthermore, these instructions use a shared file system as the content synchronization mechanism.

Follow the instructions below to configure the API-M Gateway in a distributed environment: 

Note that the configurations in this topic are done based on the following pattern. 
Image Added

Step 1 - Configure the load balancer

For more information see, Configuring the Proxy Server and the Load Balancer.

Step 2 - Configure the Gateway

When using the shared file system, all nodes have the manager worker capability. Therefore, there is no need of having a separate manager node. Follow the instructions below to set up the Gateway nodes and enable it to work with the other components in the distributed setup.

Step 3.1 - Configure the common configurations

Carryout the following configurations on the Gateway manager node.

Modify the api-manager.xml file as follows. This configures the connection to the Key Manager component.

Configure key management related communication.

In a clustered setup if the Key Manager is fronted by a load balancer, you have to use WSClient as KeyValidatorClientType in <API-M_HOME>/repository/conf/api-manager.xml. This should be configured in all Gateway and Key Manager components.

Disable the Thrift Client to optimize performance.
You need to configure this in the Gateway  <API-M_HOME>/repository/conf/api-manager.xml file.

In a clustered setup if the Key Manager is NOT fronted by a load balancer, you have to use ThriftClient as KeyValidatorClientType in <API-M_HOME>/repository/conf/api-manager.xml. This should be configured in all Gateway and Key Manager components.

Disable the Thrift Client to optimize performance.
You need to configure this in the Gateway <API-M_HOME>/repository/conf/api-manager.xml file.

Specify the ThriftClientPort and ThriftServerPort values. 10397 is the default.

Specify the hostname or IP of the Key Manager. The default is localhost. In a distributed deployment we must set this parameter in both Key Manager nodes and Gateway nodes only if the Key Manager is running on a separate machine. Gateway uses this parameter to connect to the key validation thrift service.

Step 4 - Configure the Gateway worker

Worker nodes specialize in serving requests to deployment artifacts and and reading them. They can be exposed to external clients.

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9763"
                proxyPort="80"
--------
/>
<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"
                port="9443"
                proxyPort="443"
--------
/>

<GATEWAY-MANAGER-IP> mgt.am.wso2.com 

xxx.xxx.xxx.xx3 mgt.am.wso2.com 

...

Step 3 - Optionally, configure Hazelcast

You can seamlessly deploy WSO2 API Manager using local caching in a clustered setup without Hazelcast clustering. However, there are edge case scenarios where you need to enable Hazelcast clustering. To  For more information, see Working with Hazelcast Clustering to identify whether you need Hazelcast clustering and to configure Hazelcast clustering if needed, see Working with Hazelcast Clusteringit.

Step

...

4 - Start the Gateway Nodes

...

Follow the instructions below to start the Gateway

...

nodes:

1. Comment the following configurations in the <API-M_HOME>/repository/conf/api-manager.xml file.

   Code Block

   <JMSEventPublisherParameters> <java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial> <java.naming.provider.url>repository/conf/jndi.properties</java.naming.provider.url> <transport.jms.DestinationType>topic</transport.jms.DestinationType> <transport.jms.Destination>throttleData</transport.jms.Destination> <transport.jms.ConcurrentPublishers>allow</transport.jms.ConcurrentPublishers> <transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName> </JMSEventPublisherParameters>

2. Start the WSO2 API-M Gateway manager node by typing the following command in the terminalcommand prompt.

   Code Block
   sh <GATEWAY<API-M_MANAGERGATEWAY_HOME>/bin/wso2server.sh

Step 6.2 - Start the Gateway worker

...

Update the <API-M_HOME>/repository/conf/api-manager.xml file by commenting out the following configurations.

<JMSEventPublisherParameters>
   <java.naming.factory.initial>org.wso2.andes.jndi.PropertiesFileInitialContextFactory</java.naming.factory.initial>
   <java.naming.provider.url>repository/conf/jndi.properties</java.naming.provider.url>
   <transport.jms.DestinationType>topic</transport.jms.DestinationType>
   <transport.jms.Destination>throttleData</transport.jms.Destination>
   <transport.jms.ConcurrentPublishers>allow</transport.jms.ConcurrentPublishers>
   <transport.jms.ConnectionFactoryJNDIName>TopicConnectionFactory</transport.jms.ConnectionFactoryJNDIName>
</JMSEventPublisherParameters>

Start the Gateway worker by typing the following command in the terminal.

sh <GATEWAY_WORKER_HOME>/bin/wso2server.sh -Dprofile=gateway-worker

...