You set up a cluster for WSO2 App Manager and create a distributed deployment of its four main components: Publisher, Store, Gateway, and Identity Provider. This page describes how to set up the distributed deployment in the following sections.
Understanding the App Manager architecture
App Manager uses four main components. They are Publisher, Store, Gateway and IDP. Additionally, App Manager uses four databases. The following databases are shared among the server nodes.
User Manager Database(UM_DB) - Stores information related to users and user roles. This information is shared among the IDP Server, Store, Publisher and Gateway.
APP Manager Database(AM_DB) - Stores information related to the apps along with the app subscription details.
Registry Database(REG_DB) - Shares information between the Publisher, Store and Gateway.
ES Storage Database(ES_STORAGE_DB) - Shares resources (images, etc.) between the Publisher and Store.
In the following diagram, the four components are set up in a distributed deployment, and the four databases are connected to the relevant components respectively.
Installing and configuring App Manager
The following steps describe how to download, install, and configure App Manager. You must create four instances of App Manager for this.
- Download the latest versions of WSO2 App Manager in each of the servers in the cluster for distributed deployment.
- Unzip the App Manager zipped archive, and rename each of those directories respectively as IDP, Gateway, Publisher, and Store. These four directories are located in a server of their own and will be used for each component of the API Manager. Each of these unzipped directories will be referred to as
<APPM_HOME>or<PRODUCT_HOME>in this document.
Installing and configuring the databases
The following steps describe how to download and install MySQL Server, create the databases, configure the datasources, and configure the App Manager components to connect to them.
Download and install MySQL Server.
Download the MySQL JDBC driver.
Unzip the downloaded MySQL driver zipped archive, and copy the MySQL JDBC driver JAR (
mysql-connector-java-x.x.xx-bin.jar) into the<PRODUCT_HOME>/repository/components/libdirectory of all the nodes in the cluster.- Define the host name for configuring permissions for the new database by opening the
/etc/hostsfile and adding the following line:<MYSQL-DB-SERVER-IP> carbondb.mysql-wso2.com
You would do this step only if your database is not on your local machine and on a separate server.
- Enter the following command in a terminal/command window, where
usernameis the username you want to use to access the databases:mysql -u username -p - When prompted, specify the password that will be used to access the databases with the username you specified.
Only four databases are used as shared MySQL databases, the rest of the data is stored in the internal H2 database that is shipped with the product. Create the four databases using the following commands. The
usernameandpasswordare the same as those you specified in the previous steps.mysql> create database AM_DB; mysql> use AM_DB; mysql> source <APPM_HOME>/dbscripts/appmgt/mysql.sql; mysql> source <APPM_HOME>/dbscripts/identity/mysql.sql; mysql> grant all on AM_DB.* TO username@localhost identified by "password"; mysql> create database UM_DB; mysql> use UM_DB; mysql> source <APPM_HOME>/dbscripts/mysql.sql; mysql> grant all on UM_DB.* TO username@localhost identified by "password"; mysql> create database REG_DB; mysql> use REG_DB; mysql> source <APPM_HOME>/dbscripts/mysql.sql; mysql> grant all on REG_DB.* TO username@localhost identified by "password"; mysql> create database ES_STORAGE_DB; mysql> use ES_STORAGE_DB; mysql> source <APPM_HOME>/dbscripts/storage/mysql/resource.sql; mysql> grant all on ES_STORAGE_DB.* TO username@localhost identified by "password";
Ensure that MySQL is configured so that all nodes can connect to it.
Configure the datasources for the four database as follows:
Open the
<APPM_HOME>/repository/conf/datasources/master-datasources.xmlfile in all four App Manager nodes.Enable the components to access the App Manager database by modifying the WSO2AM_DB datasource in all four
master-datasources.xmlfiles as follows:<datasource> <name>WSO2AM_DB</name> <description>The datasource used for the API Manager database</description> <jndiConfig> <name>jdbc/WSO2AM_DB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://apimgtdb.mysql-wso2.com:3306/apimgtdb?autoReconnect=true</url> <username>user</username> <password>password</password> <defaultAutoCommit>false</defaultAutoCommit> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource>Enable all components to access the user's database by configuring the WSO2UM_DB datasource in their
master-datasources.xmlfiles as follows:<datasource> <name>WSO2UM_DB</name> <description>The datasource used by user manager</description> <jndiConfig> <name>jdbc/WSO2UM_DB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://userdb.mysql-wso2.com:3306/userdb?autoReconnect=true</url> <username>user</username> <password>password</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource>Enable the Publisher and Store components to access the registry database by configuring the WSO2REG_DB data source in their
master-datasources.xmlfiles as follows:<datasource> <name>WSO2REG_DB</name> <description>The datasource used by the registry</description> <jndiConfig> <name>jdbc/WSO2REG_DB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://regdb.mysql-wso2.com:3306/regdb?autoReconnect=true</url> <username>user</username> <password>password</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource>Enable the Publisher and Store components to access the storage database by configuring the ES_Storage datasource in their
master-datasources.xmlfiles as follows:<datasource> <name>JAGH2</name> <description>The datasource used for by the Jaggery Storage Manager</description> <jndiConfig> <name>jdbc/ES_Storage</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://mysql-appm.wso2.com:3306/ES_STORAGE_DB?autoReconnect=true</url> <username>user</username> <password>password</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource>
To give the Gateway, Publisher and Store components access to the App Manager database, open the
<APPM_HOME>/repository/conf/api-manager.xmlfile in each of the components and add the following line as the first child node of the root element (if it is not already there):<DataSourceName>jdbc/WSO2AM_DB</DataSourceName>
<UiActivityPublishDataSourceName>jdbc/WSO2AM_DB</UiActivityPublishDataSourceName>To give all the components access to the user's database with shared permissions, open the
<APPM _HOME>/repository/conf/user-mgt.xmlfile in each of these nodes and add or modify thedataSourceproperty of the<configuration>element as follows:<configuration> ... <Property name="dataSource">jdbc/WSO2UM_DB</Property> </configuration> <UserStoreManager class="org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager"> <Property name="TenantManager">org.wso2.carbon.user.core.tenant.JDBCTenantManager</Property> <Property name="ReadOnly">false</Property> <Property name="MaxUserNameListLength">100</Property> <Property name="IsEmailUserName">false</Property> <Property name="DomainCalculation">default</Property> <Property name="PasswordDigest">SHA-256</Property> <Property name="StoreSaltedPassword">true</Property> <Property name="ReadGroups">true</Property> <Property name="WriteGroups">true</Property> <Property name="UserNameUniqueAcrossTenants">false</Property> <Property name="PasswordJavaRegEx">^[\S]{5,30}$</Property> <Property name="PasswordJavaScriptRegEx">^[\S]{5,30}$</Property> <Property name="UsernameJavaRegEx">^[^~!#$;%^*+={}\\|\\\\<>,\'\"]{3,30}$</Property> <Property name="UsernameJavaScriptRegEx">^[\S]{3,30}$</Property> <Property name="RolenameJavaRegEx">^[^~!#$;%^*+={}\\|\\\\<>,\'\"]{3,30}$</Property> <Property name="RolenameJavaScriptRegEx">^[\S]{3,30}$</Property> <Property name="UserRolesCacheEnabled">true</Property> <Property name="MaxRoleNameListLength">100</Property> <Property name="MaxUserNameListLength">100</Property> <Property name="SharedGroupEnabled">false</Property> <Property name="SCIMEnabled">false</Property> </UserStoreManager>To give the Publisher and Store components access to the registry database, open the
<APPM_HOME>/repository/conf/registry.xmlfile in each of these two components and configure them as follows:Note: Do not replace the following configuration when adding in the mounting configurations. The registry mounting configurations mentioned in the below steps must be added in addition to the following.
<dbConfig name="wso2registry"> <dataSource>jdbc/WSO2CarbonDB</dataSource> </dbConfig>
In the Publisher component's
registry.xmlfile, add or modify thedataSourceattribute of the<dbConfig name="govregistry">element as follows:<dbConfig name="govregistry"> <dataSource>jdbc/WSO2REG_DB</dataSource> </dbConfig> <remoteInstance url="https://publisher.appm-wso2.com"> <id>gov</id> <cacheId>user@jdbc:mysql://regdb.mysql-wso2.com:3306/regdb</cacheId> <dbConfig>govregistry</dbConfig> <readOnly>false</readOnly> <enableCache>true</enableCache> <registryRoot>/</registryRoot> </remoteInstance> <mount path="/_system/governance" overwrite="true"> <instanceId>gov</instanceId> <targetPath>/_system/governance</targetPath> </mount> <mount path="/_system/config" overwrite="true"> <instanceId>gov</instanceId> <targetPath>/_system/config</targetPath> </mount>
In the Store component's
registry.xmlfile, add or modify thedataSourceattribute of the<dbConfig name="govregistry">element as follows (note that this configuration is nearly identical to the previous step with the exception of theremoteInstanceURL):<dbConfig name="govregistry"> <dataSource>jdbc/WSO2REG_DB</dataSource> </dbConfig> <remoteInstance url="https://store.appm-wso2.com"> <id>gov</id> <cacheId>user@jdbc:mysql://regdb.mysql-wso2.com:3306/regdb</cacheId> <dbConfig>govregistry</dbConfig> <readOnly>false</readOnly> <enableCache>true</enableCache> <registryRoot>/</registryRoot> </remoteInstance> <mount path="/_system/governance" overwrite="true"> <instanceId>gov</instanceId> <targetPath>/_system/governance</targetPath> </mount> <mount path="/_system/config" overwrite="true"> <instanceId>gov</instanceId> <targetPath>/_system/config</targetPath> </mount>
Modify the /etc/hosts entries to map the relevant IP addresses to the remoteInstance URLs.
127.0.0.1 publisher.appm-wso2.com
127.0.0.1 store.appm-wso2.com
Once registry caching is enabled, sync the published apps between the Publisher and Store nodes by enabling clustering in both Store and Publisher nodes. To do this, open the
<APIM_HOME>/repository/conf/axis2/axis2.xmlfile in each of these two components and configure them as follows:<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
Configuring the connections among the components
You can now configure the inter-component relationships illustrated in the following diagram by modifying their <APPM_HOME>/repository/conf/api-manager.xml files.
- Open the
<APPM_HOME>/repository/conf/api-manager.xmlfiles in the Gateway, Publisher, IDP and Store components. - Modify each of the files as follows:
Publisher and Store:
<AuthManager> <ServerURL>https://<IP of the IDP>:9443/services//ServerURL> <Username>admin</Username> <Password>admin</Password> </AuthManager> <APIGateway> <Environments> <Environment type="hybrid"> … <ServerURL>https://<IP of the Gateway>:9443/services/</ServerURL> <Username>admin</Username> <Password>admin</Password> <GatewayEndpoint>http://<IP of the Gateway>:8280 ,https://<IP of the Gateway>:8243</GatewayEndpoint> </Environment> </Environments> … </APIGateway> <SSOConfiguration> <IdentityProviderUrl>https:/<IP of the IDP>:9443/samlsso</IdentityProviderUrl> <Configurators> <Configurator> ... <parameters> <providerURL>https:/<IP of the IDP>:9443</providerURL> <username>admin</username> <password>admin</password> </parameters> </Configurator> ….. </Configurators> </SSOConfiguration> <EntitlementServiceConfiguration> …. <Parameters> <ServerUrl>https://<IP of the IDP>:9443</ServerUrl> <Username>admin</Username> <Password>admin</Password> </Parameters> </EntitlementServiceConfiguration>Publisher only:
<APIKeyManager> <ServerURL>https://<IP of the IDP>:9443/services/</ServerURL> <Username>admin</Username> <Password>admin</Password> … <KeyValidatorClientType>ThriftClient</KeyValidatorClientType> <ThriftClientPort>10398</ThriftClientPort> <ThriftClientConnectionTimeOut>10000</ThriftClientConnectionTimeOut> <ThriftServerPort>10398</ThriftServerPort> …. </APIKeyManager>Store only:
<APIKeyManager> <ServerURL>https://<IP of the IDP>:9443/services/</ServerURL> <Username>admin</Username> <Password>admin</Password> … <KeyValidatorClientType>ThriftClient</KeyValidatorClientType> <ThriftClientPort>10399</ThriftClientPort> <ThriftClientConnectionTimeOut>10000</ThriftClilentConnectionTimeOut> <ThriftServerPort>10399</ThriftServerPort> …. </APIKeyManager>Gateway:
<APIKeyManager> <ServerURL>https://<IP of the IDP>:9443/services/</ServerURL> <Username>admin</Username> <Password>admin</Password> … <KeyValidatorClientType>ThriftClient</KeyValidatorClientType> <ThriftClientPort>10396</ThriftClientPort> <ThriftClientConnectionTimeOut>10000</ThriftClientConnectionTimeOut> <ThriftServerPort>10396</ThriftServerPort> …. </APIKeyManager> <SSOConfiguration> <IdentityProviderUrl> https:/<IP of the IDP>:9443/samlsso </IdentityProviderUrl> <Configurators> <Configurator> ... <parameters> <providerURL>https:/<IP of the IDP>:9443</providerURL> <username>admin</username> <password>admin</password> </parameters> </Configurator> ….. </Configurators> </SSOConfiguration>IDP:
<APIGateway> <Environments> <Environment type="hybrid"> … <ServerURL>https://<IP of the Gateway>:9443/services/</ServerURL> <Username>admin</Username> <Password>admin</Password> <GatewayEndpoint>http://<IP of the Gateway>:8280 , https://<IP of the Gateway>:8243</GatewayEndpoint> </Environment> </Environments> … </APIGateway>
Open the
<APPM_HOME>/repository/conf/sso-idp-config.xmlfiles in the IDP component and modify it as follows.<SSOIdentityProviderConfig> <TenantRegistrationPage>https://stratos-local.wso2.com/carbon/tenant-register/select_domain.jsp</TenantRegistrationPage> <ServiceProviders> <ServiceProvider> <Issuer>store</Issuer> <AssertionConsumerService>https://localhost:9447/store/acs</AssertionConsumerService> <SignResponse>true</SignResponse> <CustomLoginPage>/store/login.jag</CustomLoginPage> </ServiceProvider> <ServiceProvider> <Issuer>social</Issuer> <AssertionConsumerService>https://localhost:9447/social/acs</AssertionConsumerService> <SignResponse>true</SignResponse> <CustomLoginPage>/social/login</CustomLoginPage> </ServiceProvider> <ServiceProvider> <Issuer>publisher</Issuer> <AssertionConsumerService>https://localhost:9446/publisher/acs</AssertionConsumerService> <SignResponse>true</SignResponse> <CustomLoginPage>/publisher/controllers/login.jag</CustomLoginPage> </ServiceProvider> </ServiceProviders> </SSOIdentityProviderConfig>
Configuring the Publisher and Store to be on the same cluster domain
As explained in the sections above, the Publisher and Store use a shared registry for sharing app artifacts. Therefore whenever an artifact is added/modified on the Publisher, the change needs to be reflected on the Store appropriately. Due to the fact that the Store fetches artifacts from the registry cache, modifications to artifacts done on the Publisher may not immediately be reflected on the Store unless the relevant cache entry is explicitly cleared off or the cache expires. For the cache to be explicitly cleared off, the Publisher and Store need to be in the same cluster domain and need to be aware of each other's existence.
A Gateway node must be up and running for you to create apps through the Publisher.
Follow the steps below to configure the Publisher and Store to be in a single cluster domain. You need to configure both nodes as instructed below
Open the
<APPM_HOME>/repository/conf/axis2/axis2.xmlfile and scroll down to the 'Clustering' section.Set the 'enable' attribute of the
<clustering>element totrue.<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
Change the '
membershipScheme' parameter to 'wka'.<parameter name="membershipScheme">wka</parameter>
Provide a domain for the cluster.
<parameter name="domain">wso2.pub.store.domain</parameter>
Specify the 'localMemberHost' and 'localMemberPort' parameters. If on the publisher, the 'localMemberHost' should be the publisher's IP address. The port value should be the port on which the Publisher will be listening for incoming cluster messages. Same applies to the Store.
<parameter name="localMemberHost">192.168.10.1</parameter> <parameter name="localMemberPort">4000</parameter>
Specify well known member. When specifying the well known member, the Publisher should specify the Store information and the Store should specify the Publisher information. The port that we provide here should be equal to the '
localMemberPort' of the other member.<members> <member> <hostName>192.168.10.2</hostName> <port>4000</port> </member> </members>- Save and close the files and restart the servers (if running) for the changes to take effect.
Configuring SSO for the Publisher and Store
The following steps provide instructions on how to configure Single Sign-On (SSO) for the Publisher and Store components.
In the Publisher, open the
<APPM_HOME>/repository/deployment/server/jaggeryapps/publisher/config/publisher.jsonfile and modify it as follows:"ssoConfiguration": { "enabled": true, "issuer": "publisher", "identityProviderURL": "https://localhost:9444/samlsso", "keyStorePassword": "wso2carbon", "identityAlias": "wso2carbon", "responseSigningEnabled": "true", "publisherAcs": "https://localhost:9446/publisher/acs", "keyStoreName": "/repository/resources/security/wso2carbon.jks" }In the Store, open the
<APPM_HOME>/repository/deployment/server/jaggeryapps/store/config/publisher.jsonfile and modify it as follows:"ssoConfiguration": { "enabled": true, "issuer": "store", "identityProviderURL": "http://localhost:9444/samlsso", "keyStorePassword": "wso2carbon", "identityAlias": "wso2carbon", "responseSigningEnabled": "true", "storeAcs" : "https://localhost:9447/store/acs", "keyStoreName": "/repository/resources/security/wso2carbon.jks" },In the Store, you must also open the
<APPM_HOME>/repository/deployment/server/jaggeryapps/social/config/social.jsonfile and modify it as follows."ssoConfiguration": { "enabled": true, "issuer": "social", "identityProviderURL": "http://localhost:9444/samlsso", "keyStorePassword": "wso2carbon", "identityAlias": "wso2carbon", "responseSigningEnabled": "true", "storeAcs" : "https://localhost:9447/store/acs", "keyStoreName": "/repository/resources/security/wso2carbon.jks" },
Create a cluster of Gateway components
To cluster App Manager, create a cluster of Gateway components. To cluster the Gateway component, use the steps in Clustering the App Manager Gateway.
Configuring the Identity Server as the IDP
This section includes steps on how to configure the WSO2 Identity Server as the identity provider (IDP). When the Identity Server is used as the IDP component, use the same configurations mentioned in the sections above for the Publisher, Store and Gateway. Use the IP of IS server as <IP of the IDP> in above configs.
Download the WSO2 Identity Server. Unzip the downloaded file into the server and rename the directory as IDP. The unzipped directory will be referred to as <IS_HOME> in this document.
Copy the MySQL JDBC driver JAR (mysql-connector-java-x.x.xx-bin.jar) into the
<IS_HOME>/repository/components/libdirectory of all the nodes in the cluster if you have not done so already.If SQL commands in the Clustering App Manager 1.1.0 section are already executed, execute the following script also.
mysql> drop database UM_DB; mysql> create database UM_DB; mysql> use UM_DB; mysql> source <IS_HOME>/dbscripts/mysql.sql; mysql> grant all on UM_DB.* TO username@localhost identified by "password";
Configure the datasource for UM database as follows:
Open the
<IS_HOME>/repository/conf/datasources/master-datasources.xmlfile in the Identity Server component.Enable IDP components to access the user's database by configuring the WSO2UM_DB datasource in the master-datasources.xml file as follows:
<datasource> <name>WSO2UM_DB</name> <description>The datasource used by user manager</description> <jndiConfig> <name>jdbc/WSO2UM_DB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:mysql://mysql-appm.wso2.com:3306/UM_DB?autoReconnect=true</url> <username>user</username> <password>password</password> <driverClassName>com.mysql.jdbc.Driver</driverClassName> <maxActive>50</maxActive> <maxWait>60000</maxWait> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource>If The Identity Server is used as IDP, you must configure the Publisher, Store and Social service providers in the Identity Server. See Configuring a Service Provider for more information on how this configuration is done. Also see Configuring SAML2 Web Single-Sign-On since SAML2 is used to configure SSO. Create the three service providers as shown in the following configuration table.
SP Name Issuer Assertion Consumer URL Enable Response Signing Enable Assertion Signing Enable Single Logout Enable Attribute Profile publisher publisher https://<IP of publisher>:9443/publisher/acs True True True True store store https://<IP of store>:9443/store/acs True True True True social social https://<IP of store>:9443/social/acs True True True True The following configurations are an example of how this is configured for the Publisher. The following steps must be done for the Store and Social service providers as well.
Sign in. Enter your username and password to log on to the Management Console.
- Navigate to the Main menu to access the Identity menu. Click Add under Service Providers.
- Fill in the Service Provider Name and provide a brief Description of the service provider.
- Click Register.
- In the following page, expand Inbound Authentication Configuration and SAML2 Web SSO Configuration and click Configure.
- In the page that appears, do the configurations listed in the table and click Register.
- Now do the same for the Store and Social service providers as well.