Global Cluster Cache Invalidation

Introduction

In previous versions of WSO2 Governance Registry, cache invalidation was only possible within a single cluster domain. From G-Reg 5.0.0 onwards, cache invalidation is possible within different cluster domains. Since different products cannot be included in a single cluster domain, the caching data passes from one product cluster to another. For information on how to set up a cluster, see Setting up a Cluster.

The approach is to create a message broker and from each cluster, one node (coordinator) is registered to the broker. When one node gets an update, the broker is notified and the other clusters are notified by the broker to update their cache. First, you need to install the global cache invalidation feature. 

You can write a custom notification implementation to support different message broker protocols. The default implementation is the JMS protocol. 

Cache invalidation only works after enabling the configuration in the cache.properties file. Make sure that the feature is installed in the server via the feature management console. The feature can be enabled by adding the following settings to the cache.properties file located in the <GREG_HOME>/repository/conf folder:

enabled=false
class.CacheInvalidationClass=org.wso2.carbon.registry.caching.invalidator.connection.JMSNotification
initialContextFactory=org.apache.activemq.jndi.ActiveMQInitialContextFactory
providerUrl=tcp://localhost:61616
securityPrincipal=guest
securityCredentials=guest
cacheInvalidateTopic=GlobalCacheInvalidation

• enabled - This is to enable and disable global cache invalidation.

• class.CacheInvalidationClass - The invalidation class to be used for the invalidation notification. It consists of setting up connection, subscribing and publishing implementation.

• cacheInvalidationTopic - The topic name of the cache invalidation broker.

Configure with ActiveMQ

Follow the instructions below to setup and configure with ActiveMQ.

1.  Download and setup Apache ActiveMQ. Please note that ActiveMQ should be up and running before starting WSO2 G-Reg.
2. Copy the following client libraries from the <AMQ_HOME>/lib directory to the <GREG_HOME>/repository/components/lib directory. 
   

    For ActiveMQ 5.8.0 and above:

   • activemq-broker-5.8.0.jar

   • activemq-client-5.8.0.jar

   • geronimo-jms_1.1_spec-1.1.1.jar

   • geronimo-j2ee-management_1.1_spec-1.0.1.jar

   • hawtbuf-1.9.jar

   For earlier versions of ActiveMQ:

   • activemq-core-5.5.1.jar

   • geronimo-j2ee-management_1.0_spec-1.0.jar

   • geronimo-jms_1.1_spec-1.1.1.jar

3. Configure the cache.properties file in the <GREG_HOME>/repository/conf directory as follows,

   enabled=true
   class.CacheInvalidationClass=org.wso2.carbon.registry.caching.invalidator.connection.JMSNotification
   initialContextFactory=org.apache.activemq.jndi.ActiveMQInitialContextFactory
   providerUrl=tcp://localhost:61616
   securityPrincipal=guest
   securityCredentials=guest
   cacheInvalidateTopic=GlobalCacheInvalidation

Developer Guide

For developers who wants to invalidate the global cache among clusters,

1. Get the registered org.wso2.carbon.caching.impl.CacheInvalidator OSGi service and check the existence of the service.
2. Call the invalidateCache(int tenantId, String cacheManagerName, String cacheName, Serializable cacheKey) method.
3. Set the tenant ID, cache manager name, cache name and serializable key in the relevant places in the code that need to invalidate the cache.

An example of referencing the registered CacheInvalidator service can be found under the following component: https://github.com/wso2/carbon-kernel/tree/v4.4.5/core/org.wso2.carbon.registry.core

Service lookup:

See line 82: https://github.com/wso2/carbon-kernel/blob/v4.4.5/core/org.wso2.carbon.registry.core/src/main/java/org/wso2/carbon/registry/core/internal/RegistryCoreServiceComponent.java

Usage:

See line 252: https://github.com/wso2/carbon-kernel/blob/v4.4.5/core/org.wso2.carbon.registry.core/src/main/java/org/wso2/carbon/registry/core/caching/CachingHandler.java

For more information about building OSGi bundles, see this tutorial in the WSO2 Library: http://wso2.com/library/tutorials/develop-osgi-bundles-using-maven-bundle-plugin/