Transactional ESB
- Former user (Deleted)
- Rukshani Weerasinha (Unlicensed)
Introduction
Transaction plays a major role in today's business world. It adds a very important functionality to your system, which you will only see in a system crash. Having good transactional support in your system will definitely save your business in a system crash. The first section is an overview of transaction support in WSO2 ESB. Then a walk through of three complete samples, where the transaction mediator and the JMS transaction support in WSO2 ESB will be introduced. These samples will explain how you can use WSO2 ESB transaction support in a real world scenario in your production system.
What is a transaction?
A transaction is a set of operations executed as a single unit. It also can be defined as an agreement which is carried out between separate entities or objects. A transaction can be considered as indivisible or atomic when it has the characteristic of either being completed in its entirety or not at all. During the event of a failure for a transaction update, atomic transaction type guarantees transaction integrity such that any partial updates are rolled back automatically.
Transactions have many different forms, namely Financial transactions, Database transactions, etc. In the ESB point of view, we can think of two types of transactions:
- Distributed transaction
- JMS transaction
Distributed transaction
A distributed transaction is a transaction that updates data on two or more networked computer systems. It extends the benefits of transactions to applications that must update distributed data. Implementing robust distributed applications is difficult because these applications are subject to multiple failures, including failure of the client, the server, and the network connection between the client and server. For distributed transactions, each computer has a local transaction manager. When a transaction works at multiple computers, the transaction managers interact with other transaction managers via either a superior or subordinate relationship. These relationships are relevant only for a particular transaction.
JMS transactionsJMS local transaction
A local transaction represents a unit of work on a single connection to a data source managed by a resource manager. In JMS, we can use the JMS API to get a transacted session and call methods for a commit or rollback for the relevant transaction objects. This is managed internal to a resource manager. There is no external transaction manager involved in the coordination of such transactions.
Distributed JMS transaction
An external transaction manager manages the coordination of the transaction. Designing and using JMS distributed transactions is more complex than using local JMS transactions.
The transaction manager is the primary component of the distributed transaction support infrastructure, however the JDBC driver (the resource adapter) and the application server (in which you deploy your applications) should have the following two characteristics.
The driver should implement the JDBC 2.0 API(including the optional package interfaces XADataSource and XAConnection) or higher and the JTA interface XAResource.
The application server should provide a Datasource class that is implemented to interact with the distributed transaction infrastructure and a connection pooling model for performance improvements.
When JMS transactions are in place, local transactions are managed by the JMS provider itself whereas the distributed JMS transactions are managed by the XAResource enabled transaction manager in the J2EE application server.
Note
You need to check if your application server provides a XAConnectionFactory when you look for the ConnectionFactory.
So if you plan to run the following samples with different database and J2EE server environment, check for the above requirements (also this is a MUST that you need to verify if you plan to run a transactional system).
WSO2 ESB transaction support
WSO2 ESB has two kinds of transaction support.
- Transaction mediator
- JMS transport transaction
Transaction mediator
A new Synapse mediator (transaction mediator) has been added. It supports distributed transaction using Java transaction API (JTA)2. JTA allows applications to perform a distributed transaction that is transactions that access and update data on two or more networked computer resources (an example would be to have two databases or a database and a message queue such as JMS). This mediator can be used to perform a distributed transaction. The Synapse configuration has been extended to add explicit transaction markers. What this mean is you can use the Synapse configuration language to define the start, end etc., of your transaction. It is the responsibility of the user to define when to start, commit or rollback the transaction. For example, we can mark the start of a transaction at the start of a database commit and end the transaction at the end of the database commit and we can mark rollback transaction in case of a failure occurring.
Transaction mediator configuration
<transaction action="new | use-existing-or-new | fault-if-no-tx | commit | rollback | resume"/>
The transaction action attribute has the following meanings:
New - Creates a new JTA transaction or generates a fault if a transaction already exists.
Use-Existing-or-New - Creates a new JTA transaction or does nothing if a transaction exists.
Fault-if-no-tx - Generates a fault if no transaction exists or does nothing if a transaction exists.
Commit - Commits transaction or generates fault if no transaction already exists.
Rollback - Rolls back the pending operations for the session or generates fault if no transaction already exists.
Resume - Resumes transaction or generates a fault if no transaction exists.
End - Indicates the end of a transaction for the session.
Each of these actions can be used to mark the required action of the distributed transaction.
Transaction mediator scenario
We can use the following scenario to show how transaction mediator works. Assume we have a record in one database and we want to delete that record from the first database and add it to the second database (these two databases can be run on the same server or they can be in two remote servers). The database tables are defined such that the same entry cannot be added twice. So in the successful case the record will be deleted from the first table (of the first database) and will be added to the second table (of the second database) and in the failure case (in which case the record which is going to add is already in the second database) no record will be deleted from the first table and no record will be added into the second database.
System Requirements:
- Windows, Linux or Solaris operating systems;
- WO2 ESB;
- JBoss Application server;
- Derby Database;
- Apache ActiveMQ.
Since the transaction mediator is implemented using JTA you need to have a JTA provider. We will be using JBoss J2EE application server (which implements the transaction support through Arjuna TS) as the JTA provider, so we will have to deploy WSO2 ESB in JBoss Application server(AS). Apache Derby as the database server. The References section contain the links to download the required softwares.Â
Note
JBoss server and the Derby database server support the characteristics mentioned above.
Running the example
Unzip WSO2 ESB distribution to a place of your choice. And then remove the geronimo-jta_1.1_spec-1.1.0.wso2v1.jar ( This JAR file can be found in $ESB_HOME/repository/components/plugins). The reason is using of the implementation of javax.transaction.UserTransaction6 of JTA provider (here JBoss) and if we have both in class path, there is a classloading issue which causes the transaction mediator not to work.
We need to deploy WSO2 ESB on JBoss AS. The JBOSS installation path will be referred to as $JBOSS_HOME and the WSO2 ESB repo location as $CARBON_HOME.
Drop the derby client JARs (derby.jar, derbynet.jar and derbyclient.jar) into $CARBON_HOME/repository/components/lib folder and also into $JBOSS_HOME/server/default/lib (we'll use the default JBoss configuration) folder.
The full Synapse configuration is shown below (you can directly paste the following configuration into synapse configuration in the following file:
$ESB_HOME/repository/conf/synapse-config/synapse.xml
In the in-sequence, we will send a message to the service and in the out sequence, we will delete an entry from the first database and will update the second database with that entry. If we try to add an entry which is already there in the second database, the whole transaction will be roll-backed.
<definitions xmlns="http://ws.apache.org/ns/synapse">
<sequence name="myFaultHandler">
<log level="custom">
<property name="text" value="** Rollback Transaction**"/>
</log>
<transaction action="rollback"/>
<send/>
</sequence>
<sequence name="main" onError="myFaultHandler">
<in>
<send>
<endpoint>
<address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
</endpoint>
</send>
</in>
<out>
<transaction action="new"/>
<log level="custom">
<property name="text" value="** Reporting to the Database esbdb**"/>
</log>
<dbreport useTransaction="true" xmlns="http://ws.apache.org/ns/synapse">
<connection>
<pool>
<dsName>java:jdbc/XADerbyDS</dsName>
<icClass>org.jnp.interfaces.NamingContextFactory</icClass>
<url>localhost:1099</url>
<user>esb</user>
<password>esb</password>
</pool>
</connection>
<statement>
<sql>delete from company where name =?</sql>
<parameter expression="//m0:return/m1:symbol/child::text()"
xmlns:m0="http://services.samples" xmlns:m1="http://services.samples/xsd"
type="VARCHAR"/>
</statement>
</dbreport>
<log level="custom">
<property name="text" value="** Reporting to the Database esbdb1**"/>
</log>
<dbreport useTransaction="true" xmlns="http://ws.apache.org/ns/synapse">
<connection>
<pool>
<dsName>java:jdbc/XADerbyDS1</dsName>
<icClass>org.jnp.interfaces.NamingContextFactory</icClass>
<url>localhost:1099</url>
<user>esb</user>
<password>esb</password>
</pool>
</connection>
<statement>
<sql>INSERT into company values (?,'c4',?)</sql>
<parameter expression="//m0:return/m1:symbol/child::text()"
xmlns:m1="http://services.samples/xsd" xmlns:m0="http://services.samples"
type="VARCHAR"/>
<parameter expression="//m0:return/m1:last/child::text()"
xmlns:m1="http://services.samples/xsd" xmlns:m0="http://services.samples"
type="DOUBLE"/>
</statement>
</dbreport>
<transaction action="commit"/>
<send/>
</out>
</sequence>
</definitions>
You need to have two distributed Derby databases ("esbdb" and "esbdb1") to run the sample on how to set up the databases. The database table was created using the following SQL query. Note the table schema in which we cannot have the same entry twice.
Create table company(name varchar(10) primary key, id varchar(10), price double);
Add few records to the two database table
Database1:
INSERT into company values ('IBM','c1',0.0);
INSERT into company values ('SUN','c2',0.0);
Database1:
INSERT into company values ('SUN','c2',0.0);
INSERT into company values ('MSFT','c3',0.0);
Note the order of the record. You also need two data source declarations for JBoss AS for the two distributed database.
Datasource1:esb-derby-xa-ds.xml
<?xml version="1.0" encoding="UTF-8"?>
<datasources>
<xa-datasource>
<jndi-name>jdbc/XADerbyDS</jndi-name>
<isSameRM-override-value>false</isSameRM-override-value>
<xa-datasource-class>org.apache.derby.jdbc.ClientXADataSource</xa-datasource-class>
<xa-datasource-property name="portNumber">1527</xa-datasource-property>
<xa-datasource-property name="DatabaseName">esbdb</xa-datasource-property>
<xa-datasource-property name="User">esb</xa-datasource-property>
<xa-datasource-property name="Password">esb</xa-datasource-property>
<metadata>
<type-mapping>Derby</type-mapping>
</metadata>
</xa-datasource>
</datasources>
Datasource2:esb-derby1-xa-ds.xml
<?xml version="1.0" encoding="UTF-8"?>
<datasources>
<xa-datasource>
<jndi-name>jdbc/XADerbyDS1</jndi-name>
<isSameRM-override-value>false</isSameRM-override-value>
<xa-datasource-class>org.apache.derby.jdbc.ClientXADataSource</xa-datasource-class>
<xa-datasource-property name="portNumber">1527</xa-datasource-property>
<xa-datasource-property name="DatabaseName">esbdb1</xa-datasource-property>
<xa-datasource-property name="User">esb</xa-datasource-property>
<xa-datasource-property name="Password">esb</xa-datasource-property>
<metadata>
<type-mapping>Derby</type-mapping>
</metadata>
</xa-datasource>
</datasources>
Just drop the above two datasource declarations into (note that the two datasource file name should be *-xa-ds.xml ) $JBOSS_HOME/server/default/deploy folder. You also need to map the above jndi names, so drop the following jboss-web.xml configuration into $JBOSS_HOME/serer/default/deploy/esb.war/WEB-INF/.
<!DOCTYPE jboss-web PUBLIC
"-//JBoss//DTD Web Application 5.0//EN"
"http://www.jboss.org/j2ee/dtd/jboss-web_5_0.dtd">
<jboss-web>
<resource-ref>
<res-ref-name>jdbc/XADerbyDS</res-ref-name>
<jndi-name>java:/XADerbyDS</jndi-name>
</resource-ref>
<resource-ref>
<res-ref-name>jdbc/XADerbyDS1</res-ref-name>
<jndi-name>java:/XADerbyDS1</jndi-name>
</resource-ref>
</jboss-web>
Now go into $JBOSS_HOME/bin and start the server. Just run the run.sh (run.bat) script. Note that you need to set the CARBON_HOME enviornment varible pointing to the Carbon repository location. Now we can try the samples. We need to deploy the SimpleStockQuote service which comes with WSO2 ESB sampels.
Successful Scenario
First we will remove the IBM record from the first database and add it to the second database. Run the sample with following options.
ant stockquote -Daddurl=http://localhost:9000/services/SimpleStockQuoteService -Dtrpurl=http://localhost:8280/ -Dsymbol=IBM
Check both databases to see how the record is deleted from the first database and added into the second database.
Failure Scenario
Now we will try adding an entry which is already there in the second database. This time use Symbol SUN.
ant stockquote -Daddurl=http://localhost:9000/services/SimpleStockQuoteService -Dtrpurl=http://localhost:8280/ -Dsymbol=SUN
You will see how the fault sequence is executed and the whole transaction is rollback. Check both database again. You will notice that no record is deleted from the first database and no record is added into the second database.
JMS transport transaction
In addition to the transaction mediator, WSO2 ESB also has support for JMS transaction. WSO2 ESB ships with a JMS transport (which support JMS 1.1 and later) which supports both local and distributed JMS transactions. In a JMS client you can use local transactions to group message sends and receives. The JMS API Session interface provides commit and rollback methods that you can use in a JMS client.
JMS local transaction
WSO2 ESB fully support JMS local transactions.
Scenario
In the following scenario, a message is read from a JMS queue and it is processed by a back end service. While executing one sequence ESB receives a fault and this causes the JMS transaction to rollback. In the successful scenario, the transaction is committed and the request is sent to the backend service. In the following configuration, there is a class mediator which will set a property called "MESSAGE_COUNT" to the message context and it will demonstrate this behavior. Depending on the value of this variable the decision to either commit or rollback the transaction will be taken. You can download the binary ZIP of the mediator from here. Drop this mediator to $ESB_HOME/repository/components/lib.
The source of the mediator looks as shown below.
public class MessageCounterMediator extends AbstractMediator {
private static int MESSAGE_COUNT = 0;
public boolean mediate(MessageContext synCtx) {
MESSAGE_COUNT++;
synCtx.setProperty("MESSAGE_COUNT", MESSAGE_COUNT);
return true;
}
}
ESB configuration is given below.
<proxy name="StockQuoteProxy" transports="jms" startOnLoad="true">
<target>
<inSequence>
<class name="org.wso2.carbon.mediator.MessageCounterMediator"/>
<switch source="get-property('MESSAGE_COUNT')">
<case regex="1">
<property name="SET_ROLLBACK_ONLY" value="true" scope="axis2"/>
<log level="custom">
<property name="Transaction Action" value="Rollbacked"/>
</log>
</case>
<default>
<log level="custom">
<property name="Transaction Action" value="Committed"/>
</log>
<send>
<endpoint name="endpoint_urn_uuid_677F3EF4BC0AE1AF5B32399295906279-2025938318">
<address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
</endpoint>
</send>
</default>
</switch>
<property name="OUT_ONLY" value="true"/>
</inSequence>
</target>
<publishWSDL uri="file:repository/samples/resources/proxy/sample_proxy_1.wsdl"/>
<parameter name="transport.jms.ContentType">
<rules>
<jmsProperty>contentType</jmsProperty>
<default>application/xml</default>
</rules>
</parameter>
</proxy>
To start a local JMS transaction, define the following property in JMS transport Listner in axis2.xml.
<parameter name="transport.jms.SessionTransacted">true</parameter>
By default, the session is not transacted and if you want to use JMS local transaction, set the above parameter to true. Also note the following property in the failure case which will roll back the local transaction.
<property name="SET_ROLLBACK_ONLY" value="true" scope="axis2"/>
Running the example
You do not need to deploy the ESB on JBOSS to run this sample since this does not require any distributed transaction manager as in other cases.
1. Copy the JMS client JARs (activemq-core-5.2.0.jar, geronimo-j2ee-management_1.0_spec-1.0.jar, geronimo-jms_1.1_spec-1.1.1.jar) into $ESB_HOME/repository/components/lib
2. Start the activemq server.
3. Deploy and start the SimpleStockQuoteService service. You need to get the SimpleStockQuoteService service attach with this article and (SimpleStockQuoteService.aar) and place it in $ESB_HOME/samples/axis2Server/repository/services.
4. Start WSO2 ESB with above configuration
5. Run the JMS client with following command.
ant jmsclient -Djms_type=pox -Djms_dest=dynamicQueues/StockQuoteProxy -Djms_payload=MSFT
When you run the client, you can see two log lines saying that first time the traction was rolled back and in the second attempt the transaction was committed. And service will be served to the message received.
Distributed JMS transaction
WSO2 ESB has the support for distributed JMS transaction. In this case, you can use the transaction mediator to manage multiple distributed resources. An ideal candidate for this category would be handling a JMS queue and a data base server using a single transaction. A sample configuration will be very similar to the distributed transaction example configuration given in the section on distributed transactions.
Conclusion
A transaction is a set of operations executed as a single unit. It is also known as an agreement, which is carried out between separate entities or objects. It is indivisible or atomic, when it has the characteristic of either being completed in its entirety or not at all. WSO2 ESB provides support for transactional ESB which includes Transaction Mediator and JMS Transport Transaction. Transaction Mediator is based on Synapse, which supports distributed transaction support using the Java transaction API. JMS Transport Transaction is based on JMS 1.1 and later, which support both local and distributed JMS transactions.
Future work
Some efforts are in progress to add the support for using a stand alone JTA provider such as Atomikos.
For more information see Transactions.