Configuring Mediation Data Publishing

You can configure mediation data publishing in WSO2 ESB, so that you can send ESB mediation related statistics to WSO2 Business Activity Monitor (WSO2 BAM) or to WSO2 Data Analytics Server (WSO2 DAS) for advanced analysis. The BAM/DAS server receives data events via its thrift API. This is a binary protocol that enables fast data transmission between the ESB and BAM/DAS.

To view mediation statistics, you must first enable the statistics collector in the ESB. To do this,

Open <ESB_HOME>/repository/conf/carbon.xml in a text/XML editor.
Search for the StatisticsReporterDisabled property, change its value to false, and then save and close the file.

Now that you have enabled the statistics collector in the ESB, the following section describes the process of configuring mediation data publishing and also walks you through an example scenario on how mediation data publishing can be used to log mediation statistics into a Cassandra database.

Configuring mediation data publishing

Note

If you need to send ESB mediation statistics to WSO2 Data Analytics Server (WSO2 DAS), you should configure mediation data publishing by specifying DAS credentials instead of BAM credentials, and you also have to create required event streams and event receivers in WSO2 DAS manually.

For information on creating event streams in WSO2 DAS, see Understanding Event Streams and Event Tables.

For information on configuring event receivers in WSO2 DAS, see Configuring Event Receivers.

Log in to the ESB management console, click the Configure tab and then click Mediation Data Publishing. The Mediation Data Publisher Configuration screen opens.
Specify values for all the fields. Following are descriptions of the fields:

Enable Mediation Stats: Select this to enable the mediation statistics agent.
Stream Definition Configuration:
- Stream Name : Any string with alpha-numeric characters. The default name is bam_mediation_stats_data_publisher .
- Version : The stream version. This is what distinguishes different streams that have the same stream name. The default version is 1.0.0.
- Nick Name : Any string with alpha-numeric characters. This is a user-preferred nick name for the stream name, which is useful when the stream name is lengthy and you need to keep a shorter alias for convenience.
- Description : A description about the stream identified by the stream name and stream version pair. The description should also consist of alpha-numeric characters.
BAM Credentials:
- BAM URL : The IP address of the BAM server. The port must be the thrift port (e.g., tcp://127.0.0.1:7611). Add a comma-separated list of server URLs here to ensure that the load is distributed among many servers. It minimizes the effect of a failure of one node.
- Username: The BAM Thrift login user name. The default value is admin.
- Password: The BAM Thrift login password. The default value is admin.
Properties: BAM receives events from different data agents. You can associate a property to uniquely identify the events triggering from one agent. Adding properties is particularly useful in a clustered environment.

Now that you have configured mediation data publishing, let's have a look at how we can test the functionality of mediation data publishing.

To test the functionality of mediation data publishing, you need to create an endpoint service as the WSDL proxy service, and define a sample WSDL proxy service. The following sections describe how you can set up the back-end server for the ESB endpoint, how you can create a sample WSDL proxy service and finally how you can send a message from the ESB in order to test whether it is correctly stored in the Cassandra database.

Setting up the back-end server for the ESB endpoint

Follow the steps below to create an endpoint service as the WSDL proxy service:

In a command window, navigate to the <ESB_HOME>/samples/axis2Server/src/SimpleStockQuoteService directory, and then run ant to build and deploy this service to the sample Axis2 server.
Navigate to the <ESB_HOME>/samples/axis2Server directory and start the server using one of the following commands:
- Linux//Unix: ./axis2server.sh
- Windows: axis2server.bat
Go to http://localhost:9000/services/SimpleStockQuoteService?wsdl and verify that the service is running.

Adding a proxy service to WSO2 ESB

Follow the steps below to create a sample WSDL proxy service:

Click the Main tab on the Management Console. Go to Manage -> Services -> Add and then click Proxy Service. The Create Proxy Service from Template screen opens.
On the Create Proxy Service from Template screen, click WSDL Based Proxy to create a new WSDL-based proxy.
The WSDL Based Proxy screen opens. Specify values for the fields and click Create. The proxy service will be created and should appear in the Services list on the Deployed Services screen.
The fields are described below. The fields marked with * are mandatory.
- Proxy Service Name: The name of the proxy service. For example, Simple_Stock_Quote_Service_Proxy.
- WSDL URI: The WSDL URI that you created above. i.e., http://localhost:9000/services/SimpleStockQuoteService?wsdl. To ensure that the URI is valid, click Test URI.
- WSDL Service: The name of the WSDL service. In this case it is SimpleStockQuoteService.
- WSDL Port: The port of the WSDL service. In this case it is SimpleStockQuoteServiceHttpSoap11Endpoint.
On the Deployed Services page, select the proxy service that you created (i.e., Simple_Stock_Quote_Service_Proxy) to access its service dashboard.
In the proxy service's dashboard that opens, click Enable Statistics in the Specific Configuration section.

Sending messages from the ESB

Follow the steps below to send custom messages through the previously-created WSDL Proxy Service so that you can test whether the statistics are correctly stored in the Cassandra database.

Note

This is only an example, but the steps to use mediation data publishing in a production environment are similar.

Navigate to the <ESB_HOME>/samples/axis2Client directory and run the following command:
```
ant stockquote -Daddurl=https://localhost:8244/services/Simple_Stock_Quote_Service_Proxy -Dmode=fullquote -Dsymbol=testString
```
Here, the Daddurl value must be given from the HTTPS endpoint located in the Simple_Stock_Quote_Service_Proxy service's dashboard. To access the proxy service dashboard, click the Main tab on the Management Console. Go to Manage -> Services -> List and then click the Simple_Stock_Quote_Service_Proxy service on the Deployed Services screen. Instead of the testString for symbol, you can use any preferred string.
Once done, the BAM Mediation Stat Data Agent in the Simple_Stock_Quote_Service_Proxy service sends statistics from the ESB to the column family with the given stream name in key-space EVENT_KS in the Cassandra database. Use the Cassandra Explorer in BAM to see the data in the Cassandra database.

Descriptions of the above payload data values, which will be automatically generated through the ESB are as follows.

Event Property Name	Description
StreamId	The ID of the stream.
Timestamp	The timestamp of the event at the time it is published.
payload_avg_processing_time	The average time to process the payload by the ESB.
payload_count	The number of messages received by the ESB.
payload_direction	Whether the payload is an incoming/outgoing message from/to ESB. Applicable values are either `in` or `out`.
payload_fault_count	The number of occurrences that the message has encountered any fault when serving the request. If all requests are successfully served, this count will be 0.
payload_max_processing_time	The maximum time required to process the payload by the ESB.
payload_min_processing_time	The minimum time required to process the payload by the ESB.
payload_resource_id	The ID of the ESB resource type.
payload_stats_type	The ESB resource type from which the payload is published. Applicable values are `Proxy`, `Sequence`, `Endpoint` and `Any`.
payload_timestamp	The time that the request came into WSO2 service hosting server (WSO2 AS). This is not the time that the event was received by the BAM server.

Analyzing the mediation statistics

BAM mediation statistics metrics are similar to the metrics used by the general ESB mediation statistics component. Mediation statistics are widely used monitoring features where functional components of the ESB, sequences, proxy services and endpoints directly get involved in processing messages. The BAM mediation statistics feature enables a server/tenant administrator to collect runtime statistics on these three types of functional components and to view them through an interactive BAM dashboard. This feature provides a simple but effective way to determine the runtime performance of the ESB. The collected statistical data can be used to analyze network traffic patterns and draw rough estimates on how the service bus will be used in the future. One of the most important attribute of the mediation statistics feature is that it can be configured in a fine-grained manner. In other words, it allows a user to collect data on a specified set of sequences, proxy services and endpoints.

All statistics collected by the mediation component is aggregated over a time duration and is published to BAM as an event. That is, mediation statistics events are not sent for every request. Rather, the server aggregates request measures over a period of time and sends an event to BAM. For more information on the measures of mediation statistics, see the following WSO2 library article http://wso2.com/library/articles/2010/06/wso2-esb-mediation-statistics-can-numbers-tell-about-your-soa.