Collecting Data Statistics of Services

Service data analysis is crucial in understanding and optimizing their effective usage. For this reason, functionality to capture and collect statistics related to service data is integrated by default into service-hosting WSO2 products such as the WSO2 Application Server (AS), WSO2 Data Services Server (DSS), WSO2 Business Process Server (BPS), WSO2 Business Rules Server (BRS) etc. Once captured, these statistics are transferred to a data analysis product like WSO2 Business Activity Monitor (BAM) or WSO2 Complex Event Processor for processing and deriving business-critical information.

WSO2 BAM receives data events via its Thrift API. This uses a binary protocol and enables fast data transmission between the service-hosting product and BAM server. BAM's Service Data Agent can be configured early so that the statistics to be extracted from service data can be pre-defined. In the configuration panel, the user can specify the BAM server (Thrift server) related information and properties to be extracted from the service of the service-hosting product. The user should define the Event Stream related parameters uniquely identified by a name and a version.

When using data publisher API to publish data in a periodic manner to WSO2 BAM/CEP, the eviction time and eviction idle time for the connections should be higher than the periodic interval. This is required to re-use the created socket connections from the pool, avoiding closure of it and creation of new connections. The default eviction period is 5.5 seconds (5500 milliseconds). If you are publishing events in a periodic interval as more than 5.5s, you need to tune the <secureEvictionTimePeriod> parameter accordingly, in the <PRODUCT_HOME>/repository/conf/data-bridge/thrift-agent-config.xml file of the agent in the client side, by increasing this default value.

See the following topics for details:

Using on-premise deployment of WSO2 AS

If you are using an on-premise deployment of WSO2 AS, the service data publishing capability is provided by the following feature in the WSO2 feature repository:

Name : WSO2 Carbon - BAM Service Data Agent Aggregate
Identifier : org.wso2.carbon.bam.service.agent.feature.group

If it is not included in your product by default, you can add it by installing the above feature using the instructions given in section, Feature Management.

Follow the instructions below to setup service data publishing in a running Carbon instance.

Before starting AS, enable service data publishing as explained below:

Go to {AS_HOME}/repository/conf/etc/ and open bam.xml.
Look for <ServiceDataPublishing>disable</ServiceDataPublishing> and enable it.
Start the Application Server. For more information, see Starting the server.
Log on to the product's Management Console select "Configure -> Service Data Publishing."
The "Service Data Publisher Configuration" window opens. Fill it appropriately. For example,
Enable Service stats: Tick to get the activity statistics.
Enable Activity Service: This enables message tracing with BAM, CEP or other data analysis servers. Typically, a request is sent to an endpoint after being passed through some intermediate stages. Messaging tracing enables to track each messages and visualize the message body in each of these intermediate stages.
Note

Enable Activity Service feature will be implemented from the next major release of BAM.
Stream Definition Configuration:

Stream Name: Stream Name can be any string with alpha-numeric characters
Version: Stream Version distinguishes different streams with the same Stream Name. Default version should be 1.0.0.
Nick Name: This is a user preferred nick name to the Stream Name in alpha-numeric characters
Description: A description describing about the particular stream defined by Stream Name, Stream Version pair. Description should also consists of alpha-numeric characters.

BAM Credentials:

BAM URL: Enter the IP address of the BAM server. And the port should be the thrift port ( ex: tcp://127.0.0.1:7611 ). You can add a comma-separated list of server URLs to ensure that load is distributed among many servers and to minimize effects of a failure of one server node.

See the documentation on setting up multi receiver and load balancing data agent, for details on how BAM handles load distribution using multiple servers.
Username: Put the BAM Thrift log-in user name. Default value is admin.
Password: Put the BAM Thrift log-in password. Default value is admin.

Properties: A BAM deployment can receive events from different data agents. In order to uniquely identify events triggering from one agent, a property can be associated with it. This feature is particularly useful in clustered environments. Click Update once done.

The service-hosting server is now ready to collect and send service data to any data analysis server like the BAM and CEP. Specific instructions to receive this data and analyze it can be found in the respective documentation of the data analysis servers.

Using Cloud deployment of WSO2 AS

In a cloud deployment of WSO2 AS, it is not required to enable data publishing to BAM using the 'BAM Service Data Agent Aggregate' feature as explained above. Instead, you can simply add the BAM url to the <PRODUCT_HOME>/repository/conf/carbon.xml file as shown below.

<BamServerURL>https://bamhost:bamport/services/</BamServerURL>

Note that cloud deployment is enabled by the following setting in the <PRODUCT_HOME>/repository/conf/carbon.xml file:

<IsCloudDeployment>true</IsCloudDeployment>