Publishing API Runtime Statistics

This section explains how to set up WSO2 Business Activity Monitor (BAM) to collect and analyze runtime statistics from the WSO2 API Manager. The instructions here apply to BAM versions that are older than 2.4.1. To set up the same with BAM 2.4.1 or later, see APIM 1.7.0 docs or later.

To publish data from the API Manager to BAM, the Thrift protocol is used. Information processed in BAM is stored in a database from which the API Publisher retrieves information before displaying in the corresponding UI screens.

By default, org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataPublisher is configured to collect data events from WSO2 BAM. If you use a system other than WSO2 BAM to collect and analyze runtime statistics, you can write a new data publishing agent by extending APIMgtUsageDataPublisher and adding your logic to the XML. You can find the API templates inside <APIM_HOME>/repository/resources/api_templates. When writing a new data publishing agent, make sure the data publishing logic you implement has a minimal impact to API invocation.

<BAM_HOME> refers to the Business Activity Monitor installation directory.
The data source and database names used in this guide are just examples. They may vary depending on your configurations.

Follow the instructions below to set up WSO2 BAM and the API Manager.

Prerequisites

Ensure that you have the following in your environment,

JDK 1.6.* or 1.7

If you install JDK in Program Files in the Windows environment, avoid the space by using PROGRA~1 when specifying environment variables for JAVA_HOME and PATH. Else, the server throws an exception.
Cygwin ( http://www.cygwin.com ) : Required only if you use Windows. WSO2 BAM analytics framework depends on Apache Hadoop, which requires Cygwin in order to run on Windows. Install at least the basic net (OpenSSH,tcp_wrapper packages) and security related Cygwin packages. After Cygwin installation, update the PATH variable with C:/cygwin/bin and restart BAM.

Configuring WSO2 API Manager

Enable API tracking option by setting the <APIUsageTracking> element to true in <APIM_HOME>/repository/conf/api-manager.xml file. Also specify the data source used for getting BAM statistics in <DataSourceName> element. For example,
```
<APIUsageTracking>
 
    
    <Enabled>true</Enabled>   
    <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
    <ThriftPort>7612</ThriftPort> 
    <BAMServerURL>tcp://localhost:7612/</BAMServerURL>
    <BAMUsername>admin</BAMUsername>
    <BAMPassword>admin</BAMPassword>
    
    <DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>
 
</APIUsageTracking>
```
In the above configuration, the default Thrift port is increased by 1 because we apply an offset of 1 to default BAM port later in this guide.

<BAMServerURL> refers to the endpoint to which events will be published from the API Gateway. This endpoint is also known as the event receiver. You can define multiple event receiver groups, each with one or more receivers. A receiver group is defined within curly braces and receiver URLs are delimited by commas.
For example, <BAMServerURL>{tcp://localhost:7612/,tcp://localhost:7613/},{tcp://localhost:7712/,tcp://localhost:7713/}</BAMServerURL>. The example has two receiver groups defined with two receivers in each group. When a request passes through the API Gateway, an event will be published to one selected receiver in each group.

Specify the datasource definition in <APIM_HOME>/repository/conf/datasources/master-datasources.xml file as follows.

Replace <BAM_HOME> with the absolute path of BAM installation directory.
The WSO2AM_STATS_DB database is not available in <BAM_HOME>/repository/database directory at this point. It is created only after BAM starts up later in this guide.
The JNDI name must match the one given step 5 below, when configuring BAM.

  <datasource>
          <name>WSO2AM_STATS_DB</name>
          <description>The datasource used for getting statistics to API Manager</description>
          <jndiConfig>
             <name>jdbc/WSO2AM_STATS_DB</name>
          </jndiConfig>
          <definition type="RDBMS">
             <configuration>
                 <!-- JDBC URL to query the database -->
                 <url>jdbc:h2:<BAM_HOME>/repository/database/APIMGTSTATS_DB;AUTO_SERVER=TRUE</url>
                 <username>wso2carbon</username>
                 <password>wso2carbon</password>
                 <driverClassName>org.h2.Driver</driverClassName>
                 <maxActive>50</maxActive>
                 <maxWait>60000</maxWait>
                 <testOnBorrow>true</testOnBorrow>
                 <validationQuery>SELECT 1</validationQuery>
                 <validationInterval>30000</validationInterval>
            </configuration>
         </definition>
  </datasource>

Next, prepare BAM to collect and analyze statistics from API manager.

Configuring WSO2 BAM

Download WSO2 BAM 2.3.0 or later from location: http://wso2.com/products/business-activity-monitor.
Change port offset of BAM to 1 by editing the file <BAM_HOME>/repository/conf/carbon.xml file (search for the offset node).
```

 
<Offset>1</Offset>
```
This increments all ports used by the server by 1, which means the BAM server will run on port 9444. Port offset is used to increment the default port by a given value. It avoids possible port conflicts when multiple WSO2 products are run in same host.

It is not necessary to add an offset to the default port if API Manager and BAM are set up on separate servers.
Copy the file <APIM_HOME>/statistics/API_Manager_Analytics.tbox to directory, <BAM_HOME>/repository/deployment/server/bam-toolbox. If this folder is not in the BAM installation directory by default, create it.

If you use Oracle DB to store statistical data, copy API_Manager_Analytics_Oracle.tbox instead.

The API Manager Analytic Toolbox is a toolbox that is an installable archive, with a .tbox extension. It contains necessary artifacts that models a complete usecase, from collecting data, analyzing through defined Hive scripts to summarizing data through gadgets, Jaggery scripts and other dashboard components.
Configure a database to which BAM can write the analyzed information. This information is retrieved by the API Publisher before displaying on the corresponding statistical dashboards.
Specify the datasource definition in <BAM_HOME>/repository/conf/datasources/bam_datasources.xml file as follows.
- The JNDI config name given below must match the one given in step 2 above when configuring APIM.
- WSO2AM_STATS_DB is the datasource used to fetch analytical data from the database. This example uses an H2 database. If you want to use a different database, see Changing the statistics database.
Because you changed the default BAM port in step 2 above, change the Cassandra port given in JDBC connection URL in the following datasource configuration found in master-datasources.xml file. Since the port offset is 1, the Cassandra port must be 9161. For a list of default ports used by WSO2 products, see Default Ports of WSO2 Products.
```
<datasource>
           <name>WSO2BAM_CASSANDRA_DATASOURCE</name>
           <description>The datasource used for Cassandra data</description>
           <definition type="RDBMS">
               <configuration>
                   <url>jdbc:cassandra://localhost:9161/EVENT_KS</url>
                   <username>admin</username>
                   <password>admin</password>
               </configuration>
           </definition>
</datasource>
```
Note that localhost is used here as opposed to the machine IP. Cassandra is bound by default on localhost, unless you change the data-bridge/data-bridge-config.xml file. Also, if you are running BAM on a different server, the port will be different.

If you are using WSO2 BAM 2.4.0 or older, you need to change the port in WSO2BAM_UTIL_DATASOURCE also, as you did in the configuration above.

If you change the default BAM port (with a port offset), you must change the Cassandra port accordingly, before running the Hive scripts. If not, you get an unable to connect to server Cassandra exception. To overcome this, change the default Cassandra port with the offset value (default port+offset) in <BAM_HOME>/repository/conf/datasources/bam_datasources.xml file.
If you executed any Hive scripts prior to changing the default port, add the following line at the beginning of the Hive scripts and rerun:
drop table <hive_cassandra_table_name>;
Open <BAM_HOME>/repository/conf/etc/hector-config.xml file and change the port to localhost:9161. You must add the other nodes too when configuring a clustered setup.
```
Nodes>localhost:9161</Nodes>
```
The default port is incremented according to the BAM port offset. This is not necessary if you are using WSO2 BAM 2.5.0.
Restart BAM server by running <BAM_HOME>/bin/wso2server.[sh/bat].

If you want to host the BAM server on a different machine or change the running port, you must edit the <APIUsageTracking> node in <APIM_HOME>/repository/conf/api-manager.xml file as follows:

<!--API usage tracker configuration used by the BAM data publisher in API gateway.-->
    <APIUsageTracking>
        <!-- Enable/Disable the API usage tracker.-->
        <Enabled>true</Enabled>

        <!-- API Usage Data Publisher.-->
        <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>

        <!--Thrift port of the remote BAM server.-->
        <ThriftPort>7612</ThriftPort>

        <!-- Server URL of the remote BAM server used to collect statistics. Must be specified in protocol://hostname:port/ format.-->
        <BAMServerURL>tcp://localhost:7612</BAMServerURL>

        <!--Administrator username to login to the remote BAM server.-->
        <BAMUsername>admin</BAMUsername>

        <!--Administrator password to login to the remote BAM server.-->
        <BAMPassword>admin</BAMPassword>

        <!--JNDI name of the data source to be used for getting BAM statistics.This data source should be defined in the master    
            datasources.xml file in conf/datasources directory.-->
        <DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>
    </APIUsageTracking>

Changing the statistics database

To use a different database than the default H2 for statistical publishing, you must change the properties of the datasource element, and additionally delete some metadata tables created by previous executions of the Hive script, if there are any.

To delete the metadata tables,

Log in to BAM management console and select Add in Analytics menu.
Go to the Script Editor in the window that opens.

Execute the following script.

drop table APIRequestData;
drop table APIRequestSummaryData;
drop table APIVersionUsageSummaryData;
drop table APIResourcePathUsageSummaryData;
drop table APIResponseData;
drop table APIResponseSummaryData;
drop table APIRequestData;
drop table APIRequestSummaryData;
drop table APIVersionUsageSummaryData;
drop table APIResponseData;
drop table APIResponseSummaryData;

After configuring WSO2 BAM to render and produce statistics of APIs hosted and managed in the API Manager, you can view them through various statistical dashboards in the API Publisher, depending on your permission levels. For information, refer to section Viewing API Statistics.