Monitoring Access Logs

HTTP access logs help you monitor information such as the persons who access the product, how many hits are received, what the errors are, etc. This information is useful for troubleshooting errors.

All WSO2 products can enable access logs for the HTTP servlet transport. This servlet transport works on 9443/9763 ports, and it receives admin/operation requests. Therefore, access logs for the servlet transport is useful for analysing operational/admin-level access details. Additionally, in WSO2 API Manager (WSO2 API-M), WSO2 Enterprise Service Bus (WSO2 ESB), and WSO2 Enterprise Integrator (WSO2 EI)  you can generate access logs for the PassThrough and NIO transport as well. The PassThrough and NIO transport works on 8280/8243 ports and is used for API/Service invocations. By default, the access logs from both the Servlet transport and the PassThrough transport are written to a common access log file located in the <API-M_HOME>/repository/logs directory.

To configure the default behaviour of HTTP access logs in WSO2 API-M, see the following topics.

Configuring access logs for the HTTP Servlet transport

As the runtime of WSO2 products is based on Apache Tomcat, you can use the Access_Log_Valve variable in Tomcat as explained below to configure access logs for the HTTP Servlet transport:

1. Open the <API-M_HOME>/repository/conf/tomcat/catalina-server.xml file, which is the server descriptor file for the embedded Tomcat integration.

2. Customize the attributes for the Access_Log_Valve variable shown below. 

   <Valve className="org.apache.catalina.valves.AccessLogValve" 
                  directory="${carbon.home}/repository/logs"
                  prefix="http_access_" 
                  suffix=".log"
                  pattern="combined"/>

   The attributes that are used by default are explained below. See the descriptions of the Tomcat-supported Access Log Valve attributes and customize the required values.

   directory	The path to the directory that will store the access log file. By default, this location is set to ${carbon.home}/repository/logs in all WSO2 products.
   prefix	The prefix added to the log file's name. By default, this is set to: "http_access_"
   suffix	The suffix added to the log file's name. By default, this is set to: ".log"
   pattern	The attribute defines the format for the log pattern, which consists of the information fields from the requests and responses that should be logged. The pattern format is created using the following attributes: A standard value to represent a particular string. For example, "%h" represents the remote host name in the request. See the list of string replacement values supported by the Tomcat valve. %{xxx}i is used to represent the header in the incoming request (xxx=header value). %{xxx}o is used to represents the header in the outgoing request (xxx=header value). While you can use the above attributes to define a custom pattern, the standard patterns shown below can be used. common (Apache common log pattern): pattern=%h %l %u %t "%r" %s %b combined (Apache combined log pattern): pattern=%h %l %u %t "%r" %s %b "%{Referer}i" "%{User-Agent}i" Note that, by default, the "combined" pattern is enabled in WSO2 API-M.

3. Restart the server.
   According to the default configurations, a log file named http_access_.{DATE}.log  is created inside the <API-M_HOME>/repository/logs  directory. The log is rotated on a daily basis.

Configuring access logs for the PassThrough or NIO transports (Service/API invocation)

By default, access logs related to service/API invocation are disabled for performance reasons in the above products. You should enable these access log only for troubleshooting errors. Follow the steps given below to enable access logs for the PassThrough or NIO transport:

1. Add the following entry in the <API-M_HOME>/repository/conf/log4j.properties configuration file.

   log4j.logger.org.apache.synapse.transport.http.access=INFO

2. Create a file named <API-M_HOME>/repository/conf/access-log.properties with the following configuration and customize it as required.
   You can customize the format of your PassThrough or NIO access logs based on the configurations in your  access-log.properties file.

   All the supported options are in the following file. Therefore, make sure to uncomment the required options to enable them as required.

   # Default access log pattern
   #access_log_pattern=%{X-Forwarded-For}i %h %l %u %t \”%r\” %s %b \”%{Referer}i\” \”%{User-Agent}i\”
   # combinded log pattern
   #access_log_pattern=%h %l %u %t \”%r\” %s %b \”%{Referer}i\” \”%{User-Agent}i\”
   access_log_pattern=time=%t remoteHostname=%h localPort=%p localIP=%A requestMethod=%m requestURL=%U remoteIP=%a requestProtocol=%H HTTPStatusCode=%s queryString=%q
   # common log pattern
   #access_log_pattern=%h %l %u %t \”%r\” %s %b
   # file prefix
   access_log_prefix=http_gw_
   # file suffix
   access_log_suffix=.log
   # file date format
   access_log_file_date_format=yyyy-MM-dd
   #access_log_directory=”/logs”

   access_log_directory	Add this property ONLY if you want to change the default location of the log file. By default, the product is configured to store access logs in the <API-M_HOME>/repository/logs directory.
   access_log_prefix	The prefix added to the name of the log file.
   access_log_suffix	The suffix added to the name of the log file.
   access_log_file_date_format	The date format used in access logs.
   access_log_pattern	The attribute defines the format for the log pattern, which consists of the information fields from the requests and responses that should be logged. The pattern format is created using the following attributes: A standard value to represent a particular string. For example, "%h" represents the remote host name in the request. Note that all the string replacement values supported by Tomcat are NOT supported for the passthrough transport's access logs. The list of supported values are given below. %{xxx}i is used to represent the header in the incoming request (xxx=header value). %{xxx}o is used to represents the header in the outgoing request (xxx=header value). While you can use the above attributes to define a custom pattern, the standard patterns shown below can be used. common (Apache common log pattern): access_log_pattern=%h %l %u %t "%r" %s %b combined (Apache combined log pattern): access_log_pattern=%h %l %u %t "%r" %s %b "%{Referer}i" "%{User-Agent}i" By default, a modified version of the Apache combined log format is enabled in WSO2 API-M as shown below. Note that the "X-Forwarded-For" header is appended to the beginning of the usually combined log format. This correctly identifies the original node that sent the request (in situations where requests go through a proxy such as a load balancer). The "X-Forwarded-For" header must be present in the incoming request for this to be logged. access_log_pattern=%{X-Forwarded-For}i %h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-Agent}i\"

3. Restart the server. 

4. Invoke an API in WSO2 APIM.
   For testing purposes, use the artifacts in the quick start guide. 
   The access log file for the service/API will be created in the <API-M_HOME>/repository/logs directory with the following format.  

   <access_log_prefix><date>.log

   
   For example, http_gw_2018-12-19.log file. 

   Note that there will be delay in printing the logs to the access log file.

Supported log pattern formats for the PassThrough or NIO transports

          %a

          %A

          %b

          %B

          %c

          %C

          %e

          %E

          %h

          %l

          %L

          %k

          %m

          %n

          %r

          %s

          %S

          %t

          %T

          %u

          %U

          %v

          %V

          %x

          %Z