Logging

Logging is one of the most important aspects of a production-grade server. A properly configured logging system is vital in identifying errors, security threats and usage patterns. You can view system and application logs of a running WSO2 product instance in different ways as follows:

Through the Management Console.
Through the log files that are stored in <PRODUCT_HOME>/repository/logs folder. The folder contains current logs in a log file with a date stamp. Older logs are archived in wso2carbon.log file.
Through the command prompt/shell terminal that opens when running the wso2server.bat/wso2server.sh files to start the server.

WSO2 products use a log4j-based logging mechanism through Apache Commons Logging facade library. The log4j.properties file, which governs how logging is performed by the server is in <PRODUCT_HOME>/repository/conf folder. There are two ways to configuring log4j.

Manually editing the log4j.properties
Logging Configuration through the management console. Changes apply at run time.

We recommend the second approach because you do not have to restart the server for the configuration changes to apply. When you change the parameters using the Management Console, first, the server stores new values in the database and then changes the appropriate components in the logging framework, enabling logging properties to be updated immediately. All changes made to Log4j through the management console are persisted in the WSO2 Registry and are available after server restarts. Any changes to the logging configuration you make through the management console get priority over log4j.properties file settings. However, if you modify log4j.properties and restart the server, the earlier log4j configuration that persisted in the registry will be overwritten. There is also an option in the management console to restore the original Log4j configuration from the log4j.properties file.

WSO2 products store logs per service. You cannot drill down service-level logs further to filter operational or query logs. We also do not provide database level logs. However, if you get SQL errors (e.g., SQL violations in your queries), you can see those errors in ERROR logs. You can also use application logs to in case of an issue to figure out the cause.

Logging configuration

There are three main components in log4j as Loggers, Appenders, and Layouts. You can change these parameters both globally and individually, at run time.

Follow the steps below to configure logging properties using the management console.

Log in to the product's management console and select Configure > Logging.
The Logging Configuration page appears as follows:

It has the following configuration options:

Persist All Configuration Changes: Allows you to persist all modifications, which will be available even after the server restarts.
Global Log4J Configuration: This section allows you to assign a single log level and log pattern to all loggers.
- Log Level : Severity of the message. Reflects a minimum level for the logger. You can view the hierarchy of levels.
- Log Pattern : Defines the output format of the log file. This is the layout pattern that describes the log message format.
Restore Defaults button allows to overwrite the Registry with the logging configurations specified in log4j.properties file.
Configure Log4J appenders: This section allows you to configure appenders individually. Log4j allows logging requests to print to multiple destinations. These output destinations are called Appenders. You can attach several appenders to one logger.
- Name : The name of an appender. Following log appenders are configured by default:
  - CARBON_CONSOLE - Logs to the console when the server is running.
  - CARBON_LOGFILE - Writes the logs to AS_HOME/repository/logs/wso2carbon.log.
  - CARBON _MEMORY
  - CARBON_SYS_LOG - Allows separation of the software that generates messages from the system that stores them and the software that reports and analyzes them.
  - CARBON_TRACE_LOGFILE
- Log pattern - Defines the output format of the log file.
- Sys Log Host - The IP address of the system log server. The syslog server is a dedicated log server for many applications. It runs in a particular TCP port in a separate machine, which can be identified by an IP address.
- Facility - The log message type sent to the system log server.
- Threshold - Filters log entries based on their level. For example, threshold set to "WARN" will allow log entry to pass into appender if its level is "WARN," "ERROR" or "FATAL," other entries will be discarded. This is the minimum log level at which you can log a message.
The available categories of logs you can view are:
- TRACE - Designates fine-grained informational events than the DEBUG.
- DEBUG - Designates fine-grained informational events that are most useful to debug an application.
- INFO - Designates informational messages that highlight the progress of the application at coarse-grained level.
- WARN - Designates potentially harmful situations.
- ERROR - Designates error events that might still allow the application to continue running.
- FATAL - Designates very severe error events that will presumably lead the application to abort.
Configure Log4J Loggers: A Logger is an object used to log messages for a specific system or application component. Loggers are normally named, using a hierarchical dot-separated namespace and have a "child-parent" relationship. For example, the logger named "root.sv" is a parent of the logger named "root.sv.sf" and a child of "root."
When the server starts for the first time, all the loggers initially listed in the log4j.properties file appear on the logger name list. This section allows you to browse through all these loggers, define a log level and switch on/off additivity to any of them. After editing, the logging properties are read only from the database.
- Logger - The name of a logger.
- Parent Logger - The name of a parent logger.
- Level - Allows to select level (threshold) from the drop-down menu. After you specify the level for a certain logger, a log request for that logger will only be enabled if its level is equal or higher to the logger’s one. If a given logger is not assigned a level, then it inherits one from its closest ancestor with an assigned level. Refer to hierarchy of levels above.
- Additivity - Allows to inherit all the appenders of the parent Logger if set as True.
In this section, loggers can be filt ered by the first characters (use the Starts With button) or by a combination of characters (us e the Contains button).

Enabling data service DEBUG logs

To receive data service level debug logs, set the following list of loggers to DEBUG using either the UI or the log4j.properties file. Their logger level is set to INFO by default. Changing through the UI does not require a server restart.

org.wso2.carbon.dataservices.core.DBDeployer
org.wso2.carbon.dataservices.core.DBInOnlyMessageReceiver
org.wso2.carbon.dataservices.core.DBInOutMessageReceiver
org.wso2.carbon.dataservices.core.DBUtils
org.wso2.carbon.dataservices.core.admin.DataServiceAdmin
org.wso2.carbon.dataservices.core.admin.DataServiceFileUploader
org.wso2.carbon.dataservices.core.custom.datasource.AbstractCustomDataSourceReader
org.wso2.carbon.dataservices.core.custom.datasource.EchoDataSource
org.wso2.carbon.dataservices.core.description.config.SQLConfig
org.wso2.carbon.dataservices.core.description.query.SQLQuery
org.wso2.carbon.dataservices.core.engine.DataService
org.wso2.carbon.dataservices.core.engine.ParamValue
org.wso2.carbon.dataservices.core.internal.DSAxis2ConfigurationContextObserver
org.wso2.carbon.dataservices.core.internal.DataServicesDSComponent
org.wso2.carbon.dataservices.task.DSTaskAdmin
org.wso2.carbon.dataservices.task.internal.DSTaskServiceComponent

If you use the UI, in the Logging Configuration page, search for loggers that contain the name dataservice and turn their level to DEBUG. For example,