Publishing Logs to BAM

If you are using BAM to monitor the logs from the servers in your production environment, the first step is to ensure that all logs from the servers are published to BAM. To achieve this, the log4j appender should be configured in each server before they are started. That is, you must open the log4j.properties file, add 'LogEvent' to the root logger and configure the LogEvent credentials accordingly.

Note that your BAM server should be up and running before the other servers are started. To test if the logs are successfully sent to BAM, you can log into Cassandra explorer and check if there is a new column family created under EVENT_KS keyspace.

Go to <PRODUCT_HOME>/repository/conf/log4j.properties file and add LOGEVENT to log4j root logger as shown below.
```
log4j.rootLogger=INFO, CARBON_CONSOLE, CARBON_LOGFILE, CARBON_MEMORY, CARBON_SYS_LOG,LOGEVENT
```

To add data publishing URLs and credentials, modify the LOGEVENT appender’s LOGEVENT.url to the BAM Server thrift URL and also change the LOGEVENT.userName and LOGEVENT.password to the credentials from BAM as illustrated below.

log4j.appender.LOGEVENT=org.wso2.carbon.logging.service.appender.LogEventAppender
log4j.appender.LOGEVENT.url=tcp:<BAM server thrift url>
log4j.appender.LOGEVENT.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
log4j.appender.LOGEVENT.columnList=%T,%S,%H,%A,%d,%c,%p,%m,%I,%Stacktrace
log4j.appender.LOGEVENT.userName=admin
log4j.appender.LOGEVENT.password=admin
#log4j.appender.LOGEVENT.password=secretAlias:Log4j.Appender.LOGEVENT.Password
log4j.appender.LOGEVENT.truststorePath=/repository/resources/security/client-truststore.jks
log4j.appender.LOGEVENT.trustStorePassword=<trust-store-password>
#log4j.appender.LOGEVENT.trustStorePassword=secretAlias:Carbon.Security.TrustStore.Password
log4j.appender.LOGEVENT.processingLimit=1000
log4j.appender.LOGEVENT.maxTolerableConsecutiveFailure=20
log4j.appender.LOGEVENT.streamDef=streamName

Data Model for Log Event

When the logs are published to BAM, they are stored in a Cassandra cluster. The data stream, which carries the information that needs to be stored for a particular LogEvent, is always created per tenant, server and date. Therefore, when this data stream is initialised in BAM, a unique column family is also created per tenant, server and date, in a predefined keyspace of the Cassandra cluster. Also, note that in order to capture tenant information, server information and node information, the org.apache.log4j.PatternLayout is extended and wrapped with log4j LogEvent.

Given below is the information in the data stream of a LogEvent:

{'name':'log. tenantId. applicationName.date','version':'1.0.0', 'nickName':'Logs', 'description':'Logging Event',
'metaData':[{'name':'clientType','type':'STRING'} ], 
'payloadData':[
   {'name':'tenantID','type':'STRING'},
   {'name':'serverName','type':'STRING'},
   {'name':'appName','type':'STRING'},
   {'name':'logTime','type':'LONG'},
   {'name':'priority','type':'STRING'},
   {'name':'message','type':'STRING'},
   {'name':'logger','type':'STRING'},
   {'name':'ip','type':'STRING'},
   {'name':'instance','type':'STRING'},
   {'name':'stacktrace','type':'STRING'}
 ] }

Go to Log Analysis in BAM for details on how you can monitor these logs in BAM.