In order to effectively manage a web server, it is necessary to get feedback about the activity and performance of the server as well as any problems that may be occurring. The HTTPD Logs can provide everything that happens on your server, from the initial request, through the URL mapping process, to the final resolution of the connection, including any errors that may have occurred in the process.
Every time your server receives a request, it makes an entry to one or more log files. These log files are useful for a variety of purposes, from statistical analysis of your visitors to forensic analysis of an attack on your server. The HTTPD Logs sample is intended to show the capability of WSO2 BAM which can analyze the raw httpd logs and produce useful result. This sample demonstrate how you can use your logs to analyse the web traffic come to your server from different regions. It calculate the region from the ip address in logs and visualize it in gadgets.
Sample guide
1. Start the WSO2 BAM Server.
...
| No Format |
|---|
1.202.218.8 - - [20/Jun/2012:19:05:12 +0200] "GET /robots.txt HTTP/1.0" 404 492 "-" "\"Mozilla/5.0" 208.115.113.91 - - [20/Jun/2012:19:20:16 +0200] "GET /logs/?C=M;O=D HTTP/1.1" 200 1278 "-" "Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)" 123.125.71.20 - - [20/Jun/2012:19:30:40 +0200] "GET / HTTP/1.1" 200 912 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 220.181.108.101 - - [20/Jun/2012:19:31:01 +0200] "GET / HTTP/1.1" 200 912 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 123.125.68.79 - - [20/Jun/2012:19:53:24 +0200] "GET / HTTP/1.1" 200 625 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 178.154.210.252 - - [20/Jun/2012:19:54:10 +0200] "GET /?C=S;O=A HTTP/1.1" 200 663 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 74.125.126.102 - - [20/Jun/2012:20:15:28 +0200] "GET / HTTP/1.1" 200 606 "http://www.google.com/url?sa=t&rct=j&q=error&source=web&cd=1&ved=0CFAQFjAG&url=http%3A%2F%2Fwww.isrolab.com%2F&ei=GxPiT5PsL4e04AHPtgE&usg=AFQjCNHnmjmdkUV584ORIpOXz7zAPX0UHQ" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" 74.125.126.103 - - [20/Jun/2012:20:15:29 +0200] "GET /icons/blank.gif HTTP/1.1" 200 383 "http://www.isrolab.com/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" 74.125.126.93 - - [20/Jun/2012:20:15:29 +0200] "GET /icons/folder.gif HTTP/1.1" 200 460 "http://www.isrolab.com/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" |
Published event format (Stream definition)
| Code Block |
|---|
{
'name':'org.wso2.sample.httpd.logs',
'version':'1.0.0',
'nickName': 'Httpd_Log_Stream',
'description': 'Sample of Httpd logs analysis',
'metaData':[
{'name':'clientType',
'type':'STRING'
}],
'payloadData':[
{'name':'log',
'type':'STRING'
}]
} |
...