Managing Apache Logs
Apache provides several tools for managing your logs. Other Apache-specific third-party tools are available and are mentioned here. Because Apache can log requests in the CLF, most generic log processing tools can be used with Apache as well.
Earlier in the chapter, you learned how to use the HostNameLookups directive to enable or disable hostname resolution at the time the request is made. If HostNameLookups is set to off (the default), the log file will contain only IP addresses. Later, you can use the command-line logresolve utility on Unix or logresolve.exe on Windows to process the log file and convert the IP addresses to hostnames.
The logresolve utility reads log entries from standard input and outputs the result to its standard output. To read to and from a file, you can use redirection, on both Unix and Windows:
logresolve < access.log > resolved.log
Log files should not be removed while Apache is running, because the server is writing directly to them. A solution would be to use an intermediate program to log the requests. The program will, in turn, take care of rotating the logs.
Apache provides the rotatelogs program on Unix and rotatelogs.exe on Windows for this purpose. It accepts three arguments: a filename, a rotate interval in seconds, and an optional offset in minutes against UTC (Coordinated Universal Time).
TransferLog "|bin/rotatelogs /var/logs/apachelog 86400"
will create a new log file and move the current log to the /var/logs directory daily. (At the end of the command, 86400 is the number of seconds in one day.)
By the Way
If the path to the program includes spaces, you might need to escape them by prefixing them with a \ (backslash)for example, My\ Documents. This is especially common in the Windows platform.
If the name of the file includes % prefixed options, the name will be treated as input to the strftime function that converts the % options to time values. The manual page for the rotatelogs utility contains a complete listing of options, but here's an example:
TransferLog "|bin/rotatelogs /var/logs/apachelog%m_%d_%y 86400"
This command will add the current month, day, and year to the log filename.
Merging and Splitting Logs
When you have a cluster of Web servers serving similar content, perhaps behind a load balancer, you often need to merge the logs from all the servers in a unique log stream before passing it to analysis tools.
Similarly, if a single Apache server instance handles several virtual hosts, sometimes it is useful to split a single log file into different files, one per each virtual host.
Logtools is a collection of log-manipulation tools that can be found at http://www.coker.com.au/logtools/. Additionally, Apache includes the split-file Perl script for splitting logs. You can find it in the support subdirectory of the Apache distribution.
Many commercial and freely available applications are available for log analysis and reporting. Two popular open source applications are Webalizer (http://www.mrunix.net/webalizer/) and awstats (http://awstats.sourceforge.net/).
Wusage is a nice, inexpensive commercial alternative and can be found at http://www.boutell.com/wusage/.
Monitoring Error Logs
tail -f logname