log_analysis

0.46 GPL (GNU General Public License)    
  UNRATED

  560 downloads

A log file analysis engine that extracts relevant data for any of the recognised log

description

download

specs

changelog

log_analysis is a log file analysis engine that extracts relevant data for any of the recognised log messages and produces a summary that is much easier to read.

log_analysis is my solution to these problems. It goes through several different kinds of logs (currently syslog, wtmp, and sulog), over some period (defaults to yesterday). It strips out the date and PID, and throws away certain entries. Then it tries each entry against a list of perl regular expressions. Each perl regular expression is associated with a category name and a rule for extracting data. When there's a match, the data-extracting rule is applied, and filed under the category.

If a log entry is unknown, it's filed under a special category for unknowns. Identical entries for a given category are sorted and counted. There's an option to mail the output, so you can just run it out of cron. You can also save a local copy of the output. If you prefer to PGP-mail yourself the output, you can do this, too. The whole thing is designed to be easily extended, complete with an easy plug-in interface. The default mode is for reporting, but it also "real" and "gui" modes for continuous monitoring, complete with action support. Oh, and you can edit patterns in a GUI that helps write regular expressions quickly and easily.

Security

The program needs to run with permissions to read your log files in order to be useful, which usually means root. It does not default to SUID root, and I recommend not making it SUID, so just run it as root (ie. manually or out of cron). I've tried to avoid temp files everywhere that I can, and in the one case where I do use a temp file, I make sure to use the POSIX tmpnam function instead of trying to make up my own temp file algorithm. The default umask is 077. If you use action commands, there is nothing to stop you from using parts of the log message in insecure ways, so for goodness' sake, be careful.

Local extensions

log_analysis already has lots of rules, but chances are that you have log entries that aren't already covered. So, log_analysis can easily be extended via a local config file, as documented in the log_analysis manpage. There's even an easy way to do modular plug-ins.
read more   
Last updated on April 18th, 2012

#log analyzer #extracts data #analysis logs #log_analysis #log #analyzer #extracts

log_analysis

0 User reviews so far.

SUBMIT