loggerfs is the virtual file system that allows you to store logs in a database. I just released the first version of it (checkout the News) and will now be focusing on supporting more logging formats. If you don't yet know what loggerfs is, then here's a simple explanation for when you need it:
- Are you running cron scripts periodically to parse log files and put them in a database?
- Do you need to store log files remotely because you have limited space?
- Would you like a more efficient solution than simply polling the log files and waiting for new data?
If you answer yes to any of the above questions, please take a look at loggerfs. It will allow you to create virtual files to which you can then direct syslog/ apache/ squid/ etc. to log to. For example, instead of having syslog store authentication information in /var/log/auth.log, you could create a virtual file in /var/loggerfs/auth.log and then tell syslog to log to that file. Now instead of storing the information in the auth.log file, the data is actually sent to a database server that you defined in the 'logs.xml' file. Every time new information is sent to the file, it is immediately sent to the database server, which means that:
- It doesn't poll for information, it listens, which makes it a lot more efficient.
- It automatically creates the tables in the database and makes sure that all the information can be stored.
- You'll be able to easily sort/ analyze the log files once they're in the database.
I encourage you to checkout the CVS on the sourceforge project page, and let me know if you have any questions/ problems/ suggestions.
Quick Instructions (as root):
There is only 1 file that you absolutely need to edit, and that is:
(or wherever you installed loggerfs)
It contains a listing of all the log files that you'll want to have in the virtual file system. By default there's an entry for an apache log file that could contain either the apache_combined or the apache_common formats.
To start the loggerfs virtual file system, create a mount point and mount the file system (as root):
If you change the configuration files after you've mounted the file system, you will need to call the 'loggerfs-reload' script. That tells loggerfs that the configuration files have changed and need to be reloaded.
Supported Log Formats:
- Apache combined
- Apache common
- Squid 2
What's New in This Release:
· There is now some good documentation on how to install and use loggerfs.
· This release features man pages, MySQL caching, various improvements, and .deb packages for those using an OS that supports the Debian package manager.