Beaver iconBeaver 24

Python daemon that munches on logs and sends their contents to logstash
Beaver is a Python daemon that munches on logs and sends their contents to logstash.

Installation

Using PIP:

From Github:

pip install git+git://github.com/josegonzalez/beaver.git#egg=beaver

From PyPI:

pip install beaver==1

Usage

usage:

beaver [-h] [-r {worker,interactive}] [-m {bind,connect}] [-p PATH]
 [-f FILES [FILES ...]] [-t TRANSPORT]


optional arguments:

-h, --help show this help message and exit
-r {worker,interactive}, --run {worker,interactive}
 run worker or interactive mode
-m {bind,connect}, --mode {bind,connect}
 bind or connect mode
-p PATH, --path PATH path to log files
-f FILES [FILES ...], --files FILES [FILES ...]
 space-separated filelist to watch. Overrides --path
 argument
-t {amqp,redis,stdout}, --transport {amqp,redis,stdout}
 log transport method


Background

Beaver provides an lightweight method for shipping local log files to Logstash. It does this using either redis, stdin, zeromq as the transport. This means you'll need a redis, stdin, zeromq input somewhere down the road to get the events.

Events are sent in logstash's json_event format. Options can also be set as environment variables.

Examples

Example 1: Listen to all files in the default path of /var/log on standard out:

beaver

Example 2: Sending logs from /var/log files to a redis list:

REDIS_URL="redis://localhost:6379/0" beaver -t redis

Example 3: Use environment variables to send logs from /var/log files to a redis list:

REDIS_URL="redis://localhost:6379/0" BEAVER_PATH="/var/log" BEAVER_TRANSPORT=redis beaver

Example 4: Zeromq listening on port 5556 (all interfaces):

ZEROMQ_ADDRESS="tcp://*:5556" beaver -m bind

# logstash config:
input { zeromq {
 type => 'shipper-input'
 mode => 'client'
 topology => 'pushpull'
 address => 'tcp://shipperhost:5556'
 } }
output { stdout { debug => true } }


Example 5: Zeromq connecting to remote port 5556 on indexer:

ZEROMQ_ADDRESS="tcp://indexer:5556" beaver -m connect

# logstash config:
input { zeromq {
 type => 'shipper-input'
 mode => 'server'
 topology => 'pushpull'
 address => 'tcp://*:5556'
 }}
output { stdout { debug => true } }

last updated on:
August 19th, 2012, 19:17 GMT
price:
FREE!
developed by:
Jose Diaz-Gonzalez
license type:
MIT/X Consortium License 
category:
ROOT \ System \ Logging

FREE!

In a hurry? Add it to your Download Basket!

user rating

UNRATED
0.0/5
 

0/5

What's New in This Release:
  • Ensure new files are added to a transports configuration. Closes #96. Closes #101. [Jose Diaz-Gonzalez]
  • Allow float numbers for update_file_mapping_time. [Jose Diaz-Gonzalez]
  • Fix invalid casting of boolean values. [Jose Diaz-Gonzalez]
  • Perform all conversions in config.py. Closes #99. [Jose Diaz-Gonzalez]
read full changelog

Add your review!

SUBMIT