Q4M, short from Queue for MySQL, is a message queue that works as a pluggable storage engine of MySQL 5.1, designed to be robust, fast, flexible. The development started in late December of 2007, and although it is very primitive, operates quite swiftly.
Q4M logs all message transfers to disk. A power failure or OS crash would never loose a message.
Although Q4M syncs every operation to disk, it runs fast by using group commits, and checksums for data recovery. Even with HDD write cache disabled, it is able to transfer >1,000 messages per second on an ordinally Pentium 4 linux box.
Since Q4M works as a storage engine of MySQL, it is possible to JOIN queues against ordinally MySQL tables. Or you may use triggers for sending data into queue.
� MySQL 5.1.x
Installing the Source Distribution
For installing the source distribution, source code of MySQL is required.
download and build MySQL 5.1 (installation is not mandatory, but if you plan to install Q4M into a binary distribution of MySQL, make sure to use the same configuration options for building the MySQL source distribution. The configuration of MySQL can be found in bin/mysqlbug script.)
untar the Q4M distribution
run make install
Below are the important options when configuring Q4M.
set the directory of MySQL source code
set the library directory of your MySQL. It is in most cases either lib/mysql or lib subdirectory of MySQL
if your MySQL is built with --with-debug option, Q4M should be built with this option as well. Or your MySQL server will crash.
select disk flush method. The fcntl option only works on Mac OS X, which calls fcntl with F_FULLFSYNC option.
Installing the Binary Distribution
Follow the steps below to install the binary distribution.
untar the distribution
copy libqueue_engine.so to lib directory of MySQL 5.1
Testing the Installation
To test your installation, run the run_tests.pl. You can set your database location by using the DBI environment variable. DBI_USER and DBI_PASSWORD variables are also avialable. An output like below should appear.
$ DBI='dbi:mysql:database=test;host=kazdev;port=3307' ./run_tests.pl
Multireader benchmark result:
Number of messages: 6400
Number of readers: 32
Elapsed: 3.366 seconds
Throughput: 1901.245 mess./sec.
All tests successful.
Files=6, Tests=5370, 140 wallclock secs (115.80 cusr 5.92 csys = 121.72 CPU)
To evaluate Q4M, download either a binary or source distribution from the install page, and follow the installation instructions. We also have a small tutorial. You may use SQL to access Q4M queues, or there is a wrapper module available for perl (Queue::Q4M).
- This release fixes a file descriptor leak on DROP TABLE.
- It includes binaries for Linux i386, Linux x86_64, and Mac OS X 10.4 (x86) as well as a source code tarball.