metazip 0.8

metazip project is an Apache 2.0 module that allows you to serve dynamically-constructed, uncompressed zip files. Thus, your users can choose to download individual files or entire collections, but you don't have to reserve disk space for both.

metazip works by comparing a real, uncompressed archive to the files it contains. It then constructs a special .mz file that contains the "difference" between the files and the archive. This difference contains the metadata that the archive tool uses to extract the files. metazip does not interpret this metadata; it simply records it in the .mz file. After the .mz file has been created, the original archive file can be deleted. Simply place the newly-created .mz file in the same directory as the files it references. The mz command-line utility takes care of most of the details.

When the .mz file is requested, metazip "glues together" the files and the contents of the .mz file to reconstruct the archive file. Because metazip hooks into Apache's filter system, Apache handles all of the heavy lifting. metazip merely creates some high-level data structures.

Therefore, metazip's overhead is practically negligible. In addition, Apache's normal functions for resuming aborted downloads, calculating the content length, setting headers, etc. work identically for static files and for metazip's dynamically-generated output.

[Note: The only "catch" is that metazip must be prepared to handle its input in arbitrary-sized chunks (called "bucket brigades" in Apache-speak), even though in practice the input will generally be in one large buffer (because its input is the small .mz file). This is reflected in metazip's code structure (in metazip.c), but is completely transparent to you.]

Requirements:

· metazip depends on several GNU extensions, so GNU bash, gcc, and make are required. Of course, it is an Apache 2.0 module, so the Apache development environment is required. On Red Hat systems, the required package is "httpd-devel" (which includes "apr-devel" and "apr-util-devel"). On Debian systems, it is "apache2-threaded-dev" (which includes "libapr0-dev").
· Optionally, you can build metazip using GNU libtool. Just set the LIBTOOL environment variable when building.
· Optionally, you can build and debug metazip in the Eclipse development environment, using the C/C++ Development Toolkit (CDT). Project files (.project and .cdtproject) are included.

Building metazip

Building metazip should be as simple as typing "make all". This checks prerequisites, compiles all source files, links the executables, and builds the mod_metazip.so shared library.

Optionally, you can test metazip prior to installing it. Just run "make test". This invokes Apache in debug mode, and you can point your browser to http://localhost:8000/ to test metazip interactively.

If all goes well, you can install metazip using "make install". However, this may require root access, depending on where Apache is installed. Type "make -n install" to see what commands are run during installation.

Apache Configuration

metazip is easy to configure. Just add a few lines to your httpd.conf or .htaccess file:

# Load the metazip module.
LoadModule metazip_module modules/mod_metazip.so

# Associate .mz files with metazip.
AddHandler metazip .mz

When a user requests a .mz file, metazip sets the Content-Disposition header with the archive filename (without the .mz extension). Therefore, your users will always be prompted with the "correct" filename.

Optionally, you can use mod_rewrite to rewrite "virtual" requests for non-existent .zip and .tar archives to the corresponding metazip file. This allows you to integrate metazip without having to change existing links to archives. In this case, the Content-Disposition header is not necessary, since the user is requesting a .zip or .tar file, not a .mz file. When the user requests an archive that doesn't exist, mod_rewrite rewrites internally to .zip.mz or .tar.mz; however, this is completely invisible to the end user. If the user requests an archive that does exist, then it is served normally, like any static file.

To use mod_rewrite with metazip, just add the following to the .htaccess file in your document root (or within a section in httpd.conf):

# If the requested archive doesn't exist,
# try the corresponding .mz file.
< IfModule mod_rewrite.c >
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME}.mz -f
RewriteRule .zip|.tar$ %{REQUEST_URI}.mz
< /IfModule >

Note: You may need to dynamically load mod_rewrite, using the LoadModule directive in httpd.conf:

LoadModule rewrite_module modules/mod_rewrite.so

Product's homepage

Here are some key features of "metazip":

· Extremely efficient, by hooking into Apache's filter system.
· Works equally well for zip, tar, and other archive files.
· Supports resumption of aborted downloads.
· Lightweight, self-contained C implementation.
· Free, open source license (Apache).

Requirements:

· Apache

Limitations:

· Works for uncompressed archives only.
· Requires command line utility to construct .zip.mz "template" file.
· Currently only supported on Linux with GNU bash, gcc, and make.