IPC::ConcurrencyLimit 0.05

IPC::ConcurrencyLimit is a Python module that implements a mechanism to limit the number of concurrent processes in a cooperative multiprocessing environment. This is an alternative to, for example, running several daemons.

Roughly speaking, a typical setup would be the following:

 Cron starts a new process every minute.
 The process attempts to get a lock as shown in synopsis.
 If it obtains a lock, it starts working and exits when the work is done.
 If not, max_procs processes are already working, so it exits.

This has several distinct advantages over daemons.

 Processes do not run as long. Small memory leaks are less likely to become a problem.
 Rolling out new code is trivial. No need to do any daemon restarting and worrying about interrupting a unit of work.
 No complicated master/slave setups and process/thread pooling.

The implementation uses some form of locking to limit concurrency: There's simply a limited number of locks to go around. The detailed locking implementation is chosen using the type parameter to the constructor. The base distributions ships with one locking strategy only: IPC::ConcurrencyLimit::Lock::Flock for a file-locking based concurrency limit.

Among the other potential strategies that are not part of this distribution are NFS-based locking using File::SharedNFSLock or using MySQL's GET_LOCK. Both of these schemes would allow limiting concurrency across multiple hosts without a special-purpose daemon.

SYNOPSIS

 use IPC::ConcurrencyLimit;
 
 sub run {
 my $limit = IPC::ConcurrencyLimit->new(
 type => 'Flock', # that's also the default
 max_procs => 10,
 path => '/var/run/myapp', # an option to the locking strategy
 );
 
 my $id = $limit->get_lock;
 if (not $id) {
 warn "Got none of the worker locks. Exiting.";
 exit(0);
 }
 else {
 # Got one of the worker locks (ie. number $id)
 do_work();
 }
 
 # lock released with $limit going out of scope here
 }
 
 run();
 exit();


Product's homepage

Requirements:

· Perl