utcondor is a Python library that provides Python tools for distributed computing on the Condor platform at UTCS. These tools probably won't be useful to you unless you're a computer science graduate student at the University of Texas at Austin.
The goal of utcondor is a reliable implementation of a simple distributed computing model. It requires little boilerplate, and switches easily between local and remote execution.
Overview
The utcondor library supports distributed computing tasks that can be cast as a one-level parallel map: a function executed over multiple inputs on multiple machines. This model is primitive, but easy to apply and often good enough. For a trivial example, to square a range of numbers in distributed fashion:
import condor
def f(x):
return x**2
def main():
calls = [(f, [x]) for x in range(16)]
for (call, result) in condor.do(calls, 4):
print call.args, result
if __name__ == "__main__":
main()
Any arguments passed to the remotely-executed callable must be pickleable.
Installation
Install the two dependencies; pyzmq-static is probably the easiest way to install the Python bindings to ØMQ:
pip install plac
pip install pyzmq-static
Then use waf to install utcondor into your local Python installation:
./waf configure
./waf install
You're running inside a virtualenv, right?
Caveat Emptor
Be careful. Pay attention to whether Condor jobs are being successfully cleaned up. Use at your own risk.
Product's homepage
Requirements:
· Python
Find us on Google+
