Execute shell commands in a parallel and/or distributed way
  • Python 96.2%
  • Shell 3.7%
Find a file
2024-09-30 15:57:58 +09:00
bin works better this way 2014-05-09 16:23:16 +09:00
cfg added GPL header 2010-02-23 17:21:11 +09:00
lib/Pyro going fomr Pyro 3.10 to 3.11, Irmen said it would correct the bug I 2010-12-08 18:53:36 +09:00
package corrections so that it can be parsed 2014-02-14 14:57:30 +09:00
src src/parallel.py: use python2 explicitely 2024-09-30 15:57:58 +09:00
tests updated 2010-02-22 16:58:14 +09:00
BIOINFORMATICS_EXAMPLE added 2010-07-02 19:27:19 +09:00
CONTRIBUTORS completed 2010-01-09 17:05:56 +09:00
GPL_LICENSE.txt renamed 2009-11-29 16:29:04 +09:00
Makefile added install target 2010-04-30 13:30:25 +09:00
par.tgz added, I need the release to be available _now_ because 2010-07-20 10:21:57 +09:00
par.tgz.sig added, I need the release to be available _now_ because 2010-07-20 10:21:57 +09:00
README users are kindly asked to cite the corresponding publication if using 2010-09-29 20:19:19 +09:00
test_parallel.input added initial files 2009-11-29 16:21:38 +09:00
test_parallel.output.reference added initial files 2009-11-29 16:21:38 +09:00
test_parallel.sh the correct way is to use the wrapper, not the python script directly 2012-04-27 10:20:08 +09:00
TODO big update 2010-09-30 14:30:02 +09:00

If you use this software, please cite the corresponding publication:

@article{Berenger2010,
author = {Berenger, Francois and Coti, Camille and Zhang, Kam Y. J.},
title = {{PAR: A PARallel And Distributed Job Crusher}},
doi = {10.1093/bioinformatics/btq542},
year = {2010},
journal = {Bioinformatics}
}

# you need to install Pyro, cf. http://pyro.sourceforge.net
# (sudo yum install pyro _OR_ sudo aptitude install pyro)
# On the server, TCP port 7766 must be open
#                TCP port 9090 must also be open if the server is also
#                the Pyro nameserver (default case)

# if Pyro is not installed system wide, try this:
export PYTHONPATH=~/usr/Pyro-3.10/build/lib

# Some usage examples:

# run in parallel commands from test_parallel.input
cat test_parallel.input | ./parallel.py -i /dev/stdin

# same than before but with a progress bar
cat test_parallel.input | ./parallel.py -i /dev/stdin -v

# run in parallel commands from test_parallel.input and store output
# in output.log
./parallel.py -i test_parallel.input -o output.log

# same than before but with a user defined output printer
./parallel.py -i test_parallel.input -o output.log -p post_proc_example

## real world usage example
# 1) server side
./parallel.py -v -i many_commands.sh -o par_many_commands.log -s
# 2) client side, on each machine you want to join the computation
#    replace SERVER_NAME by the machine name from where you launched
#    parallel.py using -s
./parallel.py -c SERVER_NAME
# 3) be thrilled! ;)