[kepler-users] Suggestions / examples of using JobCreator, JobSubmitter, JobChecker for large batches of jobs?

David LeBauer dlebauer at illinois.edu
Fri Oct 28 11:15:50 PDT 2011


Hello,

I want to launch an ensemble of jobs on a remote machine. Can I
configure the JobSubmitter / Job Checker to wait until all of the jobs
reach either an error or leave the queue?

Essentially, I want to implement something like the following in kepler:
executable: server:/path/to/rundir/run
input files: configfiles{1..500}


rsync configfiles* server:/path/to/rundir/
ssh -T <server> "cd /path/to/rundir/; for $f in configfiles*; do qsub
-cwd -N $f -pe mpich 1 -j y -o $f.log ./run 1 $f; done"

and then, when all of the jobs have status 1 or 0, trigger the next
module (which will read the output)

Thank you,

David

-- 
David LeBauer, PhD
Energy Biosciences Institute
University of Illinois Urbana-Champaign
1206 W. Gregory Drive
Urbana, IL  61801, U.S.A.



More information about the Kepler-users mailing list