--block option to block until jobs have finished

[pipitone]

Grid engine has the -sync y option that makes qsub block until the submitted job finishes (or errors out). This can be useful when writing dynamic pipelines with stages that can't be submitted all at once with dependencies (because which later stages run depend on the outputs from earlier stages).

It appears PBS does not have this feature (and unsure about other platforms), but we could make our own by polling qstat.

[pipitone]

The PBS hack way of blocking (e.g. for job id 12345):

qsub -W depend=after:12345 -I -x true

[gdevenyi]

Discussion re: PBS adaptivecomputing/torque#268

[gdevenyi]

After bringing it up on the mailing list, and being pointed to that thread, it looks like there's been movement and it will be implemented.

Pity clusters usually run old versions for a long time.

[pipitone]

Okay, let's use my hack above for now. It should work generally, I think.

[gdevenyi]

--wait is the option for sbatch

[gdevenyi]

Need this feature to fix up all the ants scripts so they play nicer with Niagara

[gdevenyi]

Job submission for blocking needs to be done via a map or other parallel object, otherwise you get sequential blocking during submission.

[gdevenyi]

PBS hack version requires #156 implemented

[gdevenyi]

At some point PBS added -Wblock=true. It is not clear which versions of PBS (Torque, PBSPro, OpenPBS) actually support this, but it should be added

https://lists.gnu.org/archive/html/help-make/2017-10/msg00004.html

[gdevenyi]

Fixed by #213