python subsubmit.py -x SCRIPTS [SCRIPTS ...] [-n NODES [NODES ...]]
This program is used to submit one or multiple script(s) to one or multiple CPU node. One does not have to login to a computing node to execute this program. This is very useful for submitting light jobs such as
polarize etc of Tinker software.
parallel tool of GNU is used to submit the high level script files. On a certain node,
concurrent.futures model of Python is used to submit the low-level scripts files, as describe in submit.
python subsubmit.py -x ana1.sh ana2.sh ... ana1000.sh -n node90 node91 ... node100
Here is the efficiency validations on 2163
||parallel; 1 node||36|
||parallel; 2 nodes||20|
||parallel; 4 nodes||11|
||parallel; 6 nodes||8|
||parallel; 8 nodes||7|
As seen above, more than 50x accelaration is obtained by using both
GNU parallel and
concurrent. Depending on how many CPU cores your
submitting node has, the efficiency may vary. Consulting GNU parallel for more information.