running optimize_AUGUSTUS.pl with multiple cpus on SGE
Posted: Fri Nov 20, 2015 12:25 pm
Originally posted in the old forum by Anand Rao on 15.03.2013 - 18:24
Hi Mario, Katharina et al,
I am trying to sun optimize_AUGUSTUS.pl with 64 cpus on our Sun Grid Engine and so far have been unable to use > 1 cpu - even after installation of the Perl module Parallel::ForkManager on our SGE
My shell script for qsub looks like so:
There is NO error message, however qstat reveals the use of just one cpu!
So when we tried to change a few things and used openmpi
module load openmpi
we put mpirun in front of the optimize_augustus.pl Perl script we were running
And then submitted with:
It ran with 64 cpus but I killed it as it was creating core files!
core.25865: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV),
SVR4-style, from 'etraining'
How should I modify my shell script or qsub submission syntax /run parameters so that I can benefit from multi-threading, else its way too time-consuming!
Thanks in advance for your help.
Sincerely,
Anand
Hi Mario, Katharina et al,
I am trying to sun optimize_AUGUSTUS.pl with 64 cpus on our Sun Grid Engine and so far have been unable to use > 1 cpu - even after installation of the Perl module Parallel::ForkManager on our SGE
My shell script for qsub looks like so:
Code: Select all
#!/bin/bash
### Change to the current working directory:
#$ -cwd
### Job name:
#$ -N optmzAUG
#$ -S /bin/bash
module load gcc augustus
perl /share/apps/augustus-2.6/scripts/optimize_augustus.pl --species=Medicago_truncatula --cpus=16 --rounds=3 /home/aksrao/AUGUSTUS/Mtr_training/Mt3.5v5_bothUTRs.gb.train --UTR=on --metapars=/share/apps/augustus-2.6/config/species/Medicago_truncatula/Medicago_truncatula_metapars.utr.cfg --trainOnlyUtr=1
So when we tried to change a few things and used openmpi
module load openmpi
we put mpirun in front of the optimize_augustus.pl Perl script we were running
And then submitted with:
Code: Select all
qsub -pe mpi 64 optimizeAUGUSTUS.sh
core.25865: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV),
SVR4-style, from 'etraining'
How should I modify my shell script or qsub submission syntax /run parameters so that I can benefit from multi-threading, else its way too time-consuming!
Thanks in advance for your help.
Sincerely,
Anand