Originally posted by Luke in the old forum on 05.12.2013 - 11:20
I'm wondering if you can help me out with a weird memory problem I've run into run running augustus. It's the ppx extension.
I'm looping over a batch of 1000 profiles, running a fastBlockSearch and then running augustus ppx on the best hit region ~15kb max. Strangely, I'm running out of memory! I've been raising the ceiling and am up to 44gb / cpu.
I've narrowed it down to a single nasty profile, so it appears to be an isolated case.
It's quite long, and low complexity in parts.
AUGUSTUS PPX memory problem
Moderator: bioinf
Re: AUGUSTUS PPX memory problem
Originally posted by Mario in the old forum on 05.12.2013 - 11:25
This is likely to result from cases where - due to repeats - the profile or some of the blocks generate an excessive number of hits. A proper fix will require some programming effort on our side.
I suggest, you go for a quick fix:
a) Perhaps, repeat masking gets rid of that region.
b) Use ulimit -v to limit the memory of augustus and ignore the (few) cases where it crashes because of the memory limit.
c) Retry the crashed jobs with a decreased max_entropy parameter.
This may not even work when the entropy is small but the motif still occurs very often.
This is likely to result from cases where - due to repeats - the profile or some of the blocks generate an excessive number of hits. A proper fix will require some programming effort on our side.
I suggest, you go for a quick fix:
a) Perhaps, repeat masking gets rid of that region.
b) Use ulimit -v to limit the memory of augustus and ignore the (few) cases where it crashes because of the memory limit.
c) Retry the crashed jobs with a decreased max_entropy parameter.
This may not even work when the entropy is small but the motif still occurs very often.