Hello,
I'm getting some strange results from optimize_augustus.pl, essentially it is not resulting in improving the training parameters. I'm using ~ 1200 high quality models that I split into a test set (200) and training set, run etraining, run an initial prediction with augustus which yields gene level sensitivity at 0.745 and specificity 0.73. I then run the optimize_augustus.pl script, after it is complete, the gene level prediction sensitivity is now 0.733 and specificity is 0.707. So it seems like after "optimization" the prediction ability is actually worse. I've seen a very similar pattern with different genomes and gene models. I'm running augustus v3.2.1 compiled using gcc-5.
Thanks,
Jon
optimize_augustus.pl results in lower prediction accuracy??
Moderator: bioinf
Re: optimize_augustus.pl results in lower prediction accuracy??
Hi Jon,
thank you for pointing out that this can happen. I have also seen this in more than one genome annotation project.
Always make a copy of original parameters before optimize_augustus.pl
Always visualize prediction results in a browser with before/after optimize_augustus.pl
Based on visualization and on the numbers, you can make a decision whether you would like to use the "new" parameters, or the "old" ones.
Best,
Katharina
thank you for pointing out that this can happen. I have also seen this in more than one genome annotation project.
Always make a copy of original parameters before optimize_augustus.pl
Always visualize prediction results in a browser with before/after optimize_augustus.pl
Based on visualization and on the numbers, you can make a decision whether you would like to use the "new" parameters, or the "old" ones.
Best,
Katharina