The SBML discrete stochastic models test suite
Good test suites (and benchmark suites) are important for software (and model) development, but can be pretty hard to come up with for stochastic models or software relying on probabilistic algorithms.In this issue of Bioinformatics there is an application note describing such a test suite:
The SBML discrete stochastic models test suite
Evans, Gillesipe and Wilkinson
Bioinformatics 2008 24(2):285-286.
Their approach is pretty simple: compare your simulations with the expected value and see if it falls outside the expected range (from known or previously simulated sd). As such, there is not really that to it. The models are also pretty simple, so while they will be useful for catching obvious bugs in a general simulator, I am not sure they will help catching more complex bugs. Of course, you do need to have the simple stuff working before you can tackle the harder problems, so it could still be useful.
Anyway, I am not planning on implementing a general SBML simulator, so it is not that much use to me, except that I am teaching a Systems Biology class where we are using Wilkinson’s book so the models in the test suite matches the exercises I am giving my students, and I can use the test suite to test their programming. Neat.
Citation, for Research Blogging:Evans, T.W., Gillespie, C.S., Wilkinson, D.J. (2007). The SBML discrete stochastic models test suite. Bioinformatics, 24(2), 285-286. DOI: 10.1093/bioinformatics/btm566