Supplemental material

In this post John Hawks complains about the important information that is left out of papers and hidden in supplemental material.

I mention this, because Asger Hobolth and I were just discussing this yesterday.

In the olden days, ten years ago, I would simply put the two papers side by side and find the discrepancies. But nooooo, we can’t do that any more. Now, all the relevant parameters from one of the papers (you guessed it, the one published by the Nature Publishing Group) are hidden away in a supplement.

You’d think that might not be so bad, since I have the supplement. But I have to keep tracking the cross references to the paper to find out where the methods apply. It’s a pain in the neck. Nobody else ever seems to complain. But that’s because they simply don’t read the papers! AAARGGGH!

Trust me, we complain!

Asger has just spent the last week trying to reproduce a result from a paper, only to find out that a lot of crucial info was left out of both paper and the supplemental material (that contained the data, but not the filtering that was done on the data). He had to get that info from one of the authors.

Personally, I spent a couple of weeks in December trying to reconstruct a method hidden deep in the supplemental material of a Nature paper — on a project very related to the rest of John’s post, by the way — but never managed to reproduce the results from the paper.  I got close, but never quite there.

It might be taking it a bit too far to say that people don’t complain, because they don’t read the papers, but I think very few people read the supplemental material.  At least in any kind of details.

I know that I only read the supplemental material for very few of the papers I read; only those where I want to reconstruct a method, or where I don’t really believe the results and want to see how the data really support it.

Sadly, very often the supplemental information doesn’t help much there either.

Is the supplemental material even reviewed?

Post score: 8-4 = 4

Author: Thomas Mailund

My name is Thomas Mailund and I am a research associate professor at the Bioinformatics Research Center, Uni Aarhus. Before this I did a postdoc at the Dept of Statistics, Uni Oxford, and got my PhD from the Dept of Computer Science, Uni Aarhus.

2 thoughts on “Supplemental material”

  1. I could not agree more!
    Supplemental material has become the burial ground for poorly written and poorly reviewed methods and results.
    Many a time you can only make sense of the paper if you have the supplementary material by your side. This cross-referencing is a big hassle and what the heck is the point in having something called a supplement when it is, in fact, essential to the paper?

    I would love to see supplementary material disappear.

  2. Well, I have some mixed feelings about this…

    I mean, I am pissed when the supplemental material is of a poor quality and it is impossible to actually reproduce the results in the paper from the description in the supplemental material. So I think it should be properly reviewed and be of the same quality as the paper itself.

    On the other hand, there is the page limit to consider. It is not possible to put all the supplemental material in the main paper. The paper is often much shorter than the combined supplemental material.

    Even for online publications — which I’m betting will be all publications soon — where page limits is not a major problem, it just doesn’t make sense to put all the material in the main paper.

Leave a Reply