I hadn’t noticed that…
At Genomicron, Ryan Gregory refuses to participate in ResearchBlogging. Why? Because their slogan is Discussing and Creating Peer-reviewed Research. Discussing is fine, but we are not creating peer-reviewed research by blogging about it.
I hadn’t noticed this slogan — it is only on the large icon and I only use the small icons when I use it in posts about published research — but it is not something I worry too much about. I like to read discussions about published papers in blogs, but I am not kidding myself that much research is being created there.
I’ll still use the icon to highlight when I am discussing a paper — and not some more general issue.
Another, older complaint, is that blogging on peer-reviewed research it can be confused for the actual peer-review process:
As a scientist, I take the peer review system very seriously (its several problems notwithstanding) and I do not wish to see blogs perceived as even an approximation of that system. That said, blogs are a useful way to discuss research, and I am happy to see this new development in science communication.
Again, I love reading about paper discussions — it feels like a global journal club — but I agree that the actual peer-review process has very little to do with blog discussions of papers!
May 21st, 2008 at 2:36 pm
“…..I agree that the actual peer-review process has very little to do with blog discussions of papers”
I do not see why these two things need to be different. An “actual peer-review process” are evaluation of your work by your peers, – that can easily be done in a blog-format. The blog-format also provides absolute openness which in my view could be considered superior to traditional peer-reviewing where all power lies in the hand of editors and reviewers in closed rooms and anonymous settings. Sometimes the peer reviewers could use some peer-reviewing…., – I know many would agree with me there…
May 21st, 2008 at 4:00 pm
I am all for open reviews, see my post on this: Thoughts on peer-reviewing.
All I’m saying here is that there is a difference between the peer-reviewing process and blogging about a paper. I also write a bit about this in that post.
The short version is, that I am all for open reviews — I would love to see reviews published together with papers since this adds information about the concerns the reviewers had and what was done to address this — but there is a major difference between the quality control aspect of reviewing and blogging about a paper.
Not that there are not significant overlap in this. What I include in a blog post about a paper I would also include in a review report, but I spend a lot more time on a proper review that I do on a blog review.
May 22nd, 2008 at 2:17 pm
I think people who call blogging “peer review” don’t really know what scientific peer review really is. For one thing, peer review keeps papers from being published if they are not up to par (ideally, anyhow), whereas blogs publish first then comment. I have yet to see a blog post go over the figure captions, stats, reference list, interpretations of data, specific methods, and all the other very critical aspects that go into real peer review. And if it did, no one would read it.
Summaries of published papers, and even comments about them that are critical, are excellent and I strongly encourage this. However, calling this “peer review” is misleading, and I certainly resist a move to circumvent real review by substituting the blog variety by itself.
May 22nd, 2008 at 9:30 pm
I have yet to see a blog post go over the figure captions, stats, reference list, interpretations of data, specific methods, and all the other very critical aspects that go into real peer review.
I have yet to see a peer review that did this. My experience is that people who blog about my work do a much, much more thorough job of review (and are more often correct) than the normal peer review process. Too many peer reviewers either provide no helpful comments (“Paper was great, recommend acceptance!”) or abuse the opportunity.
I think that Mailund is correct that open reviews would help, but aside from the fact that most bloggers are not professionals, I don’t see that as being very different from blogging.
May 23rd, 2008 at 6:04 am
There is a lot of field-specific tradition in peer-review, I think. I changed field from computer science to bioinformatics about four-five years ago, and I noticed major differences in reviews.
First, there is a major difference in how we publish, in computer science and bioinformatics (I think the later might be more like biology, but I haven’t ever reviewed “pure” biology papers, nor submitted any). In computer science, you publish at conferences, and journal papers are mainly summing up results from previous conference papers, or just longer versions of conference papers with a few more details. The biologist I know do not even consider a conference publication to be a publication.
In computer science reviews, I typically get a lot more detailed comments about the methods, checking that everything works, suggestions for improvements, suggestions for how descriptions of algorithms could be improved, etc. Rarely, if ever, will a reviewer comment on whether the problem addressed is interesting at all…
In bioinformatics reviews, I have only gotten feedback on the details of methods a few times, but I typically get a lot of comments about the problem the methods address. For good or bad … I have on several occasions gotten algorithmic papers rejected because the reviewer thought I should make algorithms for different problems.
As for the quality of blog reviews, I think there is a strong bias here… a paper of general interest will get many blog comments, some of high and some of low quality. A technical paper, or a paper with minor contributions to the field, will probably not get any online discussion.
Really, blog commenting on papers and peer-reviewing are two different things, with different purposes. I don’t think that one can replace the other…
May 23rd, 2008 at 6:05 am
btw, John, I absolutely love your blog!
May 23rd, 2008 at 1:57 pm
As for the quality of blog reviews, I think there is a strong bias here… a paper of general interest will get many blog comments, some of high and some of low quality. A technical paper, or a paper with minor contributions to the field, will probably not get any online discussion.
Agreed. I’ve reviewed things for journals that I don’t often (if ever) read (not to mention grants). The long tail of research papers is pretty long. But this is also a labor question, since there are only four or five research-oriented blogs writing about my area. I could imagine a much larger community taking in much more specialized papers.
Really, blog commenting on papers and peer-reviewing are two different things, with different purposes. I don’t think that one can replace the other…
I can only observe that I have gotten more accurate comments on my research papers from blogs than from anonymous peer review. This is less true (but not universally so) for peer reviewers who provide their names; so I think the issue of anonymity clouds the comparison (blogs may be “anonymous” but have a reputation aspect; they’re not one-offs). It makes me more inclined toward open review, or posting preprints. This is unusual in biology, but more routine in computer science, right? Maybe that tradition makes a difference?
May 23rd, 2008 at 2:31 pm
I’m not sure open reviews (or preprints, for that matter) are more common in computer science.
I can easily see anonymity leading to laziness, though. When your reputation is not on the stake, and you are busy with your own work, taking the time to do a proper review might not have that high a priority.