Do we still need pre-publication peer-review ?

A bit over a month ago Glyn Moody wrote a blog post arguing that abundance of scientific publishing outlets removes the need for our current system of pre-publication peer-review. The post sparked an interesting discussion here on FriendFeed.

Glyn Moody tells us that we have now:

"yet another case of a system that was originally founded to cope with scarcity - in this case of outlets for academic papers. Peer review was worth the cost of people's time because opportunities to publish were rare and valuable and needed husbanding carefully"

Since we have an endless capacity to publish information online Moody argues that there is no longer a need to pre-select before publication. We can leave that all behind us and do a post-publication peer-review that is distributed by all of the readers using all sorts of article level metrics that PLoS has been promoting.

More recently Duncan wrote another blog post that has some information that I think is important for this discussion. He was trying to estimate how many articles have ever been published. In the process he noted an interesting number - the number of articles that are currently published per minute. Pubmed keeps a table with the number of articles that they have information on per year. I don't think the last couple of years are well annotated and the first decades are that reliable so I just plotted here the totals between 1966 and 2007.

It is not surprising to see that the number of articles published per year is increasing, it probably matches well our expectations. I personally feel like I never have enough time to keep up with the literature. We are currently over the 700.000 papers per year. A search on pubmed for articles published in 2009 returns 848.856 papers. Something like 1.6 papers per minute !

So, although we have no scarcity of publishing outlets we have a huge scarcity of attention. It is very literally impossible to keep up with the current literature without some sophisticated filtering system. With all of the imperfections of our current System (TM) of editorial control, subjective peer review, subjective impact evaluations, impact factors and so on, we must agree that we need a lot of help filtering through these many articles.

I have read some people arguing that we should be capable ourselves of reading papers and realizing if they are interesting/innovative or not. That is fine for the very narrow range of topics that are close to our area of interest. I have pubmed queries for my topics of interest and I do filter through these myself without relying (too much) on the journal it was published on, etc. The problem is everything else that is not within this extremely narrow range of topics or the many papers that escape my queries. I want to be made aware of important new methods and new discoveries outside my narrow focus.

Moody and many others argue that we can do the filtering after publication by the aggregated actions of all of the readers. I totally agree, it should be possible to do the filtering after publication. It should be possible but it is not in place yet. So, if we want to do away with the System .. build a better system along side it. Show that it works. I would pay for tools that would recommend me papers to read. In my mind, this is where publishers of today should be making their money, in tools that connect the readers to what they want to read, not on content that should be free to read and re-use by anyone (open access).

Review - The Shallows by Nicholas Carr

On a never ending flight from Lisbon back to San Francisco I finished reading the latest book from Nicholas Carr: "The Shallows - What the Internet is doing to our brains". The book is a very extended version of an article Carr wrote a few years ago enteitled "Is Google Making us stupid" that can be read online. If you like that article you will probably find the book interesting as well.

In the book (and article) Carr tries to convince the reader that the internet is reducing our capacity to read deeply. He acknowledges that there is no turning back to a world without the internet and he does not offer any solutions, just the warning. He explains how the internet, as many other communication revolutions (printing press, radio, etc), changes how we perceive the world. In a very material way, it changes our brain as we interact with the web and learn to use it. He argues that the web promotes skimming the surface of every web page and that the constant distractions (email, social networks) are addictive. This addiction can even be explained by an ancient species need to constantly be on the look out for changes in our environment. So, by promoting this natural and addictive shallow intake of information, the internet is pushing aside the hard and deep type of reading that has been one of mankind's greatest achievements.

After reading all of this I should be scared. I easily spend more than ten hours a day on these interwebs and my job as a researcher depends crucially on my capacity to read deeply other scientific works, reason about them, come up with hypothesis, experiments etc. So, why I am still writing this blog post instead of sitting in some corner reading some very large book ? Probably because I do not share Nicholas Carr's pessimist view. I actually agreed with a lot more things that I was expecting to before reading the book. I certainly believe that, like any other tool, the internet changes our brains as we used it. I agree also that reading online promotes this skimming behavior that the book describes. I observe the same from my own experience. What I find hard to believe is that the internet will result in the utter destruction of mankind as we know it (* unless saved by The Doctor).

It is just a personal experience but, despite my addiction to the internet, I haven't stopped reading "deeply". Not only is it a job requirement, I enjoy it. One of my favorite ways to spend saturday mornings is to get something to read and have long breakfast outside. At work I skim through articles and feeds to find what I need and when I do I print to read deeply. That is why I have piles of articles on my desk. This just to say that I found a way around my personal difficulty with deep reading on the computer screen. In other words, if it is required, we will find a way to do it. The internet habits that might be less conducive to deep thought are not worse than many any other addiction of our society and we have learned to cope with those.

I cannot imagine going back to a time when I would need to go to a library and painfully look for every single scientific article I wanted. Not to mention the impossibility of easily re-using other people's data and code. So even if a small but significant number of people can't find a way to cope with the lure of the snippets the advantages still overwhelmingly outnumber the disadvantages.

This topic and book have been covered extensively online. It is almost even evidence in itself that Carr is wrong that such a wealth of interesting and diverse opinions have shown up on the very technological platform that Carr is criticizing in the book (granted that some of these are also newspapers :). Examples:

Mind Over Mass Media (by Steven Pinker)
Carr's reply

Interview with Nick Carr and New York Time's blogger Nick Bilton

and for a different take on the topic here is an interview with Clay Shirky