Contrary to what you read in Nature, Open Access has not caused the growth in science publishing

I wasn’t planning on spending my Thanksgiving delving into PubMed statistics to refute yet another bogus claim about open access publishing. But being a vegan, I didn’t really have much else to do anyway. So…

The newest Nature has an Op-Ed from Martijn Arns, a brain researcher in the Netherlands with a title I couldn’t ignore: “Open access is tiring out peer reviewers“. He complains about poor quality (not negative) reviews he’s gotten on some recent papers, and asserts that this is due to an increase in the burden on peer reviewers with the rise of digital publishing. I agree with him – there are deep problems with the way we go about peer review – and his solution to the problem – implement post-publication peer review – is spot on.

But as the trolly title of the article (which I suspect was added by Nature and not by Arns) suggests, Arns argues that the increase in publishing volume, and therefore reviewer workload, is due to the rise of open access. It is, of course true, that open access publishing has been growing rapidly, and thus it might seem to some people that the growth in scientific publishing overall is due primarily to open access. But impressions can often be wrong, so I decided to look at some data.

I used PubMed to get data on the total number of papers published annually since 2000 and PubMed Central to determine the fraction of these articles that are open access. (The numbers aren’t perfect, since some journals have made their back content open access, slightly inflating the number of open access articles from early years. But this is a small effect.)

YearPubMedOpen AccessFraction OA
200052987134380.006
200154477840980.008
200256212245530.008
200359243554890.009
200463747475320.012
2005697839118560.017
2006744503150650.020
2007782502204680.026
2008831834325760.039
2009872229590480.068
2010935583818540.087
201110109901119100.111
201210731581465610.137
201311305541798710.159

You can see several things in these data. First, the number of papers in PubMed has increased dramatically with more than 2.1 times as many papers published in 2013 as in 2000. At the same time the fraction of articles in PubMed that are open access has increased even more dramatically, from basically nothing in 2000 to 15.9% of articles in PubMed in 2013.

Interestingly, in the last few years, the annual grown in open access papers has exceeded the annual growth in non-open access papers. The graph below plots, for each year, the annual growth in open access (the number of open access papers published in that year minus the number of open access papers published in the previous year) as a fraction of the annual growth in papers published (the total number of papers published in that year minus the total number of papers published in the previous year).

 

OA as fraction of total new papers

(Not sure what’s going on with 2009 – something is weird with that datapoint).

But does this mean that open access is driving the increase in scientific publishing output? Of course not. Just because open access is capturing market share doesn’t mean that it is driving an overall increase in the size of the market. If it were, than you would expect the annual increase in the number of papers published to be increasing as open access has risen. But this is not the case. The number of papers published has been increasing at roughly 6% since for the last 10 years, with no relationship to the number of open access papers published, which has been increasing steadily every year.

OA_Output

Thus we are really looking at two essentially independent phenomena. There is clearly a rise in the total number of papers published. And open access has been capturing an increasing fraction of this growing market. But it is simply false to assert that open access has been driving this increase in scientific output.

That said, I agree completely with Arms that there is a big problem here. It IS a huge burden to try and find reviewers for all of these paper. And more importantly, this process has gotten slower and slower. The real failure of digital publishing is not that the number of papers has increased – it’s that the time it takes to publish papers has not improved at all. It’s ridiculous that we live in a world where it is possible to share information across the globe instantaneously, but that science as an enterprise has chosen to delay the sharing of new scientific knowledge for an average of 9 months as we go through a byzantine process of pre-publication peer review.

We need to – as Arms suggests (and I have written about previously) move to post-publication peer review. But in doing so we need to focus on the problem – peer-review. And in order to do this, we have to shift away from paywalled, subscription journals which depend entirely on pre-publication peer review to justify their existence. Thus, rather than mistakenly blaming open access for creating problems with peer review, we have to recognize that it is an important part of the solution.

 

This entry was posted in open access. Bookmark the permalink. Both comments and trackbacks are currently closed.

11 Comments

  1. Posted November 27, 2014 at 7:16 pm | Permalink

    Not that it invalidates your conclusion, but I’d like to see precisely how these numbers were obtained. I think the OA figures are way too low. If I search PMC, for example, using the term “2000[PDAT]” (publication date), I get 42 171 records. If I search PubMed with the same term and specify the filter “freetext[FILT]”, I get 109 247 records for the year 2000.

    I posted more details here: http://rpubs.com/neilfws/45825. Let me know if I’ve done anything ridiculous.

    • Posted November 28, 2014 at 9:12 am | Permalink

      Neil, as I pointed out on Twitter, we’re counting different things. You’re counting free full text articles, while I’m only counting ones that are truly open access.

      • Posted November 30, 2014 at 1:01 pm | Permalink

        Thanks for the clarification and for discussing the search terms on Twitter.

        I don’t know how “truly open access” is defined. Is it to do with licensing? Do authors/journals see an “OA subset option” when they submit to PMC?

  2. Posted November 27, 2014 at 11:56 pm | Permalink

    Thanks, Mike for this important corrective. See also my own response, Open-access megajournals reduce the peer-review burden.

    Just one correction: you write:

    “the fraction of articles in PubMed that are open access has increased even more dramatically, from basically nothing in 2000 to 15.9% of articles in PubMed in 2013”

    But your table shows a growth to only 0.159%. Presumably the bug is in the table caption (“%OA”), which should be headed something like “Proportion OA” — either that, or the numbers in the table multiplied by 100.

  3. Qaz
    Posted November 28, 2014 at 12:54 pm | Permalink

    Presumably, there are more scientists publishing, which means there should be a larger pool of peer reviewers to draw from. Unless individuals are publishing more than they used to. The real question is whether (a) individuals are doing more writing and less reviewing or (b) more papers are getting rejected and being resubmitted, thus producing more reviews needed per publication.

    Neither of these reflect any issues with open access. Much more likely, they reflect the Glamourization of the science business.

    I also note that the main claim Arns seems to be saying is that the riff-raff (to quote another grumpy old guy) is reviewing his papers incompetently. There have always been bad reviews and bad reviewers. I don’t think there’s any good evidence that reviews have been getting worse. My anecdata is that they’ve been getting better.

  4. Thomas Munro
    Posted November 29, 2014 at 1:11 am | Permalink

    Great stuff, thanks. I think few people are aware of this yet, and I suspect it’s also true across Web of Science and Scopus, or soon will be.
    I’d suggest an edit for clarification, substituting “annual growth” for “number of new papers added every year”, e.g. “open access now captures more than 50% of annual growth, despite making up only 16% of new papers.”

  5. Michelle M
    Posted November 30, 2014 at 10:58 pm | Permalink

    I am all for the idea of post-publication peer review, but I’m conflicted about whether or not it should be anonymous as it is on PubPeer. Anonymity provides the perfect way for people to be vindictive and sabotage other people’s careers. But at the same time, I wouldn’t criticize an esteemed scientist’s work if my identity is public lest our paths ever cross in the future.

  6. Posted December 4, 2014 at 11:40 am | Permalink

    This is an interesting analysis. But consider the following counterhypothesis. The total number of published articles has increased at a steady rate over the past 14 years for two reasons that have coincidentally varied together: changes in hiring and growth in open access. During the early 2000s, the number of articles increased because faculty hiring was on the rise (similar number of papers published per researcher but more researchers). Moving into the late 2000s and continuing into the present, as hiring rates have slowed down due to budget crunches, the number of published articles continued to increase at a steady rate because of the growth in open access (similar number of researchers, but more papers per researcher). In other words, as new faculty hiring began to dwindle, open access publishing began to grow. This alternative account expects a constant rate of growth of total articles (because what was lost in the decelerating rate of new hires was made up for by more pubs in open access) and an increase in the number of open access articles, as you report. It also makes some predictions. First, it predicts that in countries which did not experience hiring freezes in the late 00s, the rate of growth of total articles published should have increased, all else equal. Second, it predicts that in fields with fewer open access journals, the rate of total articles published should have decreased starting in the late 2000s, all else equal (especially that hiring figures are similar across fields being compared).

    With that said, I agree there’s a problem with the peer review process and that it needs to be fixed. Post-publication peer review is an idea we should be trying out.

2 Trackbacks

  • […] This post is just a summary of some interesting online discussion from last week around open access publishing. I learned a few things about definitions and PubMed/PMC filters. It all begins with an opinion piece, “Open access is tiring out peer reviewers.” With a title like that you might expect rebuttals from people like Michael Eisen and you’d be right. […]

  • […] More to read: The Open Access publishing model has repeatedly been associated with poor sciences, in part because it is used many predatory journals. In this blog post, Ernesto Priego argues that defective peer review is not confined to open access journals. Another rebuttal of this criticism, specifically addressed to the Nature article can be found here. […]