The impact of Randy Schekman abandoning Science and Nature and Cell

Recipients of this year’s Nobel Prizes converge this week on Stockholm to receive their medals, dine with the King and Queen, and be treated like the scientific royalty they have become. For most this time is, understandably, about them and their work. So, bravo to my Berkeley colleague Randy Schekman – one of this year’s recipients of the prize in Physiology/Medicine – for using the spotlight to cast a critical eye at the system that brought him to this exalted level.

In a column in The Guardian, Randy writes:

I am a scientist. Mine is a professional world that achieves great things for humanity. But it is disfigured by inappropriate incentives [...] We all know what distorting incentives have done to finance and banking. The incentives my colleagues face are not huge bonuses, but the professional rewards that accompany publication in prestigious journals – chiefly NatureCell and Science.

He goes on to make his case for why these high-impact subscription journals are so toxic, and finishes with a pledge:

Like many successful researchers, I have published in the big brands, including the papers that won me the Nobel prize for medicine, which I will be honoured to collect tomorrow. But no longer. I have now committed my lab to avoiding luxury journals, and I encourage others to do likewise.

I gave up publishing in Science, Nature, Cell and all other subscription-based journals when I started as a junior faculty at Berkeley in 2000, and having devoted immense amounts of time and energy over the ensuing 13 years to convincing other scientists to do the same. I co-founded a publisher – PLOS – whose raison d’etre was to provide authors with an alternative to the big-name subscription publishers Randy so rightly takes to task.

Yet despite great success – my career has flourished without publications in the “big three”, and PLOS is now a major player in the publishing work – it is a measure of just how far we have to go – just how powerful the incentives to publish in “high impact” journals are – that Randy’s announcement is big news.

I hope that Randy will serve as inspiration – an example for others to follow. But, sadly, I suspect the will not. Lots of people have already dismissed his shift as the easy action of someone who had already “got his”. And of course they’re right. Even before his Nobel Prize, Randy was a science superstar whose papers would have been read even if he had done nothing more than tape printed copies to the bulletin board outside of his office. His students and postdocs don’t need a Science, Nature or Cell paper to get taken seriously – they only need a good letter from their now Nobel Laureate advisor.

I know that most people will dismiss Randy’s example, because they have done it to me. Even though I gave up subscription journals at the beginning of my independent career – before I had students, grants or tenure – most people I talk to say “Good for you. But you were trained in high-profile labs, you had Science and Nature papers as a postdoc, and you were already well known. You could get away with it. I can’t.” It’s all true. I understand why – especially in this horrible funding climate – people are unwilling to shun a game that they may despise, but which almost everybody tells them they have to play to survive. And since everything that is true about me is 100 times more true about Randy, his followers are likely to come primarily from the far upper tier of scientists.

This is sad. Because we need to listen to him. Indeed we need to take him one step further. While I admire everything eLife is doing to make the process of peer review saner, they still reject a lot of good papers that don’t meet the reviewers’ and editors’ standards of significance. As I’ve written elsewhere [1][2], we need to dispense entirely with journals and with the idea that a few reviewers – no matter how wise – can decide how significant a work is at the time. But whether you support Randy’s vision of sane pre-publication peer review, or my vision of a journal free world built around post-publication review, we have the same problem – we need more than a handful Nobel Prize winners and true believers to abandon the current system. So what’s it going to take?

Fifteen years ago, when I first became involved in reforming science publishing, the big problem was there were no alternatives. Now there are plenty – there’s eLife, PLOS, BMC and many others who are attacking various pathologies in science publishing. But still SNC maintain their allure. And they will continue to do so until people no longer believe they are the ticket to success. It’s a nasty, self-fulfilling prophesy. Most biomedical scientists send their best work to SNC, and so there’s a correlation between who gets jobs/grants/tenure and publishing in SNC, and so the next generation thinks they have to publish in SNC to get jobs/grants/tenure and on and on and on.

We could all just choose to stop. Start sending your best work to eLife instead. Or just do what we should all do and send ALL of our work to PLOS ONE, BMC and other journals that don’t consider significance in the publishing decision. We SHOULD do that. But, listening to people out there, I don’t think most scientists are ready to.

I think a better place to work is on hiring, grants and tenure. If we all commit to NEVER looking at the journal in which a paper appeared when we’re evaluating someone, and if we speak up when anyone else does it. If we really endeavor to judge people solely by the contents of their manuscripts word will slowly get out, and people will stop thinking it’s worth it to go through the slog of review at SNC. They’ll stop spending months doing pointless experiments that will make their work “sexier” to editors and reviewers.

And maybe we’ll start seeing Nobel Prize winners whose work was never published in Science, Nature or Cell – and nobody will even notice.

This entry was posted in open access and tagged , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

43 Comments

  1. Posted December 11, 2013 at 2:32 am | Permalink

    “If we all commit to NEVER looking at the journal in which a paper appeared when we’re evaluating someone …. .. They’ll stop spending months doing pointless experiments that will make their work “sexier” to editors and reviewers.”

    Well written Michael, thanks for your post! I believe that things will eventually change, as people like you move towards more senior positions and bring a cultural change. The process is painstakingly slow, though, as there are still many people around in decision positions who are very conservative.

  2. Posted December 11, 2013 at 4:38 am | Permalink

    “there’s a correlation between who gets jobs/grants/tenure and publishing in SNC”

    I’d say the system works like this: SNC -> Grants -> more SNC -> even more Grants ->…

    This pathological chain will break if funding agencies start rewarding genuine reproducible results but not flashy publications. The current money distribution system is at the core of “disfigured” science. The rich labs getting richer. Masses of postdocs going broke.

    The faults with SNC are not the reason but the consequence of the pathology.

    • GM
      Posted December 12, 2013 at 5:17 am | Permalink

      What you say is correct but it is not the ultimate reason.

      The real reason we use CNS as proxy for quality is that something has to be used as proxy for quality, and we have not yet substituted CNS with something better. When (if) we do, new pathologies will emerge, specific to whatever new system we come up with.

      As almost nobody who still remembers the times when it was not universally applied is still alive, we have come to accept peer review as the natural way of doing things. But it really only became a mandatory feature of the publication process some time after WWII. Before that journals had academic editors who decided what was published, and scientific reputation was built not so much on where you published but what you published (which is what everyone wants to go back to). But those were very different times – there were just a few journals in a given field that everyone read cover to cover and people were able to understand the research in a much wider variety of fields. It is also instructive to check how long it took people to get their PhDs and at what age they were making significant contributions. Compare to the situation now – everything is so compartmentalized, there are so many journals, and everybody is so busy that it is impossible to actually read the papers, both because people don’t have the time and because they don’t have the expertise beyond their very own narrow field. That’s why we have peer review (because it is no longer possible for editors to handle the flood of papers, both in terms of volume and the actual content) and that’s why we have proxies of quality. I have to follow something like 100+ journals to keep up to date on everything related to what I do, and unsurprisingly I fail at doing that, but my case is nowhere near as bad as some PIs I have seen who rarely even read papers. It’s even worse with administrators and funding agencies who don’t even have anything approaching the expertise of the scientists and the scientists themselves already can’t keep up

      Of course, part of it is laziness on the part of everyone involved, but the bigger factor is that the branching of science into sub-sub-sub-fields and the exponential growth in output (the numbers of papers published every year doubles every 10-15 years). The dominance of C/N/S as one way of measuring quality is the inevitable consequences of all that.

      BTW, this also means that every new journal that gets launched with the intention of eroding the dominance of CNS has in a way the opposite effect – because it means yet more papers and yet more need for somehow measuring quality without having to drink from the firehose of papers.

      • Posted December 12, 2013 at 9:03 am | Permalink

        The increasing flood of papers is not an obstacle for gradating them by quality or finding good papers. Do not underestimate the power of the online audience. For example, Amazon publishes millions of books. Is it pre-publication editorial reviews that really matter? No, it is post-publication reader reviews that ultimately decide how good a book is. And finding good books on Amazon is easier than browsing them in some physical luxury bookstore.
        It is about time to shift the peer-review from pre-publication to post-publication. However, I do have one caveat. The pre-publication peer – review overall does improve paper quality. Let’s not throw the baby out with the bathwater.
        Some mechanism should exist that allows authors to successively improve their already published papers (just like software developers can improve their product with succesive versions). When citing a paper, it will be cited with its version number
        The reason I emphasize this is that such a mechanism would allow authors to improve thier paper via the post-publication peer-review just like (or even better) the current pre-publication peer-review improves it.

  3. Posted December 11, 2013 at 5:04 am | Permalink

    There are a few of us out there that have landed TT jobs without SNC papers. We will go up for tenure and (hopefully) get grants without SNC papers. Let’s just hope that we are visible enough to demonstrate top the masses that it is possible, even without these high profile marquee pubs.

    Maybe we should start a blog highlighting those examples, particularly those that do not come from “BSD labs”. I think this would be a powerful way to counter the fears of not getting a job/grant/tenure without these pubs.

    • GM
      Posted December 12, 2013 at 5:21 am | Permalink

      There are a few exceptions of course, but they are that – exceptions.

      Last Spring I can hardly recall a faculty candidate who came to give a talk in our department who didn’t have multiple C/N/S papers (plus a solid pedigree)

      I am pretty sure this Spring it will be the same.

      Also, I have serious doubts that most of the people who make the hiring decisions ever read the blogs where these issues are discussed, and that these things are on their minds at all.

      • Posted December 12, 2013 at 7:08 am | Permalink

        point is, that there are more that you would suspect. and true, that most people making hiring decisions won’t rest the blog, This is a bottom up movement, and highlighting these examples is a good thing.

        The number of super candidates *without* CNS papers is growing. Maybe someday these arbitrary decision rules will have to be reconsidered.

  4. Posted December 11, 2013 at 5:41 am | Permalink

    This is on the mark. Young people will have to do this as a movement, with enough critical mass that–and this is key, I think–administrators (chairs, deans, funders–will recognize. This means re-educating them and, importantly, having ways to sort chaff from grain. Now, we have a misleading sorting system (e.g., ‘impact’ factors, etc.).

    If senior people start insisting on this, then we’ll get plenty of help. If CVs used for grants and promotions etc. were not allowed to include funding history or more than a few publications and required some ‘online presence’ category, we could rationalize the system.

    If grant funding and even journal decisions used a random draw among the set of ‘acceptable’ submissions, rather than ranking as they do now, there would be less unreasonable pressure to rate people on what was actually funded or accepted, and recognize that actual influence on one’s profession, properly measured, is what should count.

    Efforts are under way to develop better evaluating criteria, and they should be pushed along by all of us.

    Well, and maybe we should start giving more respect to teaching, where the actual impact is far greater than the vast bulk of published work.

  5. Benoit
    Posted December 11, 2013 at 5:49 am | Permalink

    Dear Michael,

    I completely agree with what you say.

    To understand my argument, I’d like to get to the definition of a publication: Originally it’s a manuscript that is published in a paper journal. This was the only way to disseminate knowledge over the world. Because paper is expensive and shouldn’t be wasted, journals used stringent selection policies and peer-review to optimise the value of the paper they sent all over the world.

    Having general high-impact journals and specialized lower-impact journals was also necessary because, without pubmed, the ability to browse the literature was limited.

    This selection procedure gradually acquired a new function: because only good science could be published, it was used as a proxy for assessing the scientific output of a scientist: the famous publication record.

    Nowadays internet has alleviated most of what made publications so important, yet we still want to submit our work to the judgement of professional editors, go through long peer-review processes, write rebuttal letters and make experiments that we don’t feel are necessary but are asked by reviewers.

    In the mean time, the video of a snorring cat posted on youtube raised to worldwide fame in few days, while asking 2 minutes of work for uploading.

    Thus journals, which use to be the vectors facilitating worldwide dissemination of knowledge, have become the main hindrance to the dissemination of knowledge.

    The main reason for still using these classical media of knowledge dissemination is that we became addicted to the publication record, as a mean to evaluate each other.

    As long as we are slaved to this scheme, changing the name of the journal or their funding scheme will not address the problem significantly.

    The true publication revolution will come when anyone will post anything as he likes it on youtube (or any other medium), without selection, without peer-review.

    Will we lose in quality? I don’t think so. We will have to convince the community that our data are worth it, not just an editor and three reviewers. We will be judged after our data have been released to the community, not before.

    Will we lose in our ability to browse the literature? We won’t be able to check established journals and expect that they feed us worthwhile literature. However, I’m convinced that in internet times, good data will eventually surface. The video of the snoring cat at youtube gets famous because people find it funny. There are gazillions of youtube videos, yet only a few get famous. Is it a worse system than editorial decision and peer-review? I don’t think so.

    Will we lose our ability to evaluate each other? Yes! We will have to actually read about each other science rather than counting papers on pubmed. This will be difficult but I guess is worth it.

    This revolution is coming. I don’t know how and when and I don’t know how to speed it up. But I bet that in a few decades from now, publication record will only be a bad memory.

    • GM
      Posted December 12, 2013 at 5:35 am | Permalink

      All of this sounds very nice. However, think about the reality of getting 100+ papers every day with titles and keywords that are vaguely relevant to what you’re doing. Are you going to be able to read all of them carefully and critically evaluate them? Of course not. Are you going to be able to just scan them? That’s also impossible. Are you going to be able to even read the abstract? Doubtful.

      Once you think about things from that perspective the complete “let’s get rid of the journal hierarchy and move to post-publication review” treatment becomes a lot less appealing.

      That works in small fields like theoretical physics and the various mathematical subfields, where the output is relatively modest in volume. I am not saying they have no problem with the volume of what gets out out there, especially given that those are not the kind of papers we have in biology that you can read in less than half an hour and understand almost completely – some of theirs take months to completely digest. But it’s still a lot smaller than in biology and I have serious doubts it’s going to work.

      Actually even if it works as well as in theoretical physics, it’s still not going to be a very good situation because the situation there isn’t every good either (it’s an example of the different kind of pathologies that come with a different system). What happens when papers get posted to the arXiv is that if you are the Edward Witten type everyone reads what you have posted and takes notice. But if you are not, a lot fewer people do. And this has everything to do with who you are and pretty much nothing to with the content of your work. If everyone reads what comes out of the HHMI labs while the lower-profile work gets a lot less attention, we will have a perpetuation of the same system, just through different means. We may just as well stick to what we have with the addition of open access for all journals

      • Posted December 12, 2013 at 6:13 am | Permalink

        I am not suggesting we have no system for evaluating, classifying and assessing the importance of published works. Rather I am suggesting that we do not couple this assessment with the act of publication. Instead of having assessment be something that happens at a fixed point in time by 2 or 3 reviewers, with their judgment reduced to a yes-or-no decision to publish in a particular journal, we build a system in which works are assessed from the time they are published and throughout their useful lifetime in a way that is MORE useful to potential readers than journal title is now.

        See http://www.michaeleisen.org/blog/?p=694.

    • GM
      Posted December 12, 2013 at 5:37 am | Permalink

      P.S. YouTube videos are not at all a good analogy – there is a very low entry bar for being able to evaluate YouTube videos and the ones that get a lot of views are almost invariably demonstrations of the tendency of human societies to sink to the lowest common denominator, not of the ability of large groups of people to collectively decided what is of high quality.

      Science should be, and to an extent fortunately still is, different

    • Benjamin Martin
      Posted December 28, 2013 at 9:09 pm | Permalink

      I think you just made the most compelling argument for the importance of pre-publication review and journals yet. The ideal dream scenario you describe sounds dreadful. I would not argue that the current system is perfect (or anywhere close), but given the choice between top-notch professionally edited journals or a charismatic snoring cat, I’m sticking with current system.

  6. Posted December 11, 2013 at 6:35 am | Permalink

    You say you quit publishing CNS in 2000, but you have at least 2 articles in which you are co-author long after that in Nature significantly after that:

    (1) Pennacchino et al (2006) “In vivo enhancer analysis of human conserved non-coding sequences”, doi:10.1038/nature05295

    (2) Clark, Eisen et al, (2007) “Evolution of genes and genomes on the Drosophila phylogeny”, doi:10.1038/nature06341

    And another in Cell in 2008.

    (3) Fowlkes et al (2008) “A Quantitative Spatiotemporal Atlas of Gene Expression in the Drosophila Blastoderm”, Volume 133, Issue 2, 18 April 2008, Pages 364–374

    • Posted December 11, 2013 at 6:41 am | Permalink

      Yes. But you know the way the world works. Those are not papers from my lab – they are papers from other groups that we contributed to but where we did not make the decision of where to publish. Every single paper from my lab since day 1 has either been in an open access journal (PLOS, BMC, PeerJ and a few others) or in PNAS with the open access option (which is not really open access, but given PNAS’s early support of PubMed Central, I have tried to support their efforts).

      • Posted December 11, 2013 at 8:43 am | Permalink

        Why did you not remove your name from those publications? I’m not saying you should have. It just seems like the sort of thing you’d do.

        • gagan sidhu
          Posted December 15, 2013 at 2:02 pm | Permalink

          you should have the balls to question his character under your real name, coward.

          what’s he supposed to do? hunt these people down and say “TAKE MY NAME OFF IT!! OR ELSE!!!”? Sometimes even the freedom fighters have SOME limitations; why rock the boat on an easy publication if it’s at no expense to you, and it was the other lab’s decision in the first place?

          those seem like purty nice gifts to be receiving, and i don’t have a problem with it at all. if someone wanted to put dr eisen’s name because he was apart of a big collaboration involving some of “his guys”, that shows you the amount of respect he has in the community.

          dr24hours==#hack imo.

  7. Posted December 11, 2013 at 6:40 am | Permalink

    Mike

    The point is that you have a huge amount of credibility in OA publishing, whereas Schekman has none, and the message is severely diluted, if not damaged, when it is delivered by someone with no credibility.

    Even a slight shift towards OA publishing prior to his eLife PR onslaught would have helped.

    Honestly, I am dismayed by the whole event. http://biomickwatson.wordpress.com/2013/12/10/i-think-you-might-be-a-hypocrite/

    Mick

  8. Mohamed Noor
    Posted December 11, 2013 at 6:41 am | Permalink

    I agree in principle with all of this, but there’s a facet we all ignore– the “money laundering” associated with institutional subscriptions to professional society journals. Some society conferences are either subsidized or at least backstopped by proceeds from the society that come from the journal. Similarly, other activities (such as society-sponsored workshops, various student grants or awards) come from the same. If we all go exclusively to PLoS, eLife, and BMC, some conferences will cost a lot more, there’ll be no more “travel awards” or other sorts of student awards, etc. And publication costs will be higher. So, for those whose primary funding is from NSF especially, it’ll quickly move to becoming prohibitive to both publish (where it’s worse to have “some” money then “none”) and for some people to present in at least some meetings.

    The way the system works now is that we basically launder money from universities through their “institutional subscriptions” passed to societies to make all these other activities work cheaply. In principle, there are three solutions if we all go to open-access: 1) universities funnel the funds they would have used for society journals back to investigators to fund student travel, research, & conferences, and/ or 2) NSF starts awarding more and larger grants (presumably through a massive Congressional budget increase). Sadly, both of those don’t seem very likely in the near-term. The third solution is the sad but most financially feasible one– 3) stop all those other activities.

    Again, I don’t disagree with your points at all (I’m a big proponent of DORA, as well as PLoS and BMC in particular), but I just want to point out that there are downstream effects, and it’s not as simple as deciding to publish in X vs. Y outlet. I completely agree with you, but I also think the entire system needs to be reconsidered simultaneously from the ground-up…

  9. Will
    Posted December 11, 2013 at 7:29 am | Permalink

    To me, these kinds of lofty proclamations from hyper-successful scientists won’t make much of a dent. Whether or not individual PIs agree with Schekman or not, they still think most people care about the fetishized journals. The culture has to change.

    I expect scientists will respond to data. What do studies of say, SNC-low v. SNC-high researchers suggest? I imagine SNC-high researchers dominate in getting hired and getting resources, and it’s probably not even close. But maybe if that trend was decreasing it would be encouraging to young researchers.

  10. BenK
    Posted December 11, 2013 at 7:30 am | Permalink

    Michael;
    As supportive as I am about getting scientific content into the open literature – and about providing a venue for negative results, raw data, source code, reproducibility experiments – I can’t agree with your statement about the sexy experiments required to attract attention. If anything, with fewer opportunities to highlight work across many or all disciplines, short term attention and recognition would hinge entirely on the ‘sexy’ experiments.

  11. Eskimo
    Posted December 11, 2013 at 7:33 am | Permalink

    One of the driving forces behind eLife seems to be disdain for the professional editors at Science/Nature/Cell. Schekman and Tjian have said as much. And one of the driving forces behind that is the mindset: “If that postdoc doesn’t produce lots of data and then join the R01 club, he or she is worthless.” To be consistent, the BSDs have to keep looking down on the postdocs who wash out of their labs even after they’re gone.

  12. Posted December 11, 2013 at 7:48 am | Permalink

    One of the many barriers to change is the fact that, for foreign postdocs, a paper in one of the SNC journals is a ticket to a faculty position in their home country. It is pretty hard for a PI to ignore that reality when deciding where to submit a postdoc’s papers.

  13. JP
    Posted December 11, 2013 at 10:14 am | Permalink

    Scientists of the world unite- Publish in PLOS (and other open access journals)! We all know what would be best we just need to put it into practice!

  14. RichardS
    Posted December 11, 2013 at 12:26 pm | Permalink

    I’ve noticed in seminars that some professors like to cite the journal where their work was published as a little preface to introducing the actual work (especially CNS). This humblebrag routine has always been a bit off-putting to me, and illustrates the pervasive nature of the biases that you’re fighting against (not only in hiring/grants).

  15. Denature
    Posted December 11, 2013 at 2:22 pm | Permalink

    People always pretend that the average quality of research is low in something like Nature and high in something like PLoS. The opposite is the case. Sure, there are a lot of crappy papers in Nature, Science and Cell, but way less so than in open access journals. And that despite the claim that PLoS looks for quality and Nature only for impact.
    I think everybody agrees that the system needs tweaking, but eliminating the need to publish something actually interesting and groundbreaking will just lead to a lot of science that does not even try to do something original. And because it all of a sudden will get easy to get a lot of publications (which all count “somewhat”) with very little work, the quality will actually go down. You will always need an incentive for a person to go out of their way to try to do something exceptional, otherwise nobody would try to set up this new difficult assay, come in on weekends and stay late. Why would anyone bother if just doing what everybody else does is considered “high quality craftsmanship” and aiming for something groundbreaking will only cause a lot of work with little gain? I am sorry to sound cynical, but scientists are human (even though idealists) and are as much affected by personal gain as anyone else. That this strive for personal gain is sometimes skewered towards “sexy” research with little substance is awful, but I do think it is the exception, not the norm. At the end of the day Nature/Science/Cell have a strong record of providing a platform for research that advanced human civilization and PLoS has a record of allowing people to publish studies that did not really work out but advance methodology or just make it possible for people to not chase the same non-feasable hypothesis. Both formats are needed, because you sometimes need a platform that gives you the attention you need to push an unusual idea, and sometimes you just want to keep people informed about the work that your lab has done lately so they can benefit from it.

    The people founding eLife have the luxury of getting attention wherever they publish, something most of us first have to achieve. It is a nice idea to improve the quality of science, but throwing out the editor, as in eLife, is bad. The editor is the only sane person in the submission process, and the one least affected by politics. The whole negative attitude towards editors does not come from the motivation to improve the quality of science. As most of us know a lot of Nobel laureates don’t even bother with preparing half decent talks. I have been part of a paper submission of a Nobel laureate to Nature and it was an absolute horrid experience, plagued by never being allowed to actually criticize the person who has the most power in my field. Editors are in fact least affected (even though far from being immune) by politics and that is a good thing. The motivation of the founders of eLife just comes from the fact that a Nature/Science/Cell editor is the only person in the life of a Nobel laureate/Max-Planck director who actually tells them that their latest work might not be as brilliant or solid as they think. Obviously they don’t like that (since the editor is even a “non-scientist”), so why not try to eliminate that one person to make their life even easier? And why not getting complete power back by founding their own “elite” open-access journal?

    I have followed articles in eLife and the average quality so far is far from good (especially the ones by big names). I think having more editors and having the review process more public would be a good way to have science published based on the quality and not by the name of the submitting author.

    • Joshua Schraiber
      Posted December 11, 2013 at 3:44 pm | Permalink

      Interestingly, there was very recently an article in Science Careers arguing almost exactly the opposite of this. It says that “in a knowledge-work context, too much pressure—too much emphasis on reducing wasted time or resources—can cripple knowledge-based projects, in large part by corrupting the environment in which knowledge workers thrive.”

      Essentially, being constantly under pressure to produce high impact publications could, in a very real way, reduce the innovativeness and productivity of scientists because “[k]nowledge workers are motivated by the work itself and the pleasure of doing it, by an internal drive to find answers or to make things. As most readers of this essay surely know from experience, anything that undermines that motivation—pressure to produce, meddling by management, fear of sanctions, anxiety, resentment, even gratuitous performance bonuses—worsens work performance.”

      So it might seem that getting rid of SNC-type journals would in fact reduce this kind of performance anxiety and instead give us back the insights into the natural world that we all think we do science to find out.

    • AMoreno
      Posted December 20, 2013 at 8:13 pm | Permalink

      Wow, so you are saying that the research not published in SNC did not require hundreds of hours of hard work and effort form all people involved?
      I am sure most of the manuscript submitted to SNC are scientifically sound, the issue is the “impact”. A lot of work is considered not of general interest because it is not about model or charismatic organisms. This reinforces the publication of a lot of paper on trendy topics, which in turn become even more trendy. That is the true enemy of originality, as one cannot know in advance if a new scientific endeavor will produce the flashy results that SNC requires. You either get lucky or you just give the editors of SNC what everyone knows they like.
      And just to finish, a lot of the research produced by all of us is either not conclusive or is more complicated than previously thought. We can get it published in small journals or just store it in a hard drive or lab book. That research is hard work as well and could set the bases of future breakthroughs, it deserved to see the light of day.

  16. Ryan Calsbeek
    Posted December 11, 2013 at 3:15 pm | Permalink

    I don’t see how the incentives of open access journals can be considered different from those of SNC. Paying $1500 USD to BMC per publication must certainly raise a few eyebrows, no?

    • anthony zador
      Posted December 11, 2013 at 4:40 pm | Permalink

      I agree. Although I dislike the basic business model of most commercial journals (authors volunteer their time; reviewers volunteer their time; middlemen make the profit), and I commend people who for reasons of principle decline to publish in such journals, the business model seems largely orthogonal to the issue of how CNS vanity journals distort science. In principle there is no reason Plos or elife couldnt evolve into vanity journals.

      Randy Schekman seems to believe that the issue is whether the journals have professional editors (http://malypense.wordpress.com/2012/06/22/elife-my-qa-with-editor-in-chief-randy-schekman/). Again, this seems largely irrelevant to me.

      The core issue is whether the primary filter for a result should be where it’s published. In a dream world we would all publish in journals (like PlosOne) that assess only the competence of the work, not the impact. Then other postpublication mechanisms would evolve to highlight important results.

  17. Spiny Norman
    Posted December 11, 2013 at 5:51 pm | Permalink

    Will wrote: “I expect scientists will respond to data.”

    Plate tectonics wasn’t fully accepted until the old guard was dead.

    That is how paradigm shifts occur.

  18. Spiny Norman
    Posted December 11, 2013 at 5:56 pm | Permalink

    “You will always need an incentive for a person to go out of their way to try to do something exceptional, otherwise nobody would try to set up this new difficult assay, come in on weekends and stay late.”

    That’s the biggest load of horseshit I’ve ever read.

    Good science is not done to get fancy publications. Good science is done to learn how things work.

    If that’s not a person’s motive in doing science, if that’s not the core driver, if she or he would stop tomorrow if the glam journals ssuddenly ceased to exist, I have only this to say: don’t ket the door hit your ass on the way out. And please, don’t come back.

  19. Spiny Norman
    Posted December 11, 2013 at 6:03 pm | Permalink

    “I have followed articles in eLife and the average quality so far is far from good (especially the ones by big names).”

    You won’t really know how good the work is for a decade or more. The real significance of any body of work takes time to evaluate. Work must be replicated and extended. Things that seem good early on often lead to dead ends. Things that seem pedestrian sometimes turn out to be of the utmost importance.

    If you claim that you can evaluate the “quality” of the work in a new journal that’s published, what, a hundred papers (?), based on your reading of some minor fraction of them, you don’t understand much about how science actually progresses.

  20. Vivian
    Posted December 12, 2013 at 10:11 am | Permalink

    I agree that it is important to work on hiring, grants, and tenure – because as long as those decisions are based on where you publish, then it will matter where you publish. But how can we all agree to that?

    This is a discussion we had from the earliest days of PLOS. Journals are empty shells. If we ALL decided not to publish in Nature, it would cease to publish research. Just like that. If we ALL decided to ignore where a paper is published when we make important decisions related to someone’s career and work, then where a paper is published will cease to be important.

    But how do we get to “ALL”? Having a few scientist-author leaders is great, but obviously not enough.

  21. Guy Tanentzapf
    Posted December 12, 2013 at 9:37 pm | Permalink

    “I think a better place to work is on hiring, grants and tenure. If we all commit to NEVER looking at the journal in which a paper appeared when we’re evaluating someone, and if we speak up when anyone else does it.”

    Deal! but only if we also promise to not look at what university or institute someone works in and judge how much Ivy covers the walls of said place; promise to never look where someone did their PhD/Post doc and judge how famous that big shot was; promise to not consider who their friends are, how well connected they are and who they know; and finally promise to just focus on the science and not on the how many followers that person has on twitter or how good they are at self promotion. So yes, science and only science but for everything not just the publications.

    All of those factors are as corrupting and destructive to science as the cult of CNS, and in many way much less egalitarian. Anyone can send a paper to Nature but only a select few can afford to go to Harvard and get into the right lab and get to work in the lab of someone famous.

  22. Martin kreitman
    Posted December 12, 2013 at 10:26 pm | Permalink

    In regards to evaluating a person’s work for tenure, a Chicago Dean once told Russ Lande, “We may count your publications or weigh them but we certainly won’t read them.” Let’s also do away with tenure.

  23. Irun Cohen
    Posted December 14, 2013 at 6:59 am | Permalink

    My experience leads me to suspect that “high impact” publishing, in addition to its other ills, now actually obstructs novel thinking by rewarding the continuity of mainstream ideas. When I started doing immunology research some decades ago, one could gain acceptance to Nature or Science with a small piece that called attention to data that contradicted an accepted idea or that supported a novel concept. Nowadays, SNC editors are disinclined to send for review data that contradict an entrenched view or that provide a new insight into a “closed” issue. Reviewers selected by “high impact” editors, on their part, often reject novel findings for their apparent lack of an “acceptable hypothesis” or an “explanatory mechanism”. Indeed, the safest path to SNC publication is to provide support for an accepted, mainstream idea using advanced or novel technology. Aspiring researchers, as a consequence, often choose to ignore or shelve inconvenient results; alas, science advances by challenging accepted views.

  24. miah
    Posted February 11, 2014 at 2:20 pm | Permalink

    I have to agree with both your comments and some of the comments of others here, especially the comment about not looking at the name of the school on your diploma. Coming from a medium sized school as I did, there can be a lot of pressure to publish in these top journals if you want your career to go anywhere. I mean I could publish five papers in open access journals while in a program no one really pays attention to, but am I going to compete against someone coming out of Harvard, with two publications in science and nature, and a PI with a well known name? Even if you remove the names of the journals, what’s to stop any of those other absolutely meaningless metrics from just becoming more important?

    Better yet, lets say you have succeeded in removing the journal from consideration and you will simply review the work based on its own merit. OK, what does that mean? How do you judge the merit of a paper compared to others in your pile of applicants? Or more to the point, how do you judge the intellectual contributions of the applicant to that paper versus what was just spoon fed to them? Can you really objectively say any more than, well I guess they’re capable of at least doing some of the experiments in this paper. And perhaps understanding the work itself. Do all the committee members even have the background to judge those papers on their merit alone?

    My point is that even if you manage to remove the name of the journal from consideration, in a system already built largely on nepotism and other absurd metrics, what is to stop something equally as absurd from simply taking more of a center stage? And how do a group of people objectively measure the merits of a publication? Frankly at some point I think we’d all be better off if we just admitted we have no idea what makes a good scientist, instead of just largely faking it.

  25. miah
    Posted February 11, 2014 at 2:33 pm | Permalink

    I am glad you mentioned the arguments people tend to make though, about how it’s easy because you’ve already made it, because I think that’s an important point and you dismiss it a bit easily to me, largely just saying it’s unfortunate.

    I’m glad because frankly I think if that’s still an argument people make, that’s something the open access advocates have to at least take part of the blame for. I mean for me, what’s my alternative? I didn’t go to harvard or yale. I’m not working in a lab with buckets of money. If I want to really get ahead in my field, I need to either attach my name to someone who will really open doors, or publish in science, nature, or cell. There really isn’t a third option is there– other than either accepting my career will only ever go so far, or quiting entirely and going off to sell used cars or something. That’s about it. If there was a third option I would take it, because frankly I detest the absurd competition and out-right dickishness of a lot of it, but from where I’m sitting those are my choices. And until there is an alternative, it really is going to be largely relegated to those with the luxury.

  26. Posted March 16, 2014 at 6:35 am | Permalink

    I think it’s a losing strategy to ask people to ignore journal prestige as long as we have no better indicator of the importance of new papers. Instead we need to overcome the culture of secret peer review and evaluate each others’ work openly. Blogs are a good tool for contributing to this cultural transition. We could sign and post our reviews, for example. Web-based technology for more integrated open evaluation is also emerging. Once we’ve built our own independent open evaluation system and that system is perceived to be more reliable than journal prestige, we will be independent of the publishing industry. Of course the publishing industry is working on such tools as well — and perhaps that’s a good thing. But ultimately science needs independent mechanism for evaluating primary research papers.

2 Trackbacks

  • By Ronin Institute on December 13, 2013 at 8:15 am

    [...] important topic, and has already spawned a number of responses, including interesting thoughts from Michael Eisen, Retraction Watch, Luboš Motl, PZ Myers, Junk Science, Scholarly Kitchen, and [...]

  • [...] les mastodontes tels que Nature ou Science. Les avis les plus balancés (exemples en anglais: 1 et 2) n’auront pas manqué de noter que Schekman a cependant déjà un passif de militant (par [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>