Preprints don't promote confusion – so taking them away won't fix anything
In response to my Twitter thread against Tom Sheldon’s anti-preprints article in Nature, I received more responses in support of Sheldon’s view than I expected. So I wrote an extended takedown for my blog and, of course, The Wire, pasted below.
In 1969, Franz J. Ingelfinger articulated a now-famous rule named after him in an attempt to keep the New England Journal of Medicine (NEJM), which he edited at the time, in a position to give its readers original and fully vetted research. Ingelfinger stated that the NEJM wouldn’t consider publishing a paper if it had already been publicised before submission or had been submitted to another journal at the same time. The Ingelfinger rule symbolised a journal’s attempt to recognise its true purpose and reorganise its way of functioning to stay true to it.
Would we say this is true of all scientific journals? In fact, what is a scientific journal’s actual purpose? First, it performs peer review, by getting the submissions it receives scrutinised by a panel of independent experts to determine the study’s veracity. Second, the journal publicises research. Third, it creates and maintains a record of the section of the scientific literature it is responsible for. In the historical context, these three functions have been dominant. In a more modern, economic and functional sense, scientific journals are also tasked with making profits, improving their impact metrics and making research more accessible.
As it happens, peer review is no longer the exclusive domain of the journal – nor is it considered to be an infallible institution. Second, journals still play an important part in publicising research, especially via embargoes that create hype, pointing journalists towards papers that they might otherwise not have noticed, as well as preparing and distributing press releases, multimedia assets, etc. Of course, there are still some flaws here. And third, the final responsibility of maintaining the scientific record continues to belong to the journal.
Too much breathing space
Pressures on the first two fronts are forcing journals to stay relevant in newer ways. A big source of such pressure is the availability of preprints – i.e. manuscripts of papers made available by their authors in the public domain before they have been peer-reviewed.
Preprint repositories like arXiv and biorXiv have risen in prominence over the last few years, especially the former. They are run by groups of scientists – like volunteers pruning the garden of Wikipedia – that ensure the formatting and publishing requirements are met, remove questionable manuscripts and generally – as they say – keep things going. Scientific journals typically justify their access cost by claiming that they have to spend it on peer review and printing. Preprints evade this problem because they are free to access online and are not peer-reviewed the way ‘published’ papers are. In turn, the reader who wishes to read the preprint must bear this caveat in mind.
This week, the journal Nature published a (non-scientific) article headlined ‘Preprints could promote confusion and distortion’. Authored by Tom Sheldon, a senior press manager at the Science Media Centre, London, it advanced a strange idea: that bad science was published in the press because journalists did not have “enough time and breathing space” to evaluate it. While Sheldon then urges scientists “to be part of these debates – with their eyes open to how the media works” – the more forceful language elsewhere in the article suggest that preprints should go and that that will fix the problem.
There are numerous questionable judgments embedded here. Principal among them is that embargoes are the best way to do it – and this may seem obvious from the journal’s point of view because an embargo functions like a pair of blinders, keeping a journalist focused on a journal-approved story, and reminding her that she must contact a scientist because a deadline is approaching after which all publications will ‘break’ the story. Of course, embargoes aren’t the norm; the Ingelfinger rule says that the journal will be responsible for ensuring that whatever it publishes is good-to-go.
But with a preprint, there are no deadlines; there are no pointers about which papers are good or bad; and there is no list of people to contact. The journal fears that the journalist will fumble, be overcome with agoraphobia and, as Sheldon writes, “rushing to be the first to do so … end up misleading millions, whether or not that was the intention of the authors.”
It is obvious that the Ingelfinger + embargo way of covering research will produce more legitimate reportage more often – but these rules are not the reasons why the papers are reported the way they are.
High-profile cases in which peer-review failed to disqualify bad and/or wrong papers and papers’ results being included in the scientific canon only for replication studies to completely overturn them later are proof that journals, together with the publishing culture in which they are embedded, aren’t exactly perfect.
Some scientists have even argued that embargoes should be done away with because the hype they create often misrepresents the modesty of the underlying science. Others focused their attention on universities, which often feed on the hype created by embargoes to pump out press releases making far-fetched claims about what the scientists on their payrolls have accomplished.
In turn, journalists have been finding that good journalism is the outcome only when good journalism is also the process. Give a good journalist a preprint to work with and the same level of due diligence will be applied. Plonk a bad journalist in front of an embargoed news releases and a preprint, and you will only get shoddy work both times. It is not as if journalists suspend their fact-checking process when they work with embargoed papers curated by journals and reinstate it when dealing with preprints. A publication that covers science well will quite likely cover other subjects with the same regard and sensitivity not because of the Ingelfinger rule but because of the overall newsroom culture.
Last line of defence
Moreover, an infuriating presumption in the Nature article is that the preprint flows as if by magic from the repository where it was uploaded into the hands of the “millions” now misled by it. Indeed, though it is annoying that the phrasing makes no room for a functional journalist who can step in, write about the paper and arrange for it to be publicised, it is simply frustrating that the journalistic establishment remains invisible to Sheldon’s eye even when we’re talking about an extra-journal agent messing up along the way.
It is the product of this invisibility – rather, a choice to not acknowledge evident work – that suggests to the scientific journal that it must take responsibility for ensuring all that it publishes is good and right. As a pathway to accruing more relevance, this can only be good for the journal; however, it is also a way to accrue more power, so it must not be allowed to happen. This is ultimately why taking preprints away makes no sense: journals must share knowledge, not withhold it.
By taking preprints away from journalists, Sheldon proposes to force us to subsist on journal-fed knowledge – knowledge that is otherwise impossible to access for millions in the developing world, knowledge that is carefully curated according to the journal’s interests and, most of all, pushing the idea that the journal knows what is best for us.
But journals are not the last line of defence, even though they would like to think so; journalists are. That is how journalism is structured, how it functions, how it is managed as a sector, how it is perceived as an industry. If you take control away from journalists to move beyond papers approved by a journal, we lose our ability to question the journal itself.
The only portion of the Nature article that elicits a real need for concern is when Sheldon refers to embargoes as a means of safeguarding novelty for news publishers. He quotes Tom Whipple, science editor of The Times, saying that it is impossible to compete with the BBC because the BBC’s army of reporters are able to pick up on news faster. The alternative, he implies, is to preserve embargoes because they keep the results of a paper new until a given deadline – letting journalists from publishers small and large cover it at the same time.
In fact, if it is reform that we are desperate for, this is the first of three courses of action: to keep removing the barriers instead of making access more equitable. The second is to fix university press releases. The third is to stop interrogating preprints and start questioning publishing practices. For example, is it not curious that both Nature and NEJM, as well as many other ‘prestigious’ titles, rank almost as highly on the impact index as they do on the retraction index?
Update: The following correction was made to the Nature article on July 25 (h/t @kikisandberg). I guess that’s that now.
