Unclear science

Scientific papers frequently have a line to this effect, often in the same syntax: “While we know X, why Y happens is unclear.” The papers then proceed to elucidate Y. Sometimes a paper comes along that makes you ask, “Unclear according to whom?”.

A few weeks ago, there was a dubious paper in Nature Communications in which a scholar claimed to have analysed the effects of fasting due to Ramadan on the decisions of Muslim judges in India and Pakistan. The paper had so many shortcomings as to make its findings inapplicable in the real world, chief among them that there was no way to verify whether a judge identified as Muslim (by their name) was observing the rituals of Ramadan. Yet a journal that many look up to* published the paper – and its claim that there was a tendency for such judges to be lenient – opening the door for right-wing maniacs in India to claim Nature‘s approval for their Islamophobic views.

We can’t ask that such studies not be published at all. Doing so is the authors’ prerogative and an exercise of academic freedom. The problems are that academia often seems to forget that it doesn’t operate in a vacuum and that the wider socio-political environment has tended to interpret new information in twisted ways. The studies should respond to these environs rather than presuming to be above it all. Academic freedom begins and ends within the academy walls. Scientific papers are written within but they will always be read outside.

For one, consider the value- or judgment-neutral manner in which the results of scientific studies are presented. Are we stopping to consider if that can be a problem? We should. It would be reckless of a scientific paper with potential to cause harm, especially social harm, to not have an ‘interpretation railroad’ baked into the text, and not just in the “Discussion” section, to discourage a reader from applying the results in certain contexts for specific reasons. A paper that doesn’t contemplate its limitations is not a responsible paper.

This does not – and cannot – apply to most papers, but in some cases the potential is obvious, such as a paper that considers the effect of Ramadan fasting on the pronouncements of Muslim judges in half-baked fashion.

* That Nature Communications isn’t the same as Nature may be obvious to scientists; among the people at large, the ‘prestige’ carries over almost entirely.