Root Privileges

Ranga Dias

artificial intelligence Bharatiya Janata Party blogging CERN Chandrayaan 2 climate change Covaxin COVID-19 COVID-19 pandemic Donald Trump Elon Musk Facebook fantasy fiction Gaganyaan Higgs boson India-based Neutrino Observatory Indian Council of Medical Research Indian Space Research Organisation ISRO journalism K Sivan Large Hadron Collider NASA nationalism Nobel Prizes novel coronavirus Open Access particle physics peer review Prime Minister Narendra Modi pseudoscience Quantum mechanics Science science communication science fiction science journalism science writing scientific publishing scientific research scientism social media SpaceX The Hindu The Wire Twitter

  • Rescuing superconductivity

    From a paper in Nature Reviews Physics, December 19, 2024:

    One of the forefront fields of modern superconductivity research is that on hydrides at high pressures. Over the past few years, this research has attracted considerable publicity, of which a substantial fraction has been negative. Scientific fraud has been committed and exposed, and arguments continue about specific aspects of data presented in some other papers. Among all the noise that is being generated, one might lose sight of the big-picture question of whether the field is on solid foundations or not, that is, whether high-pressure hydrides host superconductivity at all. Here, we readdress this central issue. We select and critically examine what we identify as six key papers on the topic. We have all spent substantial portions of our careers working on superconductivity, so hope that the conclusions that we reach will carry at least some weight. We also decided to include among our authorship team only people who have never worked directly on hydride superconductivity, so that our examination of the scientific facts can be as impartial as possible. We conclude that it is overwhelmingly probable that the phenomenon of hydride superconductivity is genuine.

    It’s intriguing such an exercise had to be undertaken. It’s yet another reminder that practising science isn’t simply a matter of following the facts. Science is part of the world, not separate from it, and is affected by what others think of it, especially based on perceptions of trustworthiness, self-correctability, and integrity. Self-correctability in particular went out the window the moment the holes in the Dias/Salamat saga became clear, followed by integrity. Imagine discovering a groundbreaking new natural phenomenon: usually such things revitalise fields looking for a breakthrough, but here, the field became marred by a slew of bad papers that shrunk funding opportunities and rendered young researchers trying to enter or already in the field nervous about their future.

    In fact the self-correctability and integrity issues were compounded by the actions of the journals that published the problem papers. Nature and Physical Review Letters both have submissions peer-reviewed. The process of peer review is designed to check whether the data provided match the conclusion provided, not the integrity of the data. However, the data the journals reviewed before publishing the papers was also the data independent experts reviewed to find flaws, consequently leading to the retractions. What explains this? Further, one of the papers, purporting to show superconductivity in LuNH and published in Nature in March 2023, didn’t contain enough evidence to support the conclusion, which the journal’s review missed as well. A Nature news feature reported in September that year:

    Critiques started appearing as soon as the Nature paper was published. One major line of criticism is that the Rochester team didn’t provide enough evidence to show that resistance does go to zero in its material. Dias and his colleagues state in the paper that they removed “small residual resistance” from some of their electrical measurements, but critics argue that it should not be necessary to remove background for these types of measurements, given clean readings of both a sample’s current and voltage. The problem with removing a background, says Sven Friedemann, a physicist at the University of Bristol, UK, is that it implies that the raw data do not go to zero — and therefore don’t show superconductivity.

    The same feature also quoted two scientists saying Nature’s retraction of a carbonaceous sulphur hydride paper in 2022 was “not strong enough”.

    The names of many of the authors of the review should be familiar to people who have been following the Dias/Salamat saga, including Peter Hirschfeld, Steven Kivelson, Andrew Mackenzie, and Subir Sachdev. The review reportedly began with the two possible outcomes — hydrides display superconductivity versus hydrides don’t — being equally probable and concluded in favour of the former after assessing the results reported by multiple groups. While the nominal definition of superconductivity alludes only to the fact that a material’s electrical resistance drops to zero, condensed-matter physicists perform four tests looking for different features. One is zero electrical resistance; another is that the material’s magnetisation varies through a particular pattern. On this count the reviewers assessed data from only one group, that of Mikhail Eremets & co. in 2022.

    Yet another familiar name, Jorge Hirsch, has already expressed his disapproval towards the review. “I was surprised and disappointed to see this. I speculate [they wrote] it because hydrides being superconductors would establish the validity of BCS theory, in which they firmly believe,” he told Physics. A bit of relevant background here is that Hirsch is a detractor of the popular BCS theory of superconductivity and a proponent of his own holes theory. While Physics writes that he’s already flagged some problems with the Eremets et al. paper, it doesn’t say the Eremets et al. paper raised significant doubts about the validity of his holes theory — which is to say both the study and Hirsch’s idea could be flawed rather than the study alone. Overall, if science is to remain trustworthy, scientists need to undertake exercises like this, conducting — while being seen to be conducting — impartial reviews of the prevailing evidence and considering whether it makes sense to continue working in fields beleaguered by the influence of some dishonest exponents.

    I only hope reviewers will also take a closer look at the roles journals and their misguided incentives — and the still largely blind trust the global scientific community places in them — play in sustaining scandals in science.

  • The journal’s part in a retraction

    This is another Ranga Dias and superconductivity post, so please avert your gaze if you’re tired of it already.

    According to a September 27 report in Science, the journal Nature plans to retract the latest Dias et al. paper, published in March 2023, claiming to have found evidence of near-room-temperature superconductivity in an unusual material, nitrogen-doped lutetium hydride (N-LuH). The heart of the matter seems to be, per Science, a plot showing a drop in N-LuH’s electric resistance below a particular temperature – a famous sign of superconductivity.

    Dias (University of Rochester) and Ashkan Salamat (University of Nevada, Las Vegas), the other lead investigator in the study, measured the resistance in a noisy setting and then subtracted the noise – or what they claimed to be the noise. The problem is apparently that the subtracted plot in the published paper and the plot put together using raw data submitted by Dias and Salamat to Nature are different; the latter doesn’t show the resistance dropping to zero. Meaning that together with the noise, the paper’s authors subtracted some other information as well, and whatever was left behind suggested N-LuH had become superconducting.

    A little more than a month ago, Physical Review Letters officially retracted another paper of a study led by Dias and Salamat after publishing it last year – and notably after a similar dispute (and on both occasions Dias was opposed to having the papers retracted). But the narrative was more dramatic then, with Physical Review Letters accusing Salamat of obstructing its investigation by supplying some other data as the raw data for its independent probe.

    Then again, even before Science‘s report, other scientists in the same field had said that they weren’t bothering with replicating the data in the N-LuH paper because they had already wasted time trying to replicate Dias’s previous work, in vain.

    Now, in the last year alone, three of Dias’s superconductivity-related papers have been retracted. But as on previous occasions, the new report also raises questions about Nature‘s pre-publication peer-review process. To quote Science:

    In response to [James Hamlin and Brad Ramshaw’s critique of the subtracted plot], Nature initiated a post-publication review process, soliciting feedback from four independent experts. In documents obtained by Science, all four referees expressed strong concerns about the credibility of the data. ‘I fail to understand why the authors … are not willing or able to provide clear and timely responses,’ wrote one of the anonymous referees. ‘Without such responses the credibility of the published results are in question.’ A second referee went further, writing: ‘I strongly recommend that the article by R. Dias and A. Salamat be retracted.’

    What was the difference between this review process and the one that happened before the paper was published, in which Nature‘s editors would have written to independent experts asking them for their opinions on the submitted manuscript? Why didn’t they catch the problem with the electrical resistance plot?

    One possible explanation is the sampling problem: when writing an article as a science journalist, the views expressed in the article will be a function of the scientists that I have sampled from within the scientific community. In order to obtain the consensus view, I need to sample a sufficiently large number of scientists (or a small number of representative scientists, such as those who I know are in touch with the pulse of the community). Otherwise, there’s a nontrivial risk of some view in my article being over- or under-represented.

    Similarly, during its pre-publication peer-review process, did Nature not sample the right set of reviewers? I’m unable to think of other explanations because the sampling problem accounts for many alternatives. Hamlin and Ramshaw also didn’t necessarily have access to more data than Dias et al. submitted to Nature because their criticism emerged in May 2023 itself, and was based on the published paper. Nature also hasn’t disclosed the pre-publication reviewers’ reports nor explained if there were any differences between its sampling process in the pre- and post-publication phases.

    So short of there being a good explanation, as much as we have a scientist who’s seemingly been crying wolf about room-temperature superconductivity, we also have a journal whose peer-review process produced, on two separate occasions, two different results. Unless it can clarify why this isn’t so, Nature is also to blame for the paper’s fate.

  • Scientists’ conduct affects science

    Nature News has published an excellent feature by Edwin Cartlidge on the “wall of scepticism” that arose in response to the latest superconductivity claim from Ranga Dias et al., purportedly in a compound called nitrogen-doped lutetium hydride. It seems the new paper has earned a note of concern as well, after various independent research groups failed to replicate the results. Dias & co. had had another paper, claiming superconductivity in a different material, retracted in October 2022, two years after its publication. All these facts together raise a few implications about the popular imagination of science.

    First, the new paper was published by Nature, a peer-reviewed journal. And Jorge Hirsch of the University of California, San Diego, told Nature News “that editors should have first resolved the question about the provenance of the raw data in the retracted 2020 Nature article before even considering the 2023 paper”. So the note reaffirms the role of peer-review being limited to checking whether the information presented in a paper is consistent with the paper’s conclusions, and not checking whether it is well-founded and has integrity in and of itself.

    Second, from Nature News:

    “Researchers from four other groups, meanwhile, told Nature’s news team that they had abandoned their own attempts to replicate the work or hadn’t even tried. Eremets said that he wasted time on the CSH work, so didn’t bother with LuNH. ‘I just ignored it,’ he says.”

    An amusing illustration, I think, that speaks against science’s claims to being impartial, etc. In a perfectly objective world, Dias et al.’s previous work shouldn’t have mattered to other scientists, who should have endeavoured to verify the claims in the new paper anew, given that it’s a fairly sensational claim and because it was published in a ‘prestigious’ peer-reviewed journal. But, as Eremets said, “the synthesis protocol wasn’t clear in the paper and Dias didn’t help to clarify it”.

    The reciprocal is also true: Dias chose to share samples of nitrogen-doped lutetium hydride that his team had prepared only with Russell Hemley, who studies material chemistry at the University of Illinois, Chicago, (and some other groups that he refused to name) – and that Hemley is one of the researchers who hasn’t criticised Dias’s findings. Hemley is also not an independent researcher; he and Dias worked together on the work in the 2020 paper that was later retracted. Dias should ideally have shared the samples with everyone. But scientists’ social conduct does matter, influencing decisions about how other scientists believe they should respond.

    Speaking of which: Nature (the journal) on the other hand doesn’t look at past work and attendant misgivings when judging each paper. From Nature News (emphasis added):

    The editor’s note added to the 2023 paper on 1 September, saying that the reliability of data are in question, adds that “appropriate editorial action will be taken once this matter is resolved.” Karl Ziemelis, Nature’s chief applied- and physical-sciences editor, based in London, says that he and his colleagues are “assessing concerns” about the paper, and adds: “Owing to the confidentiality of the peer-review process we cannot discuss specific details of what transpired.” As for the 2020 paper, Ziemelis explains that they decided not to look into the origin of the data once they had established problems with the data processing and then retracted the research. “Our broader investigation of that work ceased at that point,” he says. Ziemelis adds that “all submitted manuscripts are considered independently on the basis of the quality and timeliness of their science”.

    The refusal to share samples echoes an unusual decision by the journal Physical Review B to publish a paper authored by researchers at Microsoft, in which they reported discovery a Majorana zero mode – an elusive particle (in a manner of speaking) that could lead the way to building quantum ‘supercomputers’. However, it seems the team withheld some information that independent researchers could have used to validate the findings, presumably because it’s intellectual property. Rice University physics professor Douglas Natelson wrote on his blog:

    The rationale is that the community is better served by getting this result into the peer-reviewed literature now even if all of the details aren’t going to be made available publicly until the end of 2024. I don’t get why the researchers didn’t just wait to publish, if they are so worried about those details being available.

    Take all of these facts and opinions together and ask yourself: what then is the scientific literature? It probably contains many papers that have cleared peer-review but whose results won’t replicate. Some papers may or may not replicate but we’ll never know for a couple years. It also doesn’t contain replication studies that might have been there if the replicators and the original research group were on amicable terms. What also do these facts and view imply for the popular conception of science?

    Every day, I encounter two broad kinds of critical imaginations of science. One has emerged from the practitioners of science, and those studying its philosophy, history, sociology, etc. These individuals have debated the notions presented above to varying degrees. But there is also a class of people in India that wields science as an antidote to what it claims is the state’s collusion with pseudoscience, and such collusion as displacing what is apparently science’s rightful place in the Indian society-state: as the best and sole arbiter of facts and knowledge. This science is apparently a unified whole, objective, self-correcting, evidence-based, and anti-faith. I imagine this science needs to have these characteristics in order to effectively challenge, in the courts of public opinion, the government’s oft-mistaken claims.

    At the same time, the ongoing Dias et al. saga reminds us that any ‘science’ imprisoned by these assumptions would dismiss the events and forces that would actually help it grow – such as incentivising good-faith actions, acknowledging the labour required to keep science honest and reflexive, discussing issues resulting from the cultural preferences of its exponents, paying attention to social relationships, heeding concerns about the effects of one’s work and conduct on the field, etc. In the words of Paul Feyerabend (Against Method, third ed., 1993): “Science is neither a single tradition, nor the best tradition there is, except for people who have become accustomed to its presence, its benefits and its disadvantages.”