Turning peer review into modern-day holy scripture

Suddenly, the esoteric system of peer review has hit the headlines.

The Lancet, a leading British medical journal, has acknowledged that it made a serious error in publishing a study suggesting a link between the MMR vaccine and autism and bowel disease. Earlier this month, a group of leading stem cell researchers wrote an open letter pointing out the systematic abuse of peer review by a small cabal of scientists, whom they accuse of using their position to slow down the publication of the findings of their competitors.

Then there is the scandal surrounding the leaked emails of the Climatic Research Unit at the University of East Anglia (UEA) in England, and the dubious data published by the Intergovernmental Panel on Climate Change (IPCC), which further exposes a worrying trend towards the corruption of peer review.

Peer review is a system that subjects scientific and scholarly work to the scrutiny of other experts in the field. Ideally it ensures that research is only approved or published when it meets the standards of scientific rigour and its findings are sound. At its best, peer review guarantees that it is disinterested science which informs public discussion and debate. When established through peer review, the authority of science helps to clarify disputes and injects into public discussion the latest findings and research. Peer reviewing depends on a community of experts who are competent and committed to impartiality. It depends on the commitment and collaboration of scientists and scholars in a given field.

However, the individuals who constitute a ‘community of experts’ also tend to be preoccupied with their own personal position and status. Often, the colleagues they are reviewing and refereeing are their competitors and sometimes even their bitter rivals. The contradiction between working as a member of an expert community and one’s own personal interests cannot always be satisfactorily resolved.

Unfortunately, even with the best will in the world, peer reviewing is rarely an entirely disinterested process. All too often the system of peer review is infused with vested interests. As many of my colleagues in academia know, peer reviewing is frequently carried out through a kind of mates’ club, between friends and acquaintances, and all too often the question of who gets published and who gets rejected is determined by who you know and where you stand in a particular academic debate.

Peer reviewing cannot remain immune to the preoccupations, agenda and interests of the individuals who carry it out. Even when they have the best intentions, academics and scientists can overlook errors and become blind to the importance of a new but maverick contribution. They are ordinary mortals who have their fair share of prejudices, and are often no less petty or self-centred than other people can sometimes be. Nevertheless, peer reviewing has traditionally, at least, been the most effective way of exercising quality control over the proposals and output of the scholarly and scientific communities.

The experience of the past few months indicates that there are at least three different ways that the system of peer review can be undermined.

First, there is the genuine mistake. One example of this was the failure of the Lancet’s refereeing process to spot the flaws in the study associating the MMR vaccine with autism and bowel disease. Now that the Lancet has retracted this flawed study, questions need to be asked as to whether in this instance the desire to gain publicity for the Lancet influenced the decision to rush into print.

Second, there is the damaging influence of nepotism and professional jealousy. Academics and researchers are all too conscious of how their prestige and career opportunities can be enhanced by getting their work published in a major journal. Sometimes, reviewers regard the research they are refereeing as the work of a competitor and adopt the tactic of either delaying or preventing its publication. This is the accusation made by 14 stem cell researchers in a letter to several major journals in their field. The researchers claim that the peer-review process was corrupted by reviewers who deliberately stalled, and even prevented, the publication of new results so that they or their associates could publish the breakthrough first. They also accused the journals of not doing enough to prevent this stalling from taking place.

The third, and in recent years the most disturbing, threat to the integrity of the peer-review system has been the growing influence of advocacy science. In numerous areas, most notably in climate science, research has become a cause and is increasingly both politicised and moralised. Consequently, in climate research, peer review is sometimes looked upon as a moral project, where decisions are influenced not simply by science but by a higher cause. The scandal surrounding ‘Climategate’ is as much about the abuse of the system of peer review as it is about the rights and wrongs of the various claims made by advocacy researchers in and around the IPCC and the UEA.

Turning peer review into a dogma

The usual problems associated with peer review, as outlined above, have been exacerbated through the transformation of peer review into a form of authorisation. Increasingly, peer review is cited as kind of unquestioned and unquestionable authority for settling what are in fact political disputes. Consequently, the findings of peer review are looked upon, not simply as statements about the quality of research or of a scientific finding, but as the foundation for far-reaching policies that affect everything from the global economy to our individual lifestyles.

Increasingly, peer review has been turned into a quasi-holy institution, which apparently signifies that a certain claim is legitimate or sacred. And from this perspective, voices which lack the authority of peer review are, by definition, illegitimate. Peer review provides a warrant to be heard – those who speak without this warrant deserve only our scorn.

You can almost visualise peer-review dogmatists waving their warrant and demanding that their opponents be silenced. For someone like George Monbiot, the British climate-change alarmist, peer review is the equivalent of a holy scripture. Boasting of his encounter with an opponent, who challenged him to a debate on speed cameras, Monbiot wrote: ‘I accepted and floored him with a simple question.’ Predictably, the question was: ‘Has he published his analysis in a peer-reviewed journal?’

In a world where opponents can be ‘floored’ simply because they lack the authority provided by the ritual of peer review, it is not surprising that there is considerable incentive to manipulate the system of peer review, to bend it to one’s own will and needs. Andrew Dessler, a climate-change researcher, also sought to floor an opponent, who apparently wrote a ‘denier op-ed’ in the Wall Street Journal, by dismissing its value on the grounds that that newspaper is not peer reviewed. Dessler argued that, since ‘the only place’ where this ‘denier’ can present his views is in ‘non-peer-reviewed venues like conferences and press releases’, he is worthy only of censorious contempt.

Climate alarmists do not simply boast of their monopoly over peer-reviewed outlets – they also do their best to call into question peer-reviewed outlets that dare to publish research that challenges any aspect of their moral crusade. When Cambridge University Press published Bjorn Lomborg’s The Skeptical Environmentalist, it faced bitter criticism from campaigners who hinted that something had gone wrong with the publisher’s system of review. Stephen Schneider, a professor in environmental studies, asked why ‘a publisher with so excellent a reputation in natural sciences (it even published the IPCC reports) publish[ed] a polemic under its imprimatur’, and demanded to know if Cambridge University Press had ‘the book completely reviewed?’ It seems that as far as Schneider is concerned, it is simply unthinkable that a publication that questions the prevailing consensus could have been properly reviewed.

The zealous policing of peer review by campaigners is directly encouraged by the IPCC itself. As Reiner Grundman argued in (the peer-reviewed journal) Environmental Politics, the IPCC ‘characterises outside critics as unscientific as they do not publish in peer-reviewed literature’. With so many moral resources invested in the authority of peer review, it is not surprising that some supporters of the IPCC consensus adopt an almost casual attitude towards the violation of academic protocols. The leaked ‘Climategate’ emails show how one UEA scientist, Dr Keith Briffa, wrote to a colleague to ask for help in keeping a paper that he did not like out of an academic journal that he edits. US climate scientist Michael Mann has proposed that a journal should be ostracised for daring to publish a paper criticising his work. ‘I think we have to stop considering Climate Research as a legitimate peer-reviewed journal’, he argued. Phil Jones, the central figure in the Climategate scandal, promised to keep two research papers out of the IPCC report. ‘I will keep them out somehow – even if we have to redefine what the peer-review literature is’, he said.

Another dodgy dossier

Sadly today, there are far too many researchers for whom science has become an instrument for the realisation of a higher cause. As a result they are scientists in name, but moralisers in practice. The manipulative exploitation of peer review is underwritten by a culture where campaigners are permitted to have a cavalier attitude towards facts.

While the IPCC insists that its critics should be judged by the most rigorous standards of peer review, it has a more relaxed attitude towards its own publications. In recent weeks there have been a series of damaging revelations about how conclusions drawn by the IPCC’s 2007 report were based on speculation and anecdotes. So claims made about disappearing mountain ice were cobbled together from information drawn from a student’s dissertation and an article published in a mountaineering magazine. Other claims were based on information from newsletters, press releases and reports produced by environmentalist advocacy groups.

There is a powerful double standard at work here: the IPCC attacks its critics for relying on ‘grey literature’ – that is, non-peer-reviewed literature – and yet it has relied on anecdotes and speculation in its reports. We shouldn’t be too surprised about this double standard, because, fundamentally, the IPCC is not simply concerned with presenting the facts but with interpreting them, giving them meaning, giving them momentum. It continually makes conceptual leaps from facts to meaning, from findings to politics. Of course there is nothing wrong with being in the meaning business, just so long as you are honest about it and do not present yourself as the pure, impartial voice of science.

It shouldn’t be surprising that those involved in the corruption of peer review should also be happy to use anecdotes and speculation as the moral equivalent of hard scientific data. However, it is important to understand that these people fervently believe in their cause and are convinced that, far from deceiving the public, they are preserving and protecting a higher truth. Like the authors of the British government’s dodgy dossier on Iraq, they are convinced they are absolutely right. And it is this sense of righteousness that allows them not to let the absence of a few facts stand in the way of promoting their arguments as either hard intelligence or peer-reviewed science. It was the moral conviction of former US defence secretary Donald Rumsfeld that allowed him to respond to a question about the existence of weapons of mass destruction in Iraq by stating that ‘the absence of evidence is not evidence of absence’. And in a similar manner, the absence of evidence does not deter climate alarmists from practising their art.

The philosophy of the Noble Lie – revealing a ‘higher truth’ with little regard for meaningful facts – allows people to stretch the truth in good conscience. One apologist for the sordid Climategate affair has reminded the public to ‘not forget the context in which many of these emails were sent’. Apparently, ‘this is a saga that goes back to a time before the current political and media concern about climate change’. He reminds us that this was before Al Gore got his Nobel Prize and when ‘well-funded climate sceptics routinely spread disinformation’. From this perspective, the ‘context’ lightens the burden of moral reproach. Climategate is an understandable if not 100 per cent justified response to the ‘context’. Which is precisely how Noble Lies are hatched.

Today, many people can count on the authority enjoyed by climate science to avoid having to engage with the criticisms or concerns of the public. That is why, even when the emperor that is the IPCC is caught without its proper peer reviews, it can still carry on by blaming the little boy for being too sceptical.

Frank Furedi’s latest book, Wasted: Why Education Isn’t Educating, is published by Continuum Press. (Buy this book from Amazon(UK).) This is a longer version of an article first published in the Australian on 20 February.