Preventing bad reporting on health research

For anyone with medical training, mainstream media coverage of science can be an uncomfortable read. It is common to find correlational findings misrepresented as denoting causation, for example, or findings in animal studies confidently exaggerated to make claims about treatment for humans. But who is responsible for these misrepresentations?

In the linked paper (doi:10.1136/bmj.g7015) Sumner and colleagues found that much of the exaggeration in mainstream media coverage of health research—statements that went beyond findings in the academic paper—was already present in the press release sent out to journalists by the academic institution itself.1

Sumner and colleagues identified all 462 press releases on health research from 20 leading UK universities over one year. They traced 668 associated news stories and the original academic papers that reported the scientific findings. Finally, they assessed the press releases and the news articles for exaggeration, defined as claims going beyond those in the peer reviewed paper.

Since coding for exaggeration could be subjective, the authors’ structured appraisal focused on three areas: making causal claims from correlational findings in observational data, making inference about humans from studies on other animals, and giving direct advice to readers about behaviour change. This allowed an assessment of where each exaggeration first appeared. If a news story claimed a new treatment for humans, for example, but the study was on mice—and the academic paper made no claim about humans—then did the exaggeration first appear in the press release, or the newspaper article?

Over a third of press releases contained exaggerated advice, causal claims, or inference to humans. When press releases contained exaggeration, 58% to 86% of derived news stories contained similar exaggeration, compared with exaggeration rates of 10% to 18% in news articles when the press releases were not exaggerated. This was an onerous piece of research, with coding done by a large team of students, but the high concordance in exaggeration scores between blinded raters is reassuringly high.

Considerable quantitative research has already been done on the misrepresentation of medical research in mainstream media, although the amount of work funded in this area probably does not represent the true impact of media coverage on health risk behaviour and patients’ informed decision making. The HealthNewsReview website in the United States offers ongoing critical appraisal of mainstream media coverage on treatments and tests. A published summary of its first 500 appraisals2 found that most news articles failed to satisfactorily discuss the quality of the evidence or to quantify the absolute magnitude of benefits and harms. Projects in Canada3 and Australia4 reported similar findings, and an analysis of all coverage for trastuzumab (Herceptin) found uncritically positive reporting.5 In terms of story selection, evidence suggests that the media are more inclined to report exceptional causes of death6 7; that bad news generates more coverage than good news and that observational studies are more likely to be covered than trials8 (perhaps because observational research more often reflects the kinds of lifestyle choices that patients can make themselves). Press releases have also been studied: 58% from US research institutions failed to include caveats about important methodological shortcomings in the research that was being promoted9; and a cohort study of five major medical journals found that lower quality press releases were associated with lower quality news coverage.10

This is not a peripheral matter. Evidence suggests that media coverage can have an effect on the uptake of treatments and services11 12; and even on subsequent academic citations.13 Because of this, it is useful to think about practical positive steps. Improving standards among journalists has long been tried; best practice guidelines already exist for academics, journals,14 and institutional press officers,15 but these are routinely ignored. In addition to these strategies, it might be useful to build on the features of academic journals that improve standards and earn trust in science: accountability, transparency, and feedback.

Accountability is straightforward: all academic press releases should have named authors, including both the press officers involved and the individual named academics from the original academic paper. This would create professional reputational consequences for misrepresenting scientific findings in a press release, which would parallel the risks around misrepresenting science in an academic paper.

Transparency is similarly straightforward. Press releases are a crucial part of communicating science, often more impactful than the paper, but they are often only sent privately to journalists and are rarely linked from academic papers. Instead, press releases should be treated as a part of the scientific publication, linked to the paper, referenced directly from the academic paper being promoted, and presented through existing infrastructure as online data appendices, in full view of peers.

Feedback requires a modest extension of current norms. At present, researchers who exaggerate in an academic paper are publicly corrected—and held to account—in commentaries and letters to the publishing journal, through the process of post-publication peer review. This could be extended. Press releases are a key part of the publication of the science: journals should reflect this and publish commentary and letters about misrepresentations in the press release, just as they publish commentary on the academic paper itself.

Collectively this would produce an information trail and accountability among peers and the public. An immediate—albeit mischievous—opportunity also exists. Sumner and colleagues were good enough to share 462 individual coding sheets online and were generous enough to avoid naming and shaming the worst offenders. A motivated student with a spare afternoon could write the analytical code needed to extract data on those academics and institutions associated with the worst exaggerations and publish their names online, along with details of the transgressions. If funding could be found, then extending this project for a further two years would offer a much larger prize: the discovery of whether an ongoing ranking, prominently presented in public, might change academic behaviour and create an environment where researchers finally act to prevent patients and the public being routinely misled.