The knowledge system underpinning healthcare is not fit for purpose and must change
BMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h2463 (Published 03 June 2015) Cite this as: BMJ 2015;350:h2463
- Ian Roberts, professor of epidemiology and public health,
- Katharine Ker, lecturer in epidemiology,
- Phil Edwards, statistician,
- Deirdre Beecher, information scientist,
- Daniela Manno, clinical lecturer,
- Emma Sydenham, managing editor
- 1Cochrane Injuries Group, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, UK
- Correspondence to: I Roberts ian.roberts{at}lshtm.ac.uk
- Accepted 25 March 2015
Information on the effectiveness and safety of healthcare should be valid, precise, up to date, clear, and freely available. Currently none of these criteria are fully satisfied, and Cochrane systematic reviews are not the solution. In this article we explain why the knowledge system for healthcare is unfit for purpose and suggest how it should change.
The problem with systematic reviews
Because the medical literature contains a biased sample of trials, systematic reviews that are based on it are also biased.1 Despite decades of exhortation about trial publication, about half of all trials are unpublished, and even the most diligent efforts to synthesise the results from all (or an unbiased sample of) relevant trials are in vain.1 2 Even when trials are identified selective outcome reporting limits their validity.3 4 The literature is awash with low quality, underpowered, single centre trials and the trend is upwards. As Altman recognised over 20 years ago “much poor research arises because researchers feel compelled for career reasons to carry out research that they are ill equipped to perform, and nobody stops them.”5
The systematic review movement is renowned for its obsessive zeal to find every published trial, irrespective of size or quality. These efforts have increased the accessibility of many trials that should never have been conducted. Their citation in reviews perpetuates the problem by contributing to the impact factor of the journals that published them. In the United Kingdom, the funding provided to Cochrane review groups is proportional to the number of trials included in reviews, creating a financial incentive to find and include every trial regardless of its quality.
Most journal editors and systematic …
Log in
Log in using your username and password
Log in through your institution
Subscribe from £173 *
Subscribe and get access to all BMJ articles, and much more.
* For online subscription
Access this article for 1 day for:
£38 / $45 / €42 (excludes VAT)
You can download a PDF version for your personal record.