Papers

Accuracy of information on apparently credible websites: survey of five common health topics

Heinke Kunst, specialist registrar ^a, Diederik Groot, medical student ^b, Pallavi M Latthe, specialist registrar ^c, Manish Latthe, general practitioner ^d, Khalid S Khan, consultant, education resource centre ^c. 

^a Department of Respiratory Medicine, St Mary's Hospital, London W2 1NY, ^b Maastricht University, 6200 MD Maastricht, Netherlands, ^c Birmingham Women's Health Care NHS Trust, Birmingham B15 2TG, ^d Tower Hill Medical Centre, Birmingham B42 1LJ

Correspondence to: K S Khan k.s.khan{at}bham.ac.uk

The internet provides an easily accessible forum to disseminate both accurate and inaccurate health informationso it has the potential to facilitate but also to jeopardise healthcare provision. ^{1 2} Many criteria have been alleged to capture the quality of health websites, ^{3 4} but the validity of these criteria needs to be examined.⁵ The source, currency, and hierarchy of the evidence posted on a website may be used to judge its credibilitythat is, the power of inspiring belief. If these criteria were fulfilled, the contents of the website would be expected to be accurate. We determined whether websites that seem to be credible provide accurate health information.

	Methods and results

Top Methods and results Comment References

We determined the relation between credibility features and accuracy of contents of 121 websites that provided information on five common health topics: chronic obstructive pulmonary disease (23 sites), ankle sprain (36), emergency contraception (32), menorrhagia (9), and female sterilisation (21). These sites were identified either by searching each of the most commonly used engines (such as Altavista, Excite, Hotbot, Infoseek, Lycos, Northern Light, Webcrawler) or by simultaneously consulting them using a meta-search engine, Copernic 4.1 (www.copernic.com/). We selected English language websites whose content provided information about the topics. Website selection and data extractions were performed in duplicate, and agreement between the two assessors was high.

The entire contents of the selected websites were assessed for three credibility features (source, currency, and evidence hierarchy) and accuracy of contents. Source and currency are widely used to assess scientific credibility of a website. ^{3 4} The source of medical information is usually regarded as the main criterion for its credibility; sites should display the source of the information clearly. Currency is shown by websites that display the date of the original document or content posting on the internet, and that of any updates. We looked at the hierarchy of evidence posted on each website, examining whether the levels assigned to various pieces of information were related to their validity or methodological quality. This allows users to assess the strength of the recommendations being made. Our assessment showed that 113/121 (93%) websites described source, 59 (49%) currency, and 22 (18%) evidence hierarchy.

Accuracy of website contents was judged against rigorously developed, peer reviewed, and published guidelines for each of the five health topics (see table on bmj.com). The data on accuracy were extracted as a proportion of guideline statements included in the website's contents and they were converted into three accuracy levels. In level I, more than two thirds of guideline statements were covered (28/121 (24%) websites); in level II, one third to two thirds of guideline statements were covered (43 (35%) websites); in level III, less than one third of guideline statements were covered (50 (41%) websites).

We examined the relation between features of website credibility and level of accuracy of contents by cross tabulation and assessed the strength of association with Kendall's rank correlation, which adjusts for tied ranks in the data. The value of the coefficient (tau b) ranges from 1 to 1. Interpretation is subjective, but values near zero may be taken to indicate no correlation whereas values near 1 indicate a strong correlation. As shown in the table, websites with description of credibility features tended to have higher levels of accuracy of contents, but this relationship was not strong.

	Comment

Top Methods and results Comment References

Our study shows that features of website credibilitysource, currency, and evidence hierarchyhave only slight or at best moderate correlation with accuracy of information in five common health topics. Thus, apparently credible websites may not necessarily provide higher levels of accurate health information.

	Acknowledgments

We thank Julie Morris for statistical input.

KSK conceived and designed the study. HK, DG, PML, ML, and KSK searched, selected, and assessed websites. KSK and DG analysed and interpreted the data and wrote the manuscript. HK, PML, and ML revised the manuscript. KSK is guarantor for the paper.

	Footnotes

Funding: None.

Competing interests: None declared.

A table of sources of websites appears on bmj.com

	References

Top Methods and results Comment References

1.	Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the internet. Caveant lector et vieworlet the reader and buyer beware. JAMA 1997; 277: 1244-1245[CrossRef][ISI][Medline].
2.	Roberts JM, Copeland KL. Clinical websites are currently dangerous to health. Int J Med Inf 2001; 62: 181-187[CrossRef][ISI][Medline].
3.	Kim P, Eng TR, Deering MJ, Maxfield A. Published criteria for evaluating health related websites: review. BMJ 1999; 318: 647-649[Abstract/Free Full Text].
4.	Health Information Technology Institute. Criteria for assessing the quality of health information on the internet. http://hitiweb.mitretek.org/iq/onlycriteria.html (accessed 15 Jan 2002).
5.	Griffiths KM, Christensen H. Quality of web based information on treatment of depression: cross sectional survey. BMJ 2000; 321: 1511-1515[Abstract/Free Full Text].

(Accepted 4 February 2002)