Wanted—more answers than questions: literature review
BMJ 2001; 323 doi: https://doi.org/10.1136/bmj.323.7327.1462 (Published 22 December 2001) Cite this as: BMJ 2001;323:1462All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Already in my first preclinical year of medical school, as we were
learning to read scientific papers and pondering the problem of "more
questions than answers," the professor in attendance compared our
knowledge to a circle. The area of the circle is the knowledge we have --
the "answers" and the circumference -- the "questions". As our body of
knowledge grows, so too does the circumference, or the number of
questions. As past knowledge becomes "obvious" we forget to include it in
our pool of "things we know" so that the number of questions seems much
greater in comparison to the answers. In fact though the area still
remains pi-R-squared where the circumference grows as 2-pi-R.
Competing interests: No competing interests
Editor
We enjoyed Anthony David's account of his survey of "more questions
than answers" in the Christmas issue [1]. A similar phenomenon has been
commented on elsewhere [2]. Whilst his conclusion "to avoid clichés like
the plague", is attractive, there is a serious underlying issue and one we
face repeatedly during systematic reviews of the management of fractures.
More research is often called for at the back of another well-worn and
dispiriting statement: "There is insufficient evidence to inform
practice".
In our review on the conservative management of fractures of the
distal radius we decided to take another approach [3]. The review covered
31 small and mostly methodologically flawed trials evaluating several
aspects of conservative management. Given this situation, it seemed
unproductive to promote further randomised, though better quality, trials.
Instead, we considered that the way forward in this subject was for the
key players to take stock of available literature and agree on the
important questions and the methods, such as large-scale multi-centre
randomised trials using an agreed set of outcome measures, to get the
answers.
Calling for more research without knowing the questions, especially
from patients' point of view, seemed inappropriate to us.
1. David AS. Wanted - more answers than questions: literature review.
BMJ 2001; 323: 1462-3 (22-29 December).
2. Ruffman R. 'Further studies needed.' Lancet 1991; 338:1026.
3. Handoll HHG, Madhok R. Conservative interventions for treating
distal radial fractures in adults (Cochrane Review). In: The Cochrane
Library, 4, 2001. Oxford: Update Software.
Competing interests: No competing interests
Professor David ought to be congratulated for his contribution to the current issue of the BMJ. His piece is informative (although in the 9 months since his literature search, medline now returns 184 hits for English language titles/abstracts with the words 'more questions than answers'!) and entertaining. I'm sure he would be the first to agree that his piece also raises more questions than answers.
Herein lies his real stroke of genius-his choice of title [1]. A similar literature review performed in a few months time will (apparently) reveal that he has single-handedly increased by 33.3% the number of papers purporting to provide 'more answers than questions' over the last 35 years!
Competing interests: I work in the same academic department as the author!
[1] David AS. Wanted-more answers than questions:literature review. BMJ 2001; 323:1462-3.
Competing interests: No competing interests
Your Scientist Knows More Than You
Anthony David's recent intriguing article in this journal1, concluded
that medical publications confer a "negative stereotype" on medical
science because more often than not, the researchers are left with more
questions than answers. Similarly, the often stated claim that "more
research is needed," or some variant, is problematic since the fruit of
such labors is less fruit and more labor.
While David's conclusions seem to be apt in light of his data, the
"negative stereotype" that he believes is being generated by these
apparently unproductive ventures, does not seem to be shared by the
general public, which continues to place a high regard on science. To
examine this issue from a different perspective I conducted a systematic
literature review of the "popular" view of the achievements of medical
science.
Methods and Results
The data base I used as the source of this study was the Internet
search engine: "Google." Statistical significance was determined by the
binomial test. I first compared the number of "hits" for the two key
phrases David used in his study, namely, "more questions than answers,"
and "More answers than questions." The former resulted in 29,500 "hits"
compared to only 459 for the latter, a 65:1 ratio (95% Confidence
Interval: 2945-2954) which was highly statistically significant (p<.
001) . Among the questions still unanswered were those dealing with the
American television program "Survivor," welfare reform, and archeological
evidence from the Phoenician city of Sidon. Entries offering a more
positive note included those involving sports, the internet, and David's
own formidable study. Perhaps it is not so much that medical science is
left with more questions than answers, but that as a species, we are never
satisfied with what we know or what we don't know.
To address this hypothesis, I conducted a further comparison using
the phrases: "Scientists know," and "Scientists don't know." The former
resulted in 17,200 "hits" compared to 4,250 for the latter (a 4:1 ratio)
(95% CI: 17,074-17,310; p<. 001). This was far different from what
might be expected from the negative inference deduced from David's study
and from my previous inquiry. Although there is an assumption that
coming up with more questions is less preferable than certainty, the
inference from this result is that despite their creation of an ever
greater pool of ignorance, people think scientists know far more than they
don't know.
To examine this question further, I performed a secondary comparison
to determine if scientists were unique in knowing more than they don't
know. The hypothesis I tested was whether the same proportion of knowing
to not knowing scientists (4:1) was also true of the general public. After
all, scientists may be a lofty crew, but they are not the only homo
sapiens on the planet. The comparison I made was between "most people
know" and "most people don't know." While the former outpaced the latter
(60,900 to 31,900), the ratio here was only 2:1 (95%CI: 60,598-61,115;
P<.001). The same ratio was found for "awareness" ("most people are
aware," "most people are not aware"), although we are far less aware than
knowing (7,710 versus 4,960 respectively; 95%CI: 7602-7817; p<.001).
Comment
When it comes to "knowing," scientists fare far better than the
general public. The public rates scientists very near the top of all
professions in terms of ethics2, and appears to perceive scientists as
people who know more than they do, which may be one of the reasons
scientists are held is such high regard.
What does this bode for continued public support for science? One
suggestion that immediately comes to mind from David's essay is that
scientists should, as a song from the past suggests, continue to
"accentuate the positive." David, tongue in cheek, recommends that
researchers "avoid cliches like the plague." I, for one, am reluctant to
throw the baby out with the bath water. It is not cliches in general
that are at fault. It is only those that paint a negative picture that are
to blame. When it comes to "knowing," scientists fare far better than the
general public.
Acknowledgments
I thank my scientist colleague John Hannigan for his insightful
comments and cliches. I edited the former, and removed the later.
Competing Interests
ELA is a working scientist.
References
1. David, AS. Wanted–more answers than questions: literature review.
BMJ 2001; 323:1462-3.
2. FoxNews/Opinion Dynamics poll. New York Post (28 May 2000).
Competing interests: No competing interests