Finding and appraising qualitative evidence
Ignoring uncertainties about the effects of treatments has led to avoidable suffering and deaths. To reduce this suffering and premature mortality, treatment uncertainties must be acknowledged and addressed, first by reviewing systematically what is already known, and then in well designed research to reduce continuing uncertainties.
Why do we need fair tests of treatments in health care? Have not doctors, for centuries, ‘done their best’ for their patients? Sadly, there are many examples of doctors and other health professionals harming their patients because treatment decisions were not informed by what we consider now to be reliable evidence about the effects of treatments. With hindsight, health professionals in most if not all spheres of health care have harmed their patients inadvertently, sometimes on a very wide scale (Silverman 2003; Grimes 2007). Indeed, patients themselves have sometimes harmed other patients when, on the basis of untested theories and limited personal experiences, they have encouraged the use of treatments that have turned out to be harmful. The question is not whether we must blame these people, but whether the harmful effects of inadequately tested treatments can be reduced. They can, to a great extent.
Acknowledging that treatments can sometimes do more harm than good is a prerequisite for reducing unintended harm (Gregory 1772; Haygarth 1800; Fordyce 1802; Behring 1893). We then need to be more ready to admit uncertainties about treatment effects, and to promote tests of treatments to adequately reduce uncertainties. Such tests are fair tests.
People have often been harmed because treatments have been based only on theories about how health problems should be treated, without testing how the theories played out in practice.
For example, for centuries people believed the theory that illnesses were caused by ‘humoral imbalances’, and patients were bled and purged, made to vomit and take snuff, in the belief that this would end the supposed imbalances. Still, as long ago as the 17th century, a lone Flemish doctor was impertinent enough to challenge the medical authorities of the time to assess the validity of their theories by proposing a fair test of the results of their unpleasant treatments (Van Helmont 1648).
By the beginning of the 19th century, British military surgeons had begun to show the harmful effects of bloodletting for treating “fevers” (Robertson 1804; Lesassier Hamilton 1816). A few decades later, the practice was also challenged by a Parisian physician (Louis 1835). Yet at the beginning of the 20th century, orthodox practitioners in Boston, USA, who were not using bloodletting to treat pneumonia were still being judged negligent (Silverman 1980). Indeed, Sir William Osler, one of the most influential medical authorities in the world, who was generally cautious about recommending unproven treatments, advised his readers at the end of the 19th century that: “during the last decades we have certainly bled too little. Pneumonia is one of the diseases in which a timely venesection [bleeding] may save life. To be of service it should be done early. In a full-blooded, healthy man with a high fever and bounding pulse the abstraction of from twenty to thirty ounces of blood is in every way beneficial” (Osler 1892).
Although the need to test the validity of theories in practice was recognized by some people at least a millennium ago (Ibn Hindu 10th-11th century), this important principle is still too often ignored. For instance, based on untested theory, Benjamin Spock, the influential American child health expert, informed the readers of his best selling book ‘Baby and Child Care’ that a disadvantage of babies sleeping on their backs was that, if they vomited, they would be more likely to choke. Dr Spock therefore advised his millions of readers to encourage babies to sleep on their tummies (Spock 1966). We now know that this advice, apparently rational in theory, led to the cot (crib) deaths of tens of thousands of infants (Gilbert et al. 2005).
The use of drugs to prevent heart rhythm abnormalities in people having heart attacks provides another example of the dangers of applying untested theory in practice. Because heart rhythm abnormalities are associated with an increased risk of early death after heart attack, the theory was that drugs that reduced rhythm abnormalities would also reduce early deaths. Just because a theory seems reasonable doesn’t mean that it is necessarily right, however. Years after the drugs had been licensed and adopted in practice, it was discovered that they actually increase the risk of sudden death after heart attack. Indeed, it has been estimated that, at the peak of their use in the late 1980s, they may have been killing as many as 70,000 people every year in the United States alone (Moore 1995) – many more than the total number of Americans who died in the Vietnam War.
On the other hand, misplaced confidence in theoretical thinking as a guide to practice has also resulted in some treatments being rejected inappropriately because researchers did not believe that they could work. Theories based on the results of animal research, for example, sometimes correctly predict the results of treatment tests in humans, but this is not always the case (Perel et al. 2007). Based on the results of experiments in rats, some researchers became convinced that there was no point in giving clot-dissolving drugs to patients who had experienced heart attacks more than six hours previously. Had no such patients participated in some of the fair tests of these drugs we would not know that they can benefit from this treatment (Fibrinolytic Therapy Trialists’ Collaborative Group 1994).
Observations in clinical practice or in laboratory and animal research may suggest that particular treatments will or will not benefit patients; but as these and many other examples make clear, it is essential to use fair tests to find out whether, in practice, these treatments do more good than harm, or vice versa.
The text in these essays may be copied and used for non-commercial purposes on condition that explicit acknowledgement is made to The James Lind Library (www.jameslindlibrary.org).
Behring, Boer, Kossel H (1893). Zur Behandlung diphtheriekranker Menschen mit Diphtherieheilserum. Deutsche Medicinische Wochenschrift 17:389-393.
Fibrinolytic Therapy Trialists’ Collaborative Group (1994). Indications for fibrinolytic therapy in suspected acute myocardial infarction: collaborative overview of early mortality and major morbidity results from all randomised trials of more than 1000 patients. Lancet 1994;343:311-322.
Fordyce G (1802). Â A second dissertation on fever. London: J Johnson
Gilbert R, Salanti G, Harden M, See S (2005). Infant sleeping position and the sudden infant death syndrome: systematic review of observational studies and historical review of recommendations from 1940 to 2002. International Journal of Epidemiology 34:874-87.
Gregory J (1772). Lectures on the duties and qualifications of a physician. London: Strahan and Cadell.
Grimes DA (2007). Discovering the need for randomized controlled trials in obstetrics: a personal odyssey. JLL Bulletin: Commentaries on the history of treatment evaluation (www.jameslindlibrary.org).
Haygarth J (1800). Of the imagination, as a cause and as a cure of disorders of the body: exemplified by fictitious tractors, and epidemical convulsions. Bath: R. Crutwell.
Ibn Hindu (10th-11th century CE; 4th-5th century AH). Miftah al-tibb wa-minhaj al-tullab [The key to the science of medicine and the students’ guide].
Lesassier Hamilton A (1816). Dissertatio Medica Inauguralis De Synocho Castrensi (Inaugural medical dissertation on camp fever). Edinburgh: J Ballantyne.
Louis PCA (1835). Recherches sur les effets de la saignèe dans quelques maladies inflammatoires et sur l’action de l’èmètique et des vèsicatoires dans la pneumonie. Paris: Librairie de l’Acadèmie royale de mèdecine.
McPherson K (2004). Where are we now with hormone replacement therapy? BMJ 328:357-358.
Moore TJ (1995). Deadly Medicine. New York: Simon and Schuster.
Osler W (1892). Principles and Practice of Medicine. London: Appleton, p 530.
Perel P, Roberts I, Sena E, Wheble P, Briscoe C, Sandercock S, Macleod M, Mignini LE, Jayaram P, Khan KS (2007). Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ 334:197-200.
Robertson R (1804). Observations on the diseases incident to seamen, 2nd edn. Vol. 1, London: for the author.
Silverman W (1980). In: Chalmers I, McIlwaine G (eds). Perinatal Audit and Surveillance. London: Royal College of Obstetricians and Gynaecologists, 1980:110.
Silverman WA (2003). Personal reflections on lessons learned from randomized trials involving newborn infants, 1951 to 1967. JLL Bulletin: Commentaries on the history of treatment evaluation (www.jameslindlibrary.org).
Spock B (1966). Baby and Child Care. 165th printing. New York: Pocket Books, pp 163-164.
Van Helmont JB (1648). Ortus medicinæ: Id est Initia physicæ inaudita. Progressus medicinae novus, in morborum ultionem, ad vitam longam [The dawn of medicine: That is, the beginning of a new Physic. A new advance in medicine, a victory over disease, to (promote) a long life]. Amsterodami: Apud Ludovicum Elzevirium, pp 526-527.
Read more about the evolution of fair comparisons in the James Lind Library.
Finding and appraising qualitative evidence
For lecture on 3 June 2021
Resources for teaching LR etc
Paul Glasziou 08:48am Sat 03 Nov 2018
Good readable and short essay on why theory and mechanism is insufficient to predict what works. Would work well as a lesson with some examples from news articles making claims.