Negative and spurious results
I was thrilled to learn of the Journal of Negative Results in BioMedicine in Wednesday’s Science Journal, Sharon Begley’s regular column in the Wall Street Journal. How cool is it, how overdue is it, that a peer-reviewed journal is dedicated to publishing negative studies.
The peer review will help keep out the poorly designed studies, so the public will be left with a repository of reliably negative results, balancing out the attention-grabbing (and usually positive) content of the currently top-ranked biomedical journals. Let’s support this effort. Short of the push for universal open access to all published research–I’m still holding my breath–I can’t think of an effort in the publishing world more important to the advancement of scientific knowledge. Congrats and thanks to BioMed Central for publishing this journal.
If you read the WSJ article, you’ll also see mention of the Journal of Spurious Correlations. Unfortunately, it’s dedicated to the social sciences, so us biomedical science nerds won’t be reading much of this one. That’s too bad, but I hold out hope that a publisher will start a journal for clinical-science spurious correlations, which they can title…um…The Journal of Observational Studies. Just kidding! Well-designed observational studies can be of value if interpreted and used properly (see for some examples: Concato et al and Benson et al in NEJM 2000).
The inaugural issue of the Clinical Journal of Spurious Correlations could re-publish some of the literature’s classical oops of observational studies. Here are a few of my favorites:
Postmenopausal Estrogen and Progestin Use and the Risk of Cardiovascular Disease (NEJM 1996). The classic spurious correlation between use of estrogen/progestin hormone replacement therapy and the risk of coronary heart disease courtesy of the Nurse’s Health Study. The authors were well aware of potential biases affecting this study, including confounding by indication. Neverthless, these findings contributed in a big way to recommendations made by key physician groups, notably the American College of Physicians, that nearly all postmenopausal women should be offered postmenopausal hormone replacement. The eventual overthrow of this view by randomized trials–HERS and WHI–has opened many eyes to the dangers of relying on observational clinical studies as sufficient evidence to support practice guidelines.
Ventricular depolarizations (VPDs) and mortality post-MI. Several observational studies indicated that VPDs post-MI were correlated with the rate of mortality, a significant increase in risk of death was suggested. The CAST trial, preliminary results of which were published hastily in 1989, was designed to test the hypothesis that suppressing VPDs post-MI would reduce mortality. As many of you no doubt know, CAST, a randomized controlled trial, demonstrated that suppression of VPD with anti-arrhythmia drugs post-MI increased mortality substantially (2.5x to 3.5x). Two big take-homes from CAST were that biomarkers are not disease surrogates unless thoroughly validated, and correlation not only does not establish causation it can be downright misleading.
Coffee and Cancer of the Pancreas (NEJM 1981). An oldy but goody. Three or more cups of day increased the risk of pancreatic cancer by nearly three-fold over no coffee consumption. But the study wasn’t designed to examine the risk of pancreatic cancer with coffee consumption! It was an exploratory analysis. This study has since been widely taught as an example of the dangers of clinical-study fishing expeditions, particularly risky in observational studies.
Reserpine and breast cancer. In the mid-1970’s a series of smallish case-control studies showed what appeared to a modestly increased irsk of breast cancer from use of reserpine (a plant-derived indole alkaloid, which at the time was one of very few choices for treating outpatient hypertension). It wasn’t until nearly a decade later that the association was regarded as erroneous by opinion leaders (see, for example, Horwitz and Feinstein 1985). The reason for the erroneous association: exclusion bias. Most of the studies demonstrating a relationship between reserpine and breast had excluded patients with a history of cardiovascular disease. Since these exclusions weren’t equally balanced among cases and controls, it led to a spurious correlation. Although by the mid-80’s the erroneous conclusions were widely appreciated, I can tell you from personal experience that reserpine jaundice remained in the eyes of practitioners familiar with the early literature long afterwards.
