New Studies Often Reverse Existing Practices

One of every eight research articles published in a high-impact medical journal presented data from new clinical trials that yielded results contradictory to a current clinical practice, authors of a review concluded.

The results of 16 studies -- 13% of those published in the New England Journal of Medicine in 2009 -- were considered practice reversals because they came from trials that had characteristics superior to those of earlier studies, such as better design, increased statistical power, or more appropriate control groups.

Clinical practices affected by the trials ranged from cancer screening, prenatal care, medical and surgical interventions, and rehabilitation.

"Newer studies, though generally more robust than their predecessors, may not necessarily be correct," Vinay Prasad, MD, of Northwestern University in Chicago, and coauthors acknowledged in a research letter published online in Archives of Internal Medicine.

"However, on average, better-controlled and better-powered studies do provide stronger truth claims. Given the quality of studies published in the New England Journal of Medicine, we believe that the results reported are more likely to be enduring."

The authors characterized reversal as new clinical-trial results that contradict current clinical practice. Recent examples of reversal include use of hormone replacement therapy, class 1C antiarrhythmics, and the pulmonary artery catheters, they noted.

The review has been preceded by other attempts to quantify frequency and nature of reversal in medical research. For example, an analysis of highly cited journal articles found that a third of the articles were followed by other studies that contradicted the initial findings or by studies that showed smaller effects than originally reported (JAMA 2005; 294: 218-228).

"Reversal implies error or harm to patients who underwent the practice in question, during the years it was considered effective," the authors wrote in their introduction. "Reversal also undermines trust in the medical system."

In a new effort to characterize the rate and nature of reversal, Prasad and coauthors reviewed all original articles published in the NEJM during 2009. Of the 212 articles evaluated, 124 (58%) included a claim related to a medical practice.

The 124 articles included in the analysis comprised 89 investigations of new medical practices, 35 evaluations of an existing practice, 91 randomized trials, 19 prospective cohort studies, 13 retrospective cohort studies, and one case-control study.

Prasad and co-authors found that 82 (66%) of the studies yielded positive results, and 42 (33%) ended in negative outcomes. In 61 cases (49%), studies demonstrated the superiority of a new practice compared with an existing practice, 12 (10%) showed that a new practice did not improve on current practice, 16 (13%) upheld the beneficial nature of an existing practice, and 16 (13%) constituted reversal.

The remaining 19 studies were inconclusive.

Examples of reversals identified by the authors included:

• Oral prednisone for preschool children with acute virus-induced wheezing
• Mortality results from a randomized trial of prostate-cancer screening
• Randomized trials of vertebroplasty for osteoporotic fractures
• Revascularization versus medical therapy for renal-artery stenosis

"Confidence in physiologic models as the primary reason to adopt a practice initially was the most common precondition for reversal," the authors wrote.

Details of all 16 reversals are available on the journal website.

Comment