Medical research often becomes news. But sometimes the news is made to appear
more definitive and dramatic than the research warrants. This article dissects a recent health news story to highlight some common study interpretation problems we see as physician researchers and show how the research community, medical journals and the media can do better.

Raising doubts about the safety of a widely used drug like naproxen, also known as Aleve, is big health news. Add this to recently raised concerns about other drugs in the same broad category -- nonsteroidal anti-inflammatory drugs (NSAIDs) such as Vioxx, pulled off the market this fall, and Celebrex -- and the news is big indeed.

So it is no surprise that the government's decision in December to halt the Alzheimer's Disease Anti-inflammatory Prevention Trial (ADAPT) because of safety concerns about naproxen received intense media coverage. In the three days following the announcement by the National Institutes of Health (NIH), the story appeared on the front pages of all 10 top-circulation U.S. newspapers and led numerous national TV and radio newscasts.

Front-page headlines included: "Heart Risk Seen in Naproxen" (Wall Street Journal), "Tough Choice: Pain or Risk?" (USA Today), "Patients, Doctors Agonize Over Risks of Painkiller" (Los Angeles Times), "Study Links a Fourth Painkiller to an Increase in Heart Problems" (New York Times) and "Warnings About Medications' Risks Add Worry to Pain" (The Washington Post).

The headlines and the stories' prominent play overstated the nature of the evidence, inviting their audiences to believe that taking naproxen as directed for pain had been shown to be dangerous. As we explain, this may not be the case.

But the media were not the sole source of the problem. The Food and Drug Administration (FDA) and NIH played roles as well.

What Happened With the Study?

ADAPT was a large, federally funded study designed to investigate whether Celebrex or naproxen could prevent Alzheimer's disease. In January 2001 researchers began enrolling more than 2,400 healthy men and women age 70 and older who showed no signs of Alzheimer's but were considered at increased risk because of a family history of the disease.

Study participants were randomly assigned to one of three groups: One took Celebrex daily, another took naproxen, while the third took a placebo. The investigators planned to follow participants for about seven years to see how many people in each group developed Alzheimer's disease.

As in most major research studies, a committee was charged with periodically monitoring study results. It was the committee's responsibility to decide whether the study should be stopped early -- either because of the occurrence of unforeseen risks or the appearance of a dramatic treatment benefit.

This data safety monitoring committee met early in December 2004 and, according to news reports, reviewed some data raising the possibility that naproxen increased cardiovascular or cerebrovascular risk that is, risk of conditions involving the heart or brain and related blood vessels. But given the small size of the differences observed between the naproxen users and the placebo users and the possibility that they might be just a statistical fluke, the committee did not recommend stopping the study.

Later in December, spurred by growing concerns raised by other studies about risks associated with Celebrex, NIH also reviewed the ADAPT study data.

NIH reached a different conclusion: The study should be stopped. This dramatic step was taken, according to an NIH news release, to protect the safety of participants because the data "indicated an apparent increase in cardiovascular and cerebrovascular events among the participants taking naproxen when compared with those on placebo."

How Risky Was Naproxen?

To make sense of the NIH announcement, readers need answers to two basic questions:

* What are the "events" that occurred more frequently among naproxen users?

* How much more often did these events occur?

Both the official NIH announcement and a related press release by the FDA were remarkably sparse in details and failed to provide answers to either of these questions. As of this writing, virtually no one outside NIH, the ADAPT study team and possibly the FDA knows which specific events prompted the announcement or how often these events occurred.

* Risk of what? NIH halted the ADAPT study because more "cardiovascular and cerebrovascular events" occurred among patients taking naproxen than those taking placebo.

But what does that term mean? Unfortunately, the definition of the term was never released.

"Cardiovascular and cerebrovascular events" could refer to a slew of outcomes varying widely in importance. At one extreme, such "events" might refer to death from stroke or heart attack. But they could also refer to nonfatal heart attacks and strokes, some serious, some mild. Or episodes of chest pain. Or mini-strokes -- known medically as transient ischemic attacks which are generally mild and short-lived, and leave no permanent effects. Or even changes on an EKG test or a finding on an MRI scan that never produced symptoms.

All we know is that the events in question represent some combination of such incidents and perhaps others as well. The point is that not all events are equally important.

* How big was the risk? Not only were the "cardiovascular and cerebrovascular events" not defined, the number of such events was not released. Assessing risk in the absence of such information is difficult. (See "Research Basics: Quantifying Medical Risk," at right.) The announcement did not even report the relative frequency of the events that is, did the events occur twice or three times or four times as often in the naproxen group?

But thanks to the persistence of journalists, a few relevant numbers have emerged. Reporters have quoted study investigators as saying that there were 70 events altogether; and the occurrence of events among naproxen patients was 50 percent higher than among those taking placebo.

Based on these numbers and the number of patients in each study group a figure available on an ADAPT study Web site (www.jhucct.com/adapt/pdf  % 20documents/factsht.pdf) -- it is possible to calculate the likely magnitude of the naproxen risk. Assuming that Celebrex posed no increased risk to patients (the NIH announcement stated that "no significant increase in risk for Celebrex was found in this trial"), we calculate that over about three years, cardiovascular and cerebrovascular events occurred in 3.7 percent of patients taking naproxen, compared with 2.5 percent of patients taking placebo. (See "What Was the Likely Risk of Naproxen in This Study?"
below.) If the figures in the news reports are correct (that is: 70 events; 50 percent higher risk with naproxen; Celebrex and placebo have the same risk), no other mathematical solution is possible.

You don't have to take our word for it: You can do the math yourself, following the steps we outline atwww.vaoutcomes.org/washpost.php . Be warned: If the word "algebra" triggers long-forgotten high school nightmares, you may want to enlist your son or daughter for the task. Although the math involves using algebraic equations to solve for three unknowns, it is actually straightforward.

Similarly, we can compute what the worst-case scenario would look like. Assuming (absurdly) that there were no cardiovascular and cerebrovascular events in the Celebrex group (that is, Celebrex reduced the risk to zero in the study), the events occurred in 5 percent of naproxen patients compared with 3.3 percent of those assigned to placebo. To test this or other scenarios, try the interactive calculator at www.vaoutcomes.org/washpost.php .

These calculations suggest that naproxen could pose a risk, but that risk would affect less than two people per hundred over about three years. And the risk must be considered in light of who the study participants were patients age 70 and older. Because the chance that a person will experience a stroke or heart attack is strongly related to age, any added risk for younger people posed by naproxen -- if it exists -- would likely be considerably lower.

The Bottom Line

Although the precise risk of naproxen in the ADAPT study is hard to know -- because we don't know what constitutes a "cardiovascular or cerebrovascular event" -- it is possible to deduce from the information available that the size of the risk is, at most, modest.

In addition, the news from ADAPT conflicts with other credible studies that have found either no increased cardiovascular risk with naproxen or even some evidence that it actually protects the heart.

But more concerning is the nature of the NIH announcement. It failed to distinguish between naproxen users in the study -- who were taking the medication only in hopes of preventing Alzheimer's disease -- and naproxen users in the general public who take the medication because of its demonstrated effectiveness in treating musculoskeletal pain. For the study volunteers, the combination of an unknown benefit and even a small possible risk may well have been enough to justify halting the trial. We say "possible risk" because the number and type of events have yet to be verified and released by the study investigators.

But for people choosing to take naproxen for pain relief, the balance between risk and benefit is very different. Here, the benefit is known -- and directly experienced by the user -- while the risk is less certain (because many of these people are younger or are taking naproxen only intermittently).

One thing is clear. No one should have to guess what is going on.

The NIH and FDA announcements should have communicated the basic information needed to understand the nature and magnitude of the risk (if any) posed by naproxen. If it was too soon to release the answers to the two basic questions -- risk of what? and how big is the risk? -- it was too soon to frighten the public.*

Steven Woloshin, Lisa Schwartz and H. Gilbert Welch are physician researchers in the VA Outcomes Group in White River Junction, Vt., and faculty members at Dartmouth Medical School. They conduct regular seminars on how to interpret medical studies. The views expressed do not necessarily represent the views of the Department of Veterans Affairs or the United States Government.