They call it “vitamin I.” Among runners of ultra-long-distance races, ibuprofen use is so common that when scientist David Nieman tried to study the drug’s use at the Western States Endurance Run in California’s Sierra Nevada mountains he could hardly find participants willing to run the grueling 100-mile race without it.
Nieman, director of the Human Performance Lab at Appalachian State University, eventually did recruit the subjects he needed for the study, comparing pain and inflammation in runners who took ibuprofen during the race with those who didn’t, and the results were unequivocal. Ibuprofen failed to reduce muscle pain or soreness, and blood tests revealed that ibuprofen takers actually experienced greater levels of inflammation than those who eschewed the drug. “There is absolutely no reason for runners to be using ibuprofen,” Nieman says.
The following year, Nieman returned to the Western States race and presented his findings to runners. Afterward, he asked whether his study results would change their habits. The answer was a resounding no. “They really, really think it’s helping,” Nieman says. “Even in the face of data showing that it doesn’t help, they still use it.”
Nieman’s tale is no anomaly. A surprising number of medical practices have never been rigorously tested to find out if they really work. Even where evidence points to the most effective treatment for a particular condition, the information is not always put into practice. “The First National Report Card on Quality of Health Care in America,” published by the Rand Corporation in 2006, found that, overall, Americans received only about half of the care recommended by national guidelines.
A $1.1 billion provision in the federal stimulus package aims to address the issue by providing funds for comparative effectiveness research to find the most effective treatments for common conditions. But these efforts are bound to face resistance when they challenge existing beliefs. As Nieman and countless other researchers have learned, new evidence often meets with dismay or even outrage when it shifts recommendations away from popular practices or debunks widely held beliefs. For evidence-based medicine to succeed, its practitioners must learn to present evidence in a way that resonates.
Or, to borrow a phrase from politics, it’s not the evidence, stupid — it’s the narrative.
By walking into the race forum convinced that runners would alter their ibuprofen habits based on his new evidence, Nieman fell prey to what University of California, Berkeley, social psychologist Robert J. MacCoun calls the “truth wins” assumption — the idea that when someone correctly states the truth, it will be universally recognized. Ibuprofen use during a long-distance running event presents serious risks, including gastrointestinal bleeding and a condition called rhabdomyolysis, which can lead to acute kidney failure. If runners were exposing themselves to those risks without any reasonable expectation of benefit, Nieman assumed they’d want to know.
Instead, he butted up against a phenomenon that philosophers have dubbed naive realism. “It’s the idea that whatever I believe, I believe it simply because it’s true,” MacCoun says. Ultra runners honestly believe that ibuprofen reduces inflammation in the joints and muscles, and allows them to run with less pain. This explanatory story about the drug is known as a “mental model” — a conceptual framework and mental representation about how something works that helps people make sense of the world. Once a mental model is in place, the mind tends to force new information to fit within it.
“There’s this common assumption that we’re just going to educate people about the facts, and then they’re going to make use of them,” says Brendan Nyhan, a health policy researcher and political scientist at the University of Michigan. “But that’s not how people process information — they process it through their existing beliefs, and it’s hard to override those beliefs.”
Several years ago, I interviewed a runner who was hospitalized for a severe case of rhabdomyolysis after taking ibuprofen during an ultramarathon. Despite her experience, attributable in part to the 12 ibuprofen pills she popped during the 24-hour run, she continues to take ibuprofen while racing, albeit in lower doses. “Ibuprofen absolutely does work for me both in terms of pain management and decreasing joint inflammation,” she said.
Nieman’s results contradict these beliefs, and other studies show that using ibuprofen and other nonsteroidal anti-inflammatory drugs prior to exercise may actually impede tissue repair and delay the healing of bone, ligament, muscle and tendon injuries. And yet the belief that NSAIDs should help athletes perform with less pain is so ingrained that a quarter of the athletes competing at the Sydney Olympics in 2000 reported using them. The notion that a medication belonging to a class of drugs whose very name includes the word “anti-inflammatory” could actually increase inflammation strikes many runners as not just improbable, but impossible — even though that’s exactly what Nieman’s study showed via quantifiable blood markers.
But when facts contradict a strongly held belief, they’re unlikely to be accepted without a fight. “If a researcher produces a finding that confirms what I already believe, then of course it’s correct,” MacCoun says. “Conversely, when we encounter a finding we don’t like, we have a need to explain it away.”
In a series of experiments, MacCoun described fictitious studies on gun control, the death penalty and medical marijuana to volunteers. When the study results supported the volunteers’ own views on the issue, they considered the study unbiased, but when the results contradicted their existing views, they were quick to dismiss them. “If you favor gun control, and I show you a study that suggests that gun control doesn’t save lives, you think, ‘Well that researcher must be conservative and work at some kind of right-wing think tank,'” MacCoun says.
In a classic 1977 experiment, researchers asked experts to evaluate a technical manuscript. Except for the results section, all versions of the paper were identical. Reviewers not only gave the paper higher marks when it confirmed their previous views on a technical issue in their field, they were more apt to detect an inadvertent typo in the manuscript when the results contradicted their pre-existing beliefs. Studies have confirmed it again and again: We easily accept results we like and nitpick the evidence that we don’t.
The U.S. Preventive Services Task Force is an independent board of experts convened to come up with evidence-based guidelines for medical practice, without considering cost. Last fall, after poring over years of studies, the task force released new mammography recommendations. Previous guidelines had recommended yearly mammograms for women age 40 and older, but the new guidelines instead called for women age 40 to 49 to discuss the benefits and risks of mammography with their doctors to decide whether a mammogram made sense.
This recommendation, along with the call for mammograms in women age 50 and older to be done every two years, rather than annually, seemed like a radical change to many observers. Oncologist Marisa C. Weiss, founder of Breastcancer.org, called the guidelines “a huge step backwards.” If the new guidelines are adopted, “Countless American women may die needlessly from breast cancer,” the American College of Radiology said.
“We got letters saying we have blood on our hands,” says Barbara Brenner, a breast cancer survivor and executive director of the San Francisco advocacy group Breast Cancer Action, which joined several other advocacy groups in backing the new recommendations. Brenner says the new guidelines strike a reasonable balance between mammography’s risks and benefits.
Yet Brenner was not surprised by the outcry. For years, women were taught the necessity of early detection for breast cancer based on the notion that breast cancer is a relentlessly progressive disease that will inevitably kill you if you don’t remove it in time. That story about breast cancer — call it the “relentless progression” mind model — is easy to grasp, makes intuitive sense and offers a measure of comfort: Every cancer is curable as long as you catch it in time.
But it turns out that this mental model of breast cancer is wrong. Science has shown breast cancer to be far less uniform than the relentless progression model suggests, says H. Gilbert Welch of the Dartmouth Institute for Health Policy and Clinical Practice in Lebanon, N.H., and author of Should I Be Tested for Cancer?A more accurate description might be called the “uncertain future” model. Instead of starting small and gradually growing and becoming more dangerous, cancers can behave in a variety of unpredictable ways.
Most fall into one of three general patterns, which cancer expert Barry Kramer, director of the National Institutes of Health Office of Disease Prevention, has dubbed turtles, bears and birds. Turtles move so slowly that they’ll never become dangerous and so don’t require treatment. Bears can escape, but move slowly enough that you can catch them if you remain alert, and birds are so fast and flighty that the first time you spot them is when they’re flying out the window. Screening tests like mammograms can only make a difference for the bears.
Dartmouth medical professor Lisa Schwartz has interviewed women to find out how they view breast cancer and mammography with the goal of helping them make informed decisions. “What we found is that people didn’t see it as a decision,” she says. The women Schwartz studied almost universally believed in the relentless-progression model, and under that belief system, the only possible risk to mammograms and other breast cancer screenings is the chance you’ll get called back for more tests on something that turns out to be harmless. Under this mental model, there is no downside to screening because every cancer is destined to kill you and thus you can never find a cancer too early. These notions are reinforced by public service announcements promising that “early detection saves lives.”
But look at the breast cancer screening debate through the “uncertain future” lens, and an entirely different picture emerges. As of now, doctors can’t reliably determine which pattern an individual cancer will follow, and that means they must treat each one they find as if it’s a bear — lest they miss an opportunity to prevent a cancer death. But bears only represent a fraction of cancers, and it turns out that mammograms are most adept at finding turtles, Welch says. Women with turtle-like cancers can only be harmed by mammograms and other screenings because they result in the diagnosis and treatment of cancers that would have never threatened their lives. In fact, a Norwegian study published last year suggested that some breast cancers, perhaps as many as 22 percent of symptomless ones found via mammography, can regress on their own without any treatment at all.
A British Medical Journal study published last July suggests that for every life saved by a mammogram, 10 women needlessly get diagnosed and treated for cancers that never would have killed them. Meanwhile, research suggests that the most aggressive breast cancers can spread long before they’re large enough to be detectable via any currently available technology, so many women with the most deadly cancers will not be helped by a mammogram, either. The evidence favoring mammograms is strongest for women age 50 to 74 and that’s why the task force focused its recommendations on that age group. “Even if you had all the money in the world, and the best doctors and lawyers, [the decision about whether to get a mammogram] is still a close call, and different people will make different choices,” Welch says.
The dispute over the mammography guidelines is not about evidence; it’s about belief. Is the goal of cancer screening to find as many cancers as possible? Or to save as many lives as possible? How many people can be harmed to save one life? What kind of harm is acceptable? These are not easy questions, and when the people answering them are operating under different belief systems, agreement may be impossible.
The mammography debate has become a virtual religion, and the more entrenched beliefs become, the less amenable they are to new evidence. In a study to be published in Political Behavior, Nyhan and his colleague Jason Reifler at Georgia State University presented volunteers with mock news stories that included either a misleading claim from a politician or a misleading claim and a correction. Subjects presented with corrective information that ran counter to their pre-existing ideology did not update their beliefs accordingly, and the corrections actually strengthened misperceptions among some of the most strongly committed subjects.
Last summer at the Aspen Health Forum, I asked representatives from the breast cancer advocacy group Susan G. Komen for the Cure about a paper that had just come out in the British Medical Journal calculating that as many as 1 in 3 breast cancers diagnosed via a screening mammogram represented an overdiagnosis and thus had needlessly turned a healthy woman into a cancer patient.
Was her organization worried about this problem? I asked then-Komen CEO Hala Moddelmog. “I don’t think there’s evidence of overdiagnosis,” she said. Overhearing our conversation, Elizabeth Thompson, vice president of health sciences, chimed in: “We’re very concerned that insurance companies will stop funding mammograms when these kinds of studies come out.” When I asked what they were telling women about the risks associated with mammograms, Thompson continued, “We believe early detection saves lives, and we need to focus on getting out that message.”
‘I don’t want knowledge, I want certainty!” So begins the David Bowie song “Law (Earthlings on Fire),” and it provides an apt description of the human psyche, at least when it comes to medicine. Consider the problem of back pain.
Approximately 90 percent of Americans will experience low back pain at some time in their lives. “When someone comes in with acute back pain, they’re worried that there’s something very seriously wrong, and they want to know what’s causing their pain,” says Michael Von Korff, a back pain expert at the Group Health Research Institute in Seattle. Yet efforts to pinpoint the cause of back pain rarely succeed. With so many potential pain generators in the back, it’s often difficult to determine an exact source of the pain. “Is it coming from a disc, muscles? Impingement of a nerve? It’s hard to say,” Von Korff says. Only about 15 percent of all back pain episodes are diagnosable to a specific cause.
Numerous high-tech imaging systems can visualize apparent abnormalities, such as slipped discs, but these findings are more likely to instill a false sense of certainty than to pinpoint an actual cause of the pain, says Richard A. Deyo, professor of evidence-based family medicine at Oregon Health and Science University. Many of the scary-looking problems that turn up on imaging tests simply don’t correlate to pain, Deyo says. A 2005 study found bulging discs on the MRIs of 73 percent of volunteers without back pain, and a study of 558 Finns published last year showed that almost half of the study’s 21-year-old participants had at least one degenerated disc.
Visual evidence is compelling, yet studies show that imaging tests rarely improve a patient’s chances of resolving the pain. A randomized trial published in the Journal of the American Medical Association in 2003 found that back pain patients who received MRIs had more surgery but no better outcomes than similar patients who hadn’t done the scans. Rates of spinal fusion surgery climbed 220 percent between 1990 and 2001 without a concurrent improvement in outcomes or disability rates.
The overwhelming majority of back pain cases resolve on their own without aggressive treatment. About 90 percent of back pain sufferers recover within two months, and 70 percent recuperate in three weeks or less. Even herniated disks, which may seem like a problem worth fixing, often spontaneously disappear without surgery. “Usually the source of the pain isn’t critical because it’s going to be managed the same way,” Von Korff says. Conservative treatments, like pain medications, ice or heat, and exercise remain the standard treatment for most back pain.
But this evidence-based message isn’t an easy sell. “Sometimes it’s better to do less, but that answer doesn’t sit well with Americans; it sounds like a loss of resolve or capitulating to the enemy,” Berkeley psychologist MacCoun says. People go to the doctor seeking a quick cure, not advice on how to make themselves more comfortable while they wait for natural healing to occur.
And plenty of doctors are willing to offer aggressive means. “There’s a real bias for action on [the] part of patients and physicians,” Deyo says. An examination of more than 3,500 patient visits published in the Feb. 8 issue of Archives of Internal Medicine found that even though practice guidelines recommend against routine imaging for low back pain, 1 in 4 patients was nonetheless referred for imaging tests. Doctors want to offer their patients a tangible solution, and most genuinely believe the treatments they’re offering are helpful. “If you do something to a patient, chances are pretty good that he’s going to get better,” Deyo says. “So you come to believe that what you’re doing is very effective when in fact it may be only marginally so over the natural course of the condition.”
Not long ago, Deyo’s wife developed a shoulder condition that left her in terrible pain for nearly a year. After exhausting other options, she decided to look into acupuncture, but before she could start, the pain suddenly eased. “She jokes that if she had started the acupuncture two weeks earlier, she would have been convinced that acupuncture cured her,” Deyo says.
Whether or not the “do something, anything” approach is effective, aggressive action feels empowering to doctor and patient alike. In fact, studies have shown that patients who get more high-tech spine imaging are more satisfied with their care than those who don’t, even though their outcomes are no better, and in some cases worse, than those who didn’t get the imaging, Deyo says. “The people in these clinical trials have worse outcomes, but they’re more grateful — they think they had the best care.”
This is the conundrum facing those trying to inject evidence-based medicine into health care reform: How do you convince doctors and patients to dump established, well-loved interventions when evidence shows they don’t actually improve health?
First, recognize that the facts alone are unlikely to change anyone’s mind, University of Michigan political scientist Nyhan says. “People get defensive when you tell them they’re wrong,” he says. In one set of experiments, Nyhan took volunteers who believed that Saddam Hussein had weapons of mass destruction in Iraq and presented them with evidence that this belief was wrong. Instead of causing people to adjust their erroneous beliefs, the corrections often reinforced them. Presenting people with facts in conflict with their belief spurred them to re-examine all the reasons they’d held this belief in the first place, and this process of remembering served to reinforce the initial belief, despite the contrary evidence, Nyhan says.
Belief is a very difficult thing to overturn, especially when the belief is held by people with a vested interest in the old message. Sometimes these investments are monetary (back doctors make more money on procedures than on conservative treatment), but they can also be altruistic — breast cancer advocacy groups want to offer women something to protect themselves from a scary disease.
When the evidence presents a messy, unsatisfying picture, people are likely to take refuge in a more comforting story, even in the face of evidence that it’s wrong. It comes down to something the satirist Stephen Colbert calls “truthiness,” a term he coined in a 2005 episode of his Comedy Central show, The Colbert Report. “Truthiness is what you want the facts to be, as opposed to what the facts are,” Colbert said. “It is the truth that is felt deep down, in the gut.” The backlash against the new mammography guidelines stemmed in part from the truthiness of the message that mammography could prevent breast cancer. No matter that it wasn’t true, it was what people wanted to believe.
For new evidence to overcome truthiness, it must be framed in an appealing story, one that acknowledges the existing narrative. For supporters of the new mammography guidelines, that means addressing, head on, the widespread notion that breast cancer is a single, relentlessly progressive disease. “When the [new screening guidelines] came out, they might have said, ‘We know that for years you’ve heard this, and the reason we’re changing course now is that we realize that there are harms associated with screening, and we’re changing our guidelines because we want to protect you,” Schwartz says. The task force needed to emphasize in its message that the new guidelines have the same goal as the previous one — to save lives.
For truth to win, stakeholders must also have a shared vision of what the problem is, so they can mutually recognize the correct solution once it’s found, Berkeley psychologist MacCoun says. For those seeking to identify the most effective medical interventions, that means establishing agreement on what “effective” means. Should the efficacy of back pain treatments be measured by pain levels six months post-intervention, by cost, by provider profits or by patient satisfaction? This isn’t a scientific question, but a value judgment, and different criteria yield different answers.
And then there’s the question of what constitutes evidence. Proponents of comparative effectiveness research look for answers in large-scale trials, but these studies hinge on statistics about large groups of people. Such number crunching rarely has the power of personal anecdote. “Studies have shown that powerful anecdotes trump data; we see that again and again,” Nyhan says. The runners who attended Nieman’s talk were not moved by his study results, because the findings contradicted their own personal experiences, which felt truer.
Women whose breast cancers were diagnosed with a mammogram will never be persuaded by the new mammography guidelines, Breast Cancer Action’s Brenner says. “They all say, ‘If it weren’t for that mammogram, I’d be dead right now,'” she says, “even though we know from the data that this wasn’t the case for most of them.”
Science works in data and statistics, but medicine is made up of stories, says Elizabeth Rider, an assistant professor of pediatrics at Harvard Medical School. Narratives form the backbone of medicine — they’re the way people make sense of the evidence.
“Victims of overdiagnosis don’t say, ‘Look what the system did to me.’ They say, ‘Thank God the doctor saved me,'” says Thomas B. Newman, a physician and narrative medicine expert at the University of California, San Francisco. “Nobody can say I had an unnecessary mastectomy, and nobody would want to; it doesn’t make a good story.”
Howard F. Stein, a physician and medical anthropologist at the University of Oklahoma Health Sciences Center, tells the story of a farmer in Oklahoma who had come in for an appendectomy but developed a rapid heartbeat that required treatment before surgeons could operate. “The cardiologist tried to explain supraventricular tachycardia to the family, but he might as well have been talking to a rock,” Stein says. The family was very upset and couldn’t understand why they were putting off the surgery.
Finally, another doctor happened in and told the family, “His heart is shimmying like the front end of an old Chevy truck, and as long as it’s shimmying, we can’t do the surgery.” The tactic worked.
“He translated biomedical evidence into the kind of framework that would count as evidence in the patient’s world. The doctor knew damned well that the patient’s heart was not a pickup truck,” Stein says. “But this story allowed the doctor and family to come to a common understanding, without agreeing on cardiology 101.”
Yet it’s not enough to weave the facts into a story. To take hold, evidence-based messages must also meet the human need for comfort and empowerment. “Medical uncertainty is very hard, so you need to find a way to reframe it so that you can say, ‘The good thing about this is…,'” Rider says. The relentless-progression model of breast cancer has persisted in part because it offers comfort and certainty by implying that every cancer can be cured, if only women do the right thing. This message about mammograms offers a sense of empowerment and security, and women are unlikely to accept the new guidelines unless they’re presented in a way that addresses these needs.
Explanations that offer hope and empowerment will always hold more appeal than those that offer uncertainty or bad news, and when new evidence offers messy truths, they must be framed in a positive light if they’re to gain traction. You can ask doctors to give up ineffective interventions, but you must never ask them or their patients to abandon hope.
