It’s nutrition science’s turn for an accuracy and reproducibility crisis.
By Francie Diep
(Photo: Mason Masteka/Flickr)
It’s notoriously difficult to figure out which programs help people permanently lose weight. Just think of Kirstie Alley’s weight loss — and re-gain — on the Jenny Craig program, or how many programs Oprah Winfrey has tried. But what if the problem wasn’t the individuals seeking to lose weight; what if part of the problem was the scientists themselves?
Researchers conducting weight-loss studies often fail to use proper control groups, don’t take the placebo effect into consideration, and use bad statistics, according to a series of papers published last week in the journal Obesity. In other words, nutrition researchers are missing some basic science stuff, although John Ioannidis, a doctor who studies research accuracy at Stanford University and who contributed an op-ed to the series, says: “One should not take away the message that everything is wrong. I think, over the years, there has been improvement.” Nevertheless, he thinks “a large percentage” of weight loss studies feature at least one of the problems highlighted in the Obesity research. For so-called observational research, in which scientists don’t randomly assign people to programs, but instead gather data on how people eat and act normally, Ioannidis thinks “the vast majority” of that work has errors.
These problems in nutrition science add to Americans’ confusion about how to eat healthy and maintain an ideal weight. Although doctors shouldn’t depend on singlenutrition studies to advise their patients, experts Pacific Standard talked with agreed they probably do. Flawed studies contribute to weight-loss myths among scientists. All that lose-the-pudge research often makes its way to popular newspapers and magazines, which may bungle it further by presenting tentative hypotheses as photograph-ready weight-loss tips.
Luckily, there are promising solutions. Pacific Standard talked to several researchers to gather their ideas for how to shore up weight-loss research and get the science we need to solve the harder problem of maintaining weight loss.
1. Have a statistician work on every nutrition paper. A team of researchers, led by statistician Brandon George of the University of Alabama-Birmingham, suggested this in a paper in Obesity. Science is already moving in this direction, with big journals and the National Institutes of Health pledging to more closely examine the statistics in submitted papers and proposals.
2. Study how people’s environments can help them keep the weight off. Research has actually found many diets work to get people to lose weight. It’s keeping the weight off that still eludes many researchers — and dieters. This is where programs such as pedestrian-friendly city design and bans and taxes on unhealthy habits can make a difference. “You’re not going to find the answer in the lab. You’re going to find the answer at the policy level,” says Geoff Ball, a childhood obesity researcher at the University of Alberta.
3. Encourage a different kind of weight-loss study. Ioannidis advocates for a greater focus on simple, long-term trials. Scientists could identify their best leads on what plans help people sustain weight loss, have study volunteers follow those plans, then gather data from the volunteers periodically for 10 years or more. “We don’t need to spend billions of dollars to do these trials,” he says, although chances are, such a trial would be expensive and paid for with tax dollars.
Meanwhile, Kevin Fontaine, who studies health behavior at the University of Alabama-Birmingham, wants to see shorter, more intensive studies in which scientists carefully monitor what people eat. This would help ease at least one major problem in weight-loss studies: they too often rely on people reporting to researchers what they consume. Most people are notoriously bad at keeping accurate food and exercise diaries, researchshows. One science team even titled its paper on study volunteers’ food diaries, “When Something Is Not Better than Nothing.”
4. What to read in the meanwhile. For weight-loss science you can use now, we like thesestories from Vox. There are surely other good choices, too. Just avoid gimmicky diet books.
Over the past few years, scientists and journalists have been applying the pressure on scientists in various fields. The criticism that’s happening now in nutrition science echoes reproducibility and accuracy woes we’ve seen recently in the social and biomedical sciences. Hopefully that means we’ll all come out of the other side of this with better protocols for the studies we use to improve our policies, health, and personal lives.
||
