Whatever else comes of Barack Obama’s legacy on health care, financial reform or the federal deficit, his administration has seeded a little-recognized but critical shift in the way government funds and runs social programs. The insight sounds fairly obvious, although academics will recognize it as much more: The government is now trying to use evidence.
According to a new report commissioned by the National Endowment for Science, Technology and the Arts in the U.K. (because the Brits would like to figure out how to do this, too), the Obama administration is instituting “the most sweeping and potentially groundbreaking emphasis on rigorous program evaluation ever conducted by the federal government.” It has been trying to identify social programs that work, scientifically evaluate how they work and apply those models in awarding grant money.
The hope is that Uncle Sam — with the help of rigorous social science — will stop supporting teen pregnancy prevention programs that don’t actually prevent teen pregnancies, or childhood literacy programs that don’t teach all that many children how to read.
Miller-McCune’s Washington correspondent Emily Badger follows the ideas informing, explaining and influencing government, from the local think tank circuit to academic research that shapes D.C. policy from afar.
“A null finding is more common than a lot of folks appreciate,” said Jon Baron, president of the Coalition for Evidence-Based Policy and one of the co-authors of the report, alongside Ron Haskins of the Brookings Institution.
The authors cited six initiatives begun under Obama in which they conclude the role of evidence has been “unprecedented.” The Department of Health and Human Services has commissioned literature reviews on programs targeting teen pregnancies and parental home visitation. Expert panels have reviewed applicants for the Social Innovation and Investing in Innovation funds. Program guidelines for the Community College Challenge Fund and the Workforce Innovation Fund through the departments of Labor and Education have stressed the importance of evidence-based strategies and evaluation.
In the process, Baron said, the concept has taken root with career staffers at federal agencies and the Office of Management and Budget, not just with political appointees (although former OMB director Peter Orszag was a particularly enthusiastic advocate). Some efforts also began under the Bush administration, and so Baron hopes the trend won’t dissipate whenever the Obama team leaves town.
Until now, the vast majority of federal social programs have gone funded for years with little evidence of success, or little evidence of any kind. Baron identifies 10 large-scale federal social programs over the last 20 years that were evaluated with scientific research design. Nine of them — including popular programs like Head Start — turned out to have little or no impact on participants.
“Right now,” Baron said, “if you look across all areas of social policy — everything from youth development to K-12 education to teen pregnancy prevention to welfare and employment, and so on — there are probably 10 specific program models backed by strong evidence, close to definitive evidence of effectiveness, where you can say with confidence, ‘If you were to implement this in a different setting, with a similar population and be faithful to the intervention, you’d have a meaningful impact on peoples’ lives.'”
He and Haskins write that this is the most important contribution social science can make to the public good — the knowledge to design randomized control studies that would definitively identify which job-training or child-abuse prevention programs really work and which don’t.
“In business, if you build a new computer, it’s fairly straightforward to test if that computer has higher processing power, greater memory, more consumers buying it. There are some clear measures of success,” Baron said. “In social policy, it’s not quite as straightforward. You’re really asking whether this program — spending money on a home visitation program or a new kindergarten curriculum —whether the program participants do better than they would have done in the absence of the program.”
The answer also requires studying a comparable group of kindergartners that aren’t enrolled. And this is a major step that’s missing from most of the evaluation that’s already being done, where it’s being done at all. The gold standard — similar to the one used in medical trials — demands a randomized control study that can be replicated across different sites.
Organizations and agencies may resist this for fear that such studies will prove many programs unworthy of taxpayer money. And Baron and Haskins are under no illusions that evidence will one day rule all spending decisions. There will always be politics, tight budgets, lobbyists and opinion polls that factor, too, but Haskins envisions a future where there are hundreds of such scientifically proven social programs — not just 10 of them — and he adds that government could save a lot of money getting there.
Sign up for the free Miller-McCune.com e-newsletter.
“Like” Miller-McCune on Facebook.
Follow Miller-McCune on Twitter.
Add Miller-McCune.com news to your site.
