A Robot Analysis of All of America’s State of the Union Addresses

The dataset includes an address for nearly every year from 1790 to 2014.

Americans today watch the State of the Union address on TV, champing at the bit for the chance to live-tweet their reactions; Americans in 1790 likely heard nothing of it, unless they were politicians themselves. Either way, the United States’ presidents have given State of the Union addresses every year since 1790. (The only exception being 1933, when Franklin D. Roosevelt shifted the address from December to January.) The addresses are a remarkably consistent, if narrow, window into American history. And recently, a team of researchers from New York and France did a pretty cool thing with these speeches: They built a mathematical model that analyzed all of the 1.8 million words delivered in State of the Union addresses up until 2014.

The results are sort of like what a robot might see if it tried to analyze the entirety of American history. At first glance, the findings aren’t very personable, but a closer look reveals some interesting patterns. The vocabulary in State of the Union addresses stays pretty consistent from year to year, the researchers found. Over longer lengths of time, however, larger changes in specific words that presidents used begin to accumulate. One of the biggest changeovers occurred after World War I, with smaller shifts occurring after World War II and the War of 1812.

The vocabulary in State of the Union addresses stays pretty consistent from year to year.

The model works by comparing how often certain words and phrases come up in every State of the Union address. It also identifies topics, by noticing which words tend to appear together in the same paragraphs. Some of the topics disappear over time; presidents just don’t talk about settling the land or deploying the Navy like they used to. Others appear to change in form. In the beginning of the U.S.’s history, presidents often used words such as “taxation,” “appropriations,” and “public revenue”; after World War I, they were more likely to say words like “mortgage,” “welfare reform,” and “hard work.” Both those sets of words refer to domestic policy, basically, but they reflect differing views about what the government’s job is when it comes to taking care of affairs at home. An analogous shift happened in the discussion about foreign policy, from “law of nations,” “Indian tribes,” and “tranquility,” to “partners,” “hemisphere,” and “terrorists.”

Surprisingly, the Civil War didn’t appear to significantly affect the vocabulary of State of the Union addresses. The national conversation didn’t change after the Civil War, perhaps because the war didn’t solve the debate about states’ rights, the researchers write in a paper published this week in the Proceedings of the National Academy of Sciences.

What makes modern American politics, and when did American politics become modern? State of the Union addresses are an imperfect window into this question. After all, they’re always skewed toward the singular view of the most powerful man in the country. But they still reveal a nation that’s changed from busily setting itself up, via taxes and other revenues, to a world power with sophisticated public programs its citizens still debate.

Related Posts