Last September, we reported on an imaginatively designed study that attempted to document how the mood of the nation shifted in the wake of the terrorist attacks on Sept. 11, 2001.
German researchers analyzed the content of text messages sent to more than 85,000 American pagers on that day, and found indications of anger — that is, the use of words such as “hate” or “annoyed” — rose steadily as the hours went by. In contrast, the number of words indicating sadness or anxiety stayed relatively steady.
As a way of gauging the public mood, the study was groundbreaking. However, the researchers now concede it was seriously flawed.
Psychologist Cynthia Pury of Clemson University did her own analysis of the data, and discovered that 36 percent of the anger words noted in the study “were in nearly identically worded messages.” These were computer-generated instructions to reboot, and they included the word “critical.”
In this context, “critical” was a call to urgent action rather than a cutting remark. But the original researchers counted each appearance of the term as an indication of anger.
Pury published her findings in a letter to the journal Psychological Science on May 9. Four days later, the original research team, led by Mitja Back of Johannes Gutenberg University Mainz, conceded she was correct.
“We did not anticipate that emotionally irrelevant, automatically generated messages (i.e. messages that described a ‘critical’ server problem) would be incorrectly classified … as anger-related, and at the same time show a non-random time course (i.e., a dramatic increase over time),” they write. “Although this unexpected confound did not affect our findings for sadness or anxiety, it did distort our findings for anger.”
Pury’s analysis agreed with theirs, in that it “showed a strong increase in anger after the first attack,” the German researchers write. “However, this rise in anger did not continue throughout the day. Pury found a substantially lower overall correlation between anger-related words and time than we found in our original analysis.”
So…never mind. Or, at least be wary of untried methods of data collection. Perhaps Americans got angrier during the course of September 11, 2001, but this data does not prove it.
Sign up for the free Miller-McCune.com e-newsletter.
“Like” Miller-McCune on Facebook.
Follow Miller-McCune on Twitter.
Add Miller-McCune.com news to your site.
