First, there were the accolades. More than 100 instances of street harassment in a two-minute video, testifying powerfully to the routine invasion of women’s lives by male strangers.
Then, there was the criticism. How is it, people asked, that the majority of the men are black? They argued: This video isn’t an indictment of men, it’s an indictment of black men.
Now, we’ve reached the third stage: lessons for research methods classes.
Our instructor is sociologist Zeynep Tufekci, writing at Medium’s The Message. Our competing hypotheses are three:
1. Black men really do catcall more than other kinds of men.
2. The people who made this video are unconsciously or consciously racist, editing out men of other races.
3. The study was badly designed.
As Tufekci points out, any one of these could account for why so many of the catcallers were black. Likewise, all three could be at play at once.
Enter, the data wrangler: Chris Moore at Mass Appeal.
Moore and his colleagues looked for landmarks in the video in order to place every instance of harassment on the map of New York City. According to their analysis, over half of the harassment occurs on just one street—125th—in Harlem.
Did the time the producers spent in Harlem involve denser rates of harassment, supporting hypothesis #1. Did they spend an extra amount of time in Harlem because they have something against black men? That’d be hypothesis #2. Or is it hypothesis #3: they were thoughtless about their decisions as to where they would do their filming.
Honestly, it’s hard to say without more data, such as knowing how much time they spent in each neighborhood and in neighborhoods not represented in the video. But if it’s true that they failed to sample the streets of New York City in any meaningful way—and I suspect it is—then hypothesis #3 explains at least some of why black men are over-represented.
And that fact should motivate us all to do our methods right. If we don’t, we may end up offering accidental and fallacious support to ideas that we loathe.
This post originally appeared on Sociological Images, a Pacific Standard partner site, as “That Catcalling Video: Research Methods Edition.”
