Experiments reveal that a single, random upvote or downvote on Reddit has a big influence on what we read and recommend.
By Nathan Collins
(Photo: Eva Blue/Flickr)
There’s so much good stuff out there to read that you can’t possibly even glance at all of it, which is why Medium, Facebook, and other content providers have algorithms that feed you the stories you want to see, largely based on the recommendations of others. The question is, how sensitive are those algorithms to offhand, accidental, or even malicious liking, up- or down-voting, and such?
Pretty sensitive, according to a study forthcoming in ACM Intelligent Systems and Technology. A single upvote on Reddit increases a post’s score—the number of upvotes minus the number of downvotes—by 11 percent four days later. A single upvote also makes it 25 percent more likely that a post will reach a score above 2000. A downvote, meanwhile, lowers final scores by 5 percent and cuts the probability of reaching a high score nearly in half.
When developing recommendation systems like those on Reddit, the hope is that there’s a kind of collective intelligence or wisdom in the crowd: People put their heads together, pool their knowledge, and produce better recommendations than any individual would on his or her own. The problem is, whether individuals think a story is worth reading depends on more than its quality—notably, whether other people think it’s good, and whether it even appears in their feed.
“These results underscore the need for counter measures against … social engineering strategies” designed to affect what we read and recommend, the researchers write.
To quantify those sorts of effects, University of Notre Dame computer scientists Maria Glenski and Tim Weninger conducted a simple experiment. Every two minutes between September 1, 2013, and January 31, 2014, they used a computer program to find the most recent Reddit post and then chose at random whether to upvote, downvote, or—as a control—leave it alone. To see whether the timing mattered, the pair waited up to an hour after a post first showed up to give it a vote. In all, the researchers looked at 93,019 posts, then followed up four days later to determine those posts’ final scores.
A single vote, they found, had a big impact. On average, randomly upvoted posts scored a bit over 32 points—that is, they netted 32 more upvotes than downvotes—or 11 percent more than the control group. The randomly downvoted posts scored 5.2 percent lower than controls.
Interestingly, those effects didn’t depend on when the vote was cast. “We expected that immediate votes would have a larger effect than votes performed after a long delay,” Glenski and Weninger write. “However, these results show, surprisingly, that a delay in treatment generally did not have a significant effect on the mean outcome of a post’s final score.”
“The results of our experiments show that random vote perturbations through vote treatments impact the scores of posts and comments on Reddit,” the researchers write. “These results underscore the need for counter measures against … social engineering strategies” designed to affect what we read and recommend.
||
