In chess—and perhaps in other domains—mistakes have more to do with a problem’s difficulty than a player’s skill or time pressure, a new study finds.
By Nathan Collins
(Photo: Julian Finney/Getty Images for DAGOC)
To err is human, of course, but all the same it’d be nice to know why we make mistakes. There are at least three obvious reasons: because stuff is hard; because we don’t have a lot of time to figure out the best choices; and because we’re not very smart, skilled, or otherwise capable. You’d think those last two reasons are the main drivers of our foibles—and at least when it comes to chess, you’d be wrong.
In chess, “the inherent difficulty of the decision, even approximated simply by the proportion of available blunders in the underlying position, can be a more powerful source of information [about errors] than the skill or time available,” Stanford University computer scientist Ashton Anderson, Cornell University computer scientist Jon Kleinberg, and Harvard University economist Sendhil Mullainathan write in a paper to be presented next month at the 2016 Conference on Knowledge Discovery and Data Mining.
The fact that mistakes have more to do with the problem itself, as opposed to skill or time, raises questions well beyond the domain of chess.
Anderson, Kleinberg, and Mullainathan chose to study chess in part because there is an enormous amount of data on how people play the game—and in many cases a clear standard for whether players have made mistakes, thanks to computer algorithms for finding the best play in any given situation. Drawing on 24.6 million games played via the Free Internet Chess Server and another 880,000 games played by some of the best players in the world, the researchers looked at how a player’s skill—specifically, his or her Elo rating—the time remaining in the game, and the state of the chess board contributed to a player’s chance of making a mistake, in the sense of reducing their chances of winning. To make sure they could actually identify whether a move was a mistake, the team focused only on boards where fewer than six pieces remained.
Unsurprisingly, error rates increased with the number of possible mistakes, expressed as a fraction of the total number of available moves—what the researchers termed “blunder potential”—the team found. Although skill reduced error rates, it had markedly less impact than blunder potential. One additional blunderous move in a set of five, for example, had roughly the same impact on the error rate as a 600-point increase in Elo rating. To put that in context, 600 points is roughly the difference between a candidate-master rating, where chess officials start taking you seriously, and the best nine or 10 players that have every lived. Time pressure had even less effect—basically, it had no impact on error rates, unless players had less than 10 seconds to make a move.
The fact that mistakes have more to do with the problem itself, as opposed to skill or time, raises questions well beyond the domain of chess. “For a setting such as medicine, is the experience of the physician or the difficulty of the case a more important feature for predicting errors in diagnosis?” the team asks. “Or … for micro-level mistakes in a human task such as driving, we think of inexperienced and distracted drivers as a major source of risk, but how do these effects compare to the presence of dangerous road conditions?”
||
