Decision-Making Lessons from Online Chess

Many decisions involve choosing among complex options, and it may be difficult to assess the value of each alternative. In Complexity and Choice (NBER Working Paper 30002), Yuval Salant and Jörg L. Spenkuch use data on 227 million chess moves on an online chess server to study how complexity affects the quality of decision-making. They find that complex optimal options are chosen less frequently than other alternatives that are also optimal but simpler.

The researchers present a model of choice in which each option has both a true value and some degree of complexity. Decision-makers have some skill in assessing each option’s value, perhaps from intelligence or experience. They can accurately assess the value of options with complexity up to their skill level, but they only estimate with error the value of options whose complexity exceeds their skill. Decision-makers then follow a process called “satisficing.” Rather than estimating the value of all options and then choosing the highest one, satisficers set an aspiration level, or minimum value they would like to find, and accept the first option whose estimated value exceeds it.

Players appear to “satisfice,” choosing simpler moves over equally valuable but more complex ones and making more errors when the choice set includes a greater share of complex moves.

The researchers test this theory using data from Lichess, a popular online chess platform. Their data cover all human-played chess games that were used to determine player rankings between January 2013 and August 2020.

Chess is a natural setting in which to study choices among complex options. On each turn, players choose from a set of available moves. Each one has an objective value. It either allows the current player to force a win if she plays optimally, no matter what her opponent does, allows her opponent to force a win, or allows both players to force a draw. While it is computationally infeasible to classify early-game moves as win, lose, or draw, computer algorithms have classified all moves when up to six pieces are left. The researchers study chess endgames in which they can clearly identify mistakes made when players face board configurations allowing for at least one win and at least one lose or draw move.

In addition to their objective values, chess moves also have an objective measure of complexity. Win moves and lose moves have a certain “depth to mate” (DTM) — the smallest number of turns in which one of the players could force a win or be forced into a loss. The researchers argue that the higher a move’s DTM, the harder it is to identify it as a lose or a win move.

Mistakes are more common when moves are more complex. Players are less likely to choose a winning move if it is more complex, and they are less likely to mistakenly choose a losing move if it is less complex. As a result, mistakes are more common when the available moves are more complex. These choice patterns are not consistent with players evaluating all possible moves and choosing the one that they perceive to be best, but they are consistent with satisficing.

Skill offsets the costs of complexity. About 1.5 percent of chess turns in the dataset being analyzed are played by titled players like national masters and grand masters. These players make fewer mistakes than others, but only when all available winning moves are sufficiently complex. Chess games on Lichess are subject to varying time controls, and greater decision-making time also offsets the costs of complexity for choice quality, with better players benefiting more.

Finally, the researchers consider how the complexity of the set of available choices as a whole affects decision-making. They find that the choice set’s composition, not its size, affects choice quality. Mistakes are less common as win moves are added to the set of available moves, while they become more common with additional draw or lose moves. Holding the share of each category of move constant, there is no evidence that players make more mistakes as the choice set grows.

— Lucy E. Page