The Take the Best heuristic probably deserves its own post and this is it. This post is based on the paper, “Evaluating the Coherence of Take-the-Best in Structured Environments,” written by Michael D. Lee and Shunan Zhang that appeared in the July 2012 Judgment and Decision Making. Heuristic decision-making models, like Take-the-best, rely on environmental regularities. They conduct a limited search, and ignore available information, by assuming there is structure in the decision making environment. Take-the best relies on at least two regularities: diminishing returns, which says that information found earlier in search is more important than information found later; and correlated information, which says that information found early in search is predictive of information found later. Lee and Zhang develop new approaches to determining search orders, and to measuring cue discriminability, that make the reliance of Take-the-best on these regularities clear.
Lee and Zhang begin with a story about the 1992 Olympics. It was the first time professionals from the US NBA league were allowed to play in the men’s basketball competition. The US “Dream Team”, filled with stars like Michael Jordan, Magic Johnson, Larry Bird, Charles Barkley and Patrick Ewing, was one of the most dominant teams ever assembled for any sporting
competition. Their closest game was a 117–85 victory in the final over Croatia, and head coach Chuck Daly never felt the need to call a timeout during the tournament. Making predictions about the outcomes of sporting contests is notoriously difficult, but the Dream Team made some predictions easy. Imagine trying to predict whether or not the US would beat its first opponent in the tournament, Angola, and examining the players in each team, beginning with the starting five, and moving to the bench players. At some point early in the US list—maybe
after Jordan, Johnson, Bird, Barkley and Ewing—there would be no need to look further. No matter who else was on the US roster, or the Angolan roster, the outcome is already clear. The Dream Team also made predictions easy during the course of games. Such decisions about a US victory are noncompensatory, because not all of the available information is used, and so the ignored information cannot compensate for—that is, change the decision based on—the
information that is used. The remaining player rosters are not examined, and the rest of the game is not watched.
It is important to be fast, because the world is competitive. A basic reason for making decisions is to acquire resources, which are often scarce, and often contested. Making quick decisions
usually offers a competitive advantage in these situations. It is also important to be frugal, in the sense of being simple, because the world is changeable. A complicated decision making strategy will usually work well in the environment within which it was developed, but will often fail in new or altered environments. Decision making that over-fits, in this sense, is prone to failure when asked to generalize. Because real-world decision environments are continually changing, both quickly and slowly, and along many dimensions, simple decision making strategies that focus on the few stable features of the environment are likely to be the ones that succeed. Simplicity makes these decision strategies robust, just as a machine with few moving parts is unlikely to break. As Gigerenzer et al. emphasize, neither of these motivating principles are about cognitive limitations. Fast and frugal heuristics are effective to the extent they seize
on the opportunities presented by environmental regularities.
Despite its very limited search, take-the-best often matches, and sometimes exceeds, the
accuracy of benchmark statistical methods that use all of the available cue information.
However, the regularities of diminishing returns and correlated information structure in environments are key to its accuracy.
A complementary approach to accuracy, however, is to measure the internal coherence of a decision process. This is the approach Lee and Zhang adopt, and makes their evaluation of environmental regularities and take-the-best different from previous analyses. They use Hammond’s concepts of coherence and correspondence. Correspondence is getting a correct answer. Coherence involves following an effective process–a rational one. If you guess the correct answer, you have met the correspondence test, but only in certain circumstances have you met the coherence test.
For non-compensatory search, one measure of a reasonable decision process is that it terminates once the current decision is unlikely to change. If it becomes clear that further search is unlikely to change the current decision, it is sensible to stop searching. Regardless of the accuracy of that decision, further search will not change the outcome and, in that sense, the decision is internally coherent. Indeed, the definition of non-compensatory encourages a measure like this, since it captures the idea that additional information cannot change the overall decision of current information.
To demonstrate that take-the-best relies on the assumptions that the environment has diminishing returns and highly correlated information requires the ability to manipulate
the assumptions. Lee and Zhang did experiments using a richer set of search orders,
generalizing the validity search of take-the-best, so that the assumption of diminishing returns could be manipulated. In the experiments, they also divided discriminability into positive and negative components, to capture the assumption of correlated information. The experiments used the familiar question of which city is the most populous with such cues as whether it is capital or has a sports team etc. The results, for the well-studied German cities environment,
and also for Italy, the United Kingdom and the United State, show that combining diminishing returns and correlated information provide grounds for non-compensatory decision making.
When these assumptions are met, the probabilities that exhaustive search would change an initial decision are very small. The first discriminating cue favors the same alternative as exhaustive search favors. The results also show, however, the bounds on the justification for limited search. In several of the environments that were studied, when the basis of search moved away from validity, and so did not emphasize diminishing returns, more extensive search was needed. Similarly, if the correlation of information in the environment is not assumed, many more cues than the first discriminating one need to be examined to make it very unlikely a decision will change.
Asked before the Angola game what he knew about the Angolan team, US player Charles Barkley replied: “All I know about Angola is Angola’s in trouble.” That was a non-compensatory opinion, unlikely to have been changed on the basis of further reflection. It proved to be accurate, and it also was coherent.
Lee, M. D., Zhang, S.,(2012). “Evaluating the Coherence of Take-the-best in Structured
Environments.” Judgment and Decision Making, Vol. 7, No. 4, July 2012, pp. 360–372.