Customs Officer Expertise
The first paper is “Expert intuitions: How to model the decision strategies of airport customs officers?” authored by Thorsten Pachur and Gianmarco Marinello. They asked Swiss airport customs officers in Zurich and Bern and a novice control group to decide which passengers (described on several cue dimensions) they would submit to a search. Additionally, participants estimated the validities of the different cues. Then the researchers modeled the decisions using compensatory strategies, which integrate many cues, and a non-compensatory heuristic, which relies on one-reason decision making. Their analysis of the decisions of airport customs officers suggests that experts prefer simple decision strategies that rely on few cues and go without integration, whereas novices tend to use compensatory strategies that integrate multiple cues. Among the compensatory strategies, weighted-additive provided the worst account of participants’ decisions. Pachur specifically notes that this contradicts the claim that much of people’s decision making is based on an automatic weighted integration process as postulated by Glöckner & Betsch.
The results of Pachur and Marinello echo evidence that professional judges sometimes rely on simple, non-compensatory decision trees. Experts seem to rely on simple strategies even in a domain in which they make hundreds of decisions every day and obtain regular feedback. Moreover, the differences between experts and novices hold when potentially confounding factors such as age and education are properly controlled. Relative to the novices’, the experts’ representation of the cue weight distribution is more skewed; and with a skewed distribution of cue weights, the use of non-compensatory strategies is more appropriate. The greater reliance on a compensatory strategies by the novices is consistent with findings that people seem to have an initial tendency to use a compensatory strategy, which in principle allows them to explore the task more than does a non-compensatory strategy, that ignores cues.
As described above, however, the differences in cue dispersion can only partially account for the differences in strategy use. Other factors shaping the experts’ use of simple heuristics may be that customs officers have to make their decisions within a limited time frame and under considerable workload (given the large numbers of passengers generally passing through customs simultaneously). Such a situation fosters reliance on simple non-compensatory strategies. The officers’ use of take-the-best in our study might thus to some extent also reflect a decision routine spilling over from their professional work.
What are the implications of the results for current proposals of the mechanisms underlying strategy selection? The association between strategy use and cue representation indicates that some of the expert–novice differences may be couched within Marewski and Schooler’s cognitive niches framework, which proposes that strategy selection arises from an interplay between mind and environment. Second, to the extent that the experts’ reliance on simple strategies also reflects factors such as limited time or cognitive resources, more explicit processes that trade off a strategy’s cost against its expected accuracy, may also play a role.
Although the customs officers showed greater consensus in terms of both their cue ranking and their decisions there is no real evidence that their judgments were more accurate than the novices. The customs officers’ ranking agreed more with the ranking of the chief officer(Table 1), who according to internal airport statistics had the highest “success” rate (in terms of detected infringements), but the officers operate in a “wicked” learning environment whereby they receive feedback only about the passengers they screen (not about the passengers they do not screen). One cannot exclude the possibility that the stronger consensus among the experts to some extent reflects shared erroneous beliefs about the cue validities. One researcher found in her study on bail decisions by professional judges that the simple decision trees on which the judges seemed to rely were partly based on irrelevant information.
How do the custom’s officers learn about the statistical structure of the environment? Interviews with the customs officers revealed that they are given no formal instructions on how to proceed in conducting passenger checks and that there are no statistics about the predictive strength of various cues. However, there does seem to be considerable informal exchange of knowledge and subjective experience among officers.
The authors suggest their analyses add to the increasing evidence that, in contrast to common belief intuitive expertise in decision making—at least in some situations—may not reflect the consideration of multiple cues, but the use of simple heuristics.
The second paper is “Deliberation Versus Intuition: Decomposing the Role of Expertise In Judgement and Decision Making” by Koen A Dijkstra, Joop Van Der Pligt and Gerben Van Kleef. This article has an interesting premise — basically saying that real experts have both knowledge and experience. Most studies on the moderating effect of expertise distinguished between two levels: high and low expertise. Expertise is operationalized either as level of knowledge or as level of experience . The authors introduce a framework in which they consider both dimensions of expertise to explain the contradicting results that have been obtained in prior research. In this framework, they differentiate between experience and knowledge. Using the two dimensions of experience and knowledge, they distinguish among three levels of expertise. Individuals who are low in both experience and knowledge are considered to be novices. Individuals high in experience but low in knowledge are considered to be “intermediates.” Finally, experienced individuals who also have extensive knowledge are considered to be experts. In the present studies, they did not include the remaining combination of experience and knowledge (low experience and high knowledge).
In Experiment 1, they tested whether the effects of judgment mode (intuitive versus deliberative) on the accuracy of quality judgments of modern art differ as a function of expertise (novice, intermediate, and expert), as determined by the different combinations of experience and knowledge. In Experiment 2, they replicated the results of Experiment 1 in a different domain (piano performances). Experiment 3 aimed to shed more light on the process underlying the interaction between expertise and judgment mode. They induced different processing styles and asked participants with different levels of expertise to rate high-quality and low-quality poems.
The experiments showed that the impact of deliberation versus intuition on judgments of a variety of stimuli (paintings, piano performances, and poems) is moderated by experience and knowledge. Experienced individuals without formal training or professional background (“intermediates”) made poorer judgments after deliberation than when relying on intuition. Judgments of professionals and participants who had received relevant formal education (experts) did not differ as a function of judgment mode. Judgments of participants without formal education and without experience or interest in the subject matter (novices) also did not differ as a function of judgment mode.
The authors’ explanation is that knowledgeable people have a better understanding of why they feel the way they do because they can verbalize their experiences and are therefore less likely to come up with a biased set of reasons under deliberation. In contrast, unknowledgeable people are unsure of why they feel the way they do because they cannot verbalize their experience and are more likely to generate reasons that are not or only marginally related to their judgment.
The authors wonder what would be the effect of intuition and the moderating effect of experience and knowledge in rule-based decision making, such as legal judgments? Intuition depends on experiential learning, and Dijkstra et al expect legal judges to be able to (unconsciously) relate new legal cases to cases in the past and rely on intuition. The key difference between, for example, art critics and legal judges is that the latter group can rely on rules in the code of law. In this case, we could still argue that a legal judge with intermediate expertise would experience detrimental effects of deliberating because he or she ignores implicit knowledge obtained by experience.
Dijkstra et al conclude that their research indicates it might be especially harmful to deliberate, and profitable to rely on intuition, when experience is high while knowledge is relatively poor.
Comments–Customs Officer Expertise
Pachur has written papers that I like and has even coauthored papers with Glockner so when he says his results contradict the ideas of Glockner, he must believe it. Nevertheless, considering customs officers experts is problematic. They definitely operate in a wicked environment as defined by Hogarth so it is hard to learn from feedback. Their training is so limited that the wicked environment is the bulk of their education. Detecting drug smuggling may be like predicting the price of an individual stock–impossible. Randomness might be the best deterrent. Nevertheless, their experience definitely teaches them something. I am afraid it may be the expertise of a bureaucrat–learn the rules that the leadership endorses and follow them. Taking risks by trying something different is likely to get you in trouble. If you get too creative, you could lose your job. Being the customs agent most adept at rooting out smuggling could get you a promotion if there is a void at the top, but your boss may not like you threatening his supremacy, Thus, using the “checklist” is thus the most reasonable path to follow. Being a customs officer looking for drug smugglers is like a doctor trying to diagnose a rare disease, You are unlikely to get better unless someone gives you a better checklist. Like a doctor, customs officers usually have a lot of other jobs beyond searching for rare events.
I think it likely that the customs officers when faced with the same decisions that they would see every day would use the same checklist method and never even consider an automatic weighted decision process. Moreover, an automatic weighted decision process is going to encounter so much incoherence so fast that it would result in asking for deliberation to find better information which is pretty much the checklist.
Comments–Expertise in Art, Music, and Poetry
I like the idea of having several levels of expertise to compare performance and determine the process model for decision making. One value of examining experts is that they may be considered higher in the development stages of decision making so that results with experts will help explain general decision making.
However, the assignment of expertise in the three experiments was by the admission of the researchers “relatively arbitrary”. The assignment to intuitive or deliberative was random with the respective groups given different instructions. The intuitive groups were told to rely on their intuitions and not think too much while the deliberative groups were asked to provide reasons for their decisions. This seems unsophisticated and pointing subjects to over deliberation by requiring reasons. It seems to me that they were comparing intuition plus a little deliberation with intuition plus a lot of deliberation. Tighter time constraints for intuition would seem appropriate.
Their conclusion that it might be especially harmful to deliberate, and profitable to rely on intuition, when experience is high and knowledge is low may have a thread of truth. Unfortunately, they did not seem to rely on Hogarth who examined the relative accuracy of intuition and deliberation based on analytical complexity and the bias and error implied by intuitive processes (Learning, Feedback and Intuition). For example, in the Muller-Lyer illusion, intuition suggests that one line is longer than the other, while the deliberative step of measuring with a ruler shows that they are the same length. This seems like the high experience and low knowledge situation. My issue with Dijkstra et al is that they needed to look more at the character of the tasks and the determination of expertise. Nevertheless, their idea to further breakdown expertise into knowledge and experience seems worthwhile.
Pachur, T. & Marinello, G. (2013). Expert intuitions: How to model the decision strategies of airport customs officers? Acta Psychologica, 144, 97–103.
Dijkstra, K., Van Der Pligt, J., Van Kleef, G. (2013). Deliberation Versus Intuition: Decomposing the Role of Expertise In Judgement and Decision Making. J.Behav. Dec. Making, 26, 285-294.