This post is based on the paper: “Can We Trust Intuitive Jurors? Standards of Proof and the Probative Value of Evidence in Coherence-Based Reasoning,” written by Andreas Glöckner and Christoph Engel, Journal of Empirical Legal Studies, Volume 10, Issue 2, 230–252, June 2013. Standards of proof discussed in the article are not included in this post.
Glockner and Engel explain that Jury members have a difficult task. They have to make decisions based on multiple pieces of probabilistic evidence. These pieces of information are usually contradictory, essentially always incomplete, presented in multiple formats (making them hard to compare and integrate), and introduced by parties clearly intending to bias the jury. How do jury members then make meaningful decisions? Glockner and Engel suggest that there is mounting evidence that most people do not mathematically integrate evidence. Their behavior is better explained by sense making and constructing coherent stories from the evidence. Jurors attempt to create complete narratives from the pieces of evidence they hear.
Intuitive-automatic processes are valuable for efficient decision making in such situations. Because cognitive capacity for deliberate information integration is limited, rational calculations of solutions are often not possible. One alternative would be to rely on simple fast-and-frugal heuristics that, for instance, take into account one reason only. Glockner and Engel assert however, that both naïve justice intuitions and legal procedure oblige legal decision makers to take into account all relevant pieces of information. People rely on the computational power of automatic-intuitive processes underlying coherence-based reasoning to overcome this problem. Glockner and Engel think these processes enable persons to approximate rational choices surprisingly well and it has been shown that they can be applied quickly, involving little cognitive effort. These automatic processes are partially controlled by deliberate processes. According to Glockner and Engel the result of the automatic-intuitive processes is often a clear understanding (which sometimes comes as a kind of insight) of how to decide the case based on a conscious and coherent story.
Parallel constraint satisfaction (PCS) models (Glockner is a main architect of such models.) postulate that when decision makers actively construct coherent mental representations of the situation, they thereby unconsciously modify their perception of the evidence. People reason bidirectionally. Not only do they construct an interpretation from the evidence but they also reason backward from possible interpretations to the evidence. The original information is systematically distorted. Glockner and Engel assert that it has been shown that different interpretations of the evidence are weighed against each other, leading to a progressive radicalization of the evidence. Information supporting the favored interpretation is highlighted, whereas the perceived importance or reliability of information that contradicts this interpretation is reduced. These coherence shifts seem to be a fairly widespread having not only been observed for legal decisions but also that persons distort their perceptions of the attributes when selecting between apartments or evaluating weather forecasts. Considerable work on coherence-based reasoning in complex criminal cases has been conducted by Dan Simon and his co-authors. Simon’s research has shown that the mere assignment of participants to adversarial roles distorts and polarizes people’s views of the case, and people are not able to correct for such role-induced bias, even if it is in their best interest to do so. Furthermore, it has been shown that coherence shifts are transient in that evaluations of evidence recede to baseline after some time.
In the current study by Glockner and Engel, changes in the evaluation of the evidence (i.e., coherence shifts) were measured by letting participants evaluate the evidence in a pretest using social vignettes. Participants then made a verdict and rated the same evidence again. Differences in the evaluation of the evidence pre- and post verdict indicate coherence shifts. They varied the probative value of the evidence the subjects were exposed to by changing the numeric probability information in the case (e.g., the eyewitness is 80 percent vs. 95 percent vs. 99 percent certain of having seen the accused person).
The experiments included about 240 participants. Some of the results were interesting. Not surprisingly, the estimated probability of guilt was higher for persons who convicted as opposed to acquitters. The tremendous absolute difference (persons who decided guilty estimated the probability of guilt to be about 40 percent higher than persons who decided not guilty) indicates that persons really interpreted the same case in completely different ways. The probability of guilt estimation did not increase with increasing the probative value of the evidence for guilt. Thus, 80% certain evidence was just as persuasive as 99% certain evidence. In addition, when holding the general story constant, confidence is not significantly influenced by the probative value of the evidence. They also found that coherence shifts are stronger for persons who convict. Hence, convictions seem to necessitate that persons build more consistent interpretations. In addition, coherence shifts become stronger with the increasing strength of evidence that contradicts a person’s preferred verdict. Specifically, increasing numerical probabilistic values in a complex case is compensated by stronger coherence shifts.
From a more general perspective, Glockner and Engel write that the currently available knowledge about mental mechanisms indicates that erroneous decisions often do not result from “stupid” calculation errors or motivated reasoning. Instead, they result from inappropriately constructed mental representations of the problem and are byproducts of automatic-intuitive processes underlying coherence-based reasoning. Legal institutions have to both assure a good informational basis for coherence-based reasoning and take into account the possible consequences of those processes. The former can be due to biased sampling or learning experiences , anchors , hindsight effects, endowment state, and many other factors. Concerning the latter, coherence shifts in particular have to be taken into account. The consequences are overconfidence, conservatism as a result of clinging to initial interpretations, and insufficient evaluation of numerical probabilities (this study). Glockner and Engel propose that the simplest countermeasures that apply to jury members and judges are the obligation to reveal, document, and deliberately double-check the completeness of mental representations and to consider plausible alternative interpretations of the case.
The most significant finding of the experiments was that even large changes in explicitly stated probabilities, while holding the overall constellation of facts constant, did not influence conviction rates or the estimated probability of conviction. My interpretation is that people are able to pick the most likely story and not necessarily the story that will actually occur most of the time. We are better rankers than we are probability determiners.
Glockner, A., Engel, C. (2013) “Can We Trust Intuitive Jurors? Standards of Proof and the Probative Value of Evidence in Coherence-Based Reasoning.” Journal of Empirical Legal Studies, Volume 10, Issue 2, 230–252, June 2013.