Category Archives: Diverse Problem Solvers


superforecastingimagesThis post is a look at the book by Philip E Tetlock and Dan Gardner, Superforecasting– the Art and Science of Prediction.  Phil Tetlock is also the author of Expert Political Judgment: How Good Is It? How Can We Know?   In Superforecasting Tetlock blends discussion of the largely popular literature on decision making and his long duration scientific work on the ability of experts and others to predict future events.

In Expert Political Judgment: How Good Is It? How Can We Know? Tetlock found that the average expert did little better than guessing.  He also found that some did better. In Superforecasting he discusses the study of those who did better and how they did it.

Continue reading

Harnessing the Inner Crowd

innercrowdUntitledThis post is based on the paper: “Harnessing the Wisdom of the Inner Crowd,” written by Stefan M. Herzog and Ralph Hertwig that appeared in Trends in Cognitive Sciences, October 2014, Vol. 18, No. 10. This is a slightly different take on a subject addressed in the posts:  Dialectical Bootstrapping  and   Bootstrapping.    Herzog and Hertwig seem to be the go to guys on bootstrapping.  In the title they obviously refer to James Surowiecki’s The Wisdom of Crowds. (see post Smart Mobs and Diverse Problem Solvers). They explain that a lone individual can enlist the wisdom of crowds by averaging self-generated, nonredundant estimates. They review evi-
dence for this ‘wisdom of the inner crowd’, and consider how it can be produced, how its accuracy can be improved, and whether people use it to their advantage. Frankly, Figure 1, above puts the advice in one spot.

Continue reading

Does interaction matter in collective decision-making?

interactionF1.largeThis post is based on a paper: “Does interaction matter? Testing whether a confidence heuristic can replace interaction in collective decision-making.” The authors are
Dan Bang, Riccardo Fusaroli, Kristian Tylén, Karsten Olsen, Peter E. Latham, Jennifer Y.F. Lau, Andreas Roepstorff, Geraint Rees, Chris D. Frith, and Bahador Bahrami. The paper appeared in Consciousness and Cognition 26 (2014) 13–23.

The paper indicates that there is a growing interest in the mechanisms underlying the ‘‘two-heads-better-than-one’’ (2HBT1) effect, which refers to the ability of dyads to make more accurate decisions than either of their members. Bahrami’s 2010 study, using a perceptual task in which two observers had to detect a visual target, showed that two heads become better than one by sharing their ‘confidence’ (i.e., an internal estimate of the probability of being correct), thus allowing them to identify who is more likely to be correct in a given situation. This tendency to evaluate the reliability of information by the confidence with which it is expressed has been termed the ‘confidence heuristic’. I do not recall having seen the acronym 2HBT1 before, but it does recall the post Dialectical Bootstrapping in which one forms his own dyad, Bootstrapping where one uses expert judgment, and Scott Page’s work Diversity or Systematic Error? However, this is the first discussion of a confidence heuristic.

Continue reading

Dehaene: Consciousness and Decision Making

consciousimagesI love Stanislas Dehaene’s experiments, his general ideas and his book:  Consciousness and the Brain:  Deciphering How the Brain Codes our Thoughts, Viking, New York 2014 is a great synthesis and with respect to the title, it is a fine book. However, with respect to how it deals with decision making, I am mostly disappointed.

Consciousness: Informer or Informer/Decider? Although Dehaene’s Global Neuronal Workspace Theory describes what we feel as consciousness as the global sharing of information, in the book he seems to promote the idea of consciousness as the decider as well as the informer. Dehaene writes:

“My picture of consciousness imples a natural division of labor. In the basement, an army of unconscious workers does the exhausting work, sifting through piles of data. Meanwhile, at the top, a select board of executives, examining only a brief of the situation, slowly makes conscious decisions…No one can act on mere probabilities–at some point, a dictatorial process is needed to collapse all uncertainties and decide….Consciousness may be the brain’s scale tipping device—collapsing all unconscious probabilities into a single conscious sample so that we can move on to further decisions.” p89

I like the informer part, but I like the parallel constraint satisfaction (post Parallel Constraint Satisfaction Theory) idea that consciousness is asked to get more information (information search and production) which the unconscious system turns into a decision. In my scenario the visual system seems to have priority to get to the conscious level, then other sensory systems, and then the other unconscious systems push the most difficult or interesting decisions they have at any particular time through to the conscious system. Maybe there is some sort of priority ranking. Clearly, most rather mundane decisions seem to break through to consciousness only occasionally. As a part of breaking through to consciousness, more of the modular systems are alerted to the issue and maybe information can come from inside or maybe we seek information from others or examine the environment. We get the new information and the wheels of the parallel constraint system start whirring again to see if the decision can be made. Now, I do see a cognitive continuum so that yes certain decisions may stay with the board of executives. Dehaene uses the example of multidigit arithmetic. For most of us, it seems to consist of a series of introspective steps that we can accurately report. For instance, to multiply 30 by 47, I might multiply 30 by 40 and get 1200 and then add it to 7 by 30 to get 1410. But for a numerical savants that could be done in the unconscious. Nevertheless, there are certain things where consciousness does seem to be where the decisions are made. Complex multi-step questions where the emotions are more or less uninvolved might be examples.

Maybe the interesting part is the sort of phase change between the unconscious and the conscious. There is a lot happening there. Dehaene says that consciousness is doing the collapsing, but it seems to me it is already done once it reaches consciousness. Maybe that is not an important argument.  One theory is that conscious perception occurs when the stimulus allows the accumulation of sufficient sensory evidence to reach a threshold, at which point the brain ‘decides’ whether it has seen anything, and what it is. The mechanisms of conscious access would then be comparable to those of other decisions, involving an accumulation toward a threshold — with the difference that conscious perception would correspond to a global high-level ‘decision to engage’ many of the brain’s internal resources. Dehaene mentions this in a paper that was discussed in the post A Theory of Consciousness.

Consciousness Gives Us the Power of a Sophisticated Serial Computer. Dehaene is a believer in the Bayesian unconscious. “A strict logic governs the brain’s unconscious circuits–they appear ideally organized to perform statistically accurate inferences concerning our sensory inputs.” Both the unconscious and conscious systems seem to work in a linear fashion (Brunswik’s Lens Model), but the conscious system can redirect.

Dehaene states:

“This seems to be a major function of consciousness:  to collect the information from various processors, synthesize it, and then broadcast the result–a conscious symbol–to other, arbitrarily selected processors. These processors, in turn, apply their unconscious skills to this symbol, and the entire cycle may repeat a number  of times.  The outcome is a hybrid serial-parallel machine, in which stages of massively parallel computation are interleaved with a serial stage of conscious decision making and information routing.” p100

Dehaene and his colleagues have studied schizophrenics. They found a basic deficit of consciousness perception in schizophrenia. Words had to be presented for a longer time before schizophrenics reported conscious seeing. “Schizophrenics’ main problem seems to lie in the global integration of incoming information into a coherent whole.” Dehaene suggests that schizophrenics have a “global loss of top-down connectivity. This loss impairs capacity for conscious monitoring, top-down attention, working memory, and decision making. Apparently in schizophrenics, the prediction machine is not making enough predictions. With reduced top down messages, sensory inputs are never explained and error messages remain triggering multiple explanations. Schizophrenics thus see the need for complicated explanations that can lead to the far fetched interpretations of their surroundings that may express themselves as bizarre hallucinations and delusions.

Dehaene suggests that consciousness allows us to share information with others and that leads to better decisions. Dehaene’s most interesting idea is that our social abilities allow us to make decisions together and that these are better decisions. Although one can argue that language is imperfect and that much of it is used to transmit trivia and gossip, Dehaene provides evidence that our conversations are more than tabloids. This is a point that needed to be made to me. I was tending to believe that there was almost a direct tradeoff between cognitive skills and social skills and even though that tradeoff was adaptive, maybe it was close. Dehaene puts forth the argument that two heads are better than one and that consciousness makes this possible (This is also directly in line with Scott Page’s: The Difference — How the Power of Diversity Creates Better Groups, post Diversity or Systematic Error).

He cites the experiments of Iranian psychologist Bahador Bahrami. Bahrami had pairs of subjects examine two displays and were asked to decide on each trial whether the first or second contained a near threshold target image. The subjects initially made the decision independently and if they differed were asked to resolve the conflict through a brief discussion. As long as the abilities of the individuals were similar, pairing them yielded a significant improvement in accuracy. Nuances were not was shared to gain this, but simply a categorical answer (first or second display) and a judgment of confidence.

Dehaene suggests that Bayesian decision theory tells us that the very same decision rules should apply to our own thoughts and to those that we receive from others. In both cases, optimal decision making demands that each source of information, whether internal or external, should be weighted as accurately as possible, by an estimate of its reliability, before all the information is brought together into a single decision space. This sounds much like cue validities in Brunswik’s lens model or Parallel Constraint Satisfaction theory. According to Dehaene, once this workspace was opened to social inputs from other minds, we were able reap the benefits of a collective decision making algorithm: by comparing our knowledge with that of others, we achieve better decisions.







Diversity or Systematic Error?


sysquality_sys_randomI have ignored group decision making to a large extent, but bootstrapping has somehow brought me back to it–especially dialectical bootstrapping which seems to be one person group decision making. Obviously, group decision making is important.  This post will focus on political decision making. Two books from 2007, Scott Page’s: The Difference — How the Power of Diversity Creates Better Groups, Firms, Schools and Societies and Bryan Caplan’s: The Myth of the Rational Voter–Why Democracies Choose Bad Policies look at it from far apart.

Continue reading

Dialectical Bootstrapping

bootstrapsI mentioned this in my last post and could not resist it. It is based on a 2009 paper by Herzog & Hertwig,  “The Wisdom of Many in One Mind Improving Individual Judgments With Dialectical Bootstrapping.”  How can a set of individually mediocre estimates become superior when averaged? The secret is a statistical fact that, although well known in measurement theory, has implications that are often not intuitively evident . A subjective quantitative estimate can be expressed as an additive function of three components: the truth (the true value of the estimated quantity), random error (random fluctuations in the judge’s  performance), and systematic error (i.e., the judge’s systematic tendency to over- or underestimate the true value). Averaging estimates increases accuracy in two ways: It cancels out random error, and it can reduce systematic error. This reminds me of Scott Page’s diversity prediction theorem which simply states that the crowd’s error = avg error- diversity. I expect to look at systematic error and diversity in future posts, but for now how can we conduct a dialogue with ourselves and improve our predictions?

Continue reading

Smart Mobs and Diverse Problem Solvers

smartmobsHoward Rheingold is the author of Smart Mobs -the Next Social Revolution, James Surowiecki is the author of The Wisdom of Crowds, and Scott Page is the author of The Difference -How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies.  Scott Page ties them together in his prologue. Each of them looks in his own way at how diversity can help us make better decisions.  They all build on that fact of cultural evolution that it is great for each of us to make better judgments, but that for us to progress we need to make better decisions together.  I will probably look at Scott Page’s book separately at some point, but here, in his case,  I will look at a paper  that he wrote with a colleague, Lu Hong, “Groups of diverse problem solvers can outperform groups of high-ability problem solvers.”

Continue reading