Category Archives: Uncategorized


This post is based on a paper: “Learning from experience in nonlinear environments: Evidence from a competition scenario,” authored by Emre Soyer and Robin M. Hogarth, Cognitive Psychology 81 (2015) 48-73. It is not a new topic, but adds to the evidence of our nonlinear shortcomings.

In 1980, Brehmer questioned whether people can learn from experience – more specifically, whether they can learn to make appropriate inferential judgments in probabilistic environments outside the psychological laboratory. His assessment was quite pessimistic. Other scholars have also highlighted difficulties in learning from experience. Klayman, for example, pointed out that in naturally occurring environments, feedback can be scarce, subject to distortion, and biased by lack of appropriate comparative data. Hogarth asked when experience-based judgments are accurate and introduced the concepts of kind and wicked learning environments (see post Learning, Feedback, and Intuition). In kind learning environments, people receive plentiful, accurate feedback on their judgments; but in wicked learning environments they don’t. Thus, Hogarth argued, a kind learning environment is a necessary condition for learning from experience whereas wicked learning environments lead to error. This paper explores the boundary conditions of learning to make inferential judgments from experience in kind environments. Such learning depends on both identifying relevant information and aggregating information appropriately. Moreover, for many tasks in the naturally occurring environment, people have prior beliefs about cues and how they should be aggregated.

Continue reading

Cognitive Penetration

This post is based on the paper: “Priors in perception: Top-down modulation, Bayesian
perceptual learning rate, and prediction error
minimization,” authored by Jakob Hohwy (see post Explaining Away) that appeared (or is scheduled to appear) in Consciousness and Cognition, 2017. Hohwy writes in an understandable manner and is so open that he posts papers even before they are complete of which this is an example. Hohwy pursues the idea of cognitive penetration – the notion that beliefs can determine perception.

Can ‘high level’ or ‘cognitive’ beliefs  modulate perception? Hohwy methodically examines this question by trying to create the conditions under which it might work and not be trivial. For under standard Bayesian inference,  the learning rate declines gradually as evidence is accumulated, and the prior updated to be ever more accurate. The more you already know the less you will learn from the world.  In a changing world this is not optimal since when things in the environment change we should vary the learning rate. Hohwy provides this example. As the ambient light conditions improve, the learning rate for detecting a visible target should increase (since the samples and therefore the prediction error has better precision in better light). This means Bayesian perceptual inference needs  a tool for regulating the learning rate. The inferential system should build expectations for the variability in lighting conditions throughout the day, so that the learning rate in visual detection tasks can be regulated up and down accordingly.

The human brain is thus hypothesized to build up a vast hierarchy of expectations that overall help regulate the learning rate and thereby optimize perceptual inference for a world that delivers changeable sensory input. Hohwy suggests that this makes the brain a hierarchical filter that takes the non-linear time series of sensory input and seeks to filter out regularities at different time scales. Considering the distributions in question to be normal or Gaussian, the brain is considered a hierarchical Gaussian filter or HGF .

Continue reading

Fuzzy-Trace Theory Explains Time Pressure Results

This post is based on a paper:  “Intuition and analytic processes in probabilistic reasoning: The role of time pressure,” authored by Sarah Furlan, Franca Agnoli, and Valerie F. Reyna. Valerie Reyna is, of course, the primary creator of fuzzy-trace theory. Reyna’s papers tend to do a good job of summing up the state of the decision making art and fitting in her ideas.

The authors note that although there are many points of disagreement, theorists generally agree that there are heuristic processes (Type 1) that are fast, automatic, unconscious, and require low effort. Many adult judgment biases are considered a consequence of these fast heuristic responses, also called default responses, because they are the first responses that come to mind. Type 1 processes are a central feature of intuitive thinking, requiring little cognitive effort or control. In contrast, analytic (Type 2) processes are considered slow, conscious, deliberate, and effortful, and they place demands on central working memory resources. Furlan, Agnoli, and Reyna assert that Type 2 processes are thought to be related to individual differences in cognitive capacity and Type 1 processes are thought to be independent of cognitive ability, a position challenged by the research presented in their paper. I was surprised by the given that intuitive abilities were unrelated to overall intelligence and cognitive abilities as set up by typical dual process theories.

Continue reading

Perception, Action and Utility: The Tangled Skein

dawdll-e1443732985775This is the first post in quite a while. I have been trying to consolidate and integrate my past inventory of posts into what I am calling papers. This has turned out to be time consuming and difficult since I really have to write something. As part of that effort, I have been reading Surfing Uncertainty– Prediction, Action, and the Embodied MInd authored by Andy Clark. (I note that this book is not designed for the popular press–it is quite challenging.) Clark refers to the extensive literature on decision making and pointed out a part of that literature unknown to me. With that recommendation, I sought out: “Perception, Action and Utility: The Tangled Skein,” (2012) in M. Rabinovich, K. Friston & P. Varona, editors, Principles of brain dynamics:  Global state interactions. Cambridge, MA:  MIT Press written by Samuel J. Gershman and Nathaniel D. Daw.

Gershman and Daw focus on two aspects of decision theory that have important implications for its implementation in the brain:
1. Decision theory implies a strong form of separation between probabilities and utilities. In
particular, the posterior must be computed before (and hence independently of) the expected
utility. This assumption is sometimes known as probabilistic sophistication. It means that I can state how much enjoyment I would derive from having a picnic in sunny weather, independently of my belief that it will be sunny tomorrow. This framework supports a sequentially staged view of the problem –perception guiding evaluation.
2. The mathematics that formalizes decision making under uncertainty, Bayes Theorem, generally assumes Gaussian or multinomial assumptions for distributions. Gershman and Daw note that these assumptions are not generally applicable to real-world decision-making
tasks, where distributions may not take any convenient parametric form. This means that if the brain is to perform the necessary calculations, it must employ some form of approximation.
Statistical decision theory, to be plausibly implemented in the brain, requires segregated representations of probability and utility, and a mechanism for performing approximate inference.

The full story, however, is not so simple. First, abundant evidence from vision indicates that reward modulation occurs at all levels of the visual hierarchy, including V1 and even before that in the lateral geniculate nucleus. Gershman and Daw suggest that the idea of far-downstream LIP (lateral intraparietal area) as a pure representation of posterior state probability is dubious. Indeed, other work varying rewarding outcomes for actions shows that neurons in LIP are indeed modulated by the probability and amount of reward expected for an action  probably better thought of as related to expected utility rather than state probability per se. Then recall that area LIP is only one synapse downstream from the instantaneous motion energy representation in MT. If it already represents expected utility there seems to be no candidate for an intermediate stage of pure probability representation.

A di fferent source of contrary evidence comes from behavioral economics. The classic Ellsberg
paradox revealed preferences in human choice behavior that are not probabilistically
sophisticated. The example given by Ellsberg involves drawing a ball from an urn containing 30 red balls and 60 black or yellow balls in an unknown proportion. Subjects are asked to choose between pairs of gambles (A vs. B or C vs. D) drawn from the following set:
Experimentally, subjects prefer A over B and D over C. The intuitive reasoning is that in gambles A and D, the probability of winning $100 is known (unambiguous), whereas in B and C it is unknown (ambiguous). There is no subjective probability distribution that can produce this pattern of preferences. This is widely regarded as violating the assumption of probability-utility segregation in statistical decision theory. (See post Allais and Ellsberg Paradoxes).

Gershman and Daw suggest two ways that the separation between probabilities and utilities might be weakened or abandoned:

A. Decision-making as probabilistic inference
The idea here is that by transforming the utility function appropriately, one can treat it as a probability density function parameterized by the action and hidden state. Consequently, maximizing the “probability” of utility with respect to action, while marginalizing the hidden state, is formally equivalent to maximizing the expected utility. Although this is more or less an algebraic maneuver, it has profound implications for the organization of decision-making circuitry in the brain. The insight is that what appear to be dedicated motivational and valuation circuits may instead be regarded as parallel applications of the same underlying computational mechanisms over eff ectively di fferent likelihood functions.

Karl Friston builds on this foundation to assert a much more provocative concept: that for biologically evolved organisms, the desired equilibrium is by de finition just the species’ evolved equilibrium state distribution. The mathematical equivalence rests on the evolutionary argument that hidden states with high prior probability also tend to have high utility. This situation arises through a combination of evolution and ontogenetic development, whereby the brain is immersed in a   “statistical bath” that prescribes the landscape of its prior distribution. Because agents who find themselves more often in congenial states are more likely to survive, they inherit (or develop) priors with modes located at the states of highest congeniality. Conversely, states that are surprising given your evolutionary niche, like being out of water, for a fi sh, are maladaptive and should be avoided. (See post Neuromodulation.)

B. The costs of representation and computation
Probabilistic computations make exorbitant demands on a limited resource, and in a real physiological and psychological sense, these demands incur a cost that debits the utility of action. According to Gershman and Daw, humans are “cognitive misers” who seek to avoid        e ffortful thought at every opportunity, and this e ffort diminishes the same neural signals that are excited by reward. For instance, one can study whether a rat who has learned to lever press for food while hungry will continue to do so when full; a full probabilistic representation over outcomes will adjust its expected utilities to the changed outcome value, whereas representing utilities only in expectation can preclude this and so predicts hapless working for unwanted food. The upshot of many such experiments is that the brain adopts both approaches, depending on circumstances. Circumstances elicit which approach can be explained by a sort of meta-optimization over the costs (e.g. extra computation) of maintaining the full representation relative to its benefi ts (better statistical accuracy).


Coherence from the default mode network

coherenbrainThis post starts with the paper “Brains striving for coherence: Long-term cumulative plot formation in the default mode network,” authored by K. Tylén, P. Christensen, A. Roepstorff, T. Lund, S. Østergaard, and M. Donald. The paper appeared in NeuroImage 121 (2015) 106–114.

People are  capable of navigating and keeping track of all the parallel social activities of everyday life even when confronted with interruptions or changes in the environment. Tylen et al suggest that even though these situations present themselves in series of interrupted segments often scattered over huge time periods, they tend to constitute perfectly well-formed and coherent experiences in conscious memory. However, the underlying  mechanisms of such long-term integration is not well understood. While brain activity is generally traceable within the short time frame of working memory, these integrative processes last for minutes, hours or even days.

Continue reading

2nd Half of 2009 JDM Research Summary

IMG_0502This post is based on a review paper “Mindful Judgment and Decision Making,” Elke U.Weber and Eric J. Johnson, Annual Review of Psychology, 2009, 60:53–85, on the state of judgment and decision making research. The post is the second half in the outline form set up by Weber and Johnson and more or less summarizes accumulated knowledge.  All I have done is create a sort of Cliff’s Notes version. Much of it is directly quoted without proper attribution in the interest of clarity and my laziness.


1. The emotions revolution of the past decade or so has tried to correct the overemphasis on analysis by documenting the prevalence of affective processes, depicting them as automatic and essentially effort-free inputs that orient and motivate adaptive behavior.

Continue reading

What to do about emotion?

passionThis is the second post based on a paper: “Emotion and Decision Making,” that is to appear in the 2014 Annual Review of Psychology. It was written by Jennifer S. Lerner, Ye Li, Piercarlo Valdesolo, and Karim Kassam.

David Hume: “Reason is, and ought only to be, the slave of the passions, and can never pretend to any other office than to serve and obey them.”

Still, most of us have made some bad decisions under the influence of emotion. There are unwanted effects of emotion on decision making, but as Lerner et al note, they can only sometimes be reduced.

The strategies to reduce unwanted effects broadly take one of two forms: (a) minimizing the magnitude of the emotional response (e.g., through time delay, reappraisal, or inducing a counteracting emotional state), or (b) insulating the judgment or decision process from the emotion (e.g., by crowding out emotion, increasing awareness of misattribution, or modifying the choice architecture).

Continue reading

Persistence by Hogarth

regimeshiftfigure1The last post Persistence looked at persistence as being partially determined by the distribution of the waiting times for the reward. A fat tailed distribution might rationally steer one toward giving up after a short waiting period. Robin Hogarth (Educating Intuition) has recently published a paper: “Ambiguous Incentives and the Persistence of Effort: Experimental Evidence” in the Journal of Economic Behavior & Organization, Volume 100, April 2014, page 1-19, with Marie Claire Villeval that looks at economic activities where the reward is mundane–money.  It is more aimed at looking at what determines our persistence from the employers point of view, but I believe it could be more broadly applicable.

Hogarth and Villeval explore ambiguous situations where economic agents reap the benefits of engaging in an activity across time until – unknown to them – there is a shift (the regime change) in the underlying process and pursuing the activity is no longer profitable. The term regime shift was new to me in the context.  For an old city planner, regime shift meant a new mayor or change in the form of government. Apparently the ecological term more or less runs with the old definition and means abrupt long lasting non-linear change.  Hogarth has helped me understand that humans have made an evolutionary career out of understanding linear change or functions that are linear over the relevant range, while we tend to be weak at non-linear functions. How long will the investor continue to place new orders and does this depend on the regularity of his previous outcomes? How long will an employee keep working in the same firm if she no longer receives a bonus? How is the decision affected by preferences regarding risk and ambiguity and/or the regularity with which bonuses have been paid in the past?

Continue reading

Deciding Not to Decide

Hillary-Clinton-lecture-jpgThis post is an executive summary of a 2013 paper about deciding not to decide. (“Deciding Not to Decide: Computational and Neural Evidence for Hidden Behavior in Sequential Choice,”  by Sebastian Gluth, Jorg Rieskamp, and Christian Buchel, that appeared in PLoS Comput Bio 9(10). Quite frankly the detail of the paper is beyond me, but the general ideas are interesting.

Many decisions are not triggered by a single event but based on multiple sources of information. When purchasing a new computer, for instance, we certainly look at the price, but not without accounting for further aspects like capabilities, quality and appearance. According to Gluth et al, usually, these multi-attribute decisions evolve sequentially, that is, as long as the collected evidence is insufficient to motivate a particular choice we search for more information to resolve our uncertainty. Importantly, such ‘‘decisions not to decide’’ are not directly observable but can promote significant changes in behavior.

Continue reading

Spiral Model of Musical Decision Making

bach800px-VI_allemande_1This post is based on, “A spiral model of musical decision-making,” written by Daniel Bangert, Emery Schubert and Dorottya Fabian that appeared in Frontiers of Psychology on April 22, 2014. Although based on thin research, my intuition likes it, and it would seem to have applicability beyond music. It splices together ideas of Ken Hammond (post Cognitive Continuum), Jonathan Evans (post Dual Process Theories of Cognition), and Amy Baylor (post U-Shaped Intuition).

Research has shed light on how both intuition and deliberation are used by musicians. Bangert et. al. refer to Hallam who interviewed twenty-two performers about their practice habits and found differences between those who were “intuitive/serialists” who allowed their interpretation to evolve unconsciously versus “analytic/holists.” who relied on deliberate, conscious analysis of the piece. Other research has shown that while performing, musicians pay deliberate attention to certain specific musical aspects (performance cues) and also have spontaneous performance thoughts.

Continue reading