Perceptual presence

affiche - pleine pageThis post joins several others in being only tangentially related to JDM. It is based on the paper: “The felt presence of other minds: predictive processing, counterfactual predictions, and mentalizing in autism,” that appears in 2015 Consciousness and Cognition. The authors are Colin J. Palmer, Anil K. Seth and Jakob Hohwy. (Post Prediction error minimization)

A central ingredient of social experience is that we represent the mental states of other people. This sense of others’ mental states is a part of our understanding and anticipation of their behavior, and molds our own behavior correspondingly. If our friend shows up to the restaurant with a grim face, we have a sense of her mood and adjust our greeting accordingly. If she glances at our empty glass while pouring herself some wine, we have a sense of her intentions and might move our glass closer. This is the concept of mentalizing.


The authors suggest that we can conceive of mental states as just another feature of the world that comes to be modeled as the brain tries to best predict its sensory input. We represent mental states because this too allows the brain to better predict its sensory input over time. Mental state inferences are statistically constrained by representations of longer-term expectations – perhaps regarding, for example, the kind of mental states that people tend to have in a given context, or the sense of your friend’s mood that has been reflected in a variety of her behaviors since she showed up to the restaurant, or even culturally defined social contexts and norms. By minimizing prediction error, the most likely mental causes of the agent’s actions can be inferred from observation,

What differs between mentalizing and other forms of causal inference? In general, inference is made difficult by a lack of one-to-one relationships between causes and their effects. Firstly, mental states are relatively deeply hidden in the causal structure of the world. Secondly, we can expect a significant role of context in determining the relationship between mental causes and their sensory effects. A colleague’s glance at the clock during a meeting might reflect an eagerness to get home if it’s late in the day, or an annoyance at my lateness in showing up if it’s ten past nine in the morning. Deception and acting are more extreme examples of context dependency in the relationship between behaviors and the mental states that they reflect. A third challenge to causal inference regarding mental states is more specific to the social domain. When we interpret other people’s behavior, we are often aware that they are also interpreting us and trying to model our own mental states. For example, in joint action, if you and I are trying to coordinate to move a heavy table into the other room, your sense of my intentions should depend on a representation of my sense of your intentions. You know, for example, that I’m not going to try lifting the table until I think that you are also ready.

This brings us to perceptual presence. Is the recognition of a mental state in observed behavior  experienced as knowledge ‘in our head’ or as a real state of the external world? In normal circumstances perceptual content is characterized by subjective veridicality; that is, the objects of perception are experienced as real, as belonging to the world. When we perceive the tomato we perceive it as an externally existing object with a back and sides, not simply as a specific
view—a “perspectival take”—on an external scene.

But certain perceptual experiences seem to lack perceptual presence. An example is that of visual afterimages. If you gaze at a bright light and then look away you will experience a kind of residual watermark in your visual field. You can really see the afterimage in your field of view – but it isn’t perceived as a robust part of the external world and thus has little if any perceptual presence.

To account for perceptual presence, there is the idea that the hierarchical generative models implemented in the cortex include predictions about how the sensory input would change were we to interact with the world in the various possible ways that we can. These are called counterfactual predictions. For a given point in time, this is the case for both the actions that we will actually perform next and, importantly, a range of alternate actions that we could perform but won’t in this instance. To illustrate, when we look at the coffee cup on our desk the brain is engaged in predicting the sensory consequences of interacting with this object: like moving around the cup in various ways, or picking up the cup, or blocking our view of the cup with another object.

The notion of counterfactual prediction  builds upon  ‘active inference’. If the goal of the brain is to minimize prediction error: this can be achieved both by changing predictions to match the observed data and, via action, changing the sensory input to match predictions. When you drop the knife and then catch it with the other hand, you are using active inference. In this manner, prediction error minimization is an ongoing synthesis of active and perceptual inference.

One hypothesis  is that examples of perception that involve reduced perceptual presence (e.g., visual afterimages, synesthesia concurrents and certain hallucinations) are likely to be associated with a limited set of counterfactual predictions, as there are fewer ways in which we can act to change our input contingent on these kinds of (hypothesized) worldly causes. Correspondingly, perceptual presence comes about by virtue of there being a rich repertoire of
counterfactual predictions associated with the representation of the object or mental state in question.

If our friend is unhappy, this allows predictions about how the sensory input would change if I were to make a joke. If she is intending to reach out for the wine bottle to fill up her glass, this should shape predictions about the sensory effects of moving the bottle away from her, or moving my glass closer. Face-to-face conversation in particular illustrates a rich sequence of auditory and visual input that is quite directly modulated by our own utterances and expressive behaviors, the immediate sensory consequences of which are highly dependent on the succession of beliefs, intentions and emotions that we become aware of in our conversational partner throughout. The immediate sensory consequences of our actions – things we say, where we look, movements we make – are predictable in part based on the inferred mental states of those around us, and thus it is conceivable that mental state representations are associated with a broad repertoire of counterfactual predictions. Representations with a richer set of counterfactuals, whether due to the timescale or type of stimuli involved, are more likely to be experienced as real states of the perceptual external world as opposed to conceptual associations or explicit knowledge.

This leads to a hypothesis that social cognition in autism spectrum disorder is characterized by a diminished set of counterfactual predictions and the reduced perceptual presence of others’ mental states.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.