6,600 words

Sign up for the Blank Horizons mailing list here.

Definitions

1. As in previous essays, I use “perceive,” “observe,” and “experience” synonymously.
2. A “quale” (plural: “qualia”) is defined as the way that a subjective experience feels. We are all very deeply familiar with qualia; we all know what it is like to feel pain, what it is like to see the color red, and so on. I treat “quale” as a synonym of the phrase “object of consciousness.” Note that “object of consciousness,” in this essay, is referring to the form in which an object appears to our consciousness, not the actual “thing-in-itself” that may or may not underlie the appearance.

Preliminary Notes

1. This article is the fourth and final chapter in a series that was originally about the relationship between consciousness and time. However, as I discuss in the section titled “Recap,” I discovered that this relationship is inextricably linked to the unity of consciousness. The mystery of how consciousness is unified is the subject matter of the Binding Problem, hence the title of this blogpost.
2. The goal of this blog is to integrate neuroscience, philosophy, physics, and spirituality into a comprehensive framework for understanding consciousness. Up until now, the blog has addressed each of these subject matters separately, in different essays. This is the first blogpost that actually unifies perspectives from all four fields.
3. In a comment on my July blogpost, British philosopher David Pearce wrote that a proper solution to the Binding Problem (why do we have a unified consciousness?) must also solve two other core problems of consciousness: the Hard Problem of Consciousness (how does consciousness arise from the brain, if at all?) and the Problem of Temporal Experience (why do we have a single experience of ourselves that persists over time?). Accordingly, my theory on the unity of consciousness, which will be the central focus of this essay, will address all three problems (I refer to these as the Trinity of Consciousness Problems).

Recap

Back in May, I started to think about the relationship between time and consciousness. I soon realized that I was unsure of not only the solution that I wanted to propose, but also the question that I was trying to answer in the first place. I knew that there was something deeply significant yet enigmatic about the “stream of consciousness,” but I couldn’t quite put my finger on where exactly the mystery lies. After reading the entry on “Temporal Experience” in the Stanford Encyclopedia of Philosophy, I understood the essential problem, which I have called the Problem of Temporal Experience: why is it that we have an experience of succession rather than a succession of experiences? In other words, why do we have a unitary consciousness over time rather than a series of disparate, disconnected experiences? Why do I perceive, deep down, that the person who I was five minutes ago is identical to the person who I am right now? (1)

In June, I wrote that the most compelling explanation of the unity of consciousness over time claims that our memory of the immediate past is bound to our present moment of experience. That is, the brain retains “echoes” or “after-images” of our experience from up to a few seconds ago, and these are integrated with our present sensations, thoughts, feelings, and so on (for a more in-depth explanation, see the section below on the “Moments of Consciousness Model”). How does our consciousness bind, or “glue,” these “retentions” from the immediate past to our awareness of the present? How exactly does binding happen in general?

In July, I realized that the Binding Problem is highly non-trivial. A leading hypothesis in neuroscience claims that the synchronization of neuronal activity is responsible for the binding of experience. However, this view is philosophically misguided. We have one, unified experience at a given moment in time, yet the neurons that encode perceptual features are separate entities, even though they may be firing electrical impulses at the same time. Indeed, neurons that do the same thing may not necessarily be the same thing, but the various subjective phenomena in our consciousness are one experience. Furthermore, we don’t even have to appeal to philosophy in order to demonstrate that the neuroscientific explanation is inadequate. There are neurons in my brain that are firing at the same time as those in my grandmother’s brain, but there isn’t a single consciousness that unifies the two of us (…or so we think).

This blogpost will seek to offer a novel, philosophically satisfying solution to the Binding Problem.

Why the Binding Problem matters, part I: the nature of consciousness

Before I move forward, I want to address an important question: what’s the significance of the unity of consciousness? While it is too early to offer concrete details on the practical implications of a successful theory of the unity of consciousness, such a theory would lay the foundation for a theoretical understanding of consciousness in general. Why?

Consciousness differs from physical matter – or, depending on your metaphysics, other kinds of physical matter – in this fundamental sense: we determine the unit of physical matter by dividing it into smaller and smaller components, until we eventually get to an irreducible element that we call the atom. On the other hand, the unit of consciousness cannot be derived from “splitting” consciousness into tinier pieces. Indeed, to speak of “splitting” consciousness is actually nonsensical, since consciousness is always unified. It is impossible to have multiple, disjoint consciousnesses of the world; that is, the field of visual awareness that encompasses the laptop in front of me cannot be entirely separate from the field of visual awareness that includes the window above me. Even split-brain patients, in whom the neurons connecting the left and right hemispheres of the brain are severed, do not experience a divided consciousness. As the philosopher Tim Bayne writes, “perceptual features cannot enter consciousness without first being bound together to form percepts of unified objects.” This “binding constraint” seems to be one of the fundamental truths about the nature of consciousness. Thus, we could even state that the unit of consciousness – insofar as consciousness can even be broken down into units (“prerequisite” may be a more fitting word) – is the unity of consciousness. Hence, any robust theory of consciousness must explain how consciousness is unified and thereby propose a solution to the Binding Problem.

Furthermore, when we say that we are “more conscious,” we don’t mean that there are more objects populating our field of conscious awareness, but instead that there is greater unity between different perceptual objects. When a person experiences a life-threatening emergency, which typically results in a heightened state of consciousness, perceptual features that he would normally ignore suddenly start to capture his attention. Someone who is trying to escape from a fire will likely notice many details about his surrounding environment that would otherwise seem insignificant to him: the orientation of the nearest doorknob, the width of the windows, and so on. The number of objects in his environment has not increased as a result of the fire; rather, because the scope of his attention has expanded, he has bound more of these objects to his field of conscious awareness. Thus, the “amount” of consciousness that a person has appears to be inextricably linked to the amount of binding within his conscious experience.

 

Why the Binding Problem matters, part II: The Moments of Consciousness Model

As I said earlier, I believe that a solution to the Binding Problem will also solve the Problem of Temporal Experience. I want to elaborate on this idea.

In his book “The Mind Illuminated: A Complete Meditation Guide Integrating Buddhist Wisdom and Brain Science,” Culadasa (formerly known as John Yates) puts forward the “Moments of Consciousness Model.” The model claims that we are only conscious of a single object of attention at any given moment of time. Fascinatingly, the strongest – or, at least, the oldest – evidence for the Moments of Consciousness Model stems from the experience of advanced meditators. When these meditators concentrated very, very intensely on their breathing, they noticed that they couldn’t attend simultaneously to both the feeling of air passing through their nostrils and the expansion of their lungs as they inhaled.

However, this model may seem highly counter-intuitive. After all, aren’t I conscious of multiple objects of attention at the same time? For instance, as I look out into the world, aren’t I conscious of not only the furniture that I see before me, but also the sounds that I hear around me, the feeling of my back against the chair that I’m sitting on, and so forth?

Yes, Culadasa would agree that I am conscious of all these manifold sensations. However, he also argues that a single moment of experience is very, very brief, perhaps no more than several milliseconds in some cases. However, many consecutive moments of experience get bound together in “working memory,” which is essentially a type of short-term memory. The length of time that a moment of experience is retained in working memory is probably anywhere from 10 to 100 times longer than the length of the moment itself. In other words, up to a hundred moments may have to pass before a single object of attention fades from working memory. Furthermore, at any given moment, we are conscious of not merely the present object of attention, but also the entirety of our working memory.

For instance, at moment t₁, I attend to my visual perception of the laptop in front of me. At moment t₂, my eyes make a microsaccade – that is, they make a tiny movement to – the window above the table. (Note that these microsaccades are often barely perceptible, and they happen very rapidly, hence we typically fail to notice that our eyes are quickly switching between many different objects of attention.)  At moment t₃, my attention shifts away from my visual perception to my auditory awareness of the background noise in my room. Moments t₁, t₂, and t₃ are all distinct from one another, but I am conscious of all three of them – and many more – because they are all bound together in working memory.

Thus, we experience the flow of time because every moment of experience contains, and is therefore bound to, our working memory of prior moments. Without working memory, we would have a succession of disparate experiences rather than a single, unified experience of succession.

The general intuition

What does it mean for consciousness to be unified? In order to give an answer, perhaps we need to consider a meta-question first: what does it mean for any two things to be unified?

Here is the crucial insight: unity implies invariance. Unified objects are one and the same with one another because there is something that does not vary between them. When different political states are unified into a single country or empire, the states do not vary from each other in virtue of the fact that they belong to one governing entity. This entity is the invariant “glue” that binds together the various states. Thus, in order to determine what unifies two objects together, we need to search for invariance.

In physics, the study of invariance is essentially equivalent to the study of “symmetry.” Symmetry is actually a very general term, and most of its applications in physics have very little to do with the symmetry that we are ordinarily familiar with. However, before describing more complex symmetries and their relationship to the unity of consciousness, I will begin by discussing a commonplace example of symmetry.

Let’s consider the rotational symmetries of a square. A rotation of the square by 90, 180, 270, or 360 degrees will leave the square invariant; that is, the square remains unchanged after any one of these rotations. All four rotations of the square are unified with one another because they look identical to one another.

How does this example relate to consciousness? Let’s say that there are only four objects of consciousness in my field of awareness: my laptop, the window above me, the background noise in my room, and the weight of my backpack as it presses down on my shoulders. Additionally, I represent the mathematical structure of my consciousness of each of these objects with a square. (Perhaps you’re wondering: What on earth does it mean to represent the mathematical structure of my consciousness of a laptop with a square? I’ll address this question in the next section.) Furthermore, all four of the squares are rotations of one another, such that the squares all appear identical. They differ only in the permutation of their vertices. That is, if one were to (arbitrarily) assign a number to each vertex of the square, as I have in the diagram below, the arrangement of the numbers is the only feature that varies between the squares.

Fig 1. In the above example, the unity of consciousness is encoded in the “rotational” symmetries of a mathematical structure underlying objects of consciousness.

This example reveals a broader insight: different objects of consciousness are unified with one another because there is a single, invariant mathematical structure that describes all of them. In other words, different objects of consciousness are distinct “rotations” of an underlying mathematical structure. When the “rotations” leave the structure unchanged, the objects are unified with one another. (Note the scare quotes around “rotations”; as I will explain later, rotations are a specific category of a more general kind of transformation. Phenomenal binding is encoded in the symmetries that are preserved through these transformations.)

This might seem like not only a deeply speculative claim, but also one that is dangerously vague. How does one determine the mathematical structure of an object of consciousness, and what does it even mean for consciousness to have a mathematical structure? Why do the rotational symmetries of that structure – and not other kinds of symmetries – correspond to the unity of consciousness? Given that the mathematical structure of an object of consciousness almost certainly isn’t as simple as a polygon, let alone a square, what would it actually mean to rotate the structure? I will answer all of these questions in the sections below.

A more concrete example

My approach to the Binding Problem presupposes that there is a mathematical object that corresponds to consciousness. This philosophical assumption is known as qualia formalism, an idea first explicitly developed by the Qualia Research Institute (QRI) in 2016 (disclaimer: I interned for QRI last summer). As Mike Johnson, CEO of QRI, writes, qualia formalism “is a formal way of saying that consciousness is in principle quantifiable- much as electromagnetism, or the square root of nine is quantifiable.” The term “mathematical object” is incredibly broad; it could refer to a set of equations, a matrix, a probability distribution, or anything else that can be codified in the language of mathematics. However, QRI also established the principle of qualia structuralism, the notion that the mathematical object that describes consciousness has a rich structure and perhaps even a geometry. Qualia structuralism narrows the set of mathematical objects that could describe consciousness – a set of equations may not necessarily have a geometry – but the “geometry of consciousness” might still seem like an ambiguous concept. What’s a concrete example?

Certain objects of consciousness actually have a very well-defined geometry. For instance, all perceivable colors can be described as a combination of three values: red (R), green (G), and blue (B). (2) Hence, each color can be defined as a point on a coordinate system that consists of three axes: R, G, and B. There are 256 different shades of red, green, and blue in this representation, so the lowest value for each coordinate is 0 and the highest is 255. As seen in the diagram below, the geometry of (consciously) perceived colors, when these colors are plotted in “RGB space,” is a cube. Furthermore, I will tentatively define a color quale as a vector in RGB space.

Fig 2. RGB color space. Image taken from here.

We can also characterize the geometry of (consciously) perceived spatial orientations as a cube. The spatial orientation of a “rigid body” (e.g. an airplane) is defined in terms of three values: its angle in the xy-plane (i.e. the angle of its rotation around the vertical axis), its angle in the xz-plane (i.e. the angle of its rotation around the side-to-side axis), and its angle in the yz-plane (i.e. the angle of its rotation around the front-to-back axis). (3) Each coordinate has a minimum value of 0 and a maximum value of 360.

Fig 3. Orientation space.

Evidently, perceptual color space and perceptual orientation space share the same rotational symmetries; for instance, both spaces will remain invariant if they are rotated about the vertical axis by 90 degrees (I admit that this is an imperfect example, since the axes of color space have a different length than the axes of orientation space). Now, anyone who has the ability to see is visually conscious of both color and spatial orientation at the same time; in other words, color qualia and orientation qualia are bound together. I claim that the binding of color and orientation arises from the shared rotational symmetries of their corresponding vector spaces. In particular, the coordinate axes of orientation space are a rotation of the coordinate axes of color space, and vice versa. An orientation quale is encoded by a vector in orientation space, and it becomes bound to a color quale when the coordinate axes that define the (orientation-encoding) vector get rotated into the coordinate axes of color space.

For instance, in the diagram below, I have drawn an arbitrary vector in orientation space. I then rotate the coordinate axes for orientation space by 90 degrees, and I now have the coordinate axes for color space. Thus, the vector, which formerly represented consciousness of orientation, now encodes consciousness of color. Notice that I have not changed the vector; I have merely changed the coordinate axes of the vector. This is a crucial point: orientation qualia and color qualia are actually encoded by identical vectors, hence they are bound together in conscious experience. Orientation and color appear to be distinct from one another solely because they are defined by different coordinate axes.

Fig 4. The coordinate axes of RGB space are a rotation of the coordinate axes of the orientation space. Thus, the same vector is expressed by a different set of axes; more precisely, each set of axes is associated with certain basis functions, so the same vector can be represented by a different linear combination of those functions. In the diagram, o₁, o₂, and o₃ are the basis functions associated with orientation space, and r₁, r₂, and r₃ are the basis functions for RGB color space. c₁, c₂, and c₃ are “scalar” coefficients.

Perhaps the role of rotational symmetries in phenomenal binding still remains unclear. After all, one could rotate the coordinate axes of a vector space by any amount without changing the vector itself. Why do the coordinate axes of orientation space need to be rotated by 90, 180, 270, or 360 degrees, as opposed to any other angle, in order to bind orientation qualia to color qualia? The reason is quite simple: if one were to rotate the coordinate axes of orientation space by any other angle, certain vectors would then lie outside the bounds of the space (see Fig 5).

Fig 5. Certain vectors lie within orientation space when it is rotated 0 degrees, but not when it is rotated 50 degrees.

It is important to note that the example in this section does not generalize to all objects of consciousness. Most perceptual spaces cannot be described as cubes; that is, they cannot be decomposed into three simple axes. Some spaces may not even have a Euclidean geometry. For example, Andrés Gomez Emilsson, director of research at QRI, speculates that the space of smells may have a hyperbolic geometry. In order to formulate a robust theory of phenomenal binding, we must find a broad yet precise way of characterizing “qualia spaces” and the symmetries that relate them to one another.

Rotations all the way down: panpsychism and fundamental physics

How would one go about the gargantuan, seemingly impossible task of devising such a theory? The answer depends on our underlying philosophy of consciousness. A neuroscientist who subscribes to materialism would likely argue that the structure must be instantiated in the brain. A panpsychist, on the other hand, would claim that the structure cannot be localized to the brain, since consciousness is actually interwoven into the fabric of the universe. Everything is conscious, according to this view, and when coupled with the notion that there must be a mathematical formalism that describes consciousness, panpsychism naturally yields the conclusion that the structure of consciousness must be encoded in fundamental physics. I have offered a defense of panpsychism in a previous essay. While panpsychism sounds crazy, it is actually a highly viable theory of consciousness, as long as it is distinguished from animism: the view that everything is alive and therefore possesses agency, intentionality, thoughts, emotions, etc. Elementary particles almost certainly are not endowed any of these attributes, but according to (my take on) panpsychism, they have a very fundamental kind of consciousness, perhaps something akin to the feeling of presence or “being there.”

In quantum mechanics, which describes the most fundamental level of reality, states of physical systems are described by vectors in a Hilbert space, a finite- or infinite-dimensional vector space that is equipped with a structure known as the inner product. (The inner product is essentially a way of multiplying two vectors together.) The inner product can yield a function for measuring distance in Hilbert space, in which case we can meaningfully speak of the space’s geometry. Hence, it is possible to rotate the coordinate axes of a Hilbert space. In the language of quantum mechanics, such rotations are known as unitary transformations.

These unitary transformations are essentially generalizations of the “symmetry group” of an n-dimensional sphere, i.e. the set of transformations under which such a sphere would remain invariant. (4) Just as a sphere looks the same no matter the angle at which it is rotated, the Hilbert space remains invariant under unitary transformations. (It is worth noting, however, that Hilbert spaces should not be visualized as spheres, even ones of arbitrarily high dimensions.)

If panpsychism is true, the mathematical structure of consciousness is the Hilbert space. (Does this claim therefore imply that the brain is instantiating a Hilbert space, one that is somehow far more “complex” than the Hilbert space of quantum phenomena? Yes, but more on that to follow.) Furthermore, the state space of each quale – that is, the “map” of all the possible states of that quale – is determined by the coordinate axes of the Hilbert space; these axes are known as basis functions in mathematics. I claim that phenomenal binding is encoded in unitary transformations between the basis functions of an underlying Hilbert space.

Before I offer a detailed example, it would be helpful to review some postulates of quantum mechanics. Every “observable” quantity has an associated mathematical function known as an operator.  The possible states of the observable are known as the “eigenfunctions” of the operator. Furthermore, the eigenfunctions serve as the basis functions of the state space of the observable. The vectors in the Hilbert space are linear combinations of the eigenfunctions.

For instance, the eigenfunctions of the position operator are the basis functions of “position space.” Likewise, the eigenfunctions of the momentum operator are the basis functions of “momentum space.” (Position and momentum are the two most basic observables of quantum mechanics.) Additionally, performing a unitary transformation on the position eigenfunctions yields the momentum eigenfunctions. (5) In other words, it is possible to “rotate” from the basis functions of position space to the basis functions of momentum space. When this rotation (i.e. unitary transformation) takes place, the same vector that had previously represented the momentum state of the system now expresses the position state. Crucially, the structure in which the vector resides – the Hilbert space – has not changed. Position space and momentum space are one and the same Hilbert space; they are distinct from one another merely because they represent the vectors in that space with different basis functions.

How does this example relate to consciousness? According to my interpretation, observables are just objects of consciousness, i.e. qualia, and operators are their mathematical encoding. The eigenfunctions of the operators for position and momentum qualia can be viewed as rotations of a single, underlying Hilbert space. Position and momentum qualia are bound with each other because the Hilbert space always remains invariant under these rotations. This interpretation yields the conclusion that quantum systems have a unified consciousness of both position and momentum.

As consciousness becomes more “complex” – that is, as it becomes closer to the consciousness that we humans are familiar with – it becomes capable of observing a greater variety of qualia. We humans are aware of not only position and momentum but also color, emotion, touch, etc. As the number of observables increases dramatically, so does the number of corresponding operators. For any operator that is associated with an actual observable, it is a necessary condition that the basis functions of the operator are unitary transformations of the basis functions for the operators associated with all the other observables that are being perceived. Indeed, in order for a quale to enter a person’s consciousness, it must be bound with all the other qualia that he is consciously observing. (7)

However, no matter how “advanced” consciousness is, the mathematical structure that describes it will always be a Hilbert space. This begs a question that I raised previously: given that most of the complexity of consciousness supposedly emerges from activity in the brain, is the brain encoding a Hilbert space? Aside from purely philosophical speculation, is there any reason to think that the answer is yes?

As far as I am aware, very few reputable neuroscientists, if any, have even sought to demonstrate that states of the brain can be represented as vectors in a Hilbert space. (Here is one exception.) I do not claim to have any empirical evidence suggesting that such a representation exists, but in the following section, I will present some (hopefully) compelling theoretical justifications for the notion that the state space of brain activity can be accurately characterized as a Hilbert space.

A Hilbert space formulation of neuroscience, based on the Free Energy Principle

First, an important clarification: I am not arguing that there are real quantum phenomena that are taking place in the brain. I disagree with the scholars who think that the unity of consciousness is a result of actual quantum entanglement. (Incidentally, I also doubt that consciousness causes the “collapse of the wave function,” the process by which superpositions of multiple quantum states reduce to single values during measurements.) As I explained in a previous blogpost, quantum entanglement decays in less than 10^(-13) seconds, which is far shorter than the timescale of most, if not all, neurobiological processes. Rather, I assert that the mathematical formalism of quantum mechanics can also be applied directly to the study of consciousness, and hence, the brain. Structures like the Hilbert space are defined independently of their role in quantum mechanics. Thus, a Hilbert space formulation of neuroscience does not necessarily imply that neural processes are quantum in nature.

I claim that the Hilbert space representation of neural activity is motivated by the Free Energy Principle (FEP), one of the few overarching, theoretical frameworks for understanding how the brain works. (FEP was developed by legendary neuroscientist Karl Friston.)  FEP claims that the brain is fundamentally seeking to minimize free energy, an information theory quantity that corresponds to prediction error: the difference between the brain’s predictions of sensory signals and the signals themselves. In order to make these predictions, the brain must encode a probability distribution of the causes of sensory signals. Optimizing the parameters of the distribution, also known as the “sufficient statistics,” corresponds to a reduction of prediction error. For instance, a person who is trying to “feel out” his location in a pitch-black room might be (unconsciously) computing a probability distribution of the nearest exit. Based on his past experience, the “peak” of the distribution, which would be the mean if the distribution were Normal, might be 5 inches to the left of his current position. If he reaches out to his left and feels a table instead, he will update and thereby optimize the probability distribution.

This example may make FEP seem more like a theory of decision-making than a theory of consciousness. While Friston has been careful to state that FEP is not an explanation of the origins of consciousness (personal communication), he does believe that consciousness is actually nothing more than the brain’s process of making predictions about the world. For example, the conscious perception of red is the brain’s prediction of the type of color that is present in the environment, based on the sensory, i.e. visual, signals that the brain is receiving. The brain makes these predictions on the basis of probability distributions that represent the likeliness that the perceived color is red, or blue, or green, etc. Extending Friston’s line of thinking, we could argue that different probability distributions represent different states of a certain quale.

Furthermore, I argue that the mathematical encoding of a quale is an operator in a Hilbert space of these probability distributions. The basis functions of the space are determined by the eigenfunctions of the operator associated with each distribution, and the vectors of the Hilbert space are the distributions themselves. Additionally, the basis functions for any two distributions are unitarily equivalent – that is, the basis function for one is a unitary transformation of the other.

Why should the distributions be represented as elements in a Hilbert space, though, and not just any ordinary vector space? The human brain – and, I suspect, the brains of other animals as well – likely encodes distributions of many different shapes. (Examples of shapes include Normal, Poisson, multinomial, binomial, etc.) Given the utterly immense variety of qualia that we are capable of experiencing, it is highly improbable, if not downright impossible, that all the probability distributions in the brain, which are essentially encoding the possible states of qualia, could be described with a small set of shapes. Moreover, it could be the case that the probability distributions instantiated by the brain have shapes that statisticians haven’t “discovered” yet. Yet many neuroscientists presuppose that the distributions are either Normal or multinomial, and while others do not make such strong assumptions about the shapes of the distributions, some may nevertheless presume that neural variability is “Poisson-like,” which may not be true. How does the brain optimize an arbitrary distribution that could, in principle, have any shape?

As I hinted above, each distribution is completely determined by a set of sufficient statistics, which are parameters like mean and variance. A Normal distribution is fully defined by its mean and variance, but these two statistics alone would not be sufficient for characterizing an arbitrary distribution. In order to compute all the necessary statistics of an arbitrary distribution, the brain would need to have some method of storing infinitely many higher-order statistics (i.e. “moments”). In the words of statistician Eric Xing, “higher order moments bring increasingly greater resolution power for characterizing arbitrary distributions, which leads to the intuition that an infinite dimensional vector consisting of moments will absolutely capture any distribution (emphasis added).” 

Hence, an infinite-dimensional vector space like the Hilbert space can represent arbitrary distributions. While, as Xing notes, “storing or manipulating a vector of infinite dimensions [in an actual physical system] is impossible,” there is actually a neat mathematical trick that makes it possible to “embed” a distribution in Hilbert space and evaluate its moments without operating in infinite dimensions (see footnote 6 for the technical details.)

A physical implementation of Hilbert space?

The neuroscientist David Marr famously distinguishes between three levels of analysis: computational, algorithmic, and implementational. Thus far, I’ve primarily given an algorithmic account of binding: that is, I’ve specified the form in which the brain represents qualia (i.e. expectation values of probability distributions) and the algorithms with which those representations are computed (i.e. the math involved in calculating those expectation values). However, I haven’t offered an implementational account. In other words, I haven’t addressed the question: how does the brain encode a Hilbert space of probability distributions at the level of neurons, synapses, neurotransmitters, and so on?

Given the astounding complexity of the brain, and hence the difficulty of drawing clear trends about its activity, it will probably take a lifetime’s worth of work – if not multiple lifetimes – to provide a definitive answer. For inspiration, we should turn to Friston’s theory of the neurobiological implementation of the Free Energy Principle (see the “Neuronal Implementation” section in this paper). In particular, he argues that the sufficient statistics, i.e. moments, of the brain’s probability distributions depend on three variables: representations of environmental states (R); the causes of those states (C); and uncertainty (U), in order to account for random fluctuations that may affect the brain’s assessment of the environment. According to Friston, R could be encoded by the firing of action potentials; C by the strength of the connections by neurons; and U by the sensitivity of post-synaptic neurons to pre-synaptic input. If the probability distribution is Gaussian, then the process of optimizing its moments is encoded in the act of “hierarchical message-passing,” in which “forward-driving” connections from lower-level neurons to higher-level neurons convey prediction error, and backward connections mediate predictions.

Friston and I differ in a couple key regards: I believe that the moments of the brain’s probability distributions could be determined by a potentially infinite number of variables. Additionally, I think that the distributions may not necessarily be Gaussian, hence hierarchical message-passing may not be sufficient for optimizing the moments of all possible distributions. However, Friston’s ideas lay an important foundation for further progress on this research.

Putting it all together: a very preliminary theory of consciousness

At the beginning of this essay, I said that I would seek to address the Trinity of Consciousness Problems. While I certainly don’t believe that I have solved any of them, I have tried to offer some preliminary thoughts on a very general approach to these deep mysteries.

The Hard Problem of Consciousness

• Metaphysics: Everything is conscious (though most things are significantly less conscious than living creatures) and consciousness has a mathematical structure.
• Science: The mathematical structure of consciousness is a Hilbert space. The possible states of a quale, i.e. an object of consciousness, are determined by the basis functions of the Hilbert space. A quale itself is mathematically encoded by an operator in the space. The eigenfunctions of the operator are equivalent to the basis functions of the space, which determine the possible states of the quale; each state  can be expressed mathematically as a linear combination of these functions. In the brain, the basis functions, as well as their linear combinations, correspond to probability distributions.

The Binding Problem 

According to the Moments of Consciousness Model, our attention focuses on a single quale at each moment of time. Working memory contains not only the quale that we are attending to in the present, but also objects from the immediate past. Furthermore, we have a single, unified consciousness of all the various qualia stored in working memory. That is, our consciousness of one quale is the same as our consciousness of all the other qualia. For instance, my consciousness of the table in front of me is one and the same consciousness as my consciousness of the background noise around me. This statement does not imply that I am incapable of distinguishing between the table and the background noise. But if the contrary is true, then how can I say that my consciousness of one is identical to my consciousness of the other?

We can think of consciousness as a structure that contains a single quale. Consciousness is unified because the structure does not change, even though the quale does. As I mentioned above, the structure of consciousness is the Hilbert space, and the possible states of the quale are mathematically encoded in the basis functions of the space. Each quale corresponds to a different set of basis functions; for instance, “table” qualia and “background noise” qualia are associated with separate bases. Changing the quale that someone is conscious of amounts to rotating the basis functions of the Hilbert space. When the object of my consciousness shifts from the table to the background noise, the Hilbert space rotates from the “table” basis functions to the “background noise” basis functions. The formal term for a rotation, in this case, is a unitary transformation. Critically, the Hilbert space remains invariant after unitary transformations, thus the structure of consciousness stays the same while the object of consciousness varies from moment to moment.

The Problem of Temporal Experience

The solution to the Binding Problem is inextricably linked to the solution to the Problem of Temporal Experience, since qualia are bound across time. Although it appears as though we experience multiple qualia at the same time, we actually experience only one quale per moment, according to the Moments of Consciousness Model. Therefore, binding occurs between qualia that are perceived at successive moments of time; a quale experienced in the present moment is bound to the quale observed in the preceding moment. Our conscious experience of the present is unified with our experience of the immediate past, hence we have a single consciousness of the passage of time rather than a different consciousness at each moment.

---

(1) Note that the unity of consciousness over time breaks down for some people over long timeframes. I have spoken to some people who do not identify with who they were as children.

(2) Readers who are very familiar with the subjective experience of color (I’m looking at you, Andrés) may note that the RGB color space is actually not the best way of characterizing perceptual color space. There are colors with different RGB codes that appear identical to one another. Consider, for instance, RGB = (50, 200, 50) and RGB = (50, 201, 50), which are pictured below, respectively. A better alternative to the RGB cube is the CIELAB color space, which was constructed based on “just-noticeable differences” between colors. As Wikipedia states, “uniform changes of components in the CIELAB color space aim to correspond to uniform changes in perceived color.”

(3) These “Euler angles” are more commonly known as “yaw,” “pitch,” and “roll,”respectively.

(4) Note that “group” here is not being used in the colloquial sense, but rather in the mathematical sense.

(5) This unitary transformation is known as the Fourier transform. While I’ve already established that a unitary transformation can be thought of as a rotation of Hilbert space, it is nevertheless worth noting that Fourier transforms can be treated as rotations of basis vectors, as this website beautifully explains.

(6) Probability distributions are embedded in a specific kind of Hilbert space: a Reproducing Kernel Hilbert Space (RKHS). As Wikipedia explains, “An RKHS is associated with a kernel that reproduces every function in the space in the sense that for any x in the set on which the functions are defined, “evaluation at x” can be performed by taking an inner product with a function determined by the kernel.” In this case, the functions that define the RKHS  are not exactly the distributions themselves, but rather feature functions, which are determined by the kernel of the map from the distribution to the RKHS. Linear combinations of the feature functions then serve as the basis functions for the RKHS. We can compute the moment of a distribution by calculating the inner product of the basis function with the feature function mapping of the moment. For a more precise mathematical formulation of these ideas, read Xing’s paper.