Interlude

Mental Meanderings on Cognitive Computations

As we will be following tradition, this part of script collection will serve us to get lost in open ended thoughts, before we can tackle the more elusive section of it. We will do this by opening with the computational features of our brain by setting up the stage through the foundation of object-oriented computer programming, which is logic based on set theory.

As we continue in this tradition of meandering through abstract and theoretical concepts, let us further explore the principles of set theory and their uncanny ability to shape our thinking about neural computation and cognition. The utility of set theory is its ability to systematise how we understand relationships, categories, and interactions, concepts foundational to cognitive neuroscience. In doing so, we find ourselves at the intersection of mathematics, logic, and the very fabric of mental processes.

Take, for example, the basic notion of set membership, a fundamental principle in set theory: an element either belongs to a set, or it does not. This binary inclusion parallels the all-or-nothing nature of neuronal firing. A neuron either reaches the threshold to fire an action potential, or it remains at rest. In this way, the principle of set inclusion mirrors the brain’s decision-making processes, where specific stimuli either activate a neuron (include it in the ‘set’ of active neurons) or do not. Set membership however does not remain purely binary when applied to biological computation. Consider graded potentials in neural activity: though a neuron only fires when the threshold is reached, subthreshold potentials still contribute to the neuron’s likelihood of firing. This is where set theory becomes more nuanced, invoking the idea of fuzzy sets, where an element can partially belong to a set based on a degree of membership. In neuroscience, this corresponds to the probabilistic activation of neurons, a model that can be represented by Boltzmann machines, where neurons are stochastically activated based on weighted inputs.

Next, let us wander through the concept of power set, the set of all subsets of a given set. This concept can be loosely connected to the structure of synaptic plasticity and memory formation. Imagine a set of neurons engaged in processing a particular experience or task. The power set of this group represents all possible subsets, combinations of neurons that might work together in various arrangements to encode and recall different aspects of that experience. This provides a powerful analogy for understanding the plasticity of memory networks, where the brain can dynamically reconfigure neural circuits to handle new information or retrieve old memories.

The brain’s flexibility in rewiring itself, forming new synaptic connections, and pruning others, operates much like how power sets represent the possible reconfigurations of a set’s elements. For example, learning a new skill could be understood as adding new elements (neurons) to a neural set, while refining that skill might involve selecting the most efficient subsets of neurons for rapid recall and execution. The Cartesian product of two sets gives us all possible ordered pairs of elements from the two sets. In neuroscience, this relates intriguingly to the idea of multimodal integration, where different sensory modalities, such as sight, sound, and touch, combine in the brain to form a cohesive perception of the world. For instance, the integration of visual and auditory information happens in the superior colliculus and the auditory cortex, where neurons take input from both sensory streams to form paired associations. This pairing of disparate elements from different sets (i.e., sensory modalities) enables the brain to cross-reference inputs and improve the accuracy of perception. In much the same way, the Cartesian product shows how combinations of elements from different sets can generate new, meaningful relationships.

The union of two sets, a basic operation that combines all elements from both sets, reflects how the brain processes overlapping information from different experiences or contexts. Consider how semantic networks in the brain are built, each word or concept could be thought of as a set of associated meanings and connotations. When two concepts overlap, their union represents the merging of all possible associations and meanings, allowing for flexible thinking and analogical reasoning. The brain, much like a union of sets, combines information from multiple sources to derive new insights or solutions to problems. This becomes especially significant in creative thought processes, where novel ideas often emerge from the union of previously distinct concepts. In the neural landscape, this could be seen as the activation of cross-domain networks, where neurons representing seemingly unrelated ideas or experiences come together to form new cognitive structures. Returning to Russell’s Paradox, which arises from the contradiction of sets containing themselves, we find parallels in the domain of self-referential thinking and meta-cognition. The paradox forces us to grapple with the limits of set theory, much like how self-awareness challenges our understanding of the brain’s ability to think about itself. Recursive thoughts, the brain contemplating its own processes, create a feedback loop not unlike the set of all sets that do not contain themselves. This recursive nature of thought is critical for functions like reflection, planning, and theory of mind, yet it also poses profound challenges for neuroscientific modelling. The paradox of a set containing itself in logic can be likened to the complexity of feedback loops in the brain, where neurons activate other neurons in a way that eventually influences their own activity. This recursive structure is essential for memory consolidation, learning, and even consciousness, but it also illustrates the inherent limitations of any system attempting to self-model or engage in self-referential computation. Finally, the concept of complement in set theory, the idea of everything that is not in a set, offers a useful lens through which to view inhibitory processes in the brain. Just as the complement of a set represents all elements not contained within it, inhibitory neurons act to suppress neural activity that is not currently relevant to the task at hand. This selective inhibition is crucial for focused attention, where the brain actively ignores irrelevant stimuli to maintain cognitive efficiency. Set complements, then, offer a mathematical framework for understanding how the brain balances excitation and inhibition to create a coherent stream of thought or behaviour.

Thus, set theory, in all its abstract beauty, offers a flexible and profound framework for modelling the computational nature of the brain. It allows us to understand how neurons can be grouped, how relationships between neural circuits are formed, and how the brain handles both simple categorisations and complex recursive processes. Like any powerful tool, it reveals both the strengths and limitations of logical systems, urging us to approach the mysteries of the mind with both rigor and humility. As we advance further into computational neuroscience, these mathematical principles will continue to guide us, offering a logical map for navigating the complexities of cognition, consciousness, and beyond. Let us try with one of the most compelling models existing to describe how memory works called Hopfield networks that neuroscientists and computational scientists cherish alike. Hopfield networks stand as a pivotal model in computational neuroscience, offering a simplified yet powerful representation of how human memory functions. Named after John Hopfield, who introduced them in the early 1980s, these networks are a type of recurrent artificial neural network that can store information in a distributed manner and retrieve it through pattern recognition. Hopfield networks operate through energy minimisation principles, making them particularly insightful when applied to the neural dynamics of memory formation and retrieval.

Hopfield networks are composed of a set of binary units (or neurons) that are fully interconnected, meaning each neuron is connected to every other neuron in the network, but not to itself. The neurons update their states based on the weighted sum of inputs they receive from all other neurons, modified by synaptic weights that define the strength of these connections. The connections are symmetric, which means that the influence one neuron has on another is reciprocal, reinforcing the bidirectional nature of synaptic communication in real neural systems. The functioning of a Hopfield network can be described by an energy landscape, where each pattern stored in the network corresponds to a stable state or 'attractor' within this landscape. The network's operation relies on updating neuron states iteratively, moving through the landscape by minimising energy until it settles into one of these attractor states, which corresponds to the retrieval of a stored memory. This energy minimisation concept closely resembles how neural systems might operate to stabilise a particular pattern of activation that represents a memory trace. Hopfield networks employ a learning rule akin to Hebbian learning, often summarised as "neurons that fire together, wire together." This rule is instrumental in setting the weights between neurons during the training phase, allowing the network to imprint the patterns it needs to store. The weights are adjusted based on the correlation between neuron activations, effectively embedding patterns into the synaptic structure of the network. In computational terms, the weight matrix of a Hopfield network is updated to encode memories as energy minima. When a new pattern is presented, the network dynamically adjusts the synaptic weights to incorporate this pattern into its memory structure. Importantly, each added pattern slightly reshapes the energy landscape, meaning that the capacity of a Hopfield network is finite and depends on the number of neurons, too many patterns can lead to interference, where memories become blended or distorted. This aspect of limited capacity mirrors real biological memory systems, where the overlap of similar memories can cause interference, leading to errors in retrieval. Such parallels make Hopfield networks a compelling model for understanding both the strengths and limitations of human memory.

The retrieval process in Hopfield networks is driven by the network's tendency to fall into attractor states, points in the energy landscape that correspond to stored patterns. When presented with a noisy or incomplete input, the network iterates through state updates, progressively minimising the energy until it converges on the closest attractor, effectively reconstructing the stored pattern. This error-correcting property of Hopfield networks is akin to human memory's ability to recall familiar experiences even from partial or degraded cues. For example, if you hear a few notes of a familiar melody, your brain can often reconstruct the entire tune, even if the input is incomplete or distorted. This retrieval process underscores the robust, associative nature of human memory, where context and fragments are often sufficient to trigger full recall. Hopfield networks, with their associative and error-correcting capabilities, offer a computational analogy for human memory systems, where information is stored in the synaptic weights of neural circuits. The network’s dynamics reflect several key features of biological memory. Like neural circuits, Hopfield networks store memories in a distributed fashion across the connections between neurons, rather than at a specific location. This distributed coding provides resilience against damage, akin to the way human memories persist even when some neurons are lost. The ability of Hopfield networks to retrieve memories based on partial inputs mirrors how humans recall information based on contextual cues. This content-addressable feature contrasts with conventional computer memory, which relies on specific addresses to access stored information. The weight adjustments in Hopfield networks during learning resemble synaptic plasticity, the biological process through which connections between neurons strengthen or weaken based on experience. This plasticity is central to forming memories in the brain, making Hopfield networks an elegant computational model for studying memory encoding and retrieval. The concept of attractors in Hopfield networks provides a framework for understanding the stability of memories. Once a memory pattern is encoded, the network's energy landscape ensures that the pattern remains stable, even in the face of perturbations, much like how our minds retain stable recollections despite distractions or noise.

This is where the concept of memory formation in our brain becomes exciting, when entropy and information theory plug reality into neurocomputational theory and theoretical neuroscience by returning once again to the Boltzmann theorems. Boltzmann Machines represent a class of stochastic neural networks that offer a sophisticated bridge between Hopfield Networks and Reinforcement Learning, which we will explore shortly. Developed as a generalisation of Hopfield Networks, Boltzmann Machines extend the capabilities of neural computation by incorporating probabilistic elements that enable richer and more flexible learning. By modelling complex distributions over data, Boltzmann Machines capture the underlying statistical dependencies in ways that deterministic models like Hopfield Networks cannot, thus laying foundational principles that underpin more advanced techniques used in Reinforcement Learning. Boltzmann Machines are characterised by their fully connected architecture, where each neuron, or node, connects symmetrically to every other node. This interconnected structure enables the network to represent complex energy landscapes, which are utilised for learning and inference. Through processes of stochastic sampling and iterative weight adjustments, Boltzmann Machines adjust their internal representation of data distributions, making them powerful tools for tasks involving pattern recognition, feature extraction, and generative modelling.

Hopfield Networks and Boltzmann Machines both belong to the family of energy-based models, where learning is conceptualised as minimising an energy function. In Hopfield Networks, this energy function is deterministic, guiding the system towards local minima that represent stable states or memories. However, Hopfield Networks are limited in their capacity to handle overlapping or complex data patterns due to their fixed, binary state dynamics. Boltzmann Machines overcome these limitations by introducing stochasticity through a probabilistic framework. Instead of converging deterministically, Boltzmann Machines use random sampling governed by the Boltzmann distribution, which allows the network to explore a wider range of possible states. This stochastic nature prevents the model from getting trapped in local minima and enables it to capture more intricate and high-dimensional data patterns, a critical advantage when dealing with real-world problems. The probabilistic dynamics of Boltzmann Machines mirror the thermal noise seen in physical systems, where temperature governs the likelihood of state transitions. In the context of the neural network, this 'temperature' parameter controls the level of exploration versus exploitation, a concept central to both Boltzmann Machines and Reinforcement Learning algorithms.

Learning in Boltzmann Machines involves adjusting the connection weights between neurons to minimise the overall system energy and approximate the distribution of the input data. This process is typically achieved through the Stochastic Gradient Descent algorithm, where the weight updates are driven by the difference between positive and negative phase correlations of neurons. In the positive phase, the system is clamped to the input data, allowing the network to learn the correlations that exist in the observed data. In the negative phase, the system can run freely, generating data based on its current weight configuration, and thereby learning the correlations in its world model. This iterative process, where the model's internal state is continually refined to align with the observed data, parallels the exploration-exploitation trade-off central to Reinforcement Learning. Here, the network ‘explores’ potential configurations during the negative phase, while ‘exploiting’ known data patterns during the positive phase. The concept of ‘temperature’ plays a crucial role in Boltzmann Machines, acting as a parameter that modulates the level of randomness in state transitions. This is analogous to the exploration parameter in Reinforcement Learning algorithms, where higher exploration fosters learning novel patterns, and lower exploration focuses on refining known solutions. Reinforcement Learning, much like Boltzmann Machines, involves balancing exploration, trying new actions to discover better outcomes and exploitation, leveraging known actions to maximise rewards. This balance is directly influenced by the probabilistic sampling dynamics inherited from Boltzmann Machines. In Reinforcement Learning, techniques such as Softmax Action Selection are closely related to the probabilistic decision-making frameworks used in Boltzmann Machines, where actions are chosen based on their relative probabilities determined by the underlying value or policy function.

Boltzmann Machines provide an early computational basis for these mechanisms by demonstrating how systems can learn optimal states through probabilistic exploration, which has directly influenced Reinforcement Learning strategies. For instance, the Boltzmann Exploration strategy in Reinforcement Learning utilises temperature-modulated probabilities to decide on action selections, allowing agents to explore actions that might not be optimal under current knowledge but could yield better long-term rewards. Boltzmann Machines are also powerful generative models, capable of reconstructing data distributions by learning the complex dependencies among variables. This generative capability aligns with the Reinforcement Learning requirement of modelling the environment or state transitions probabilistically, facilitating decisions that anticipate not just immediate rewards but future states and outcomes.

The Restricted Boltzmann Machine, a simplified version of the Boltzmann Machine, has been particularly influential. Restricted Boltzmann Machines reduce computational complexity by limiting connections between neurons to bipartite graphs, making them more efficient for real-world applications. These networks have been used extensively in Deep Learning as feature detectors and in pre-training deep neural networks, a testament to their ability to encode complex patterns that are crucial in both supervised and reinforcement learning contexts. This model provides a bridge to understanding how neural networks in the brain might encode experiences probabilistically, balancing exploration of new behavioural strategies with exploitation of learned responses. The exploration dynamics in Boltzmann Machines echo the dopaminergic modulation observed in reinforcement learning tasks in animals, where variable reinforcement schedules enhance adaptive behaviour. By modelling these dynamics, Boltzmann Machines serve as computational tools for investigating how uncertainty and variability in reward signals shape learning processes in both artificial and biological systems. Their probabilistic framework not only extends the capacity of neural networks to represent and learn from data but also informs the development of sophisticated Reinforcement Learning algorithms that drive decision-making in uncertain conditions. The interplay between energy-based learning, probabilistic sampling, and adaptive behaviour encapsulates the essence of both Boltzmann Machines and Reinforcement Learning, highlighting their interconnected roles in advancing our understanding of artificial intelligence and cognitive neuroscience. 

So, Reinforcement Learning is yet another computational method that involves learning to make decisions, particularly through trial and error, guided by the pursuit of maximising cumulative rewards. This approach parallels the brain's fundamental mechanisms for learning from interaction with the environment, encoding behavioural adaptations that enhance survival and goal attainment. The foundation of Reinforcement Learning in biological systems lies in the dopaminergic pathways of the brain, particularly within the basal ganglia. Dopamine neurons encode reward prediction errors, the difference between expected and received outcomes, serving as the brain's internal signal for adjusting behaviour. This feedback loop drives synaptic plasticity, where changes in the strength of connections between neurons modify future responses, aligning with the Reinforcement Learning principle of updating policies to optimise rewards. This process is crucial in both lower-order motor learning and higher cognitive functions such as decision-making and strategy formulation. Neurobiologically, Reinforcement Learning processes engage multiple brain regions, including the prefrontal cortex, which is involved in planning and evaluating actions, and the striatum, which integrates sensory inputs with reward signals to influence action selection. The interaction between these regions facilitates the development of complex behavioural repertoires, demonstrating a highly evolved neural system capable of learning from both positive reinforcement and punishment, much like Reinforcement Learning algorithms balance exploration and exploitation. From a computational standpoint, Reinforcement Learning can be formally described through algorithms such as Q-learning and Policy Gradient methods. These algorithms simulate the process by which an agent, either biological or artificial, learns to map states of the environment to actions that maximise reward. Q-learning, for instance, involves learning a value function that estimates the expected utility of actions, while Policy Gradient methods directly adjust the probabilities of actions in a way that improves performance over time. These models provide insights into how the brain might encode and update behavioural strategies, drawing parallels between synaptic adjustments in neurons and the iterative updates in algorithmic policies. The significance of Reinforcement Learning extends beyond simple task learning; it encapsulates adaptive behaviour that underpins complex human activities, from navigating social dynamics to mastering intricate skills.

In both artificial and biological systems, the representation of state, action, and reward is crucial. In the human brain, the representation of these components is distributed across neural networks that encode environmental context, anticipated outcomes, and the emotional valence of actions. The orbitofrontal cortex, for example, plays a key role in assessing the value of different actions, integrating sensory input with reward expectations to guide behaviour. Reinforcement Learning elucidates how humans learn from delayed rewards, a capacity that underlies patience, planning, and impulse control. Temporal Difference Learning, a key Reinforcement Learning model, mirrors the brain’s ability to propagate reward signals backward in time, influencing earlier actions that contribute to the ultimate outcome. This capacity is reflected in the neural dynamics observed during tasks requiring foresight and delayed gratification, highlighting the deep integration of Reinforcement Learning principles within human cognition. Reinforcement Learning has become a cornerstone of modern artificial intelligence, particularly in applications where agents must adapt to complex, dynamic environments. Autonomous systems, from self-driving cars to robotic manipulators, leverage Reinforcement Learning to refine their decision-making in real-time, learning to navigate, interact, and optimise their actions based on sensory feedback. These computational models provide a framework for understanding the parallel evolution of intelligent behaviour in biological and artificial domains. In robotics, Reinforcement Learning enables machines to perform tasks with a level of flexibility and adaptability that echoes human motor learning. For instance, robotic arms can learn to manipulate objects with precision through iterative practice, akin to how humans refine their motor skills through repeated trial and correction. The continual feedback loop, sensing, acting, evaluating, and adjusting, underscores the shared principles of adaptive learning across both natural and synthetic systems.

Exploring Reinforcement Learning within the context of the human brain offers profound insights into the neural basis of learning, motivation, and behavioural adaptation. It challenges traditional views of fixed neural circuitry, emphasising the dynamic and plastic nature of the brain's decision-making architecture. Reinforcement Learning models align closely with observed neural firing patterns, particularly in dopaminergic neurons, providing a computational lens through which to interpret behavioural experiments in both animals and humans. Moreover, Reinforcement Learning’s framework allows for a deeper understanding of disorders characterised by maladaptive learning, such as addiction and compulsive behaviours. Aberrant reward processing and disrupted prediction error signalling in the brain can be framed as dysfunctions within a Reinforcement Learning model, offering potential avenues for therapeutic interventions that target these fundamental learning processes. Reinforcement Learning embodies a bridge between neuroscience and artificial intelligence, reflecting the adaptive nature of the brain's learning mechanisms. By studying Reinforcement Learning, we not only advance artificial systems but also gain invaluable insights into the complexities of human cognition, decision-making, and the neural substrates that underlie these processes. As Reinforcement Learning continues to evolve, its integration into both scientific inquiry and practical application stands as a testament to the shared principles of learning that span across both biological and computational realms.

As a final thought, we should submit the idea of Hopfield networks as a cognitive mirror. Hopfield networks offer an abstract yet profoundly insightful model for understanding memory formation and retrieval in the brain. Their structure and dynamics encapsulate key aspects of neural processing, from distributed representation to error correction, making them a valuable tool for both theoretical exploration and practical application in cognitive neuroscience. The question remains how this model aligns with memory models mentioned in the last Interlude, particularly that of Singer W.

Hopfield networks align intriguingly with Singer's conceptualisation of memory, particularly in their treatment of how information is stored and retrieved in dynamic, fluid systems rather than rigid, static structures. Singer proposes that neocortical activity should not be viewed as a simple pipeline where pre-encoded, subcortical signals merely flow upward to deliver information. Instead, he suggests that the neocortex operates more like a still pond, with neuronal activity creating ripples that propagate across a complex and highly interconnected network. Each neuronal firing event acts like a single drop that generates a wave, influencing a broader pattern of neural activity. This perspective resonates deeply with the principles underlying Hopfield networks, where each unit's state influences the entire system through a web of reciprocal connections. In Hopfield networks, memories are not stored in a fixed location but rather exist as stable patterns within the network's dynamic interactions. Similarly, Singer's metaphor highlights the fluid and emergent nature of cognitive states, where the 'hills and valleys' of neocortical activity represent momentary configurations of thought and memory. Just as Hopfield networks find stable attractor states through energy minimisation, the human neocortex achieves momentary coherence from the complex interference patterns of neuronal ripples, leading to the emergent properties of thought, memory, and consciousness. This alignment underscores a fundamental aspect of memory: it is not simply the retrieval of static data, but an ongoing process shaped by dynamic neural interactions, constantly reconfigured by the context of the present moment and the network's history of experiences. In this way, Singer’s and Hopfield’s views converge on the idea that memory and cognition are not static but are continuously evolving, shaped by the interplay of numerous, interconnected neuronal signals, a harmonious dance of stability and change that defines human thought.

As we contemplate the complexities of human memory, Hopfield networks remind us of the elegance inherent in the brain's computational strategies. They capture the essence of how we store, access, and even misremember the patchwork of our experiences. By delving into the computational mechanics of these networks, we gain a clearer understanding of the neural underpinnings of memory, revealing not only the remarkable efficiency of our cognitive architecture but also the fundamental challenges that come with it. These models guide us as we unravel the mysteries of the mind, providing a bridge between the abstract world of mathematics and the tangible reality of human thought.