Predictable and unpredictable deviance detection in the human hippocampus and amygdala

Abstract:

Our brains extract structure from the environment and form predictions given past experience. Predictive circuits have been identified in wide-spread cortical regions. However, the contribution of medial temporal structures in predictions remains under-explored. The hippocampus underlies sequence detection and is sensitive to novel stimuli, sufficient to gain access to memory, while the amygdala to novelty. Yet, their electrophysiological profiles in detecting predictable and unpredictable deviant auditory events remain unknown. Here, we hypothesized that the hippocampus would be sensitive to predictability, while the amygdala to unexpected deviance. We presented epileptic patients undergoing presurgical monitoring with standard and deviant sounds, in predictable or unpredictable contexts. Onsets of auditory responses and unpredictable deviance effects were detected earlier in the temporal cortex compared with the amygdala and hippocampus. Deviance effects in 1–20 Hz local field potentials were detected in the lateral temporal cortex, irrespective of predictability. The amygdala showed stronger deviance in the unpredictable context. Low-frequency deviance responses in the hippocampus (1–8 Hz) were observed in the predictable but not in the unpredictable context. Our results reveal a distributed network underlying the generation of auditory predictions and suggest that the neural basis of sensory predictions and prediction error signals needs to be extended.

Authors:

• Athina Tzovara

• Tommaso Fedele

• Johannes Sarnthein

• Debora Ledergerber

• Jack J. Lin

• Robert T. Knight

Date: 2024

DOI: https://doi.org/10.1093/cercor/bhad532

View PDF

Ramping dynamics and theta oscillations reflect dissociable signatures during rule-guided human behavior

abstract:

Contextual cues and prior evidence guide human goal-directed behavior. The neurophysiological mechanisms that implement contextual priors to guide subsequent actions in the human brain remain unclear. Using intracranial electroencephalography (iEEG), we demonstrate that increasing uncertainty introduces a shift from a purely oscillatory to a mixed processing regime with an additional ramping component. Oscillatory and ramping dynamics reflect dissociable signatures, which likely differentially contribute to the encoding and transfer of different cognitive variables in a cue-guided motor task. The results support the idea that prefrontal activity encodes rules and ensuing actions in distinct coding subspaces, while theta oscillations synchronize the prefrontal-motor network, possibly to guide action execution. Collectively, our results reveal how two key features of large-scale neural population activity, namely continuous ramping dynamics and oscillatory synchrony, jointly support rule-guided human behavior.

Authors:

• Jan Weber

• Anne-Kristin Solbakk

• Alejandro O. Blenkmann

• Anais Llorens

• Ingrid Funderud

• Sabine Leske

• Pål Gunnar Larsson

• Lugoslav Ivanovic

• Robert T. Knight

• Tor Endestad

• Randolph F. Helfrich

Date: 2024

DOI: https://doi.org/10.1038/s41467-023-44571-7

View PDF

Awake ripples enhance emotional memory encoding in the human brain

Abstract:

Enhanced memory for emotional experiences is hypothesized to depend on amygdala-hippocampal interactions during memory consolidation. Here we show using intracranial recordings from the human amygdala and the hippocampus during an emotional memory encoding and discrimination task increased awake ripples after encoding of emotional, compared to neutrally-valenced stimuli. Further, post-encoding ripple-locked stimulus similarity is predictive of later memory discrimination. Ripple-locked stimulus similarity appears earlier in the amygdala than in hippocampus and mutual information analysis confirms amygdala influence on hippocampal activity. Finally, the joint ripple-locked stimulus similarity in the amygdala and hippocampus is predictive of correct memory discrimination. These findings provide electrophysiological evidence that post-encoding ripples enhance memory for emotional events.

Authors:

• Haoxin Zhang

• Ivan Skelin

• Shiting Ma

• Michelle Paff

• Lilit Mnatsakanyan

• Michael A. Yassa

• Robert T. Knight

• Jack J. Lin

Date: 2024

DOI: https://doi.org/10.1038/s41467-023-44295-8

View PDF

Differential contribution of sensorimotor cortex and subthalamic nucleus to unimanual and bimanual hand movements

Abstract:

Why does unilateral deep brain stimulation improve motor function bilaterally? To address this clinical observation, we collected parallel neural recordings from sensorimotor cortex (SMC) and the subthalamic nucleus (STN) during repetitive ipsilateral, contralateral, and bilateral hand movements in patients with Parkinson’s disease. We used a cross-validated electrode-wise encoding model to map electromyography data to the neural signals. Electrodes in the STN encoded movement at a comparable level for both hands, whereas SMC electrodes displayed a strong contralateral bias. To examine representational overlap across the two hands, we trained the model with data from one condition (contralateral hand) and used the trained weights to predict neural activity for movements produced with the other hand (ipsilateral hand). Overall, between-hand generalization was poor, and this limitation was evident in both regions. A similar method was used to probe representational overlap across different task contexts (unimanual vs. bimanual). Task context was more important for the STN compared to the SMC indicating that neural activity in the STN showed greater divergence between the unimanual and bimanual conditions. These results indicate that SMC activity is strongly lateralized and relatively context-free, whereas the STN integrates contextual information with the ongoing behavior.

Authors:

• Christina M. Merrick

• Owen N. Doyle

• Natali E. Gallegos

• Zachary T. Irwin

• Joseph W. Olson

• Christopher L. Gonzalez

• Robert T. Knight

• Richard B. Ivry

• Harrison C. Walker

Date: 2023

DOI: https://doi.org/10.1093/cercor/bhad492

View PDF

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

Abstract:

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Authors:

• Colin W. Hoy

• David R. Quiroga-Martinez

• Eduardo Sandoval

• David King-Stephens

• Kenneth D. Laxer

• Peter Weber

• Jack J. Lin

• Robert T. Knight

Date: 2023

DOI: https://doi.org/10.1038/s41467-023-44248-1

View PDF

Periodic attention deficits after frontoparietal lesions provide causal evidence for rhythmic attentional sampling

Abstract:

Contemporary models conceptualize spatial attention as a blinking spotlight that sequentially samples visual space. Hence, behavior fluctuates over time, even in states of presumed ‘‘sustained’’ attention. Recent evidence has suggested that rhythmic neural activity in the frontoparietal network constitutes the functional basis of rhythmic attentional sampling. However, causal evidence to support this notion remains absent. Using a lateralized spatial attention task, we addressed this issue in patients with focal lesions in the frontoparietal attention network. Our results revealed that frontoparietal lesions introduce periodic attention deficits, i.e., temporally specific behavioral deficits that are aligned with the underlying neural oscillations. Attention guided perceptual sensitivity was on par with that of healthy controls during optimal phases but was attenuated during the less excitable sub-cycles. Theta-dependent sampling (3–8 Hz) was causally dependent on the prefrontal cortex, while high-alpha/low-beta sampling (8–14 Hz) emerged from parietal areas. Collectively, our findings reveal that lesion-induced high-amplitude, low-frequency brain activity is not epiphenomenal but has immediate behavioral consequences. More generally, these results provide causal evidence for the hypothesis that the functional architecture of attention is inherently rhythmic.

Authors:

• Isabel Raposo

• Sara M. Szczepanski

• Kathleen Haaland

• Tor Endestad

• Anne-Kristin Solbakk

• Robert T. Knight

• Randolph F. Helfrich

Date: 2023

DOI: https://doi.org/10.1016/j.cub.2023.09.065

View PDF

Decision and response monitoring during working memory are sequentially represented in the human insula

Abstract:

Emerging research supports a role of the insula in human cognition. Here, we used intracranial EEG to investigate the spatiotemporal dynamics in the insula during a verbal working memory (vWM) task. We found robust effects for theta, beta, and high frequency activity (HFA) during probe presentation requiring a decision. Theta band activity showed differential involvement across left and right insulae while sequential HFA modulations were observed along the anteroposterior axis. HFA in anterior insula tracked decision making and subsequent HFA was observed in posterior insula after the behavioral response. Our results provide electrophysiological evidence of engagement of different insula subregions in both decision-making and response monitoring during vWM and expand our knowledge of the role of the insula in complex human behavior.

Authors:

• Anaïs Llorens

• Ludovic Bellier

• Alejandro O. Blenkmann

• Jugoslav Ivanovic

• Pål G. Larsson

• Jack J. Lin

• Tor Endestad

• Anne-Kristin Solbakk

• Robert T. Knight

Date: 2023

DOI: https://doi.org/10.1016/j.isci.2023.107653

View PDF

Multiple memory systems for efficient temporal order memory

Abstract:

We report distinct contributions of multiple memory systems to the retrieval of the temporal order of events. The neural dynamics related to the retrieval of movie scenes revealed that recalling the temporal order of close events elevates hippocampal theta power, like that observed for recalling close spatial relationships. In contrast, recalling far events increases beta power in the orbitofrontal cortex, reflecting recall based on the overall movie structure.

Authors:

• Anna Jafarpour

• Jack J. Lin

• Robert T. Knight

• Elizabeth A. Buffalo

Date: 2023

DOI: https://doi.org/10.1002/hipo.23550

View PDF

Electrophysiological signatures of inequity-dependent reward encoding in the human OFC

Abstract:

Social decision making requires the integration of reward valuation and social cognition systems, both dependent on the orbitofrontal cortex (OFC). How these two OFC functions interact is largely unknown. We recorded intracranial activity from the OFC of ten patients making choices in a social context where reward inequity with a social counterpart varied and could be either advantageous or disadvantageous. We find that OFC high-frequency activity (HFA; 70–150Hz) encodes self-reward, consistent with previous reports. We also observe encoding of the social counterpart’s reward, as well as the type of inequity being experienced. Additionally, we find evidence of inequity-dependent reward encoding: depending on the type of inequity, electrodes rapidly and reversibly switch between different reward-encoding profiles. These results provide direct evidence for encoding of self- and other rewards in the human OFC and highlight the dynamic nature of encoding in the OFC as a function of social context.

Authors:

• Deborah Marciano

• Brooke R. Staveland

• Jack J. Lin

• Ignacio Saez

• Ming Hsu

• Robert T. Knight

Date: 2023

DOI: https://doi.org/10.1016/j.celrep.2023.112865

View PDF

Human REM sleep recalibrates neural activity in support of memory formation

Abstract:

The proposed mechanisms of sleep-dependent memory consolidation involve the overnight regulation of neural activity at both synaptic and whole-network levels. Now, there is a lack of in vivo data in humans elucidating if, and how, sleep and its varied stages balance neural activity, and if such recalibration benefits memory. We combined electrophysiology with in vivo two-photon calcium imaging in rodents as well as intracranial and scalp electroencephalography (EEG) in humans to reveal a key role for non-oscillatory brain activity during rapid eye movement (REM) sleep to mediate sleep-dependent recalibration of neural population dynamics. The extent of this REM sleep recalibration predicted the success of overnight memory consolidation, expressly the modulation of hippocampal—neocortical activity, favoring remembering rather than forgetting. The findings describe a non-oscillatory mechanism how human REM sleep modulates neural population activity to enhance long-term memory.

Authors:

• Janna D. Lendner

• Niels Niethard

• Bryce A. Mander

• Frank J. van Schalkwijk

• Sigrid Schuh-Hofer

• Hannah Schmidt

• Robert T. Knight

• Jan Born

• Matthew P. Walker

• Jack J. Lin

• Randolph F. Helfrich

Date: 2023

DOI: DOI: 10.1126/sciadv.adj1895

View PDF

Dynamic expectations: Behavioral and electrophysiological evidence of sub-second updates in reward predictions

Abstract:

Expectations are often dynamic: sports fans know that expectations are rapidly updated as games unfold. Yet expectations have traditionally been studied as static. Here we present behavioral and electrophysiological evidence of sub-second changes in expectations using slot machines as a case study. In Study 1, we demonstrate that EEG signal before the slot machine stops varies based on proximity to winning. Study 2 introduces a behavioral paradigm to measure dynamic expectations via betting, and shows that expectation trajectories vary as a function of winning proximity. Notably, these expectation trajectories parallel Study 1’s EEG activity. Studies 3 (EEG) and 4 (behavioral) replicate these findings in the loss domain. These four studies provide compelling evidence that dynamic sub-second updates in expectations can be behaviorally and electrophysiologically measured. Our research opens promising avenues for understanding the dynamic nature of reward expectations and their impact on cognitive processes.

Authors:

• Déborah Marciano

• Ludovic Bellier

• Ida Mayer

• Michael Ruvalcaba

• Sangil Lee

• Ming Hsu

• Robert T. Knight

Date: 2023

DOI: https://doi.org/10.1038/s42003-023-05199-x

View PDF

Music can be reconstructed from human auditory cortex activity using nonlinear decoding models

Abstract:

Music is core to human experience, yet the precise neural dynamics underlying music perception remain unknown. We analyzed a unique intracranial electroencephalography (iEEG) dataset of 29 patients who listened to a Pink Floyd song and applied a stimulus reconstruction approach previously used in the speech domain. We successfully reconstructed a recognizable song from direct neural recordings and quantified the impact of different factors on decoding accuracy. Combining encoding and decoding analyses, we found a right-hemisphere dominance for music perception with a primary role of the superior temporal gyrus (STG), evidenced a new STG subregion tuned to musical rhythm, and defined an anterior–posterior STG organization exhibiting sustained and onset responses to musical elements. Our findings show the feasibility of applying predictive modeling on short datasets acquired in single patients, paving the way for adding musical elements to brain–computer interface (BCI) applications.

Authors:

• Ludovic Bellier

• Anaïs Llorens

• Déborah Marciano

• Aysegul Gunduz

• Gwerwin Schalk

• Peter Brunner

• Robert T. Knight

Date: 2023

DOI: https://doi.org/10.1371/journal.pbio.3002176

View PDF

Distinct ventral stream and prefrontal cortex representational dynamics during sustained conscious visual perception

Abstract:

Instances of sustained stationary sensory input are ubiquitous. However, previous work focused almost exclusively on transient onset responses. This presents a critical challenge for neural theories of consciousness, which should account for the full temporal extent of experience. To address this question, we use intracranial recordings from ten human patients with epilepsy to view diverse images of multiple durations. We reveal that, in sensory regions, despite dramatic changes in activation magnitude, the distributed representation of categories and exemplars remains sustained and stable. In contrast, in frontoparietal regions, we find transient content representation at stimulus onset. Our results highlight the connection between the anatomical and temporal correlates of experience. To the extent perception is sustained, it may rely on sensory representations and to the extent perception is discrete, centered on perceptual updating, it may rely on frontoparietal representations.

Authors:

• Gal Vishne

• Edden M. Gerber

• Robert T. Knight

• Leon Y. Deouell

Date: 2023

DOI: https://doi.org/10.1016/j.celrep.2023.112752

View PDF

Grasp-specific high-frequency broadband mirror neuron activity during reach-and-grasp movements in humans

Abstract:

Broadly congruent mirror neurons, responding to any grasp movement, and strictly congruent mirror neurons, responding only to specific grasp movements, have been reported in single-cell studies with primates. Delineating grasp properties in humans is essential to understand the human mirror neuron system with implications for behavior and social cognition. We analyzed electrocorticography data from a natural reach-and-grasp movement observation and delayed imitation task with 3 different natural grasp types of everyday objects. We focused on the classification of grasp types from high-frequency broadband mirror activation patterns found in classic mirror system areas, including sensorimotor, supplementary motor, inferior frontal, and parietal cortices. Classification of grasp types was successful during movement observation and execution intervals but not during movement retention. Our grasp type classification from combined and single mirror electrodes provides evidence for grasp-congruent activity in the human mirror neuron system potentially arising from strictly congruent mirror neurons.

Authros:

• Alexander M. Dreyer

• Leo Michalke

• Anat Perry

• Edward F. Chang

• Jack J. Lin

• Robert T. Knight

• Jochem W. Rieger

Date: 2022

DOI: https://doi.org/10.1093/cercor/bhac504

View PDF

Orbitofrontal cortex governs working memory for temporal order

Abstract:

How do we think about time? Converging lesion and neuroimaging evidence indicates that orbitofrontal cortex (OFC) supports the encoding and retrieval of temporal context in long-term memory1, which may contribute to confabulation in individuals with OFC damage2. Here, we reveal that OFC damage diminishes working memory for temporal order, that is, the ability to disentangle the relative recency of events as they unfold. OFC lesions reduced working memory for temporal order but not spatial position, and individual deficits were commensurate with lesion size. Comparable effects were absent in patients with lesions restricted to lateral prefrontal cortex (PFC). Based on these findings, we propose that OFC supports understanding of the order of events. Well-documented behavioral changes in individuals with OFC damage2 may relate to impaired temporal-order understanding.

Authors:

• Elizabeth L. Johnson

• William K. Chang

• Callum D. Dewar

• Donna Sorensen

• Jack J. Lin

• Anne-Kristin Solbakk

• Tor Endestad

• Pal G. Larsson

• Jugoslav Ivanovic

• Torstein R. Meling

• Donatella Scabini

• Robert T. Knight

Date: 2022

DOI: https://doi.org/10.1016/j.cub.2022.03.074

View PDF

Automatic Sensory Predictions: A Review of Predictive Mechanisms in the Brain and Their Link to Conscious Processing

Abstract:

The human brain has the astonishing capacity of integrating streams of sensory information from the environment and forming predictions about future events in an automatic way. Despite being initially developed for visual processing, the bulk of predictive coding research has subsequently focused on auditory processing, with the famous mismatch negativity signal as possibly the most studied signature of a surprise or prediction error (PE) signal. Auditory PEs are present during various consciousness states. Intriguingly, their presence and characteristics have been linked with residual levels of consciousness and return of awareness. In this review we first give an overview of the neural substrates of predictive processes in the auditory modality and their relation to consciousness. Then, we focus on different states of consciousness - wakefulness, sleep, anesthesia, coma, meditation, and hypnosis - and on what mysteries predictive processing has been able to disclose about brain functioning in such states. We review studies investigating how the neural signatures of auditory predictions are modulated by states of reduced or lacking consciousness. As a future outlook, we propose the combination of electrophysiological and computational techniques that will allow investigation of which facets of sensory predictive processes are maintained when consciousness fades away.

Authors:

• Ruxandra I. Tivadar

• Robert T. Knight

• Athina Tzovara

Date: 2021

DOI: https://doi.org/10.3389/fnhum.2021.702520

View PDF

Prefrontal Lesions Disrupt Posterior Alpha–Gamma Coordination of Visual Working Memory Representations

Abstract:

How does the human brain prioritize different visual representations in working memory (WM)? Here, we define the oscillatory mechanisms supporting selection of “where” and “when” features from visual WM storage and investigate the role of pFC in feature selection. Fourteen individuals with lateral pFC damage and 20 healthy controls performed a visuospatial WM task while EEG was recorded. On each trial, two shapes were presented sequentially in a top/bottom spatial orientation. A retro-cue presented mid-delay prompted which of the two shapes had been in either the top/bottom spatial position or first/second temporal position. We found that cross-frequency coupling between parieto-occipital alpha (α; 8–12 Hz) oscillations and topographically distributed gamma (γ; 30–50 Hz) activity tracked selection of the distinct cued feature in controls. This signature of feature selection was disrupted in patients with pFC lesions, despite intact α–γ coupling independent of feature selection. These findings reveal a pFC-dependent parieto-occipital α–γ mechanism for the rapid selection of visual WM representations.

Authors:

• Saeideh Davoudi

• Mohsen Parto Dezfouli

• Robert T. Knight

• Mohammad Reza Daliri

• Elizabeth L. Johnson

Date: 2021

DOI: https://doi.org/10.1162/jocn_a_01715

View PDF

Intracranial recordings demonstrate medial temporal lobe engagement in visual search in humans

Abstract:

Visual search is a fundamental human behavior, which has been proposed to include two component processes: inefficient search (Search) and efficient search (Pop-out). According to extant research, these two processes map onto two separable neural systems located in the frontal and parietal association cortices. In the present study, we use intracranial recordings from 23 participants to delineate the neural correlates of Search and Pop-out with an unprecedented combination of spatiotemporal resolution and coverage across cortical and subcortical structures. First, we demonstrate a role for the medial temporal lobe in visual search, on par with engagement in frontal and parietal association cortex. Second, we show a gradient of increasing engagement over anatomical space from dorsal to ventral lateral frontal cortex. Third, we confirm previous work demonstrating nearly complete overlap in neural engagement across cortical regions in Search and Pop-out. We further demonstrate Pop-out selectivity manifesting as activity increase in Pop-out as compared to Search in a distributed set of sites including frontal cortex. This result is at odds with the view that Pop-out is implemented in low-level visual cortex or parietal cortex alone. Finally, we affirm a central role for the right lateral frontal cortex in Search.

Authors:

• S. J. Katarina Slama

• Richard Jimenez

• Sujayam Saha

• David King-Stephens

• Kenneth D Laxer

• Peter B Weber

• Tor Endestad

• Pål G Larsson

• Anne-Kristin Solbakk

• Jack J Lin

• Robert T Knight

Date: 2021

DOI: https://doi.org/10.1162/jocn_a_01739

View PDF

Electrophysiological Decoding of Spatial and Color Processing in Human Prefrontal Cortex

Abstract:

The prefrontal cortex (PFC) plays a pivotal role in goal-directed cognition, yet its representational code remains an open problem with decoding techniques ineffective in disentangling task-relevant variables from PFC. Here we applied regularized linear discriminant analysis to human scalp EEG data and were able to distinguish a mental-rotation task versus a color-perception task with 87% decoding accuracy. Dorsal and ventral areas in lateral PFC provided the dominant features dissociating the two tasks. Our findings show that EEG can reliably decode two independent task states from PFC and emphasize the PFC dorsal/ventral functional specificity in processing the where rotation task versus the what color task.

Authors:

• Byoung-Kyong Min

• Hyun-Seok Kim

• Wonjun Ko

• Min-Hee Ahn

• Heung-Il Suk

• Dimitrios Pantazis

• Robert T Knight

Date: 2021

DOI: https://doi.org/10.1016/j.neuroimage.2021.118165

View PDF

Frontotemporal Regulation of Subjective Value to Suppress Impulsivity in Intertemporal Choices

Abstract:

Impulsive decisions arise from preferring smaller but sooner rewards compared with larger but later rewards. How neural activity and attention to choice alternatives contribute to reward decisions during temporal discounting is not clear. Here we probed (1) attention to and (2) neural representation of delay and reward information in humans (both sexes) engaged in choices. We studied behavioral and frequency-specific dynamics supporting impulsive decisions on a fine-grained temporal scale using eye tracking and MEG recordings. In one condition, participants had to decide for themselves but pretended to decide for their best friend in a second prosocial condition, which required perspective taking. Hence, conditions varied in the value for themselves versus that pretending to choose for another person. Stronger impulsivity was reliably found across three independent groups for prosocial decisions. Eye tracking revealed a systematic shift of attention from the delay to the reward information and differences in eye tracking between conditions predicted differences in discounting. High-frequency activity (175-250 Hz) distributed over right frontotemporal sensors correlated with delay and reward information in consecutive temporal intervals for high value decisions for oneself but not the friend. Collectively, the results imply that the high-frequency activity recorded over frontotemporal MEG sensors plays a critical role in choice option integration.

Authors:

• Stefan Dürschmid

• Andre Maric

• Marcel S Kehl

• Robert T Knight

• Hermann Hinrichs

• Hans-Jochen Heinze

Date: 2021

DOI: https://doi.org/10.1523/JNEUROSCI.1196-20.2020

View PDF