The use of intracranial recordings to decode human language: Challenges and opportunities

Abstract:

Decoding speech from intracranial recordings serves two main purposes: understanding the neural correlates of speech processing and decoding speech features for targeting speech neuroprosthetic devices. Intracranial recordings have high spatial and temporal resolution, and thus offer a unique opportunity to investigate and decode the electrophysiological dynamics underlying speech processing. In this review article, we describe current approaches to decoding different features of speech perception and production – such as spectrotemporal, phonetic, phonotactic, semantic, and articulatory components – using intracranial recordings. A specific section is devoted to the decoding of imagined speech, and potential applications to speech prosthetic devices. We outline the challenges in decoding human language, as well as the opportunities in scientific and neuroengineering applications.

Authors:

• Stephanie Martin

• José del R. Millán

• Robert T. Knight

• Brian N. Pasley

Date: 2019

DOI: http://dx.doi.org/10.1016/j.bandl.2016.06.003

View PDF

Roles of ventral versus dorsal pathways in language production: An awake language mapping study

Abstract:

Human language is organized along two main processing streams connecting posterior temporal cortex and inferior frontal cortex in the left hemisphere, travelling dorsal and ventral to the Sylvian fissure. Some views propose a dorsal motor versus ventral semantic division. Others propose division by combinatorial mechanism, with the dorsal stream responsible for combining elements into a sequence and the ventral stream for forming semantic dependencies independent of sequential order. We acquired data from direct cortical stimulation in the left hemisphere in 17 neurosurgical patients and subcortical resection in a subset of 10 patients as part of awake language mapping. Two language tasks were employed: a sentence generation (SG) task tested the ability to form sequential and semantic dependencies, and a picture-word interference (PWI) task manipulated semantic interference. Results show increased error rates in the SG versus PWI task during subcortical testing in the dorsal stream territory, and high error rates in both tasks in the ventral stream territory. Connectivity maps derived from diffusion imaging and seeded in the tumor sites show that patients with more errors in the SG than in the PWI task had tumor locations associated with a dorsal stream connectivity pattern. Patients with the opposite pattern of results had tumor locations associated with a more ventral stream connectivity pattern. These findings provide initial evidence using fiber tract disruption with electrical stimulation that the dorsal pathways are critical for organizing words in a sequence necessary for sentence generation, and the ventral pathways are critical for processing semantic dependencies.

Authors:

• Stephanie K Ries

• Vitória Piai

• David Perry

• Sandon M Griffin

• Kesshi Jordan

• Roland Henry

• Robert T. Knight

• Mitchel S Berger

Date: 2019

DOI: https://doi.org/10.1016/j.bandl.2019.01.001

View PDF

The use of intracranial recordings to decode human language: Challenges and opportunities

ABSTRACT

Decoding speech from intracranial recordings serves two main purposes: understanding the neural correlates of speech processing and decoding speech features for targeting speech neuroprosthetic devices. Intracranial recordings have high spatial and temporal resolution, and thus offer a unique opportunity to investigate and decode the electrophysiological dynamics underlying speech processing. In this review article, we describe current approaches to decoding different features of speech perception and production – such as spectrotemporal, phonetic, phonotactic, semantic, and articulatory components – using intracranial recordings. A specific section is devoted to the decoding of imagined speech, and potential applications to speech prosthetic devices. We outline the challenges in decoding human language, as well as the opportunities in scientific and neuroengineering applications.

AUTHORS

• Stéphanie Martin

• José del R. Millán

• Robert T. Knight

• Brian Pasley

Date: 2016

DOI: 10.1016/j.bandl.2016.06.003

View PDF