The use of intracranial recordings to decode human language: Challenges and opportunities

ABSTRACT

Decoding speech from intracranial recordings serves two main purposes: understanding the neural correlates of speech processing and decoding speech features for targeting speech neuroprosthetic devices. Intracranial recordings have high spatial and temporal resolution, and thus offer a unique opportunity to investigate and decode the electrophysiological dynamics underlying speech processing. In this review article, we describe current approaches to decoding different features of speech perception and production – such as spectrotemporal, phonetic, phonotactic, semantic, and articulatory components – using intracranial recordings. A specific section is devoted to the decoding of imagined speech, and potential applications to speech prosthetic devices. We outline the challenges in decoding human language, as well as the opportunities in scientific and neuroengineering applications.

AUTHORS

• Stéphanie Martin

• José del R. Millán

• Robert T. Knight

• Brian Pasley

Date: 2016

DOI: 10.1016/j.bandl.2016.06.003

View PDF

Word pair classification during imagined speech using direct brain recordings

ABSTRACT

People that cannot communicate due to neurological disorders would bene t from an internal speech decoder. Here, we showed the ability to classify individual words during imagined speech from electrocorticographic signals. In a word imagery task, we used high gamma (70–150 Hz) time features with a support vector machine model to classify individual words from a pair of words. To account for temporal irregularities during speech production, we introduced a non-linear time alignment into the SVM kernel. Classi cation accuracy reached 88% in a two-class classi cation framework (50% chance level), and average classi cation accuracy across fteen word-pairs was signi cant across ve subjects (mean = 58%; p < 0.05). We also compared classi cation accuracy between imagined speech, overt speech and listening. As predicted, higher classi cation accuracy was obtained in the listening and overt speech conditions (mean = 89% and 86%, respectively; p < 0.0001), where speech stimuli were directly presented. The results provide evidence for a neural representation for imagined words in the temporal lobe, frontal lobe and sensorimotor cortex, consistent with previous ndings in speech perception and production. These data represent a proof of concept study for basic decoding of speech imagery, and delineate a number of key challenges to usage of speech imagery neural representations for clinical applications.

AUTHORS

• Stéphanie Martin

• Peter Brunner

• Iñaki Iturrate

• José del R. Millán

• Gerwin Schalk

• Robert T. Knight

• Brian Pasley

Date: 2016

DOI: 10.1038/srep25803

View PDF