02272nas a2200277 4500008004100000245008600041210006900127260001200196520140600208653003301614653002901647653002001676653000901696653002501705653002401730653002001754653002201774100001401796700001401810700002401824700001501848700001901863700001901882700001601901856007701917 2015 eng d00aBrain-to-text: Decoding spoken sentences from phone representations in the brain.0 aBraintotext Decoding spoken sentences from phone representations c06/20153 aIt has long been speculated whether communication between humans and machines based on natural speech related cortical activity is possible. Over the past decade, studies have suggested that it is feasible to recognize isolated aspects of speech from neural signals, such as auditory features, phones or one of a few isolated words. However, until now it remained an unsolved challenge to decode continuously spoken speech from the neural substrate associated with speech and language processing. Here, we show for the first time that continuously spoken speech can be decoded into the expressed words from intracranial electrocorticographic (ECoG) recordings.Specifically, we implemented a system, which we call Brain-To-Text that models single phones, employs techniques from automatic speech recognition (ASR), and thereby transforms brain activity while speaking into the corresponding textual representation. Our results demonstrate that our system can achieve word error rates as low as 25% and phone error rates below 50%. Additionally, our approach contributes to the current understanding of the neural basis of continuous speech production by identifying those cortical regions that hold substantial information about individual phones. In conclusion, the Brain-To- Text system described in this paper represents an important step toward human-machine communication based on imagined speech.10aautomatic speech recognition10abrain-computer interface10abroadband gamma10aECoG10aElectrocorticography10apattern recognition10aspeech decoding10aspeech production1 aHerff, C.1 aHeger, D.1 ade Pesters, Adriana1 aTelaar, D.1 aBrunner, Peter1 aSchalk, Gerwin1 aSchultz, T. uhttp://journal.frontiersin.org/article/10.3389/fnins.2015.00217/abstract02134nas a2200241 4500008004100000022001400041245009300055210006900148260001200217300001100229490000700240520137400247653003001621653002501651653002001676653001701696653003501713100002601748700002401774700002601798700002001824856004801844 2015 eng d a1741-255200aWord-level language modeling for P300 spellers based on discriminative graphical models.0 aWordlevel language modeling for P300 spellers based on discrimin c04/2015 a0260070 v123 a

OBJECTIVE: In this work we propose a probabilistic graphical model framework that uses language priors at the level of words as a mechanism to increase the performance of P300-based spellers.

APPROACH: This paper is concerned with brain-computer interfaces based on P300 spellers. Motivated by P300 spelling scenarios involving communication based on a limited vocabulary, we propose a probabilistic graphical model framework and an associated classification algorithm that uses learned statistical models of language at the level of words. Exploiting such high-level contextual information helps reduce the error rate of the speller.

MAIN RESULTS: Our experimental results demonstrate that the proposed approach offers several advantages over existing methods. Most importantly, it increases the classification accuracy while reducing the number of times the letters need to be flashed, increasing the communication rate of the system.

SIGNIFICANCE: The proposed approach models all the variables in the P300 speller in a unified framework and has the capability to correct errors in previous letters in a word, given the data for the current one. The structure of the model we propose allows the use of efficient inference algorithms, which in turn makes it possible to use this approach in real-time applications.

10aBrain Computer Interfaces10ainference algorithms10alanguage models10aP300 speller10aprobabilistic graphical models1 aSaa, Jaime, F Delgado1 ade Pesters, Adriana1 aMcFarland, Dennis, J.1 aÇetin, Müjdat uhttp://www.ncbi.nlm.nih.gov/pubmed/25686293