02766nas a2200205   4500008004100000245009500041210007100136260001200207490000700219520209000226653002902316653002102345653003002366653002102396653002702417100002502444700002402469700001902493856004802512       2014                        eng d00aA general method for assessing brain–computer interface performance and its limitations.0 ageneral method for assessing brain–computer interface performanc  c03/20140 v113 aObjective. When researchers evaluate brain–computer interface (BCI) systems, we want quantitative answers to questions such as: How good is the system's performance? How good does it need to be? and: Is it capable of reaching the desired level in future? In response to the current lack of objective, quantitative, study-independent approaches, we introduce methods that help to address such questions. We identified three challenges: (I) the need for efficient measurement techniques that adapt rapidly and reliably to capture a wide range of performance levels; (II) the need to express results in a way that allows comparison between similar but non-identical tasks; (III) the need to measure the extent to which certain components of a BCI system (e.g. the signal processing pipeline) not only support BCI performance, but also potentially restrict the maximum level it can reach. Approach. For challenge (I), we developed an automatic staircase method that adjusted task difficulty adaptively along a single abstract axis. For challenge (II), we used the rate of information gain between two Bernoulli distributions: one reflecting the observed success rate, the other reflecting chance performance estimated by a matched random-walk method. This measure includes Wolpaw's information transfer rate as a special case, but addresses the latter's limitations including its restriction to item-selection tasks. To validate our approach and address challenge (III), we compared four healthy subjects' performance using an EEG-based BCI, a 'Direct Controller' (a high-performance hardware input device), and a 'Pseudo-BCI Controller' (the same input device, but with control signals processed by the BCI signal processing pipeline). Main results. Our results confirm the repeatability and validity of our measures, and indicate that our BCI signal processing pipeline reduced attainable performance by about 33% (21 bits/min). Significance. Our approach provides a flexible basis for evaluating BCI performance and its limitations, across a wide range of tasks and task difficulties.10abrain-computer interface10ainformation gain10ainformation transfer rate10aNeuroprosthetics10aperformance evaluation1 aHill, Jeremy, Jeremy1 aHäuser, Ann-Katrin1 aSchalk, Gerwin  uhttp://www.ncbi.nlm.nih.gov/pubmed/24658406