Studies that decode imagined speech from scalp EEG typically validate their pipelines with an overt-speech positive control, yet null results rarely state what effect size the pipeline could have detected. The authors audited three public corpora—ds003626, KaraOne, and Track 3 of the 2020 International Brain-Computer Interface Competition—by injecting class-dependent effects of known amplitude into real recordings to measure the detection floor of five decoder families.
On ds003626, two lip electrodes decoded overt speech at 0.7430 accuracy and the eight external channels at 0.7068, versus 0.4231 for the 128-channel scalp montage, with no decodable signal surviving in the 8–30 Hz band. On the other two corpora, within-subject accuracy rose monotonically with frequency into the range dominated by muscle activity. The authors conclude that the overt-speech positive control is dominated by myogenic signal and cannot validate a decoding pipeline.
The detection-floor measurements also varied sharply by decoder: band-power logistic regression detected an injected 8–30 Hz desynchronization of 0.54 µV under a pre-specified criterion, whereas a compact convolutional network (EEGNet) detected nothing within subject up to 4.31 µV. In addition, the ds003626 dataset had serious pitfalls—condition codes were the reverse of the intuitive reading, the class encoding changed between public releases, and condition was a deterministic function of time within each session.
The authors recommend that speech-decoding studies report band-limited and artifact-channel accuracy, the direction of accuracy across frequency, and a measured detection floor. Without such reporting, they argue, null results are uninterpretable and positive controls may simply reflect muscle activity rather than neural speech decoding.