New benchmark reveals the promise and limitations of EEG foundation models

Artificial intelligence models trained on large-scale data have transformed language and vision applications, raising a fundamental question for brain-computer interfaces: can similarly scaled models learn general representations of brain activity? Electroencephalography (EEG), which records electrical activity from the scalp, provides an important pathway for decoding human intentions and cognitive states, but building reliable EEG models remains challenging because signals vary substantially across individuals, devices and tasks.

Foundation models have recently emerged as a promising direction for EEG analysis. By pre-training on large collections of largely unlabeled recordings, these models aim to learn transferable neural representations that can be adapted to diverse applications, including motor imagery, emotion recognition, clinical diagnosis and cognitive-state monitoring. However, direct comparisons between existing EEG foundation models have been difficult because studies often use different datasets, preprocessing strategies, training objectives and evaluation protocols.

In a new study published in National Science Review, researchers from Huazhong University of Science and Technology and Zhongguancun Academy introduced EEG-FM-Compass, a comprehensive framework for understanding and evaluating EEG foundation models. The researchers first reviewed 55 representative models and organized their designs according to data standardization strategies, model architectures and self-supervised learning objectives.

The team then developed a benchmark to evaluate 12 open-source EEG foundation models alongside competitive specialist models trained from scratch. The benchmark covered 13 public datasets spanning nine brain-computer interface paradigms, including motor imagery, P300, SSVEP, clinical EEG detection, emotion recognition, visual decoding, fatigue detection, sleep stage analysis and workload detection. The evaluation considered both cross-subject generalization, where models were tested on unseen participants, and few-shot adaptation, where models were adjusted using only limited data from a new user.

The study revealed several important findings about the current state of EEG foundation models. First, although pre-trained models demonstrated the ability to transfer knowledge across different tasks, most required task-specific adaptation to achieve reliable performance. Simply using frozen representations from pre-trained encoders was often insufficient, suggesting that current models have not yet achieved fully general-purpose EEG representations.

Second, specialized models trained from scratch remained highly competitive. Lightweight architectures designed for specific EEG tasks matched or even exceeded the performance of much larger foundation models in many scenarios. For example, EEGNet, a model containing only thousands of parameters, achieved strong overall performance despite being far smaller than many foundation models. These results indicate that model size alone does not determine effectiveness in EEG decoding.

Third, the researchers found that larger EEG foundation models did not consistently provide better generalization. Unlike the scaling trends observed in language and vision models, current EEG foundation models did not show a clear relationship between parameter size and downstream performance. The findings suggest that improving EEG model capability may require not only increasing model scale, but also developing larger and higher-quality EEG datasets, better pre-training strategies and more effective adaptation methods.

“EEG foundation models have demonstrated promising potential, but achieving truly generalizable representations of brain activity remains an open challenge,” said the researchers. “A systematic understanding of their strengths and limitations is essential for developing future brain-computer interface technologies.”

The study provides a comprehensive view of the progress and challenges of EEG foundation models and offers a benchmark for future research. The authors suggest that continued advances in data collection, model design and efficient adaptation will be critical for enabling more reliable and practical brain-computer interfaces.

The article, titled “EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models,” was accepted for publication in National Science Review. The authors are Dingkun Liu, Yuheng Chen, Zhu Chen, Zhenyao Cui, Yaozhi Wen, Jiayu An, Jingwei Luo and Dongrui Wu. The researchers are affiliated with Huazhong University of Science and Technology, Zhongguancun Academy and the Institute of Automation at the Chinese Academy of Sciences.

---

---