EEG dementia re-evaluation · results explorer
The 2025 model reached 99% validation accuracy on folds that leaked subjects. Split over subjects, the best models reach about 60%.
Loading the exported results…
In 2025 the team's graph transformer was selected by 5-fold cross-validation over 15-second chunks, so chunks of one recording sat in both training and validation. Validation accuracy climbed to 99%. The re-evaluation splits every fold over subjects.
Log: the Spring 2025 team model's own training run, parsed line by line from legacy/…/training_log.txt. That model's code is mostly Subhrajit Dey's. Every other number on this page comes from Keshav's 2026 re-evaluation.
The bars use the metric each source reports: chunk accuracy for the 2025 runs, subject-level balanced accuracy for the re-evaluation (whisker: 95% bootstrap CI over subjects). The re-evaluation did not re-run the graph transformer; its bar is the best pre-specified model, a soft vote of three feature models.
Each dot is one subject's saved out-of-fold P(AD), P(CN), P(FTD), from a model that never saw that subject; its corner region is the predicted class.
Subject-level balanced accuracy, mean of 10 repeats of nested 5-fold CV, with 95% bootstrap CIs over subjects. The deep models were pre-registered; chance is 33.3%.
A permutation test shuffles the diagnoses across subjects and reruns the whole nested pipeline, inner model selection included. The grey bars are what the pipeline scores when there is nothing to learn.
The observed value is the mean of the 10 repeats, while each permutation runs one 5-fold CV, whose null is wider, so these p-values are conservative (as the repository's README notes).