REVELIO: Universal Multimodal Task Load Estimation for Cross-Domain Generalization
Organizations: Department Digital Health and Analytics, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, 91058 Erlangen, Germany · Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen Nuremberg, 91052 Erlangen, Germany · Institute of AI for Health Helmholtz Zentrum München, 85764 Munich, Germany
Abstract
Adaptive interactive systems that respond to user mental state require task load estimation that stays reliable outside its training context. Models built in one setting rarely transfer, limiting real-world use. This work presents Robust Estimation Via End-to-End Learning of Multimodal Observations (REVELIO), a unified framework, dataset and evaluation protocol for cross-domain task load estimation. Cardiac, electrodermal, muscle, respiration, skin temperature, eye tracking and facial signals were recorded in 96 sessions, spanning a laboratory working memory task, a driving simulator and two games under consistent hardware and annotation. Multimodal fusion reached up to 0.87 area under the receiver operating characteristic curve, above single input eye tracking (0.83), facial action units (0.74) and biosignals (0.62). Multi-domain training stayed calibrated on domains seen in training, whereas removing the target domain caused the classifier to collapse in some applications. REVELIO provides a standardized foundation for measuring modality contributions and cross-domain generalization toward deployable workload-aware systems.
Figures & tables
| Ref. | Physiological | Subjective | Behavior | Performance | Subjects | Setup | Stimulus | Concept |
| ( Yannakakis et al.,, 2010 ) | EDA, HR | - | - | in game score | 8F, 28M | gaming setup | maze video game | challenge, anxiety, boredom |
| ( Lin and Imamiya,, 2006 ) | ET, ECG | NASA-TLX | ET, mouse clicks | game score | 1F, 9M | desktop workspace | action puzzle video game | cognitive load |
| ( Karpouzis et al.,, 2015 ) | - | Scaling and forced choice test | AUs, head movement | duration per level | 30F, 28M | – | jump and run video game | engagement, frustration |
| ( He et al.,, 2019 ; Kumar et al.,, 2022 ) | EEG, ECG, EDA, ET | NASA-TLX | ET | mean and std. of speed | 15F, 18M | driving simulator | driving tasks and modified -back | cognitive load |
| ( Gjoreski et al., 2020b, ) | ACC, EDA, TEMP, PPG | NASA-TLX | - | time on task, number of answers, game points | 46 (23 per task) | lab | standardized tests ( -back), video game | cognitive load |
| ( Meteier et al.,, 2021 ) | ECG, EDA, RESP | NASA-TLX | - | non-driving related task performance | 49F, 40M, 1 other | driving simulator | driving & verbal backward counting | cognitive load |
| Dimension | -back | Overcooked! 2 | Hogwarts Legacy |
| Primary process | working memory updating | planning, task switching, monitoring | visuospatial tracking, motor coordination |
| Input modality | visual (symbols) | visual (2D scene) | visual (3D first-person) |
| Response type | button press | button sequences | continuous joystick |
| Time pressure | fixed-pace | none to survival | none to race |
| Divided attention | single-task | baseline to dual-task | dual visual routes |
| Visuospatial complexity | minimal | 2D kitchen | 3D open-world |
| Application | Post-hoc pairwise (Holm-corrected) | |||
| -back | , , | |||
| Overcooked! 2 | , , | |||
| Hogwarts L. |
| Metric | Post-hoc pairwise (Holm-corrected) | |||
| Precision | : , : , : | |||
| Recall | : , : , : | |||
| Reaction Time | : , : , : |
| Metric | Post-hoc pairwise (Holm-corrected) | |||
| Successful Recipes per Cook | : | |||
| Tip per Cook | : | |||
| Recipe Points per Cook [k] | : |
| 2-2,6-4 2-3,9-2 2-4 2-5,4-2,5-8,7-8 2-6,4-5,8-11 2-7 2-8,6-6 2-9,5-2 2-10,3-11,4-10,8-6,9-9 2-11 3-2,5-9 3-3,3-10,4-8,8-10 3-4,6-5,10-8 3-5,4-4 3-6,7-9 3-7 3-8 3-9,5-4,5-5,5-11 4-3 4-6,6-9 4-7,8-8 4-9,9-5 4-11 5-3 5-6 5-7,10-7 5-10 6-2 6-3 6-7 6-8 6-10,10-4 6-11,7-4 7-2 7-3,10-9 7-5 7-6,8-2 7-7 7-10 7-11 8-3 8-4 8-5 8-7,10-2 8-9 9-3 9-4 9-6 9-7 9-8 9-10 9-11 10-3 10-5 10-6 10-10 10-11 | n-back (Driving) | n-back Dual Task | n-back (Gaming) | n-back (Combined) | Driving Application | Hogwarts Legacy | Overcooked! 2 | Gaming | Combined Applications | Complete | ||||||||||
| RSP | 0.50 0.059 | 0.52 0.052 | 0.50 0.073 | 0.57 0.047 | 0.65 0.063 | 0.45 0.090 | 0.53 0.110 | 0.61 0.054 | 0.64 0.028 | 0.65 0.013 | ||||||||||
| EMG | 0.57 0.021 | 0.66 0.033 | 0.50 0.021 | 0.58 0.031 | 0.71 0.032 | 0.61 0.065 | 0.53 0.028 | 0.54 0.034 | 0.66 0.030 | 0.64 0.016 | ||||||||||
| EDA | 0.57 0.047 | 0.68 0.036 | 0.58 0.072 | 0.65 0.040 | 0.63 0.042 | 0.77 0.078 | 0.66 0.073 | 0.70 0.078 | 0.64 0.035 | 0.66 0.038 | ||||||||||
| SKT | 0.61 0.110 | 0.57 0.074 | 0.54 0.052 | 0.54 0.076 | 0.69 0.028 | 0.48 0.116 | 0.57 0.111 | 0.57 0.123 | 0.58 0.035 | 0.54 0.048 | ||||||||||
| PPG | 0.56 0.094 | 0.45 0.104 | 0.50 0.075 | 0.50 0.048 | 0.53 0.085 | 0.64 0.085 | 0.61 0.073 | 0.64 0.041 | 0.55 0.048 | 0.51 0.043 | ||||||||||
| ECG | 0.58 0.060 | 0.56 0.058 | 0.51 0.022 | 0.60 0.042 | 0.77 0.032 | 0.67 0.085 | 0.57 0.054 | 0.71 0.032 | 0.76 0.023 | 0.73 0.026 | ||||||||||
| 2-2,2-11,8-3 2-3 2-4,2-5 2-6,9-9 2-7 2-8 2-9,2-10,3-8,8-7 3-2,3-3,3-11,7-5 3-4 3-5 3-6,5-10,13-8 3-7 3-9,12-8 3-10 4-2,11-8 4-3,10-3 4-4,10-4,11-4 4-5,11-9 4-6 4-7 4-8,5-6,10-2 4-9,10-6,13-3 4-10,4-11,11-5 5-2,10-8 5-3,9-2,12-10 5-4,9-4 5-5 5-7,6-11,9-8 5-8,8-10 5-9,6-10 5-11 6-2,6-5 6-3 6-4 6-6,8-5,8-6 6-7 6-8,6-9 7-2 7-3 7-4,9-10,9-11 7-6 7-7 7-8 7-9,8-8,8-11 7-10,7-11,9-3 8-2 8-4 8-9 9-5 9-6,12-4,12-7 9-7 10-5,12-3 10-7,11-6 10-9,13-2 10-10,13-7 10-11,12-6,13-11 11-2 11-3 11-7,13-6 11-10 11-11 12-2 12-5 12-9 12-11 13-4 13-5 13-9 13-10 | n-back (Driving) | n-back Dual Task | n-back (Gaming) | n-back (Combined) | Driving Application | Hogwarts Legacy | Overcooked! 2 | Gaming | Combined Applications | Complete | ||||||||||
| Heart Comb. | 0.55 0.099 | 0.53 0.121 | 0.47 0.080 | 0.47 0.070 | 0.56 0.053 | 0.52 0.104 | 0.60 0.059 | 0.61 0.057 | 0.61 0.037 | 0.55 0.051 | ||||||||||
| Biosignals | 0.62 0.054 | 0.62 0.072 | 0.52 0.084 | 0.57 0.067 | 0.69 0.007 | 0.71 0.040 | 0.61 0.067 | 0.69 0.059 | 0.61 0.034 | 0.62 0.034 | ||||||||||
| Eye Comb. | 0.78 0.072 | 0.87 0.033 | 0.84 0.028 | 0.85 0.026 | 0.84 0.036 | 0.88 0.080 | 0.78 0.077 | 0.85 0.052 | 0.83 0.031 | 0.83 0.031 | ||||||||||
| Head Rotation | 0.78 0.054 | 0.70 0.029 | 0.68 0.037 | 0.65 0.024 | 0.78 0.033 | 0.58 0.032 | 0.59 0.075 | 0.58 0.037 | 0.68 0.027 | 0.67 0.015 | ||||||||||
| Head Position | 0.53 0.042 | 0.50 0.043 | 0.63 0.070 | 0.54 0.041 | 0.59 0.031 | 0.60 0.073 | 0.54 0.085 | 0.54 0.067 | 0.58 0.050 | 0.58 0.028 | ||||||||||
| Head Comb. | 0.72 0.075 | 0.63 0.053 | 0.64 0.038 | 0.62 0.060 | 0.74 0.045 | 0.64 0.053 | 0.51 0.042 | 0.56 0.057 | 0.64 0.041 | 0.64 0.035 | ||||||||||
| 3-2,3-4,8-4,13-11 3-3 3-5,7-5,9-8,9-11 3-6 3-7 3-8,3-10,10-7 3-9,10-12 3-11,8-7 3-12 3-13 4-2 4-3 4-4 4-5,8-6 4-6,7-4,7-8,10-11 4-7 4-8,6-5,8-10,10-9 4-9 4-10,4-13,13-3 4-11,6-2,14-13 4-12,10-8 5-2,12-4 5-3,6-10,11-5 5-4 5-5,5-12,11-9,12-12,14-8 5-6 5-7,5-11 5-8,14-3 5-9,11-7,12-11,12-13 5-10,12-3 5-13,12-2 6-3 6-4,13-2 6-6,10-4 6-7,13-5,14-11 6-8,11-4,11-6,12-6,12-7 6-9 6-11,7-2,8-11 6-12 6-13,8-12,9-2,9-3 7-3,7-10 7-6 7-7,9-7 7-9,9-6 7-11 7-12,9-5 7-13 8-2,10-6 8-3 8-5,10-3 8-8,8-9 8-13,9-10 9-4 9-9 9-12 9-13 10-2 10-5 10-10 10-13 11-2,11-3,11-11,13-9 11-8,12-10 11-10 11-12,14-6 11-13 12-5 12-8 12-9 13-4 13-6 13-7 13-8 13-10,14-12 13-12 13-13 14-2,14-7 14-4,14-5 14-9,14-10 | Complete | n-Back | Driving | Gaming | ||||||||
| xLSTM | Conv- Next | Trans- former | xLSTM | Conv- Next | Trans- former | xLSTM | Conv- Next | Trans- former | xLSTM | Conv- Next | Trans- former | |
| Heart Comb. | 0.67 0.026 | 0.55 0.051 | 0.66 0.066 | 0.55 0.046 | 0.47 0.070 | 0.51 0.018 | 0.64 0.044 | 0.56 0.053 | 0.64 0.056 | 0.63 0.046 | 0.61 0.057 | 0.62 0.059 |
| Biosignals | 0.71 0.015 | 0.62 0.034 | 0.68 0.033 | 0.63 0.004 | 0.57 0.067 | 0.60 0.042 | 0.73 0.028 | 0.69 0.007 | 0.72 0.033 | 0.72 0.068 | 0.69 0.059 | 0.72 0.073 |
| Eye Comb. | 0.84 0.021 | 0.83 0.031 | 0.85 0.028 | 0.85 0.023 | 0.85 0.026 | 0.86 0.022 | 0.83 0.053 | 0.84 0.036 | 0.87 0.031 | 0.86 0.054 | 0.85 0.052 | 0.86 0.053 |
| Head Rotation | 0.73 0.047 | 0.67 0.015 | 0.73 0.033 | 0.73 0.028 | 0.65 0.024 | 0.72 0.044 | 0.83 0.055 | 0.78 0.033 | 0.84 0.053 | 0.59 0.026 | 0.58 0.037 | 0.56 0.058 |
| Head Position | 0.59 0.060 | 0.58 0.028 | 0.57 0.048 | 0.55 0.044 | 0.54 0.041 | 0.52 0.025 | 0.57 0.030 | 0.59 0.031 | 0.58 0.041 | 0.56 0.048 | 0.54 0.067 | 0.60 0.073 |
| Domain | AUROC 1 | AUROC 2 | 95 % CI | ||
| Pupil diameter vs. ECG | |||||
| Complete | 0.717 | 0.708 | |||
| -back (Combined) | 0.825 | 0.597 | |||
| -back (Driving) | 0.824 | 0.580 | |||
| -back (Gaming) | 0.825 | 0.526 | |||
| Driving Application | 0.674 | 0.739 | |||
| Domain | ECG | PPG | EDA | EMG | RSP | SKT | Eye Mv. | Pupil | ||||||||
| Complete | * | * | * | * | * | * | * | * | ||||||||
| -back (Combined) | * | * | * | * | * | * | * | |||||||||
| -back (Driving) | * | * | * | * | * | * | * | |||||||||
| -back (Gaming) | * | * | * | * | * | * | * | |||||||||
| Driving Application | * | * | * | * | * | * | * | |||||||||
| Gaming | * | * | * | * | * | * | * | |||||||||
| Trained on | Evaluated on | xLSTM | ConvNeXt | Transformer |
| Complete | Complete | 0.08 0.05 | 0.06 0.06 | 0.09 0.03 |
| -back | 0.09 0.05 | 0.06 0.05 | 0.11 0.02 | |
| Driving | 0.10 0.05 | 0.06 0.05 | 0.09 0.04 | |
| Gaming | 0.08 0.05 | 0.08 0.07 | 0.09 0.04 | |
| D. & G. | 0.08 0.04 | 0.06 0.06 | 0.09 0.03 | |
| -back | Complete | 0.23 0.03 | 0.18 0.02 | 0.24 0.03 |
| Domain | compl. | ||||
| -back (Driving Study) | 0.780 | 0.713 | 0.700 | ||
| -back (Gaming Study) | 0.845 | 0.807 | 0.752 | ||
| -back (Combined) | 0.831 | 0.808 | 0.746 | ||
| Driving Application | 0.851 | 0.845 | 0.484 | ||
| Hogwarts Legacy | 0.896 | 0.863 | 0.765 | ||
| Overcooked! 2 | 0.774 | 0.759 | 0.687 |
| Target | In-domain | LODO | Base | Small | ||
| -back | 0.766 | 0.713 | ||||
| Driving Application | 0.879 | 0.411 | ||||
| Overcooked! 2 | 0.784 | 0.719 | ||||
| Hogwarts Legacy | 0.878 | 0.786 |
| Target | In-cohort | LOCO | Base | Small | ||
| Driving study | ||||||
| -back | 0.787 | 0.689 | ||||
| Driving Application | 0.888 | 0.429 | ||||
| Pooled | 0.859 | 0.490 | ||||
| Gaming study | ||||||
| -back | 0.796 | 0.579 | ||||
| Domain | Page’s | ||||||
| Game -back | 37 | 5 | 0.171 | 0.591 | |||
| Drive -back | 51 | 5 | 0.224 | 0.688 | – | ||
| Combined -back | 83 | 5 | 0.202 | 0.647 | – | ||
| Overcooked! 2 | 35 | 4 | 0.175 | 0.666 | |||
| Hogwarts Legacy | 36 | 4 | 0.284 | 0.795 | – |
| Domain | Page’s | Mono. | ||||||
| ConvNeXt | ||||||||
| Game -back | 0.295 | 0.626 | 0.713 | Yes | ||||
| Drive -back | 0.342 | 0.545 | 0.595 | Yes | ||||
| Combined -back | 0.386 | 0.576 | 0.649 | Yes | ||||
| Overcooked! 2 | 0.432 | 0.425 | 0.591 | No | ||||
| Hogwarts Legacy | 0.350 | 0.609 | 0.667 | Yes | ||||
| Measure | Task Load | Predicted load | NASA- RTLX | Reaction time | F1-Score |
| Task Load | – | ||||
| Predicted load | *** | – | |||
| NASA- RTLX | *** | *** | – | ||
| Reaction time | *** | *** | *** | – | |
| F1-score | *** | ** * | *** | *** | – |
| Domain | Cohort | Sessions | Participants |
| -back (Driving Study) | driving | 51 | 51 |
| -back Dual Task | driving | 51 | 51 |
| -back (Gaming Study) | gaming | 44 | 37 |
| -back (Combined) | both | 95 | 83 |
| Driving Application | driving | 51 | 51 |
| Hogwarts Legacy | gaming | 44 | 37 |
| Group | % positive | % outer mono. | |
| Game -back | 37 | 86 | 95 |
| Drive -back | 51 | 94 | 96 |
| Combined -back | 83 | 90 | 95 |
| Overcooked! 2 | 35 | 97 | 97 |
| Hogwarts Legacy | 36 | 92 | 97 |
| Cohort | M1 NASA- RTLX | M3 NASA- RTLX | M2 NASA- RTLX | M2 Reaction time | M2 F1-score | |
| Drive -back | 51 (306) | *** | *** | |||
| Game -back | 37 (111) | *** | *** |
| Domain | Encoder | Best single | Fusion | Single | ||
| Complete | x LSTM | Eye Mv. | 0.822 | 0.765 | ||
| ConvNeXt | Eye Mv. | 0.825 | 0.733 | |||
| Transformer | Eye Mv. | 0.833 | 0.776 | |||
| -back (Combined) | x LSTM | Pupil | 0.836 | 0.841 | ||
| ConvNeXt | Pupil | 0.831 | 0.804 | |||
| Transformer | Pupil | 0.838 | 0.829 |
| Domain | Single source | compl. | |||
| -back (Driving Study) | -back (Gaming Study) | 0.713 | 0.730 | ||
| -back (Driving Study) | -back (Combined) | 0.713 | 0.771 | ||
| -back (Driving Study) | Driving Application | 0.713 | 0.778 | ||
| -back (Driving Study) | Hogwarts Legacy | 0.713 | 0.709 | ||
| -back (Driving Study) | Overcooked! 2 | 0.713 | 0.547 | ||
| -back (Driving Study) | Gaming | 0.713 | 0.667 |
| Diversity | Quantity | |||
| Source | Base | Small | Base | Small |
| Target: -back | ||||
| Driving Application | ** | ** | ** | |
| Overcooked! 2 | ** | ** | ||
| Hogwarts Legacy | ||||
| Target: Driving Application | ||||
| 3-4, 5-4 8-4, 12-4 4-4 10-4 6-4 2-4 11-4 7-4 13-4 9-4 Modality | Train Domain | Eval Domain | AUROC ( ) | |||
| \Block 4-1Biosignals | Drive N-Back | In-Domain (Baseline) | ||||
| Drive N-Back | Game N-Back | |||||
| Game N-Back | In-Domain (Baseline) | |||||
| Game N-Back | Drive N-Back | |||||
| \Block 4-1Biosignals, Eye | Drive N-Back | In-Domain (Baseline) | ||||
| Drive N-Back | Game N-Back | |||||
| 2-4 5-4 9-4 10-4 6-4, 13-4 7-4 4-4 3-4 11-4 8-4 12-4 Modality | Train Domain | Eval Domain | ECE ( ) | |||
| \Block 4-1Biosignals | Drive N-Back | Drive N-Back | ||||
| Drive N-Back | Game N-Back | |||||
| Game N-Back | Drive N-Back | |||||
| Game N-Back | Game N-Back | |||||
| \Block 4-1Biosignals, Eye | Drive N-Back | Drive N-Back | ||||
| Drive N-Back | Game N-Back | |||||
| Modality | Train domain | Eval domain | AUROC | Encoder range | Pairs | |
| Biosignals | Drive N-Back | Game N-Back | . | 15/15 | ||
| Game N-Back | Drive N-Back | . | 9/15 | |||
| Biosignals, Eye | Drive N-Back | Game N-Back | . | 14/15 | ||
| Game N-Back | Drive N-Back | . | 15/15 | |||
| Bio., AUs, ET | Drive N-Back | Game N-Back | . | 15/15 | ||
| Game N-Back | Drive N-Back | . | 14/15 | |||
| Size | Kernels | Strides | Channels |
| tiny | 7, 7, 7 | 3, 3, 3 | 16, 32, 32 |
| small | 7, 7, 7 | 3, 3, 2 | 32, 64, 64 |
| base | 7, 7, 7 | 3, 2, 2 | 64, 128, 128 |
| Size | Hidden Size | Layers |
| tiny | 64 | 2 |
| small | 128 | 4 |
| base | 256 | 8 |
| Size | MLSTM Heads | SLSTM Heads |
| tiny | 2 | 2 |
| small | 4 | 4 |
| base | 8 | 8 |
| Size | Layers | Dim. | Heads | feed-forward network (FFN) | Dropout |
| tiny | 6 | 32 | 8 | 1024 | 0.05 |
| small | 8 | 64 | 8 | 2048 | 0.10 |
| base | 12 | 128 | 16 | 3072 | 0.15 |
| Size | Stage | in | out | layers |
| tiny | 1 | 96 | 192 | 3 |
| 2 | 192 | 384 | 3 | |
| 3 | 384 | 768 | 9 | |
| 4 | 768 | - | 3 | |
| small | 1 | 96 | 192 | 3 |
| 2 | 192 | 384 | 3 |
| 4-3,5-3,6-3,6-4,6-5,7-3,7-4,7-5,7-6,8-3,8-4,9-3,9-4,10-3,10-4 2-3,3-3,4-4,5-4,5-5,6-6,7-7,8-5,9-5,10-5,10-6 3-4,4-5,7-8,8-6,9-6 2-4,5-6,6-7,10-7 3-5,4-6,9-7 2-5,6-8,8-7 3-6,4-7,5-7,10-8 | 32 | 64 | 128 | 256 | 512 | 1024 | |||
| ConvNeXt | base | 5.4 | 9.1 | 15.9 | |||||
| small | 4.0 | 6.8 | 11.8 | 22.3 | |||||
| tiny | 2.4 | 3.9 | 6.5 | 11.9 | 22.8 | ||||
| Transformer | base | 2.0 | 3.1 | 5.7 | 10.5 | 20.4 | |||
| small | 1.1 | 1.6 | 2.8 | 4.8 | 8.8 | 16.9 | |||
| tiny | 1.2 | 0.9 | 1.3 | 2.0 | 3.3 | 6.0 | |||
| 2-2 2-3,5-4 2-4 2-5 2-6,9-4 2-7,5-10 2-8,5-5,6-10 2-9 2-10 2-11,4-8,6-7 3-2 3-3,4-5,4-10 3-4,3-9,10-9 3-5,7-8 3-6,7-6 3-7,6-8,7-2 3-8 3-10,7-9 3-11,4-11,8-6,9-9 4-2 4-3 4-4 4-6 4-7,8-11,10-11 4-9,7-10,10-10 5-2 5-3,7-3 5-6,9-7 5-7,7-4 5-8,5-9 5-11,6-6,6-11 6-2,10-4 6-3 6-4 6-5 6-9 7-5 7-7 7-11 8-2 8-3 8-4 8-5 8-7,10-2 8-8 8-9 8-10 9-2,10-8 9-3 9-5 9-6 9-8 9-10 9-11 10-3 10-5 10-6 10-7 | n-back (Driving) | n-back Dual Task | n-back (Gaming) | n-back (Combined) | Driving Application | Hogwarts Legacy | Overcooked! 2 | Gaming | Combined Applications | Complete | ||||||||||
| RSP | 0.50 0.112 | 0.51 0.054 | 0.49 0.057 | 0.55 0.059 | 0.63 0.109 | 0.58 0.043 | 0.55 0.062 | 0.56 0.067 | 0.64 0.017 | 0.66 0.013 | ||||||||||
| EMG | 0.60 0.049 | 0.65 0.045 | 0.53 0.060 | 0.60 0.026 | 0.74 0.043 | 0.59 0.057 | 0.48 0.034 | 0.53 0.027 | 0.68 0.030 | 0.67 0.012 | ||||||||||
| EDA | 0.65 0.063 | 0.62 0.050 | 0.56 0.073 | 0.64 0.049 | 0.66 0.044 | 0.75 0.086 | 0.66 0.071 | 0.73 0.094 | 0.65 0.030 | 0.67 0.041 | ||||||||||
| SKT | 0.63 0.136 | 0.61 0.047 | 0.51 0.093 | 0.55 0.018 | 0.73 0.044 | 0.54 0.101 | 0.59 0.110 | 0.59 0.089 | 0.58 0.050 | 0.55 0.017 | ||||||||||
| PPG | 0.51 0.072 | 0.49 0.065 | 0.46 0.049 | 0.43 0.031 | 0.55 0.115 | 0.66 0.092 | 0.59 0.065 | 0.66 0.066 | 0.55 0.035 | 0.56 0.026 | ||||||||||
| ECG | 0.59 0.100 | 0.61 0.033 | 0.54 0.039 | 0.59 0.014 | 0.74 0.038 | 0.67 0.059 | 0.60 0.038 | 0.68 0.075 | 0.73 0.049 | 0.69 0.053 | ||||||||||
| 2-2,6-4 2-3 2-4 2-5,6-5,8-6,8-8,8-9 2-6,2-10 2-7,2-9 2-8,6-9,7-8,8-10,9-8 2-11,3-10,12-9 3-2 3-3 3-4 3-5 3-6,5-5 3-7 3-8,9-5,9-10,13-8 3-9,5-11 3-11 4-2,10-2,10-5 4-3,11-6 4-4 4-5,13-6 4-6,11-10 4-7 4-8,10-4,13-7,13-10 4-9 4-10,4-11,5-3,13-3 5-2,5-6 5-4 5-7 5-8,8-5 5-9,6-7,6-11,7-9 5-10,11-8,12-2 6-2 6-3 6-6,6-10,9-9,12-8 6-8,9-7 7-2 7-3 7-4 7-5 7-6 7-7 7-10 7-11 8-2 8-3,8-11 8-4 8-7 9-2,13-4 9-3,12-7 9-4 9-6 9-11 10-3 10-6,11-7 10-7 10-8 10-9,10-11 10-10,11-9 11-2,13-5 11-3 11-4 11-5 11-11 12-3 12-4 12-5,13-9 12-6 12-10 12-11 13-2 13-11 | n-back (Driving) | n-back Dual Task | n-back (Gaming) | n-back (Combined) | Driving Application | Hogwarts Legacy | Overcooked! 2 | Gaming | Combined Applications | Complete | ||||||||||
| Heart Comb. | 0.58 0.039 | 0.48 0.104 | 0.48 0.112 | 0.55 0.046 | 0.64 0.044 | 0.63 0.150 | 0.56 0.052 | 0.63 0.046 | 0.64 0.030 | 0.67 0.026 | ||||||||||
| Biosignals | 0.67 0.053 | 0.70 0.040 | 0.52 0.053 | 0.63 0.004 | 0.73 0.028 | 0.72 0.055 | 0.63 0.104 | 0.72 0.068 | 0.66 0.065 | 0.71 0.015 | ||||||||||
| Eye Comb. | 0.83 0.043 | 0.86 0.025 | 0.82 0.068 | 0.85 0.023 | 0.83 0.053 | 0.88 0.071 | 0.79 0.052 | 0.86 0.054 | 0.84 0.029 | 0.84 0.021 | ||||||||||
| Head Rotation | 0.83 0.059 | 0.84 0.062 | 0.62 0.089 | 0.73 0.028 | 0.83 0.055 | 0.55 0.060 | 0.54 0.091 | 0.59 0.026 | 0.75 0.051 | 0.73 0.047 | ||||||||||
| Head Position | 0.52 0.062 | 0.46 0.086 | 0.58 0.043 | 0.55 0.044 | 0.57 0.030 | 0.59 0.080 | 0.56 0.073 | 0.56 0.048 | 0.57 0.038 | 0.59 0.060 | ||||||||||
| Head Comb. | 0.69 0.062 | 0.71 0.054 | 0.60 0.042 | 0.65 0.041 | 0.74 0.065 | 0.61 0.038 | 0.57 0.066 | 0.59 0.049 | 0.68 0.055 | 0.66 0.036 | ||||||||||
| 2-2,2-4,10-7 2-3,3-8,6-2 2-5,3-5 2-6,4-5,4-6 2-7,5-4 2-8,5-8 2-9 2-10,9-4 2-11 3-2 3-3 3-4,9-2 3-6,8-8 3-7 3-9 3-10,7-11 3-11 4-2,4-3,4-8 4-4 4-7 4-9,5-6 4-10 4-11 5-2 5-3 5-5 5-7 5-9 5-10,6-7 5-11,6-11,10-9 6-3 6-4 6-5 6-6 6-8 6-9 6-10 7-2 7-3,7-5 7-4 7-6 7-7,9-5 7-8 7-9,9-9 7-10 8-2,10-2 8-3 8-4 8-5 8-6 8-7 8-9 8-10,9-8 8-11,10-10 9-3,9-10 9-6,10-6 9-7 9-11 10-3 10-4 10-5,10-11 10-8 | n-back (Driving) | n-back Dual Task | n-back (Gaming) | n-back (Combined) | Driving Application | Hogwarts Legacy | Overcooked! 2 | Gaming | Combined Applications | Complete | ||||||||||
| RSP | 0.50 0.061 | 0.53 0.035 | 0.50 0.076 | 0.59 0.025 | 0.64 0.102 | 0.50 0.088 | 0.56 0.040 | 0.58 0.065 | 0.62 0.032 | 0.65 0.022 | ||||||||||
| EMG | 0.62 0.050 | 0.64 0.051 | 0.54 0.076 | 0.59 0.052 | 0.75 0.045 | 0.58 0.091 | 0.53 0.068 | 0.52 0.009 | 0.70 0.026 | 0.67 0.010 | ||||||||||
| EDA | 0.64 0.069 | 0.64 0.028 | 0.52 0.034 | 0.64 0.032 | 0.64 0.034 | 0.76 0.077 | 0.64 0.064 | 0.72 0.099 | 0.65 0.027 | 0.67 0.044 | ||||||||||
| SKT | 0.62 0.104 | 0.59 0.042 | 0.50 0.116 | 0.51 0.051 | 0.72 0.037 | 0.50 0.040 | 0.56 0.054 | 0.60 0.103 | 0.58 0.047 | 0.55 0.018 | ||||||||||
| PPG | 0.53 0.078 | 0.48 0.125 | 0.46 0.107 | 0.46 0.070 | 0.56 0.079 | 0.58 0.177 | 0.63 0.098 | 0.69 0.070 | 0.55 0.017 | 0.55 0.024 | ||||||||||
| ECG | 0.57 0.069 | 0.60 0.034 | 0.52 0.033 | 0.60 0.034 | 0.72 0.072 | 0.74 0.024 | 0.56 0.045 | 0.67 0.031 | 0.73 0.058 | 0.70 0.050 | ||||||||||
| 2-2 2-3 2-4 2-5 2-6,9-5,12-8 2-7 2-8 2-9 2-10 2-11,7-11 3-2 3-3 3-4,6-5,8-5 3-5,5-4,6-4 3-6,3-9 3-7,11-8 3-8,7-5,13-8 3-10 3-11,9-3 4-2,13-11 4-3,11-6 4-4,5-2,5-3,10-11,11-5 4-5 4-6 4-7,4-10,10-7 4-8 4-9 4-11,11-10 5-5,7-2 5-6,10-10 5-7 5-8 5-9,8-2 5-10 5-11 6-2,6-8 6-3 6-6 6-7,8-10,8-11 6-9,6-10 6-11 7-3,13-9 7-4 7-6 7-7 7-8,8-4,8-9 7-9,8-3,8-6,9-8 7-10 8-7 8-8 9-2 9-4 9-6 9-7 9-9 9-10,9-11 10-2,12-3 10-3,11-7 10-4 10-5,11-9 10-6 10-8 10-9 11-2 11-3 11-4,12-6 11-11 12-2,13-7 12-4 12-5 12-7 12-9 12-10 12-11 13-2,13-5 13-3 13-4 13-6 13-10 | n-back (Driving) | n-back Dual Task | n-back (Gaming) | n-back (Combined) | Driving Application | Hogwarts Legacy | Overcooked! 2 | Gaming | Combined Applications | Complete | ||||||||||
| Heart Comb. | 0.56 0.058 | 0.44 0.092 | 0.46 0.085 | 0.51 0.018 | 0.64 0.056 | 0.68 0.067 | 0.56 0.050 | 0.62 0.059 | 0.66 0.041 | 0.66 0.066 | ||||||||||
| Biosignals | 0.62 0.018 | 0.69 0.037 | 0.52 0.124 | 0.60 0.042 | 0.72 0.033 | 0.75 0.126 | 0.63 0.080 | 0.72 0.073 | 0.65 0.067 | 0.68 0.033 | ||||||||||
| Eye Comb. | 0.81 0.034 | 0.87 0.038 | 0.83 0.035 | 0.86 0.022 | 0.87 0.031 | 0.87 0.061 | 0.80 0.050 | 0.86 0.053 | 0.87 0.021 | 0.85 0.028 | ||||||||||
| Head Rotation | 0.83 0.073 | 0.83 0.040 | 0.60 0.066 | 0.72 0.044 | 0.84 0.053 | 0.60 0.030 | 0.59 0.047 | 0.56 0.058 | 0.74 0.036 | 0.73 0.033 | ||||||||||
| Head Position | 0.49 0.071 | 0.48 0.051 | 0.60 0.016 | 0.52 0.025 | 0.58 0.041 | 0.54 0.129 | 0.49 0.073 | 0.60 0.073 | 0.59 0.024 | 0.57 0.048 | ||||||||||
| Head Comb. | 0.72 0.086 | 0.72 0.087 | 0.62 0.064 | 0.63 0.048 | 0.73 0.050 | 0.57 0.105 | 0.51 0.088 | 0.53 0.053 | 0.67 0.048 | 0.67 0.040 | ||||||||||
| 3-2 3-3,3-4 3-5,6-4,7-4,11-13 3-6 3-7,4-7,8-5 3-8,5-9,7-12 3-9,5-6,9-3 3-10,5-7,5-10 3-11,7-10 3-12 3-13 4-2,4-4,5-2,9-8,10-11 4-3,9-9,10-12 4-5,8-7 4-6 4-8,8-8,10-7 4-9,8-12 4-10,9-2,11-2 4-11,11-11 4-12,6-3,6-6 4-13 5-3,5-8 5-4 5-5 5-11 5-12,8-4,10-6 5-13,6-10 6-2,7-2,7-8 6-5 6-7 6-8,8-3 6-9 6-11,6-13 6-12 7-3 7-5 7-6 7-7 7-9 7-11 7-13 8-2 8-6 8-9,10-2 8-10 8-11 8-13,10-13 9-4 9-5 9-6 9-7 9-10 9-11,9-12,9-13 10-3 10-4 10-5,11-3 10-8 10-9 10-10,11-9,11-10 11-4,11-6,11-7 11-5 11-8 11-12 | Complete | n-Back | Driving | Gaming | ||||||||
| xLSTM | Conv- Next | Trans- former | xLSTM | Conv- Next | Trans- former | xLSTM | Conv- Next | Trans- former | xLSTM | Conv- Next | Trans- former | |
| RSP | 0.66 0.013 | 0.65 0.013 | 0.65 0.022 | 0.55 0.059 | 0.57 0.047 | 0.59 0.025 | 0.63 0.109 | 0.65 0.063 | 0.64 0.102 | 0.56 0.067 | 0.61 0.054 | 0.58 0.065 |
| EMG | 0.67 0.012 | 0.64 0.016 | 0.67 0.010 | 0.60 0.026 | 0.58 0.031 | 0.59 0.052 | 0.74 0.043 | 0.71 0.032 | 0.75 0.045 | 0.53 0.027 | 0.54 0.034 | 0.52 0.009 |
| EDA | 0.67 0.041 | 0.66 0.038 | 0.67 0.044 | 0.64 0.049 | 0.65 0.040 | 0.64 0.032 | 0.66 0.044 | 0.63 0.042 | 0.64 0.034 | 0.73 0.094 | 0.70 0.078 | 0.72 0.099 |
| SKT | 0.55 0.017 | 0.54 0.048 | 0.55 0.018 | 0.55 0.018 | 0.54 0.076 | 0.51 0.051 | 0.73 0.044 | 0.69 0.028 | 0.72 0.037 | 0.59 0.089 | 0.57 0.123 | 0.60 0.103 |
| PPG | 0.56 0.026 | 0.51 0.043 | 0.55 0.024 | 0.43 0.031 | 0.50 0.048 | 0.46 0.070 | 0.55 0.115 | 0.53 0.085 | 0.56 0.079 | 0.66 0.066 | 0.64 0.041 | 0.69 0.070 |