cs.HCMar 11, 2026

A Platform-Agnostic Multimodal Digital Human Modelling Framework: Neurophysiological Sensing in Game-Based Interaction

Authors: Daniel J. Buxton, Mufti Mahmud, Jordan J. Bird, Thomas Hughes-Roberts, David J. Brown

Organizations: Nottingham Trent University, Nottingham, NG11 8NS, United Kingdom · King Fahd University of Petroleum and Minerals, Dhahran 31261, Kingdom of Saudi Arabia

Abstract

Digital Human Modelling (DHM) is increasingly shaped by advances in AI, wearable biosensing, and interactive digital environments, particularly in research addressing accessibility and inclusion. However, many AI-enabled DHM approaches remain tightly coupled to specific platforms, tasks, or interpretative pipelines, limiting reproducibility, scalability, and ethical reuse. This paper presents a platform-agnostic DHM framework designed to support AI-ready multimodal interaction research by explicitly separating sensing, interaction modelling, and inference readiness. The framework integrates the OpenBCI Galea headset as a unified multimodal sensing layer, providing concurrent EEG, EMG, EOG, PPG, and inertial data streams, alongside a reproducible, game-based interaction environment implemented using SuperTux. Rather than embedding AI models or behavioural inference, physiological signals are represented as structured, temporally aligned observables, enabling downstream AI methods to be applied under appropriate ethical approval. Interaction is modelled using computational task primitives and timestamped event markers, supporting consistent alignment across heterogeneous sensors and platforms. Technical verification via author self-instrumentation confirms data integrity, stream continuity, and synchronisation; no human-subjects evaluation or AI inference is reported. Scalability considerations are discussed with respect to data throughput, latency, and extension to additional sensors or interaction modalities. Illustrative use cases demonstrate how the framework can support AI-enabled DHM and HCI studies, including accessibility-oriented interaction design and adaptive systems research, without requiring architectural modifications. The proposed framework provides an emerging-technology-focused infrastructure for future ethics-approved, inclusive DHM research.

Figures & tables

Explore similar work

CardsList
  1. Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis

    Aug 20, 2026Liang Xu, Chengqun Yang, Zili Lin +6Human Motion Generation

  2. Archon: A Unified Multimodal Model for Holistic Digital Human Generation

    May 28, 2026Chong Bao, Shichen Liu, Lijun Yu +9Unified Multimodal ModelsMultimodal Pretraining

  3. OPENJ: A Conceptual Framework for Open-Source Digital Human Modeling and Ergonomic Assessment in a CAD Environment

    May 5, 2026Sinan Bank, Casey E. Eaton