cs.HCNov 8, 2026

Feel My Focus: Gaze-Triggered Digital Twin Creation for Remote Collaboration

Authors: Shih-Yu Ma, Suibi Che-Chuan Weng, Ming-Yuan Pai, Tom Yeh, Ellen Yi-Luen Do

Organizations: Department of Computer Science, University of Colorado Boulder, Boulder, CO, USA · ATLAS Institute, University of Colorado Boulder, Boulder, CO, USA · Department of Music, National Taiwan Normal University, Taipei, Taiwan

Abstract

We present Feel My Focus, a gaze-triggered mixed reality (MR) demonstration that enables rapid creation and sharing of lightweight 3D digital twins during remote collaboration. Unlike systems that use gaze primarily as a referential pointer, Feel My Focus treats sustained visual attention as a generative action: a fixation on a physical object initiates single-image capture, attention-guided segmentation, and one-shot 3D reconstruction, allowing a remote partner to inspect and discuss the object in situ. The system integrates gaze-tracking glasses, modular segmentation and reconstruction services, and an MR client for runtime streaming and interaction. This design prioritizes responsiveness and experiential continuity over photometric fidelity. We report observations from a public demonstration, highlighting design trade-offs, failure modes, and user control considerations for deploying gaze-triggered generative systems in real-world collaborative settings.

Figures & tables

Explore similar work

CardsList
  1. Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints

    Mar 12, 2026Lijun Guo, Haoyu Zhao, Xingyue Zhao +5Articulated Object Reconstruction3D Reconstruction

  2. SyncTwin: Fast Digital Twin Construction and Synchronization for Safe Robotic Manipulation

    Jan 14, 2026Ruopeng Huang, Boyu Yang, Wenlong Gui +3Digital TwinsSafe Robot Manipulation

  3. Int3DNet: Scene-Motion Cross Attention Network for 3D Intention Prediction in Mixed Reality

    Mar 9, 2026Taewook Ha, Woojin Cho, Dooyoung Kim +1Mixed Reality