cs.HCNov 2, 2026

Touching Projected Agents through the DAT Screen -System Design and Questions of Interaction Boundaries-

Authors: Takaya Tamamoto, Tomoko Yonezawa

Organizations: Kansai University, Takatsuki, Japan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Touching a virtual agent involves more than detecting a finger on a display: the contact surface can deform and resist the user's push, and the agent can respond to that action. We describe the Desktop Amid Touch (DAT) screen, which combines spatial projection onto a semi-transparent, deformable mesh with touch sensing and motorized control of resistance to indentation. An initial prototype detected a single touch and estimated its position and depth from tension changes around a flexible projection screen. A subsequent prototype added motorized tension control to vary the screen's resistance to indentation. We describe the tension-controlled DAT implementation and its demonstrations, distinguishing what has been observed from what remains an open design question. These capabilities raise a design question: should touchable virtual agents respond as though they had preferences or boundaries, and under what conditions should such behavior be implemented on systems like DAT. The present paper describes a system and frames this question; it does not establish how users interpret an agent's refusal or consent.

Figures & tables

Figure 1. A user reaching toward an agent projected on the DAT screen. A projected agent appears on a semi-transparent mesh as a user's finger approaches it.
Figure 2. DAT screen, hardware layout, and tension sensor. Adapted from ( Tamamoto and Yonezawa, 2025 ) . Photograph of the DAT mesh and projector, schematic showing sensors and servomotors around the screen, and a close-up of the sliding-potentiometer tension sensor.
Figure 3. Screen regions for tension control. Adapted from ( Tamamoto and Yonezawa, 2025 ) . Diagram dividing the DAT screen into left, center, and right regions and showing the corresponding servomotors and tension sensors.
Figure 4. On-site demonstration of DAT. Two views of the projected agent: its default standing state and its seated state after a fall, with its eyes closed.
More in this topic

Human-Computer Interaction

See all 36 papers

Explore similar work

CardsList

  1. Tactile: Giving Computer-Using Agents Hands and Feet

    Jul 16, 2026Yong Liu, Zhenyi Zhong, Zhanpeng ShiComputer-Use AgentsGUI Agents

    Shanghai Jiao Tong University

  2. Mirror Skin: In Situ Visualization of Robot Touch Intent on Robotic Skin

    Dec 12, 2025David Wagmann, Matti Krüger, Chao Wang +2Human-Robot Interaction

    Saarland University, Saarland Informatics Campus Saarbrücken, Germany · Honda Research Institute Europe Offenbach/Main, Germany

  3. Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization

    Feb 24, 2026Jiachen Zhu, Lingyu Yang, Rong Shan +6Mobile GUI AgentsAI Agent Benchmarks

    Shanghai Jiao Tong University Shanghai, China · Carnegie Mellon University Pittsburgh, Pennsylvania, USA