cs.HCFeb 26, 2026

Simulation-based Optimization for Augmented Reading

Authors: Yunpeng Bai, Shengdong Zhao, Antti Oulasvirta

Organizations: National University of Singapore Singapore · City University of Hong Kong Hong Kong, China · Aalto University Helsinki, Finland

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Augmented reading systems aim to adapt text presentation to improve comprehension and task performance, yet existing approaches rely heavily on heuristics, opaque data-driven models, or repeated human involvement in the design loop. We propose framing augmented reading as a simulation-based optimization problem grounded in resource-rational models of human reading. These models instantiate a simulated reader that allocates limited cognitive resources, such as attention, memory, and time under task demands, enabling systematic evaluation of text user interfaces. We introduce two complementary optimization pipelines: an offline approach that explores design alternatives using simulated readers, and an online approach that personalizes reading interfaces in real time using ongoing interaction data. Together, this perspective enables adaptive, explainable, and scalable augmented reading design without relying solely on human testing.

Figures & tables

Figure 1. Resource-rational reading models enable principled optimization of text user interfaces across a wide design space. ( Left ) A reading agent interacts with text-based user interfaces by allocating visual attention over time, guided by perceptual input, memory state, task objectives, and an adaptive policy. ( Right ) A design space for augmented reading defined by user groups, text presentation designs, and task demands. The same underlying model can be instantiated to evaluate diverse reading scenarios, illustrated here with three examples: ( top ) a reader with restricted visual field (tunnel vision), ( middle ) reading supported by visually salient text cues (e.g., font size, type, and color), and ( bottom ) multitasking contexts such as reading while driving from an in-vehicle display. Together, the model enables principled comparison and optimization of text UI designs across users, tasks, and environments.
Figure 2. Optimization paradigms for augmented reading. (a) Human-in-the-loop optimization. Traditional text UI design relies on human readers to generate behavioral data and feedback, making optimization costly and difficult to scale. (b) Simulation-based offline optimization. A simulated reading agent represents human readers to evaluate and optimize candidate text UIs, enabling systematic exploration of the design space without repeated user studies. (c) Simulation-based online optimization. Human interaction data are used to initialize a personalized simulated agent that continues reading on the user’s behalf, supporting real-time evaluation and adaptive UI optimization during ongoing reading.
Figure 3. Gallery of simulation-based optimization for augmented reading. Three scenarios illustrate how a simulated reading agent predicts human-like reading behavior (red dots indicate fixations on text; blue dots indicate attention to the surrounding environment) and task performance (summarized by model-predicted metrics in orange boxes) under different contexts, and how these predictions guide UI optimization. Across (a) reading while walking on AR, (b) time-limited reading, and (c) visual search while driving, the model identifies bottlenecks in workload, comprehension, or safety. Targeted design interventions, such as increasing layout sparsity, reducing content load, or enhancing visual saliency lead to improved predicted reading behavior and task performance after optimization. (d) Individualized reading support demonstrates how reader-specific simulation reveals inefficient lexical access (e.g., slow reading speed and dense fixations), enabling personalized adaptations such as enhanced lexical saliency to improve predicted reading speed and comprehension.
More in this topic

Human-AI Interaction

Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.

See all 329 papers

Explore similar work

CardsList

  1. Evaluating Adaptive Personalization of Educational Readings with Simulated Learners

    Apr 17, 2026Ryan T. Woo, Anmol Rao, Aryan Keluskar +1Adaptive LearningPersonalized Learning

    School of Computing and Augmented Intelligence, Arizona State University

  2. Combining Retrieval-Augmented Text Generation with LLMs for Reading Content Recommendations

    Jun 12, 2026Sooyeon Kim, Piotr S. MaciągRetrieval-Augmented GenerationControllable Text Generation

    Institute of Computer Science, Warsaw University of Technology, Nowowiejska 15/19, 00-661 Warsaw, Poland

  3. Designing Textual Interfaces To Enhance Llm Reasoning On Audio-Visual Question Answering

    Oct 14, 2026Yi Wang, Tianzi Wang, Tong Lei +2Multimodal ReasoningAutomatic Prompt Optimization

    School of Informatics, University of Edinburgh, UK · Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong, Hong Kong SAR · Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, China