cs.HCJul 12, 2026

Motif: Discovering and Automating Personal Web Workflows

Authors: Shaokang JiangDaye Nam

Organizations: University of California, Irvine Irvine, California, USA

Abstract

Recent advances in LLMs and existing work on programming by demonstration have made it possible for end users to create automations by explicitly demonstrating their behavior to LLMs. However, these approaches rely on the assumption that users know what to automate and what is capable of being automated. Additionally, automation via LLM agents is often expensive compared with programs. We introduce Motif, a system that passively observes everyday browser activity to discover recurring interaction patterns that are programmable, makes recommendations to users whenever a pattern is discovered and generate a program to install after user confirmation. Users can review, and refine the program using natural language. We evaluated Motif in a multi-day study, comparing its ambient discoveries against automations users attempted to build via ``vibe coding.'' With eight participants, Motif discovered more automatable patterns than users recognized. Most of them matched participants' routines and were useful. Follow-up surveys showed most would continue using Motif-generated programs.

Explore similar work

CardsList
  1. cotomi Act: Learning to Automate Work by Watching You

    May 4, 2026Masafumi Oyamada, Kunihiro Takeoka, Kosuke Akimoto +5Computer-Use AgentsBrowser Agent