cs.HCFeb 28, 2026
SaveTexterial: A Text-as-Material Interaction Paradigm for LLM-Mediated Writing
Organizations: Microsoft Research Redmond, WA, USA · MIT Media Lab Cambridge, MA, USA · University of Toronto Toronto, ON, Canada
Abstract
What if text could be sculpted and refined like clay -- or cultivated and pruned like a plant? Texterial reimagines text as a material that users can grow, sculpt, and transform. Current generative-AI models enable rich text operations, yet rigid, linear interfaces often mask such capabilities. We explore how the text-as-material metaphor can reveal AI-enabled operations, reshape the writing process, and foster compelling user experiences. A formative study shows that users readily reason with text-as-material, informing a conceptual framework that explains how material metaphors shift mental models and bridge gulfs of envisioning, execution, and evaluation in LLM-mediated writing. We present the design and evaluation of two technical probes: Text as Clay, where users refine text through gestural sculpting, and Text as Plants, where ideas grow serendipitously over time. This work expands the design space of writing tools by treating text as a living, malleable medium.
Figures & tables
Figure 1. We explore how materials can inspire new interactions with text. Building on insights from a formative study, we propose Texterial , a conceptual framework and prototype implementations that treat text-as-material . Our work uses materiality to foster a new way of interacting with generative AI. Shows a framework for material-inspired text interaction. On the left, icons for materials (paint, yarn, clay, photograph, plant) are paired with operations (e.g., pinch, merge, pluck, plant, water). In the center, a plant metaphor is illustrated where dragging creates a branching fern of words. On the right, pressing on text highlights and emphasizes localized words. Together, the diagram illustrates how materials, operations, and interactions combine to treat text as a malleable medium that fosters a new way of interacting with generative AI.
Figure 2. Elicitation materials from our formative study. The inspirational cards include entry-level crafts (gardening, pottery, photography) and advanced ones (embroidery, beekeeping), as well as inanimate (clay, threads), living (plants, bees), and hybrid materials (photographed scenes). Depicts a collection of square inspirational cards used in a formative study. The cards feature images of crafts and materials such as embroidery, gardening, pottery, photography, threads, plants, bees, and colorful paint. They represent a mix of entry-level and advanced crafts, inanimate materials, living elements, and hybrid photographed scenes.
| ID | Occupation | Creativity | Experience | Type of Writing |
| P1 | Model maker (15 yrs) | 38 | 12 | N/A |
| P2 | Designer (20+ yrs) | 43 | 21 | Creative briefs, presentations |
| P3 | Principal PM Manager (1 yr) | 37 | 25 | Emails, essays, scientific articles, journaling |
| P4 | NLP Researcher (20+ yrs) | 45 | 29 | Scientific articles, short stories, poetry |
Table 1. Formative study participants. The creativity score (min: 1, max: 50, higher is more creative) and experience with writing score (min: 6, max: 30, higher is more experienced) are based on Renom et al. ( Renom et al., 2022 ) . Formative study participants. Creativity scores range from 1 to 50 (higher means more creative), and writing experience scores range from 6 to 30 (higher means more experienced). P1: Model maker with 15 years of experience. Creativity score 38, writing experience score 12. No regular type of writing. P2: Designer with over 20 years of experience. Creativity score 43, writing experience score 21. Writes creative briefs and presentations. P3: Principal PM Manager with 1 year of experience. Creativity score 37, writing experience score 25. Writes emails, essays, scientific articles, and journaling. P4: NLP Researcher with over 20 years of experience. Creativity score 45, writing experience score 29. Writes scientific articles, short stories, and poetry.
Figure 3. Norman’s stages-of-action ( Norman Donald, 2013 ) model expanded, with areas (in red) where interactions with LLMs are particularly difficult as identified by Subramonyam et al. ( Subramonyam et al., 2024 ) . The figure presents an expanded version of Norman’s stages-of-action model, illustrating where interactions with large language models (LLMs) are especially difficult. The diagram is arranged vertically, beginning at the top with “Goal,” followed by “Intention,” “Action specification,” “Execution,” “Perception,” “Interpretation,” and finally “Evaluation,” which loops back toward the goal. These seven stages are connected with arrows to show the iterative process of acting on and assessing a system. \parTwo large vertical brackets frame the process. On the left side, the upper portion is labeled “Gulf of envisioning,” and the lower portion is labeled “Gulf of execution.” On the right side, a bracket spans the perceptual and interpretive steps and is labeled “Gulf of evaluation.” At the bottom of the diagram, the “Software system” is shown as the target of execution and the source of perceptual feedback. \parThe figure highlights two regions with dashed red outlines to indicate areas where interactions with LLMs are particularly challenging, as identified by Subramonyam et al. The first highlighted region, labeled “A: Cognitive task processes,” spans the space between a user’s goal and intention formation. The second region, labeled “B: System mental model,” covers the area where a user forms an intention, specifies an action, and attempts to interpret the system’s behavior. These regions represent steps that depend heavily on how users conceptualize both their own cognitive tasks and the system’s internal workings—areas that become more complex when dealing with probabilistic, opaque AI models.
Figure 4. Texterial conceptual framework: LLMs allow users to operate on the semantics, structure and style components of text at different levels of abstraction. The diagram shows three components—Semantics, Structure, and Style—arranged horizontally. Each component is represented as a stack of layered blocks above raw text, with arrows indicating movement between layers. The vertical axis represents levels of abstraction, from raw text at the bottom to higher abstraction at the top.
Figure 5. Texterial conceptual framework: Text operations users/models perform. composing combines textual components together; abstracting moves to a meta-layer of any of the components; ideating takes a single seed and generates new concepts; condensing makes a component’s content shorter; and finally transforming preserves the main qualities of a component, but expresses the content in a different way. Many of these operations have inverse functions: for example, instead of composing , users might isolate a single component into multiple distinct components. Framework of five text operations performed by users or models. \parf₁ = compose: Combining or isolating elements. Example: f₁(“water”, “fire”) = “steam”. \parf₂ = abstract: Moving between abstract and concrete. Example: f₂(“blossoms”) = “feeling of life”. \parf₃ = ideate: Creating related alternatives. Example: f₃(“tree”) = “bush”, “ferns”, “garden”…. \parf₄ = condense: Contracting or elaborating text. Example: f₄(“after walking for many ages in the forest”) = “a long walk”. \parf₅ = transform: Changing form or expression. Example: f₅(“sunlight passes through leaves”) = “sunshine drips through leafy silence”. \parEach operation is illustrated with simple block diagrams showing the direction of composition, abstraction, creation, contraction, elaboration, or transformation.
| Gulf | Chat Interface | Text-as-Material |
| Gulf of Envisioning ( Subramonyam et al., 2024 ) | High gulf | Very low gulf |
| Do I know what it can do and how to communicate my intent? | Blank prompt can be intimidating; users may not know what to ask or how to begin. Requires prior intent or inspiration. | The interface invites play and experimentation; users can begin without a clear goal or assumptions. Supports emergent thinking and discovery. |
| Gulf of Execution ( Norman Donald, 2013 ) | Moderate gulf | Low gulf |
| Can I do what I want to do? | Users must translate intent into language; success depends on prompt clarity. Requires mental effort to formulate effective instructions. | Users manipulate text directly with gestures; actions are intuitive and discoverable. Users feel in control through embodied interaction. |
| Gulf of Evaluation ( Norman Donald, 2013 ) | Moderate gulf | Low gulf |
| Did the system do what I expected? | Feedback is textual and may be verbose or ambiguous. Users may struggle to judge correctness or relevance. | Immediate visual and tactile feedback; changes are visible and interpretable. Users can assess results in real time. |
Table 2. Analysis of how text-as-material has the potential to modify the interaction loop when working with LLMs for text manipulations, compared to a traditional chat interface. Analysis of how text-as-material has the potential to modify the interaction loop when working with LLMs for text manipulations, compared to a traditional chat interface. Three gulfs are discussed: envisioning, execution, and evaluation. Gulf of Envisioning – Question: Do I know what it can do and how to communicate my intent? Chat interface: Rated high gulf. A blank prompt can be intimidating; users may not know what to ask or how to begin, requiring prior intent or inspiration. Text-as-material: Rated very low gulf. The interface invites play and experimentation; users can begin without a clear goal or assumptions, supporting emergent thinking and discovery. Gulf of Execution – Question: Can I do what I want to do? Chat interface: Rated moderate gulf. Users must translate intent into language, and success depends on prompt clarity. Requires mental effort to craft effective instructions. Text as material: Rated low gulf. Users manipulate text directly with gestures; actions are intuitive and discoverable, giving users a stronger sense of control. Gulf of Evaluation – Question: Did the system do what I expected? Chat interface: Rated moderate gulf. Feedback is purely textual and may be verbose or ambiguous, making it harder to judge correctness or relevance. Text as material: Rated low gulf. Provides immediate visual and tactile feedback, with changes visible and interpretable in real time.
Figure 6. Examples of some of the key conceptual differences between AI-mediated text manipulation when using a prompt-based (green) vs. material-based (purple) interaction paradigm, and the perceived affordances ( Norman Donald, 2013 ; Hartson, 2003 ) and the stages of the human action model ( Norman Donald, 2013 ) they primarily tie to. The figure compares conceptual and mental shifts between prompt-based and material-based interactions with LLMs, mapping each to different stages of the human action model. The prompt-based paradigm (shown in green) frames the system through an agent-like metaphor: the model “understands,” responds, and uses memory and context from prior turns. Interaction is turn-based—I ask, it answers—and unfolds linearly as a transcript. Input is language-driven, requiring users to shape output through words, and control depends on prompt engineering, where users learn how to phrase their intent. Feedback is slow and often opaque, with hidden model mechanics that must be inferred. \parThe material-based paradigm (shown in purple) treats text as a manipulable material with affordances and constraints, behaving like clay or fabric. Cognitive framing shifts toward embodied reasoning, where gestures, posture, and physical movement influence outcomes. Interaction takes the form of direct manipulation—touching, dragging, stretching, or pressing—and the workflow is non-linear, allowing users to rearrange, merge, tear apart, or branch text fragments visually. Input is gestural, and control becomes expressive, with gestures conveying nuance such as pressure, direction, or speed. Feedback is immediate and sensory, supported by visual cues, animations, or haptics. \parAlong the bottom of the figure, a color-coded legend links each paradigm to stages of Norman’s action model, including intention, action specification, execution, perception, and evaluation, showing which stages each type of affordance (cognitive, physical, and sensory) primarily supports.
Figure 7. In our Text as Clay interface, users tangibly sculpt existing drafts through expressive touch gestures. Localized, typographic animations emphasize the clay-like effect and highlight different edits the model made to the text. Users can interact with voice to avoid detracting from the tactile experience. Shows the “Text as Clay” interface, where users sculpt drafts through touch gestures. Gestures and their functions include: Place Emphasis: Long press highlights text. Merge/Combine: Drag blocks together. Abstract: Smudge with a finger. Refine: Pinch inward to tighten or clarify. Split into Chunks: Separate text with fingers. Elaborate: Stretch fingers outward to expand content. Summarize: Squash fingers inward to compress text. Typography animates locally to emphasize the clay-like quality of edits, while voice interaction is supported to complement tactile control.
Figure 8. Text as Clay Storyboard. The user is editing their story to make it more engaging: (a) They merge a paragraph into the existing block to seamlessly combine the two, and (b) separate another paragraph into two pieces by ripping. (c) They can use their finger to smudge text, making it more abstract, and (d) pinching makes text more concrete. Finally, (e) stretching out elaborates on the text. Storyboard of the “Text as Clay” interface. (a) User drags text blocks to merge them into one. (b) User rips text apart to split it into two chunks. (c) User smudges text with a finger to abstract or blur meaning. (d) User pinches text inward to refine and tighten expression. (e) User stretches text outward to elaborate, making it longer and more detailed. The sequence demonstrates how tactile gestures—dragging, ripping, smudging, pinching, and stretching—sculpt text in a clay-like manner.
Figure 9. Example interaction flow when merging two "clay" text blocks ( compose ). The direction of approach and overlap between blocks is captured in the language model’s prompt to guide generation. Diagram showing how two text blocks (“My cat is in the garden…” and “The neighbour’s dog runs…”) can be merged using a clay-like interaction metaphor. The user drags one block toward the other, with the approach direction determining order and the overlap amount shaping the focus of change. Two alternatives are illustrated, each generating a different merged prompt. Prompts explicitly encode overlap percentage and order, producing final outputs where the two sentences are blended into a cohesive passage.
Figure 10. Through our Text as Plants interface, users continually shape their "garden" of ideas, watering, pruning, preserving, composting, and allowing ideas to grow serendipitously over time. Users can interact via voice and pen-touch gestures. Sound effects, such as plucking, and fluttering animations emphasize the materiality of the garden. Shows the “Text as Garden” interface, where ideas are visualized as plants that users can nurture and reshape. \parUsers interact through gardening metaphors: \parPruning: Dragging a leaf and dropping it to remove. \parWatering: Drawing a line with two fingers to help ideas grow. \parGrafting: Dragging part of a stem onto another plant to connect ideas. \parComposting: Dragging leaves into soil to turn old ideas into seeds. \parPreserving: Touch and hold to keep an idea. \parModify text: Double-clicking to edit words. \parGrowth over time: Plants generate new ideas as they expand. \parThe garden metaphor is reinforced with animations (like leaves fluttering) and sound effects (like plucking), emphasizing text as a living, material medium.
Figure 11. Text as Plants Storyboard: (a) To write a children’s story, a writer is planting new ferns of ideas for the setting, main character traits, and side characters. (b) They water plants to generate further ideas, and (c) pluck a few of the ideas they don’t like (d) They can read details of each generated idea, and directly edit the text. (e) Finally, they combine two ideation plant branches by grafting a part of one plant onto another. Storyboard of the “Text as Plants” interface. (a) Users long-press on the soil and speak to add new text; intent of the writing can also be edited. (b) Watering gesture with two fingers helps ideas grow, here used to expand “Mama duck character traits and personality.” (c) A stem emerges labeled “PEACEFUL,” with text ideas branching around it. (d) Users rearrange and position text fragments to refine meaning. (e) New ideas are added, such as “names for ducklings,” reinforcing the garden metaphor for nurturing text.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12. An example walkthrough of our conceptual framework using a case study of an artist-in-residence preparing a written proposal for an installation. This storyboard illustrates steps users might take in a typical writing process, and how these align with components of our framework. DESCRIPTION
Figure 13. Low-fidelity prototypes of various material-inspired interaction techniques.
| Conceptual lens | Material affordances & constraints | Embodied reasoning | Direct manipulation | Non-linear workflow | Gestural-driven input | Expressive control | Immediate sensory feedback |
| Merge / combine blocks (drag & overlap) | “I can join/knead clay. Smearing two lumps together blends the seam. More overlap yields a more unified form; minimal overlap keeps pieces distinguishable.” | Approach direction sets ordering. A slow, deliberate push suggests careful integration; a quick shove suggests rough stitching. Where and how much I overlap determines what becomes the seam versus what remains intact. | User drags one text block onto another. Contact initiates merging, and overlap defines the blending region (the seam), rather than issuing a verbal instruction. | User can merge, rip to re-separate, then merge again with a different direction or overlap. | Drag vector (source target), drop position, overlap ratio, and anchor points corresponding to the intersection region (seam). | Drag direction determines ordering. Overlap magnitude controls degree of integration. Seam location biases where merge is strongest. | Blocks visually react on contact with spreading/bending at the contact edges and color becomes denser. After release, the merged text animates locally at the seam, highlights the change. |
| Texterial operation. COMPOSE (f 1 ) combines content, and under high overlap | |||||||
| Implementation hook (prompt routing + control variables). Detect / tag overlap. During drag, compute collision intensity from overlap extent (e.g., line/region intersection). Mark the visually overlapping lines in each block with <overlap> … </overlap> tags. Route by collision type + intensity. • Vertical merge (source above target). Call getVerticalCollisionPrompt(topTextWithTags, bottomTextWithTags, i, context) . Enforce ordering: TOP BOTTOM in the final text. Edits should concentrate within/near the tagged overlap region; avoid changes outside unless necessary for coherence. Output length (top+bottom). • Full blend (extreme overlap). For near-complete overlap, call getFullBlendCollisionPrompt(firstText, secondText, i, context) to generate a single unified passage that blends both sources (not simple insertion), with length (first+second). • Horizontal insertion (source into target). When the dragged block is inserted into a target block, call getHorizontalCollisionPrompt(mainTextWithTags, insertTextWithTags, pos, i, context, insertLineText?) . Use pos (or an anchor line) to place the insertion; apply blending primarily within/near <overlap> lines. Encode intensity guidance. Convert to intensityPercent and select a blending regime: light ( ) minimal edits for a smooth transition; moderate ( ) natural flow with moderate editing; heavy ( ) extensive merging/weaving. Constraints / outputs. Return only the combined text (no markup/tags/markdown). Keep length the same or shorter than the two inputs. If userWritingContext is provided, prepend it as a conditioning line and keep edits relevant to that project. | |||||||
Table 3. Single-metaphor example of a concrete technique. — merge/combine blocks (drag & overlap)
Figure 14. Pinching to refine ( concretize ) DESCRIPTION
Figure 15. Smear to abstract ( abstract ) DESCRIPTION
Figure 16. Press to emphasize DESCRIPTION
Figure 17. Stretch to elaborate ( elaborate ) DESCRIPTION
Figure 18. Squash to summarize ( condense ) DESCRIPTION
Figure 19. Split into chunks DESCRIPTION
Figure 20. Planting a new idea fern via voice DESCRIPTION
Figure 21. Watering to promote growth of new ideas ( ideate ) DESCRIPTION
Figure 22. Composting to merge ideas together ( compose ) DESCRIPTION
Figure 23. Preserving or highlighting favorite ideas DESCRIPTION
Figure 24. Pruning to refine idea space DESCRIPTION
Figure 25. Grafting to organize idea chunks together ( compose ) DESCRIPTION