cs.HCApr 6, 2026
SaveReFinE: Streamlining UI Mockup Iteration with Research Findings
Organizations: University of Washington · Allen Institute for AI
Abstract
Although HCI research papers offer valuable design insights, designers often struggle to apply them in design workflows due to difficulties in finding relevant literature, understanding technical jargon, the lack of contextualization, and limited actionability. To address these challenges, we present ReFinE, a Figma plugin that supports real-time design iteration by surfacing contextualized insights from research papers. ReFinE identifies and synthesizes design implications from HCI literature relevant to the mockup's design context, and tailors this research evidence to a specific design mockup by providing actionable visual guidance on how to update the mockup. To assess the system's effectiveness, we conducted a technical evaluation and a user study. Results show that ReFinE effectively synthesizes and contextualizes design implications, reducing cognitive load and improving designers' ability to integrate research evidence into UI mockups. This work contributes to bridging the gap between research and design practice by presenting a tool for embedding scholarly insights into the UI design process.
Figures & tables
Figure 1. An overview of the ReFinE -aided design workflow. Based on a user’s selected UI mockup on the Figma canvas, the system generates \footnotesize{1}⃝ clusters of research insight informed by HCI research papers. Designers can examine \footnotesize{2}⃝ each cluster to understand the synthesized design implications before reviewing \footnotesize{3}⃝ action items and their visualizations, which project how potential design edits could be made to the interface. Designers can use these to perform designerly iteration, modifying or adapting the proposed changes at their own discretion. They can also explore the \footnotesize{4}⃝ source papers that informed each cluster.
Figure 2. Pipeline for generating the components provided by ReFinE
Figure 3. Contents of the cluster and their formulation
Figure 4. An example of mockup reconstruction in our pipeline. Although the reconstructed HTML may not perfectly match the original mockup, it faithfully mirrors the components in the original mockup and allows for visualization of action items.
Figure 5. An example of an action item and its visualization. With the reconstructed HTML, ReFinE selects the appropriate screen(s) from the design mockup and visualizes how each action item can be applied. Users can toggle between the before and after visualization, assisting designers in grasping the differences. Users can also bookmark each action item.
Figure 6. Design outputs from iterating two example mockups with ReFinE -generated action items. For each example, we present the extracted design context, an example of retrieved design implications, and action items used for the iteration. For conciseness, a single screen pair (before/after) is shown for each mockup, and edits are highlighted with a yellow background.
| System step | Latency (seconds) |
|---|---|
| Eliciting design context dimensions | |
| Mockup reconstruction* | |
| Retrieval & clustering | |
| Generating translated insights* | |
| Generating action items | |
| Visualizing action item |
Table 1. Overall system latency analysis (metrics are computed per task stdev, *: per unit; otherwise per design)
Figure 7. Perceived workload, communication quality, and the design output metrics from our user study. Participants in the ReFinE condition reported significantly reduced workload compared to the baseline condition, while finding the communicated design insights to be significantly improved. Additionally, participants made significantly more edits interacting with ReFinE , while reaching iteration more quickly. Error bars indicate standard errors. (**: , ***: ; †: reverse item)
Figure 8. P4’s example of implementing an action item. After recognizing its potential usefulness, P4 creatively adapted the action item to a different screen, redesigning it with alternative UI components.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Visual similarity | Latency (seconds) |
|---|---|---|
| Gemini 2.0 Flash ( Google, 2024b ) | ||
| Claude 3.5 Sonnet ( Anthropic, 2024 ) | ||
| GPT-4o ( OpenAI, 2024 ) |
Table 2. Comparison of VLMs for HTML reconstruction using image inputs (metrics are computed per-screen stdev)
| Input modality | Visual similarity | Latency (seconds) |
|---|---|---|
| Image | ||
| JSON | ||
| Image + JSON |
Table 3. Comparison of different input modalities on HTML reconstruction (base model is Gemini 2.0 Flash, metrics are computed per screen stdev)
| Method | Accuracy | Latency (seconds) |
|---|---|---|
| Complete HTML regeneration | ||
| Edit-only generation |
Table 4. Comparison of methods for generating visualizations of action items applied to designs (metrics are computed per action item stdev)
| Relevance rating | Design implication | Action item | Annotator’s explanation |
|---|---|---|---|
| 5 - Highly relevant | Reduce user burden by minimizing text input in form-heavy tasks | Implement a one-click autofill feature that uses stored preferences for all checkout fields | Directly addresses and solves the burden |
| 4 - Substantially relevant | Make it easier for users to navigate dense information in reports | Add a ‘Back to Top’ button on the settings screen | Helps with navigation, but not specific to navigating documents |
| 3 - Moderately relevant | Striking the balance between insufficient and overwhelming transparency will enable users to trust the system | Add a ‘Report’ button to the comment section to enhance user feedback and platform moderation | May support trust, but unclear link to transparency or balance |
| 2 - Slightly relevant | Allow users to control the visibility of their personal data in shared platforms | Let users toggle switches in the profile settings to show/hide recent activity | Some connection to visibility, but lacks clarity on shared context |
| 1 - Not relevant | Help users build trust in financial transactions through transparency | Display an autoplay video on the app with product promotions | Unrelated to trust or transparency |
Table 5. Examples of relevance ratings assigned by annotators when evaluating the connection between design implications and action items This table illustrates examples of relevance ratings from 1 (Not relevant) to 5 (Highly relevant), assigned by annotators assessing how well action items align with design implications. Each row includes a rating, a design implication, a corresponding action item, and a brief explanation justifying the assigned relevance.
Figure 9. Prototype mockups used in our user study