Coding agents increasingly receive UI screenshots as part of bug reports, yet degraded image quality - caused by low resolution, compression artifacts, or small rendered text - can prevent them from identifying the relevant UI elements and source files. Vision-language models become less reliable as image quality deteriorates, while OCR-assisted approaches may propagate recognition errors from degraded inputs. Both approaches are limited when the visual evidence required for localization is no longer clearly observable in the query screenshot. We propose augmenting coding agents with repository-rendered reference images: crisp screenshots generated from the same codebase that the agent is tasked with modifying. These references provide readable visual evidence of the application's UI while preserving the current, potentially bug-containing implementation state. Because they are generated from the available repository rather than requiring a known bug-free version, they can be produced automatically for reproducible application states without manual image annotation. We evaluate the approach on a controlled benchmark derived from SWE-bench Multimodal by injecting visual UI bugs into a real web application. We vary screenshot degradation and reference-image provision and measure whether the coding agent identifies the correct source file. The results show that reference images improve file localization especially for degraded image quality, increasing the gold file match rate under fourfold downsampling with JPEG compression. We confirmed with further analysis that the proposed approach recognizes text approximately 12 percentage points better than language biased OCR under the same degradation.
Figures & tables
Figure 1: Problem setting for source-file localization from degraded UI screenshots. (a) The coding agent extracts visual evidence from a clear screenshot and retrieves the correct source file. (b) Image degradation can corrupt visual evidence and misdirect repository search, resulting in incorrect file localization. (c) Repository-rendered reference images provide additional visual evidence that supports reliable interpretation of the degraded screenshot and source-file localization.
Figure 2: Overview of the proposed method. An OCR-assisted approach first extracts text from the degraded screenshot and provides the transcription alongside the image to the coding agent. Our method instead renders high-quality reference images from the current repository and provides them together with the degraded screenshot and task description. For larger reference catalogs, optional retrieval selects a subset of relevant references.
Figure 3: Example task instance (CSS color mutator). The image on the left is the degraded query screenshot (320 px), while the five images on the right comprise the reference catalog rendered from the bug-injected codebase via Playwright.
Query
None
Top 1 reference
All references
Undegraded
33
—
—
640 px
28
28
36
480 px
22
22
36
320 px
28
28
42
Table 1: Gold file match (%) across query degradation levels and reference provision settings. Top 1 reference selects one image from the catalog via embedding similarity; All references provides all five images.
Figure 4: Mean BoW-F1 by bug type at (a) 640 px, (b) 480 px, and (c) 320 px. The reference-augmented methods increasingly outperform the OCR-based baselines as degradation becomes more severe.
Level
Added content
No img
With img
Δ
L0
(generic description)
3
9
+6
L1
+ symptom category
9
12
+3
L2
+ screen name
9
14
+5
L3
+ component name
26
23
−3
L4
+ specific values
36
35
−1
Table 2: Gold file match (out of 36) by issue description level and screenshot attachment. Δ = (With img) − (No img).
Visual modality has recently been explored as a way to compress textual tokens, including rendering code as images for static code understanding. We study whether this representation can serve as operational context for agentic coding, where an agent must navigate repositories, edit source files, and verify executable patches. Using SWE-bench Verified, we evaluate rendered code in repository-level repair workflows and introduce controlled agent settings to separate unguided repository exploration from more structured repair stages. Our results show a mixed picture. Rendered code consistently reduces prompt-token cost, but the savings do not increase linearly with the nominal visual compression ratio. It largely preserves end-to-end repair accuracy, but does not overcome the performance limits of the underlying model or agent architecture, and can become unstable under aggressive compression. Further analysis suggests that visual code is most useful when raw source reading is a major bottleneck; once repository localization is structured, much of the remaining cost comes from patch--test trial-and-error, where visual compression has limited leverage. Overall, our study positions rendered code as a viable but conditional compression mechanism for realistic coding agents.
Weijie Liang, Yuanfeng Song, Xing Chen +3
The Hong Kong University of Science and Technology, Hong Kong, China · ByteDance, China
Large language model (LLM)-powered coding agents have made rapid progress in automating software engineering tasks, yet repository-level issue resolution remains challenging. Beyond generating a plausible patch, an agent must localize relevant code across interdependent files and maintain repository context that is both sufficient and focused. Code graphs expose non-local relations, but linear text interfaces obscure their topology; rendering the full repository graph yields visual representations that are too dense to perceive reliably, whereas a one-shot local view becomes stale as exploration proceeds. We present RepoAtlas, a training-free module that maintains evolving multimodal repository views through a select--project--refresh loop over a repository code graph. RepoAtlas combines evidence from the issue with the agent's current exploration state to select a task-relevant region under a fixed budget, projects the selected structure into complementary visual and textual representations, and refreshes the view when changes in the exploration state render it outdated. We evaluate RepoAtlas on SWE-bench Verified, where it improves the resolve rate by 2.4 points while reducing input tokens and model calls by 5.8% and 7.8% on average, relative to the strongest multimodal graph baseline, with consistent gains across three models of different families and scales.
A prerequisite for coding agents to perform tasks on large repositories is code localization - the identification of relevant files, classes, and functions to work on. While repository-level code localization has been performed using embedding-based retrieval approaches such as vector search, recent work has focused on developing agents to localize relevant code either as a standalone precursor to or interleaved with performing actual work. Most prior methods on agentic code search equip the agent with complex, specialized tools, such as repository graphs derived from static analysis. In this paper, we demonstrate that, with an effective reinforcement learning recipe, a coding agent equipped with nothing more than a standard Unix terminal can be trained to achieve strong results. Our experiments on three benchmarks (SWE-Bench Verified, Pro, and Lite) reveal that our models consistently achieve superior or competitive performance over 2-18x larger base and post-trained LLMs and sometimes approach performance provided by closed models like Claude Sonnet, even when using specialized scaffolds. Our work particularly focuses on techniques for re-purposing existing coding agent environments for code search, reward design, and RL optimization. We release the resulting model family, CodeScout, along with all our code and data for the community to build upon.
Lintang Sutawika, Aditya Bharat Soni, Bharath Sriraam R R +8
Carnegie Mellon University · Independent · OpenHands