cs.CVAug 28, 2026

FigMirror: Ground It, Code It, Plot It

Authors: Xiaohan Zhao, Jiacheng Liu, Yaxin Luo, Zhiqiang Shen

Organizations: VILA Lab, Department of Machine Learning, MBZUAI

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Converting scientific figures into executable code has gained increasing attention, yet existing methods primarily focus on reproducing the reference figure itself. A more practical setting is to plot new data while preserving the visual style of a reference figure (e.g., color scheme and typography). Prior approaches mimic the reference through pixel-level optimization and struggle to carry its style to new data. We show that the key to this task lies in the coordinate grounding and coding capabilities present in modern computer-use models. We propose FigMirror, an agentic framework that unlocks these capabilities through Grounded Measurement, which locates visual elements by coordinates and measures their properties through executable code. We further introduce PlotTwin-Bench, an expert-curated benchmark with fine-grained code and image-level style metrics. Experiments show that FigMirror consistently outperforms existing methods on reference-conditioned style transfer. All plots in this paper are generated by FigMirror, except those produced by other methods for comparison. Our code and data are available at: https://github.com/VILA-Lab/FigMirror.

Figures & tables

Figure 1: Reference-conditioned style transfer. FigMirror preserves the visual style of a reference scientific figure while adapting it to new target data.
Figure 2: GUIs and scientific figures share localized, code-rendered elements with sharp boundaries, flat fills, and text. This structural similarity allows coordinate grounding learned for GUI elements to localize plot elements such as ticks, legends, and marks for measurement and review.
Figure 3: The FigMirror pipeline. ✓ = resolved, ? = open. The Drawer resolves each open style attribute with Grounded Measurement (locate the element, read its exact value with code) and renders the user data into a template; the Reviewer checks it visually against the reference, routing attributes to the Revision or Preserve List, until no revision remains and the template is final.
Figure 4: Grounded Reviewer feedback. Plain text leaves the repair target ambiguous; a note tied to a marked region names it exactly, so the next Drawer revises the local mismatch and leaves matched attributes untouched.
Figure 5: Benchmark composition and evaluator alignment. (a) Five most common chart types in each source. (b) Automated score SS versus human win rate across the five methods.
Figure 6: Style transfer on two references. Each group places FigMirror beside the reference, followed by four baselines. In A, FigMirror reproduces the panel hierarchy and relative spacing of a dense composition. In B, it matches the joint hexbin layout, marginal histograms, threshold lines, color scale, and typography.
 Hand-curatedAugmented
MethodScodeS_{\mathrm{code}}SvisionS_{\mathrm{vision}}SSScodeS_{\mathrm{code}}SvisionS_{\mathrm{vision}}SS
FigMirror64.477.272.784.072.376.4
ChartIR ( Xu et al., 2025 )56.663.961.383.663.170.3
ChartGalaxy ( Li et al., 2025b )58.658.458.580.758.166.0
Plot2Code ( Wu et al., 2025a )53.456.855.664.943.050.7
METAL ( Li et al., 2025a )53.750.551.666.243.551.5
Table 1: Main results on scientific figure style transfer. We report the code score ScodeS_{\mathrm{code}} , the vision score SvisionS_{\mathrm{vision}} , and the combined score SS on the two reference sources of PlotTwin-Bench : hand-curated figures and augmented figures. All methods use GPT-5.5.
Figure 7: Grounded Measurement and box-guided refinement. (a) Three measurements from separate runs: each dashed box marks the probed region on a reference; below it, the probe code and its returned values. (b) One transfer run over three iterations; labels name the Drawer edit applied in the next column, and unboxed attributes stay in the Preserve List.
VariantScodeS_{\mathrm{code}}SvisionS_{\mathrm{vision}}SS
FigMirror64.477.272.7
w/o Reviewer62.774.970.6
w/o Grounded Measurement67.470.069.0
w/o FigMirror skill (naked Codex)60.461.461.0
Table 2: Ablations on style transfer. We implement each variant in the Codex harness by editing the corresponding skill prompts and removing the relevant tool access. The no-Reviewer variant retains the Drawer’s self-checks.
Max iterationsScodeS_{\mathrm{code}}SvisionS_{\mathrm{vision}}SS
164.575.871.8
3 (default)64.477.272.7
567.078.374.3
Table 3: Iteration budget in the Codex setting. All rows share the same model, data, and evaluation.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Full chart-type distribution. Counts for all 12 chart types in the augmented and hand-curated sources.
Figure 9: Reference enrichment examples. Each row places the seed figure on the left and its augmented reference on the right. The augmented versions add coordinated panels and retain the seed’s principal marks and encodings.
CategoryDimensionsExample changes
Data shape (6)Series count; point density; category cardinality; matrix dimension; panel count; per-series density imbalanceAdd a panel; add a series; increase category count
Axis & scale (5)Scale type; x-axis type; axis-unit normalization; value-sign polarity; dual-scale requirementLinear →\rightarrow log; numeric →\rightarrow time; positive-only →\rightarrow mixed-sign
Semantic & categorical (5)Label-domain swap; label-length inflation; ordering-principle swap; hierarchy introduction; axis-semantics swapShort →\rightarrow long labels; flat →\rightarrow grouped; replace the category domain
Appendix
Table 4: Variation menu for story-first target-data construction.
ComponentRole
Launch templateFrames the transfer task, runtime inputs, and the final artifact contract
Skill routerEntry point, required inputs, workflow, and non-negotiable constraints
OrchestratorWires the Drawer–Reviewer loop, state handoff, and stop condition
DrawerTurns reference style into grounded measurements and plotting code
ReviewerAudits the rendered draft and returns preserve and revision signals
Appendix
Table 5: Core FigMirror transfer-generation prompts. Listings show excerpts.
More in this topic

Image Style Transfer

Image style transfer changes an image’s visual appearance to match an artistic, photographic, or other reference while preserving its content.

See all 61 papers

Explore similar work

CardsList

  1. SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code

    Sep 8, 2026Wentao Li, Yibo Wu, Yizhe Chen +5Scientific VisualizationCode Generation Evaluation

    Tsinghua University

  2. PlotGround: Grounding Plot Digitization in Real Scientific Figures and Their Source Data

    Oct 5, 2026Yaohui Zhang, Binxu Li, Haoyi Duan +7Scientific VisualizationChart QA

    Generative Expert Labs, Inc · Stanford University · Princeton University

  3. Figures as Programs: Recursive Generation of Editable Scientific Figures

    Sep 1, 2026Yepeng Liu, Dasen Dai, Chengzhi Liu +7Scientific VisualizationProgram Synthesis

    1UC Santa Barbara · 2CUHK · 3Show Lab, National University of Singapore +2