Reconstructing numerical simulations from control systems research papers is often hindered by underspecified parameters and ambiguous implementation details. We define the task of Paper to Simulation Recoverability, the ability of an automated system to generate executable code that faithfully reproduces a paper's results. We curate a benchmark of 500 papers from the IEEE Conference on Decision and Control (CDC) and propose RESCORE, a three component LLM agentic framework, Analyzer, Coder, and Verifier. RESCORE uses iterative execution feedback and visual comparison to improve reconstruction fidelity. Our method successfully recovers task coherent simulations for 40.7% of benchmark instances, outperforming single pass generation. Notably, the RESCORE automated pipeline achieves an estimated 10X speedup over manual human replication, drastically cutting the time and effort required to verify published control methodologies. We will release our benchmark and agents to foster community progress in automated research replication.
Figures & tables
Fig. 1: RESCORE framework automates code recovery from control system papers. After filtering, expert screening, and annotation, Analyzer, Coder, and Verifier LLM agents operate in a closed loop to generate, execute, and refine simulation code using feedback, followed by evaluation.
Fig. 2: Prompt for Analyzer Agent. The agent performs two tasks: transcribe red-boxed equations into readable format, analyze and describe behavior of the system using simulations from the paper. problem_statement , params and init_conditions are defined by domain expert during annotation.
Fig. 3: Prompt for Coder Agent. The agent performs both initial code generation and iterative code repair. In repair mode, the current code and the feedback agent’s visual diagnosis are appended to the user message.
Fig. 4: Prompt for Verifier Agent: The agent receives the generated and target plots side-by-side. A confirmed match terminates the loop; otherwise a structured diagnosis feeds back into the Coder Agent for the next iteration.
Label
Count
% of Total
Recoverable
271
54.2%
Not recoverable
120
24.0%
Indecisive (no label)
109
21.8%
Total
500
100%
TABLE I: Qwen-3 paper filtering results across 500 candidate papers.
Single-Pass (FRS)
RESCORE (FRS)
Year ( n )
FRS-H
FRS-L
DA (%)
FRS-H
FRS-L
DA (%)
2021 (38)
1.80
2.45
64.9
2.09
2.89
66.2
2022 (41)
1.82
2.24
60.6
2.26
2.78
66.5
2023 (46)
1.65
2.13
57.1
2.21
2.70
65.4
2024 (40)
1.61
2.12
59.5
2.00
2.55
67.3
2025 (29)
1.81
2.24
56.1
2.34
2.90
60.6
TABLE II: One-pass vs. RESCORE across CDC 2021-2025 ( n=194 ). FRS Human is avg. of both raters. Best values in a row are bold .
Metric
Value
FRS Human 1 (mean)
2.05
FRS Human 2 (mean)
2.29
% papers FRS ≥3 (H1 / H2)
34.0% / 41.2%
% papers FRS =4 (H1 / H2)
19.6% / 23.7%
Human inter-rater: exact
69.6%
Human inter-rater: within-1
97.4%
TABLE III: Human annotation stats for Figure Reconstruction Score.
Fig. 5: Confusion matrices: Human raters vs. LLM grader on RESCORE outputs. Off-diagonal mass above diagonal shows LLM optimism.
Fig. 6: End-to-end attrition of the 500 candidate papers. Domain expert filtered some false positives. Of the 79 papers whose feedback loop converges, 59 (74.7%) achieve FRS ≥ 3; among the 115 exhausted papers only 3 (2.6%) do so; loop convergence predicts reconstruction quality.
Fig. 7: Distribution of papers for which the RESCORE loop converged and the iterations of coder agent needed to recover the simulation code.
Target
Single-pass
RESCORE
Data-Driven Reachability with Christoffel Functions [ 30 ]
Plot from paper
FRS = 1.0
FRS = 4.0
Identification of Piecewise Affine Systems with Online Deterministic Annealing [ 31 ]
Plot from paper
FRS = 2.0
FRS = 4.0
TABLE IV: Examples of visual-feedback improvement. Each row shows ground-truth ( Target ), LLM’s first-attempt output before any feedback ( Single-pass ), and output after the RESCORE feedback loop ( RESCORE ). Human FRS scores are beneath each plot.
Fig. 8: Mean FRS by category (RESCORE, n=194 ). Categories assigned by keyword match of paper titles. Human FRS avg. of both raters.
Fig. 9: Case study for Connected Cruise Control system from [ 33 ] . Inputs (Left): Agentic pipeline is provided with expert-annotated equations, problem statement, and target simulation Figure 2 from [ 33 ] . Single-Pass (Center): The simulation code compiles without syntax errors but has semantic flaws, such as implementing an incorrect control barrier function and a rigid step-braking profile, resulting in visual discrepancies. (Right): Guided by visual feedback, the RESCORE closed-loop diagnoses mismatches, corrects underlying mathematical structure, reconstructing the simulation.