cs.LGMar 10, 2026

Towards a Neural Debugger for Python

Authors: Maximilian Beck, Jonas Gehring, Jannik Kossen, Gabriel Synnaeve

Organizations: Johannes Kepler University Linz, Institute for Machine Learning · Meta FAIR CodeGen Team

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Training large language models (LLMs) on Python execution traces grounds them in code execution and enables the line-by-line execution prediction of whole Python programs, effectively turning them into neural interpreters (FAIR CodeGen Team et al., 2025). However, developers rarely execute programs step by step; instead, they use debuggers to stop execution at certain breakpoints and step through relevant portions only while inspecting or modifying program variables. Existing neural interpreter approaches lack such interactive control. To address this limitation, we introduce neural debuggers: language models that emulate traditional debuggers, supporting operations such as stepping into, over, or out of functions, as well as setting breakpoints at specific source lines. We show that neural debuggers -- obtained via fine-tuning large LLMs or pre-training smaller models from scratch -- can reliably model both forward execution (predicting future states and outputs) and inverse execution (inferring prior states or inputs) conditioned on debugger actions. Evaluated on CruxEval, our models achieve strong performance on both output and input prediction tasks, demonstrating robust conditional execution modeling. Our work takes first steps towards future agentic coding systems in which neural debuggers serve as a world model for simulated debugging environments, providing execution feedback or enabling agents to interact with real debugging tools. This capability lays the foundation for more powerful code generation, program understanding, and automated debugging.

Figures & tables

Figure 1: Neural Debugger Data Pipeline. Our pipeline prepares training data for neural debuggers by transforming stack-frame sequences recorded via sys.settrace in three steps: (1) we construct a state tree (Section 4.1 ) from frame events; (2) we sample trajectories by traversing the state tree using a data-generating action policy; and (3) we tokenize each trajectory using our formal neural debugger language grammar (Section 4.2 ).
Figure 2: State-action structure of Code World Model (CWM) and neural debuggers. In CWM ( FAIR CodeGen Team et al., 2025 ) the actions are viewed as code that modifies the variable states, while in neural debuggers actions influence the program state by controlling program execution analogous to traditional debuggers.
Figure 3: Transition model. We visualize the state transitions as traversal on the forward and inverse state tree. Left: Python code. Middle: Forward state tree with three levels indicated by indentation. Right: Corresponding inverse state tree. The blue numbers illustrate the correspondences between forward and inverse state tree.
Figure 4: Formal neural debugger language grammar. | indicates an OR-statement, {} indicate none or more elements, and : denotes an assignment. Whitespaces are shown for illustration purposes only. <|.|> indicate special tokens, LOCALS is the local variable dictionary, ARGS are return or exception arguments, and SRC denotes the source line.
Figure 5: Average token, action and event counts of forward debugger trajectory datasets. We show the mean function-level counts in turquoise and the repository-level counts in yellow, with the boxes indicating the 25% and 90% range. While the average action counts are similar due to the same action policy, repository-level trajectories contain more function calls, more exceptions, and longer token sequences.
Figure 6: Evolution of the exact match (em) next state prediction accuracy per debugger action on the function-level validation set during training. (a) Forward execution prediction. (b) Inverse execution prediction. Figure B.1 shows the corresponding results for the repository-level validation set, which lead to the same conclusions.
Figure 7: Exact match (em) next state prediction by state component per debugger action on the function-level validation set. Most prediction errors are in the local variables and return or exception arguments, while source lines and state events are predicted reliably. Figure B.2 shows the corresponding results for the repository-level data. Conclusions are the same.
ModelTokensInputOutput
inv_step_callstep_returnbreakpoint
1.8B Transformer Pretrain50B40.734.444.9
1.8B Transformer Pretrain150 B150\text{\,}\mathrm{B}53.648.057.7
32B CWM Finetune50B66.577.983.2
Table 1: CruxEval input and output pass@1 scores for single step prediction with neural debugger actions inv_step_call , step_return , and breakpoint (greedy decoding). All models are trained on debugger trace data only.
Figure 8: CruxEval input and output exact match @k scores for increasing prediction horizon (see Section A.2 for a discussion on exact match vs. pass metric). We use the same prompts (see Section B.3 ) as for Table 1 and insert step actions to increase number of revealed program states in the prompt. Long prediction horizons correspond to a high fraction of skipped frames (close to 1) and short prediction horizons to a low fraction (close to 0). We generate responses with temperature 0.6 and top-p 0.95. Across all models, the prediction accuracy decreases as the prediction horizon increases, with a steeper decline for smaller models trained from scratch..
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1 : Length distributions of the debugger trace dataset. We show the length distribution histograms of debugger trajectories in number of tokens and actions, and show the distribution of the code length in number of characters. Even though the number of actions are similar for function-level and repository-level data, the average token count for repository-level trajectories is significantly higher (see also Section 4.3 ). The reasons for the longer trajectories are more arbitrary python objects in their local variable state dictionaries and larger code context (see right column).
Policy Probstep_intostep_overstep_returnbreakpointcontinue
0.50.350.10.20.10.05
0.50.50.5---
Appendix
Table A.1 : Action probabilities for the action policy mix used to generate our debugger trace dataset.
ModelTokensInputInput
inv_step_callinv_step_call
  exact_match@1pass@1
1.8B Transformer Pretrain50B14.340.7
1.8B Transformer Pretrain150 B150\text{\,}\mathrm{B}17.753.6
32B CWM Finetune50B23.166.5
Appendix
Table A.2 : Comparison of CruxEval input exact_match@1 and pass@1 scores for single step prediction with the neural debugger action inv_step_call (greedy decoding) corresponding to Table 1 . The exact_match@1 (em@1) scores are considerably lower than pass@1 score, since exact match only compares to the reference inputs, and does not account for potential ambiguities in input predictions.
Figure B.1 : Evolution of the exact match (em) next state prediction accuracy per debugger action on the repository-level validation set during training. (a) Forward execution prediction. (b) Inverse execution prediction. Figure 6 shows the corresponding results for the function-level validation set. The results on both validation sets lead to similar conclusions.
Figure B.2 : Exact match (em) next state prediction by state component per debugger action on the repository-level validation set. Figure 7 shows the corresponding results for the function-level validation set. Conclusions remain unchanged.
Figure B.3 : CruxEval output prompt for the step_return and the breakpoint actions. The outcome of the step_return action is the frame with the return event and argument, while the outcome of the breakpoint action is the line event frame, including the local variables dictionary at the source line of the return statement. Predictions are generated by the 1.8 B1.8\text{\,}\mathrm{B} parameter neural debugger trained on 150 B150\text{\,}\mathrm{B} debugger trace tokens. Some line breaks are inserted for illustration purposes.
Figure C.1 : An example of a forward and inverse debugger trace for a function counting the occurrences of t in s . We take only step_into or inv_step_into actions to visit every frame. While the forward debugger trace is deterministic given its inputs, the inverse debugger trace with return argument n=2 is an example with infinitely many input combinations for s and t .
More in this topic

Code Language Models

Code language models are AI models that generate, explain, edit, and reason about source code across programming tasks.

See all 114 papers

Explore similar work

CardsList
Apr 19, 2026

Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?

Unlike code completion, debugging requires localizing faults and applying targeted edits. We observe that frontier LLMs often regenerate correct but over-edited solutions during debugging. To evaluate how far LLMs are from precise debugging, we introduce the Precise Debugging Benchmark (PDB) framework, which automatically converts any coding dataset into a debugging benchmark with precision-aware evaluation. PDB generates buggy programs by synthesizing verified atomic bugs and composing them into multi-bug programs. We define two novel metrics, edit-level precision and bug-level recall, which measure how many necessary edits are made and how many bugs are resolved. We release two evaluation benchmarks: PDB-Single-Hard on single-line bugs, and PDB-Multi on multi-line bugs. Experiments show that frontier models, such as GPT-5.1-Codex and DeepSeek-V3.2-Thinking, achieve unit-test pass rates above 76% but exhibit precision below 45%, even when explicitly instructed to perform minimal debugging. Finally, we show that iterative and agentic debugging strategies do not substantially improve precision or recall, highlighting the need to rethink post-training pipelines for coding models.

Miaosen Chai, Wang Bill Zhu, Shangshang Wang +5

University of Southern California · Microsoft · University of Wisconsin–Madison +1

LLM EvaluationAutomated Program Repair
AskPDF
Mar 11, 2026

Self-Execution Simulation Improves Coding Models

A promising research direction in enabling LLMs to generate consistently correct code involves addressing their inability to properly estimate program execution, particularly for code they generate. In this work, we demonstrate that Code LLMs can be trained to simulate program execution in a step-by-step manner and that this capability can be leveraged to improve competitive programming performance. Our approach combines supervised fine-tuning on natural language execution traces, textual explanations grounded in true execution, with reinforcement learning using verifiable rewards. We introduce two complementary objectives: output prediction given code and inputs, and solving competitive programming tasks with either ground-truth or self-predicted execution feedback. These objectives enable models to perform self-verification over multiple candidate solutions, and iterative self-fixing by simulating test execution. Across multiple competitive programming benchmarks, our method yields consistent improvements over standard reasoning approaches. We further present ablations and analysis to elucidate the role of execution simulation and its limitations.

Gallil Maimon, Ori Yoran, Felix Kreuk +4

FAIR team, Meta · Hebrew University of Jerusalem · Inria

Execution-Guided Code GenerationLLM Self-CorrectionRL for Code Generation
AskPDF
May 12, 2026

StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning

Existing code reasoning methods primarily supervise final code outputs, ignoring intermediate states, often leading to reward hacking where correct answers are obtained through inconsistent reasoning. We propose StepCodeReasoner, a framework that introduces explicit intermediate execution-state supervision. By automatically inserting structured print-based execution-trace anchors into code, the model is trained to predict runtime states at each step, transforming code reasoning into a verifiable, stepwise execution modeling problem. Building on this execution-aware method, we introduce Bi-Level GRPO, a reinforcement learning algorithm for structured credit assignment at two levels: inter-trajectory, comparing alternative execution paths, and intra-trajectory, rewarding intermediate accuracy based on its impact on downstream correctness. Extensive experiments demonstrate that StepCodeReasoner achieves SOTA performance in code reasoning. In particular, our 7B model achieves 91.1% on CRUXEval and 86.5% on LiveCodeBench, outperforming the CodeReasoner-7B baseline (86.0% and 77.7%) and GPT-4o (85.6% and 75.1%). Furthermore, on the execution-trace benchmark REval, our model scores 82.9%, outperforming baseline CodeReasoner-7B (72.3%), its 14B counterpart (81.1%), and GPT-4o (77.3%). Additionally, our approach also improves code generation performance, demonstrating that explicit execution modeling enhances both code reasoning and code generation.

Hao Wang, Rui Li, Lei Sha +1

Beihang University, Beijing, China · Peking University, Beijing, China · Zhongguancun Laboratory, Beijing, China +1

RL for Language Model ReasoningExecution-Guided Code GenerationReinforcement Learning with Verifiable RewardsStep-Level Credit Assignment
AskPDF