Towards a Neural Debugger for Python

Maximilian Beck; Jonas Gehring; Jannik Kossen; Gabriel Synnaeve

cs.LGMar 10, 2026

Towards a Neural Debugger for Python

Authors: Maximilian Beck, Jonas Gehring, Jannik Kossen, Gabriel Synnaeve

Organizations: Johannes Kepler University Linz, Institute for Machine Learning · Meta FAIR CodeGen Team

Abstract

Training large language models (LLMs) on Python execution traces grounds them in code execution and enables the line-by-line execution prediction of whole Python programs, effectively turning them into neural interpreters (FAIR CodeGen Team et al., 2025). However, developers rarely execute programs step by step; instead, they use debuggers to stop execution at certain breakpoints and step through relevant portions only while inspecting or modifying program variables. Existing neural interpreter approaches lack such interactive control. To address this limitation, we introduce neural debuggers: language models that emulate traditional debuggers, supporting operations such as stepping into, over, or out of functions, as well as setting breakpoints at specific source lines. We show that neural debuggers -- obtained via fine-tuning large LLMs or pre-training smaller models from scratch -- can reliably model both forward execution (predicting future states and outputs) and inverse execution (inferring prior states or inputs) conditioned on debugger actions. Evaluated on CruxEval, our models achieve strong performance on both output and input prediction tasks, demonstrating robust conditional execution modeling. Our work takes first steps towards future agentic coding systems in which neural debuggers serve as a world model for simulated debugging environments, providing execution feedback or enabling agents to interact with real debugging tools. This capability lays the foundation for more powerful code generation, program understanding, and automated debugging.

Figures & tables

Figure 1: Neural Debugger Data Pipeline. Our pipeline prepares training data for neural debuggers by transforming stack-frame sequences recorded via sys.settrace in three steps: (1) we construct a state tree (Section 4.1 ) from frame events; (2) we sample trajectories by traversing the state tree using a data-generating action policy; and (3) we tokenize each trajectory using our formal neural debugger language grammar (Section 4.2 ).

Figure 2: State-action structure of Code World Model (CWM) and neural debuggers. In CWM ( FAIR CodeGen Team et al., 2025 ) the actions are viewed as code that modifies the variable states, while in neural debuggers actions influence the program state by controlling program execution analogous to traditional debuggers.

Figure 3: Transition model. We visualize the state transitions as traversal on the forward and inverse state tree. Left: Python code. Middle: Forward state tree with three levels indicated by indentation. Right: Corresponding inverse state tree. The blue numbers illustrate the correspondences between forward and inverse state tree.

Figure 4: Formal neural debugger language grammar. | indicates an OR-statement, {} indicate none or more elements, and : denotes an assignment. Whitespaces are shown for illustration purposes only. <|.|> indicate special tokens, LOCALS is the local variable dictionary, ARGS are return or exception arguments, and SRC denotes the source line.

Figure 5: Average token, action and event counts of forward debugger trajectory datasets. We show the mean function-level counts in turquoise and the repository-level counts in yellow, with the boxes indicating the 25% and 90% range. While the average action counts are similar due to the same action policy, repository-level trajectories contain more function calls, more exceptions, and longer token sequences.

Panel of Figure 6 — Figure 6: Evolution of the exact match (em) next state prediction accuracy per debugger action on the function-level validation set during training. (a) Forward execution prediction. (b) Inverse execution prediction. Figure B.1 shows the corresponding results for the repository-level validation set, which lead to the same conclusions.

Panel of Figure 7 — Figure 7: Exact match (em) next state prediction by state component per debugger action on the function-level validation set. Most prediction errors are in the local variables and return or exception arguments, while source lines and state events are predicted reliably. Figure B.2 shows the corresponding results for the repository-level data. Conclusions are the same.

Model	Tokens	Input	Output
Model	Tokens	inv_step_call	step_return	breakpoint
1.8B Transformer Pretrain	50B	40.7	34.4	44.9
1.8B Transformer Pretrain	$150\text{\,}\mathrm{B}$	53.6	48.0	57.7
32B CWM Finetune	50B	66.5	77.9	83.2

Table 1: CruxEval input and output pass@1 scores for single step prediction with neural debugger actions inv_step_call , step_return , and breakpoint (greedy decoding). All models are trained on debugger trace data only.

Panel of Figure 8 — Figure 8: CruxEval input and output exact match @k scores for increasing prediction horizon (see Section A.2 for a discussion on exact match vs. pass metric). We use the same prompts (see Section B.3 ) as for Table 1 and insert step actions to increase number of revealed program states in the prompt. Long prediction horizons correspond to a high fraction of skipped frames (close to 1) and short prediction horizons to a low fraction (close to 0). We generate responses with temperature 0.6 and top-p 0.95. Across all models, the prediction accuracy decreases as the prediction horizon increases, with a steeper decline for smaller models trained from scratch..

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1 : Length distributions of the debugger trace dataset. We show the length distribution histograms of debugger trajectories in number of tokens and actions, and show the distribution of the code length in number of characters. Even though the number of actions are similar for function-level and repository-level data, the average token count for repository-level trajectories is significantly higher (see also Section 4.3 ). The reasons for the longer trajectories are more arbitrary python objects in their local variable state dictionaries and larger code context (see right column).

Policy Prob	step_into	step_over	step_return	breakpoint	continue
0.5	0.35	0.1	0.2	0.1	0.05
0.5	0.5	0.5	-	-	-

Appendix

Table A.1 : Action probabilities for the action policy mix used to generate our debugger trace dataset.

Model	Tokens	Input	Input
Model	Tokens	inv_step_call	inv_step_call
		exact_match@1	pass@1
1.8B Transformer Pretrain	50B	14.3	40.7
1.8B Transformer Pretrain	$150\text{\,}\mathrm{B}$	17.7	53.6
32B CWM Finetune	50B	23.1	66.5

Appendix

Table A.2 : Comparison of CruxEval input exact_match@1 and pass@1 scores for single step prediction with the neural debugger action inv_step_call (greedy decoding) corresponding to Table 1 . The exact_match@1 (em@1) scores are considerably lower than pass@1 score, since exact match only compares to the reference inputs, and does not account for potential ambiguities in input predictions.

Panel of Figure 13 — Figure B.1 : Evolution of the exact match (em) next state prediction accuracy per debugger action on the repository-level validation set during training. (a) Forward execution prediction. (b) Inverse execution prediction. Figure 6 shows the corresponding results for the function-level validation set. The results on both validation sets lead to similar conclusions.

Panel of Figure 14 — Figure B.2 : Exact match (em) next state prediction by state component per debugger action on the repository-level validation set. Figure 7 shows the corresponding results for the function-level validation set. Conclusions remain unchanged.

Figure B.3 : CruxEval output prompt for the step_return and the breakpoint actions. The outcome of the step_return action is the frame with the return event and argument, while the outcome of the breakpoint action is the line event frame, including the local variables dictionary at the source line of the return statement. Predictions are generated by the

1.8\text{\,}\mathrm{B}

parameter neural debugger trained on

150\text{\,}\mathrm{B}

debugger trace tokens. Some line breaks are inserted for illustration purposes.

Figure C.1 : An example of a forward and inverse debugger trace for a function counting the occurrences of t in s . We take only step_into or inv_step_into actions to visit every frame. While the forward debugger trace is deterministic given its inputs, the inverse debugger trace with return argument n=2 is an example with infinitely many input combinations for s and t .

Code Language Models

Code language models are AI models that generate, explain, edit, and reason about source code across programming tasks.

See all 114 papers

Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?
Apr 19, 2026Miaosen Chai, Wang Bill Zhu, Shangshang Wang +5LLM Evaluation Automated Program Repair
University of Southern California · Microsoft · University of Wisconsin–Madison +1
Self-Execution Simulation Improves Coding Models
Mar 11, 2026Gallil Maimon, Ori Yoran, Felix Kreuk +4Execution-Guided Code Generation LLM Self-Correction
FAIR team, Meta · Hebrew University of Jerusalem · Inria
StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
May 12, 2026Hao Wang, Rui Li, Lei Sha +1RL for Language Model Reasoning Execution-Guided Code Generation
Beihang University, Beijing, China · Peking University, Beijing, China · Zhongguancun Laboratory, Beijing, China +1

Apr 19, 2026J/K move · Enter open · S save

Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?

Miaosen Chai, Wang Bill Zhu, Shangshang Wang, Yejia Liu+4

University of Southern California · Microsoft · University of Wisconsin–Madison+1

LLM Evaluation Automated Program Repair

Open paper AskPDF arXiv

Unlike code completion, debugging requires localizing faults and applying targeted edits. We observe that frontier LLMs often regenerate correct but over-edited solutions during debugging. To evaluate how far LLMs are from precise debugging, we introduce the Precise Debugging Benchmark (PDB) framework, which automatically converts any coding dataset into a debugging benchmark with precision-aware evaluation. PDB generates buggy programs by synthesizing verified atomic bugs and composing them into multi-bug programs. We define two novel metrics, edit-level precision and bug-level recall, which measure how many necessary edits are made and how many bugs are resolved. We release two evaluation benchmarks: PDB-Single-Hard on single-line bugs, and PDB-Multi on multi-line bugs. Experiments show that frontier models, such as GPT-5.1-Codex and DeepSeek-V3.2-Thinking, achieve unit-test pass rates above 76% but exhibit precision below 45%, even when explicitly instructed to perform minimal debugging. Finally, we show that iterative and agentic debugging strategies do not substantially improve precision or recall, highlighting the need to rethink post-training pipelines for coding models.

Abstract

Code Language Models

Explore similar work

Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?

Self-Execution Simulation Improves Coding Models

StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning