Static Program Slicing Using Language Models With Dataflow-Aware Pretraining and Constrained Decoding

Pengfei He; Shaowei Wang; Tse-Hsun Chen; Muhammad Asaduzzaman

cs.SEApr 9, 2026

Static Program Slicing Using Language Models With Dataflow-Aware Pretraining and Constrained Decoding

Authors: Pengfei He, Shaowei Wang, Tse-Hsun Chen, Muhammad Asaduzzaman

Organizations: Department of Computer Science, University of Manitoba, Winnipeg, Canada · School of Engineering and Computer Science, Concordia University, Montreal, Canada · School of Computer Science, University of Windsor, Windsor, Canada

Abstract

Static program slicing is a fundamental software engineering technique for isolating code relevant to specific variables. While recent learning-based approaches using language models (LMs) show promise in automating slice prediction, they suffer from inaccurate dependency modeling and unconstrained generation, where LMs fail to capture precise data flow relations and produce slices containing hallucinated tokens and statements. To address these challenges, we propose Sliceformer, a novel approach that reformulates static program slicing as a sequence-to-sequence task using small language models such as CodeT5+. Sliceformer introduces two key innovations that directly target the identified limitations. First, to improve dependency modeling, we design dataflow-aware pretraining objectives that leverage data flow graphs (DFG) to teach models data dependencies through dataflow-preserving statement permutation and dataflow-aware span corruption. Second, to eliminate hallucination, we develop a constrained decoding mechanism that enforces both lexical and syntactic constraints. We evaluate Sliceformer on Java and Python program slicing benchmarks, demonstrating consistent improvements over state-of-the-art baselines with up to 22% gain in ExactMatch.

Figures & tables

Panel of Figure 1 — Figure 1: Motivating examples of incorrect program slices produced by LMs.

Figure 2: Illustration of dataflow-aware pre-training objectives. Left: given a snippet, we generate a statement permutation that preserves the dataflow of the original program. Right: we select a variable

a

with a dataflow chain

[X]\rightarrow a\rightarrow[Y]

, then mask its parent

[X]

(where it comes from) and child

[Y]

(where it flows to). The pre-training objective is to recover both

[X]

and

[Y]

Figure 3: TSED scores at statement boundaries for syntactically correct versus erroneous program slices based on the same input code snippet. The scores are derived from Example (3) in Figure 1 . When a syntactic error is introduced into the generated slice (Line 6), the TSED score deviates from its expected monotonic increasing pattern and instead decreases at the statement boundary.

Methods	Java				Python
Methods	Acc-D	ExactMatch	CodeBLEU	TSED	Acc-D	ExactMatch	CodeBLEU	TSED
LLM-based slicer (GPT-4/Zero-shot)	14.76	0.00	20.10	35.53	12.34	0.00	18.45	32.18
LLM-based slicer (GPT-4/RAG)	51.70	0.00	65.18	59.93	48.22	0.00	61.34	56.71
LLM-based slicer (GPT-4/COT)	56.84	0.00	68.41	59.68	53.17	0.00	65.23	57.42
LLM-based slicer (GPT-5/Zero-shot)	18.42	0.00	24.67	39.21	15.78	0.00	22.13	36.54
LLM-based slicer (GPT-5/RAG)	55.13	7.00	68.92	62.45	51.68	11.00	64.77	59.33

Table 1: Effectiveness comparison among different static learning-based program slicing methods.

Figure 4: The ablation analysis results highlight the contribution of each component. A larger performance drop indicates greater importance. We rank the component importance from left to right. Abbreviations: Perm. = statement permutation; Corr. = span corruption; Syn. = syntactic constraint; Lex. = lexical constraint.

Methods	Size	Runtime (per task)
NS-slicer (CodeBERT)	125M	0.105s
NS-slicer (GraphCodeBERT)	125M	0.135s
CodeT5+	770M	0.289s
Sliceformer	770M	0.296s
CodeLlama-7B (SFT)	7B	5.75s
Qwen3-8B (SFT)	8B	6.52s

Table 2: Inference latency comparison among different program slicing methods on Java dataset.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Java
	train	valid	test
Entries	30.8K	3.5K	8.7K
Avg. tokens	64	64	66
Avg. slocs	19	18	19
Python
	train	valid	test

Appendix

Table 3: Statistics of the two language datasets

Figure 5: Workflow of constrained decoding

Model	Architecture	Pretraining	Super Fine-tuning	Constrained Decoding	ExactMatch
CodeT5	Encoder–Decoder	$\times$	✓	$\times$	82.80
CodeT5	Encoder–Decoder	✓	✓	✓	85.12
CodeLlama-7B	Decoder-Only	$\times$	✓	$\times$	75.27
CodeLlama-7B	Decoder-Only	$\times$	✓	✓	78.40
Qwen3-8B	Decoder-Only	$\times$	✓	$\times$	80.55
Qwen3-8B	Decoder-Only	$\times$	✓	✓	83.11

Appendix

Table 4: Comparison of transformer architectures and their compatibility with Sliceformer components (Java).

Panel of Figure 6 — Figure 6: An erroneous example predicted by NS-slicer.

Code Language Models

Code language models are AI models that generate, explain, edit, and reason about source code across programming tasks.

See all 114 papers

CodeAlchemy: Synthetic Code Rewriting at Scale
Jun 8, 2026Ankit Gupta, Aditya Prasad, Rameswar PandaSynthetic Data Pretraining Synthetic Data Generation
MIT-IBM Watson AI Lab, IBM Research
CODEBLOCK: Learning to Supervise Code at the Right Granularity
Jun 10, 2026Zhijie Deng, Ling Li, Jinlong Pang +5Supervised Fine-Tuning Code Language Models
Hong Kong University of Science and Technology (Guangzhou) · UC Santa Cruz · Ant Group +1
Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation
Jul 8, 2026Didula Samaraweera, Anjana Supun, Srinath PereraRL for Code Generation Code Generation
WSO2 Santa Clara, CA, USA

Jun 8, 2026J/K move · Enter open · S save

CodeAlchemy: Synthetic Code Rewriting at Scale

Ankit Gupta, Aditya Prasad, Rameswar Panda

MIT-IBM Watson AI Lab, IBM Research

Synthetic Data Pretraining Synthetic Data Generation Code Language Models Code Generation Benchmarks

Open paper AskPDF arXiv

Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats. While synthetic data has proven transformative for language models, code remains largely unexplored beyond limited quality improvements. We present CodeAlchemy, a synthetic data generation framework that transforms publicly sourced code into semantically-rich training data through 5 strategies: CodeEnhance (quality-aware rewriting), CodeQA (template-based problems), CodeDev (developer tasks), CodeDialogue (multi-turn conversations), and CodeTrace (execution traces). We process 3 corpora across 15 languages to generate 500B+ tokens of synthetic data plus 350B reasoning tokens, orders of magnitude more than prior efforts. CodeTrace instruments and executes 1.3M+ files across 14 languages and 5K libraries, capturing control flow, state tracking, and library knowledge. We introduce DevEval (developer tasks) and TraceEval (execution prediction) benchmarks; frontier models like Claude Sonnet 4.5 achieve only 5.6% exact match on TraceEval, revealing critical gaps in semantic understanding. Our 3B models achieve 83.5% on HumanEval, 63.2% on MBPP, 8.09% win rate on DevEval, and 15.36 ROUGE-2 on TraceEval, outperforming frontier models 10x the size including 27B Gemma-3 and 32B Granite-4.0.

Abstract

Code Language Models

Explore similar work

CodeAlchemy: Synthetic Code Rewriting at Scale

CODEBLOCK: Learning to Supervise Code at the Right Granularity

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation