Self-Improving Code Generation via Semantic Entropy and Behavioral Consensus

Huan Zhang; Wei Cheng; Wei Hu

cs.SEMar 31, 2026

Self-Improving Code Generation via Semantic Entropy and Behavioral Consensus

Authors: Huan Zhang, Wei Cheng, Wei Hu

Organizations: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · State Key Laboratory for Novel Software Technology, National Institute of Healthcare Data Science, Nanjing University, Nanjing, China

Abstract

Improving the code generation capabilities of large language models (LLMs) typically relies on supervised fine-tuning or preference optimization, both of which require costly external resources such as powerful teacher models or reliable test units. However, in real-world scenarios, it is much harder to obtain reference solutions and test oracles than problem descriptions and test inputs. In this paper, we tackle a challenging yet realistic question: Can a code language model improve itself without access to a superior teacher and a test oracle? To answer this, we propose ConSelf, a self-improving approach built upon two key ideas. First, we introduce code semantic entropy, a novel metric that measures problem-level uncertainty by assessing the functional diversity of program behaviors, enabling a curriculum construction with the most learnable problems. Second, we present consensus-driven direct preference optimization (Con-DPO), a preference-based fine-tuning method that weights each preference pair by its behavioral consensus, thereby mitigating the impact of noisy self-generated supervision. Experiments on various benchmarks and backbone LLMs demonstrate that ConSelf significantly outperforms baselines, validating the effectiveness of semantic entropy-based curriculum construction and consensus-driven optimization in improving code generation without external supervision.

Figures & tables

Figure 1. The dilemma of learning from “noisy data” generated for an intractable problem. When all self-generated solutions are flawed, any learning method (SFT/DPO) becomes futile, highlighting the need to identify and filter out such problems. The dilemma of learning from ``noisy data'' generated for an intractable problem. When all self-generated solutions are flawed, any learning method (SFT/DPO) becomes futile, highlighting the need to identify and filter out such problems.

Figure 2. Overview of the ConSelf approach. The model generates code samples for each problem by observation-guided sampling, estimates code semantic entropy to filter problems, and fine-tunes itself on consensus-driven preference pairs. Overview of the \textsc{ConSelf}\xspaceapproach. The model generates code samples for each problem by observation-guided sampling, estimates code semantic entropy to filter problems, and fine-tunes itself on consensus-driven preference pairs. — Figure 2. Overview of the ConSelf approach. The model generates code samples for each problem by observation-guided sampling, estimates code semantic entropy to filter problems, and fine-tunes itself on consensus-driven preference pairs. Overview of the ConSelf\xspaceapproach. The model generates code samples for each problem by observation-guided sampling, estimates code semantic entropy to filter problems, and fine-tunes itself on consensus-driven preference pairs.

Figure 3. The prompt template for observation generation and example observations generated for the count_primes(n) problem. The prompt guides the model to produce diverse insights, serving as varied conditions for code generation. The prompt template for observation generation.

Figure 4. The prompt template for code generation. The model generates $n_{\text{code}}$ candidate programs conditioned on each observation to enhance solution diversity. The prompt template for code generation.

Backbone LLMs	Methods	HumanEval	MBPP	EvalPlus		LiveCodeBench				Avg.	RI $\uparrow$
Backbone LLMs	Methods	HumanEval	MBPP	HE+	MBPP+	Easy	Medium	Hard	Overall	Avg.	RI $\uparrow$
CodeLlama-7B	Base model	35.98	52.91	31.10	44.80	23.06	1.21	0.27	7.80	34.52	-
	PFPO	34.55	53.09	29.27	44.97	23.66	1.01	0.54	8.11	33.99	$-1.54\%$
	Self-SFT	32.93	50.88	27.24	43.03	20.19	0.91	0.02	6.82	32.18	$-6.78\%$
	Self-DPO	33.54	53.17	28.86	45.33	19.35	1.11	0.36	7.73	33.73	$-2.29\%$
	ConSelf	36.59	55.03	31.50	46.74	24.85	1.31	0.27	8.45	35.66	3.30%

Table 1. Performance comparison across different LLMs and methods on four code generation benchmarks. All results are reported using pass@1 scores (%). RI denotes relative improvement over the base model. Performance comparison across different LLMs and methods on four code generation benchmarks.

Figure 5. Comparison of the number of training examples generated by different methods for each model. A bar chart comparing the number of training examples for three models: CodeLlama, DeepSeek-Coder, and Qwen2.5-Coder. For each model, there are four bars. The bar for \textsc{ConSelf}\xspaceis consistently short. — Figure 5. Comparison of the number of training examples generated by different methods for each model. A bar chart comparing the number of training examples for three models: CodeLlama, DeepSeek-Coder, and Qwen2.5-Coder. For each model, there are four bars. The bar for ConSelf\xspaceis consistently short.

Figure 6. Area Under ROC Curve (AUC) for discriminating solvable ( $\text{pass rate}>0$ ) problems and unsolvable ( $\text{pass rate}=0$ ) ones. An AUC of 0.5 indicates no discriminative ability. Area Under ROC Curve (AUC) for discriminating solvable ($\text{pass rate} > 0$) problems. — Figure 6. Area Under ROC Curve (AUC) for discriminating solvable ( $\text{pass rate}>0$ ) problems and unsolvable ( $\text{pass rate}=0$ ) ones. An AUC of 0.5 indicates no discriminative ability. Area Under ROC Curve (AUC) for discriminating solvable ( $\text{pass rate} > 0$ ) problems.

Metrics	Pearson ( $p$ )	Spearman ( $p$ )	Kendall ( $p$ )
CodeLlama-7B
Crash rate	$-0.344$ (6e-181)	$-0.240$ (9e-83)	$-0.207$ (5e-85)
Code sem. entropy	$-0.327$ (4e-195)	$-0.292$ (1e-133)	$-0.195$ (8e-133)
Naive sem. entropy	$-0.265$ (2e-110)	$-0.229$ (5e-95)	$-0.180$ (1e-65)
Neg. log-likelihood	$0.084$ (4e-26)	$0.061$ (1e-14)	$0.041$ (1e-14)
Token-level entropy	$0.078$ (1e-22)	$0.058$ (1e-13)	$0.039$ (1e-13)

Table 2. Correlation coefficients and

p

-values between each metric and pass rate for each model. Background color indicates correlation strength and direction (blue = desirable, and red = undesirable). “0” indicates

p<10^{-300}

. Correlation coefficients and

p

-values between each metric and pass rate for each model.

Metrics	CodeLlama	DeepSeek-Coder	Qwen2.5-Coder
Neg. log-likelihood	31.95	60.62	72.85
Token-level entropy	32.18	60.44	72.90
Naive sem. entropy	33.65	61.75	72.20
Code sem. entropy	35.66	63.26	74.39

Table 3. End-to-end average performance comparison of curricula curated by different uncertainty metrics. End-to-end average performance comparison of curricula curated by different uncertainty metrics.

Backbone LLMs	Methods	HumanEval	MBPP	EvalPlus		LiveCodeBench				Avg.	RD
Backbone LLMs	Methods	HumanEval	MBPP	HE+	MBPP+	Easy	Medium	Hard	Overall	Avg.	RD
CodeLlama-7B	ConSelf	36.59	55.03	31.50	46.74	24.85	1.31	0.27	8.45	35.66	-
	w/o CW	35.77	53.70	30.49	45.41	24.25	1.51	0.27	8.37	34.75	$-2.55\%$
	w/o CC	34.15	53.44	28.66	45.24	23.30	1.21	0.27	7.95	33.89	$-4.96\%$
DeepSeek-Coder-6.7B	ConSelf	78.46	76.98	72.15	68.78	48.75	10.88	0.90	19.92	63.26	-
	w/o CW	77.44	75.40	71.95	67.20	46.95	10.47	0.81	19.17	62.23	$-1.63\%$

Table 4. Ablation results. RD denotes relative degradation in performance compared to ConSelf . Ablation results.

Figure 7. Impact of the filtering threshold

\tau

on DeepSeek-Coder-6.7B training data. (a) Number of training samples and average performance improvement over the base model. (b) Golden count and golden rate in the filtered dataset. Impact of the filtering threshold

\tau

on training data. Lower thresholds reduce sample quantity but improve data quality, with optimal performance at

\tau=0.3

Language Model Self-Improvement

Language model self-improvement studies how models use their own outputs, feedback, and experience to improve with less external supervision.

See all 171 papers

Embarrassingly Simple Self-Distillation Improves Code Generation
Apr 1, 2026Ruixiang Zhang, Richard He Bai, Huangjie Zheng +3Language Model Decoding Language Model Distillation
Apple
ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning
Mar 6, 2026Juyong Jiang, Jiasi Shen, Sunghun Kim +3LLM Code Generation LLM Self-Correction
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · Amazon AGI +1
Self-Execution Simulation Improves Coding Models
Mar 11, 2026Gallil Maimon, Ori Yoran, Felix Kreuk +4Execution-Guided Code Generation LLM Self-Correction
FAIR team, Meta · Hebrew University of Jerusalem · Inria

Apr 1, 2026J/K move · Enter open · S save

Embarrassingly Simple Self-Distillation Improves Code Generation

Ruixiang Zhang, Richard He Bai, Huangjie Zheng, Navdeep Jaitly+2

Apple

Language Model Decoding Language Model Distillation Language Model Self-Improvement LLM Code Generation

Open paper AskPDF arXiv

Can a large language model (LLM) improve at code generation using only its own raw outputs, without a verifier, a teacher model, or reinforcement learning? We answer in the affirmative with simple self-distillation (SSD): sample solutions from the model with certain temperature and truncation configurations, then fine-tune on those samples with standard supervised fine-tuning. SSD improves Qwen3-30B-Instruct from 42.4% to 55.3% pass@1 on LiveCodeBench v6, with gains concentrating on harder problems, and it generalizes across Qwen and Llama models at 4B, 8B, and 30B scale, including both instruct and thinking variants. To understand why such a simple method can work, we trace these gains to a precision-exploration conflict in LLM decoding and show that SSD reshapes token distributions in a context-dependent way, suppressing distractor tails where precision matters while preserving useful diversity where exploration matters. Taken together, SSD offers a complementary post-training direction for improving LLM code generation. Our code is available at https://github.com/apple/ml-ssd

Abstract

Language Model Self-Improvement

Explore similar work

Embarrassingly Simple Self-Distillation Improves Code Generation

ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

Self-Execution Simulation Improves Coding Models