LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting

Yewen Li; Zhiyi Lyu; Peng Jiang; Qingpeng Cai; Fei Pan; Bo An; Peng Jiang

cs.CLMar 5, 2026

LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting

Authors: Yewen Li, Zhiyi Lyu, Peng Jiang, Qingpeng Cai, Fei Pan, Bo An, Peng Jiang

Organizations: Kuaishou Technology Beijing, China · Nanyang Technological University Singapore, Singapore

Abstract

The growing scale of ad auctions on online advertising platforms has intensified competition, making manual bidding impractical and necessitating auto-bidding to help advertisers achieve their economic goals. Current auto-bidding methods have evolved to use offline reinforcement learning or generative methods to optimize bidding strategies, but they can sometimes behave counterintuitively due to the black-box training manner and limited mode coverage of datasets, leading to challenges in understanding task status and generalization in dynamic ad environments. Large language models (LLMs) offer a promising solution by leveraging prior human knowledge and reasoning abilities to improve auto-bidding performance. However, directly applying LLMs to auto-bidding faces difficulties due to the need for precise actions in competitive auctions and the lack of specialized auto-bidding knowledge, which can lead to hallucinations and suboptimal decisions. To address these challenges, we propose a hierarchical Large autoBidding Model (LBM) to leverage the reasoning capabilities of LLMs for developing a superior auto-bidding strategy. This includes a high-level LBM-Think model for reasoning and a low-level LBM-Act model for action generation. Specifically, we propose a dual embedding mechanism to efficiently fuse two modalities, including language and numerical inputs, for language-guided training of the LBM-Act; then, we propose an offline reinforcement fine-tuning technique termed GQPO for mitigating the LLM-Think's hallucinations and enhancing decision-making performance without simulation or real-world rollout like previous multi-turn LLM-based methods. Experiments demonstrate the superiority of a generative backbone based on our LBM, especially in an efficient training manner and generalization ability.

Figures & tables

Figure 1. Overview of the training scheme for our hierarchical large auto-bidding model. Training Stage I (left): train the LBM-Act with language-guided decision training via a dual embedding mechanism to fuse two modalities; Training Stage II (right): reinforcement fine-tuning the LBM-Think via group relative-Q policy optimization.

AuctionNet		USCB	CQL	IQL	BCQ	DT	DT-Q	DiffBid	DiffBid-Q	LBM(P)	LBM(GQPO)
Dense	Conversions ( $\uparrow$ )	267	336	322	321	364	371	316	319	376 $\pm$ 3.0	382 $\pm$ 3.3
Dense	Score ( $\uparrow$ )	157	171	281	270	329	334	214	260	320 $\pm$ 2.2	348 $\pm$ 2.3
Sparse	Conversions ( $\uparrow$ )	28.5	30.2	36.0	31.1	31.3	33.8	31.3	31.5	37.4 $\pm$ 0.3	38.5 $\pm$ 0.3
Sparse	Score ( $\uparrow)$	17.5	22.2	30.0	30.5	29.6	31.1	23.1	25.1	32.9 $\pm$ 0.2	33.4 $\pm$ 0.2

Table 1. Comparison between our LBM methods and previous non-LLM-based methods in AuctionNet.

	Method	Budget Utilization ( $\uparrow$ )		CPA Ratio ( $\downarrow$ )		Conversions ( $\uparrow$ )		Score ( $\uparrow$ )
	Method	Dense	Sparse	Dense	Sparse	Dense	Sparse	Dense	Sparse
LLM	Prompting	0.743	0.722	0.878	0.903	289	24.2	286	23.4
	SFT	0.765	0.741	0.882	0.804	297	27.0	286	26.9
	GRPO	0.860	0.773	0.945	0.816	327	29.9	292	29.7
	LLM-DT	0.902	0.861	0.936	0.847	366	36.0	328	32.5
	Prompt-LLM-DT	0.914	0.970	0.931	0.965	355	35.8	322	30.3

Table 2. Comprehensive evaluation of LLM-based auto-bidding methods in AuctionNet.

	DT-score					LBM
Reweight $w$	0	0.1	0.2	0.5	1.0
Budget Utlization ( $\uparrow$ )	0.884	0.839	0.818	0.780	0.632	0.938
CPA Ratio ( $\downarrow$ )	0.895	0.846	0.826	0.831	0.725	0.960
Conversions ( $\uparrow$ )	364	354	352	329	278	382
Score ( $\uparrow$ )	329	340	343	323	276	348

Table 3. Performance with various reward setting.

Panel of Figure 2 — Figure 2. Relationship between the CPA ratio and model behavior by 1000 random samples. When the CPA ratio exceeds 1, it indicates that the CPA constraint is not satisfied, suggesting a high likelihood of decreasing the bidding parameter, i.e. , $\Delta a<0$ . When the CPA ratio is smaller than 1, it will be more likely to increase the bidding parameter $\Delta a>0$ . However, this relationship is not clearly reflected in DT and pretrained LLM, but is more evident in LLM finetuned with GQPO.

Instruction	Budget Utilization( $\uparrow$ )	CPA Ratio( $\downarrow$ )	Conversions( $\uparrow$ )	Score( $\uparrow$ )
Increase	0.969	1.028	368	315
Decrease	0.894	0.924	373	325
base	0.938	0.960	382	348

Table 4. Language instruction’s impact on performance.

	Budget Ratio	0.5	0.75	1.0	1.25	1.5
Dense	DT	186	234	364	393	445
Dense	LBM(P)	198	285	376	462	520
Sparse	DT	18.2	27.5	31.3	43.5	50.0
Sparse	LBM(P)	21.3	28.5	37.4	43.6	50.6

Table 5. Generalization to various budget settings. Performance under conversions.

Figure 3. Training loss curve of LLM-DT and LBM-Act, supporting the efficiency of language-guided learning.

	Dense		Sparse
Model Size	Conversions	Score	Conversions	Score
3B	376	320	37.4	32.9
7B	375	320	37.3	34.6
32B	374	318	37.1	33.5

Table 6. Impact of LBM-Think’s model size.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4. Illustration of the inference procedure of LBM with a detailed case of the prompt and CoT. For a timestep $t$ , the LBM-Think first generates a CoT at timestep $t-1$ within the interval $\Delta t$ . Then, when timestep $t$ arrives, the LBM-Act receives the pre-generated CoT along with the detailed numerical sequence information to generate an action, which involves adjusting the bidding parameter.

Panel of Figure 5 — Figure 5. Visualization of attention scores varying across different positions and heads. For simplicity, we let positions from #0 to #6 as the language part, while the remaining positions are the numerical part. The two modalities are effectively fused using different attention heads. For instance, heads such as #4, #5, #6, and #10 in the final layer predominantly focus on the language part, whereas other heads are more attuned to the numerical parts. Interestingly, we observed that the attention in the middle layers distinctly separates the language parts (positions 0-6) from the numerical parts, with position 7 marking the beginning of the numerical part.

Params	AuctionNet	AuctionNet-Sparse
Trajectories	479,376	479,376
Delivery Periods	9,987	9,987
Time steps in a trajectory	48	48
State dimension	16	16
Action dimension	1	1
Action range	[0, 493]	[0, 8178]

Appendix

Table 7. The parameters of AuctionNet and AuctionNet-sparse.

	LBM-Think			LBM-Act
#params	3B	7B	32B	0.5B
Inference Latency	2.5s	3.6s	8.9s	63ms

Appendix

Table 8. Inference Latency of different modules.

Figure 6. Finetune the LLM using GRPO so that it first performs step-by-step reasoning and then outputs an action. However, after around 150 training steps, the model only outputs the action without performing additional reasoning.

Hyperparameters	Value
Batch size	128
Number of steps	400000
Sequence length	10
Learning rate	1e-4
Number of attention layers	6
Number of heads	8

Appendix

Table 9. The detailed hyperparameters of Q-value networks

Automated Bidding

Automated bidding studies algorithms that choose auction bids over time while balancing value, budgets, and performance constraints.

See all 36 papers

AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization
Jul 19, 2026Yuejia Dou, Hesong Wang, Xinyu Zhang +6Automated Bidding Large Language Model-Guided Optimization
Gaoling School of Artificial Intelligence, Renmin University of China Beijing, China · Alibaba Group Beijing, China
On the Role of Language Representations in Auto-Bidding: Findings and Implications
May 7, 2026Guanyu Zhu, Jining Luan, Hanwen Du +11Automated Bidding Online Advertising
City University of Hong Kong · South China Agricultural University · University of Electronic Science and Technology of China +11
Generative Auto-Bidding with Unified Modeling and Exploration
May 19, 2026Mingming Zhang, Feiqing Zhuang, Na Li +7Automated Bidding Online Advertising
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University · Taobao & Tmall Group of Alibaba

Jul 19, 2026J/K move · Enter open · S save

AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization

Yuejia Dou, Hesong Wang, Xinyu Zhang, Tianyu Wang+5

Gaoling School of Artificial Intelligence, Renmin University of China Beijing, China · Alibaba Group Beijing, China

Automated Bidding Large Language Model-Guided Optimization Online Advertising Self-Improving Agents RL for LLM Agents

Open paper AskPDF arXiv

Auto-bidding plays an essential role in online advertising, automatically adjusting bids for advertisers to optimize their commercial goals. The emerging AI-Generated Bidding (AIGB) paradigm widely adopts generative modeling to optimize bidding strategies, yet suffers from the limited mode coverage of offline datasets and inadequate task-state understanding, hindering effective exploration of optimal strategies. Large Language Models (LLMs), with prior world knowledge and reasoning capabilities, offer a promising approach to overcome these limitations. However, directly applying LLMs to auto-bidding tasks faces inherent challenges in limited numerical precision, hallucinations, and inference latency. To address these limitations, we propose AIGB-R1, a hierarchical self-evolving auto-bidding framework aiming to enhance AI-Generated Bidding via LLMs' Reasoning capabilities, comprising a high-level Planner module for macro-level strategy planning and a low-level Executor module for fine-grained decision-making. Building upon this, we design an experience-driven self-evolving loop, enabling autonomous strategy exploration and optimization from accumulated experience. We adopt a two-stage pipeline of offline pre-training and post-training alignment, and build an interactive bidding simulation environment for strategy rollout. Furthermore, we propose Decoupled Group Relative Policy Optimization (D-GRPO) to achieve end-to-end optimization via advantage decoupling. Experimental results on a large-scale public dataset demonstrate the effectiveness of AIGB-R1.

Abstract

Automated Bidding

Explore similar work

AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization

On the Role of Language Representations in Auto-Bidding: Findings and Implications

Generative Auto-Bidding with Unified Modeling and Exploration