LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting

Yewen Li; Zhiyi Lyu; Peng Jiang; Qingpeng Cai; Fei Pan; Bo An; Peng Jiang

cs.CLMar 5, 2026

LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting

Authors: Yewen Li, Zhiyi Lyu, Peng Jiang, Qingpeng Cai, Fei Pan, Bo An, Peng Jiang

Organizations: Kuaishou Technology Beijing, China · Nanyang Technological University Singapore, Singapore

Abstract

The growing scale of ad auctions on online advertising platforms has intensified competition, making manual bidding impractical and necessitating auto-bidding to help advertisers achieve their economic goals. Current auto-bidding methods have evolved to use offline reinforcement learning or generative methods to optimize bidding strategies, but they can sometimes behave counterintuitively due to the black-box training manner and limited mode coverage of datasets, leading to challenges in understanding task status and generalization in dynamic ad environments. Large language models (LLMs) offer a promising solution by leveraging prior human knowledge and reasoning abilities to improve auto-bidding performance. However, directly applying LLMs to auto-bidding faces difficulties due to the need for precise actions in competitive auctions and the lack of specialized auto-bidding knowledge, which can lead to hallucinations and suboptimal decisions. To address these challenges, we propose a hierarchical Large autoBidding Model (LBM) to leverage the reasoning capabilities of LLMs for developing a superior auto-bidding strategy. This includes a high-level LBM-Think model for reasoning and a low-level LBM-Act model for action generation. Specifically, we propose a dual embedding mechanism to efficiently fuse two modalities, including language and numerical inputs, for language-guided training of the LBM-Act; then, we propose an offline reinforcement fine-tuning technique termed GQPO for mitigating the LLM-Think's hallucinations and enhancing decision-making performance without simulation or real-world rollout like previous multi-turn LLM-based methods. Experiments demonstrate the superiority of a generative backbone based on our LBM, especially in an efficient training manner and generalization ability.

Figures & tables

Figure 1. Overview of the training scheme for our hierarchical large auto-bidding model. Training Stage I (left): train the LBM-Act with language-guided decision training via a dual embedding mechanism to fuse two modalities; Training Stage II (right): reinforcement fine-tuning the LBM-Think via group relative-Q policy optimization.

AuctionNet		USCB	CQL	IQL	BCQ	DT	DT-Q	DiffBid	DiffBid-Q	LBM(P)	LBM(GQPO)
Dense	Conversions ( $\uparrow$ )	267	336	322	321	364	371	316	319	376 $\pm$ 3.0	382 $\pm$ 3.3
Dense	Score ( $\uparrow$ )	157	171	281	270	329	334	214	260	320 $\pm$ 2.2	348 $\pm$ 2.3
Sparse	Conversions ( $\uparrow$ )	28.5	30.2	36.0	31.1	31.3	33.8	31.3	31.5	37.4 $\pm$ 0.3	38.5 $\pm$ 0.3
Sparse	Score ( $\uparrow)$	17.5	22.2	30.0	30.5	29.6	31.1	23.1	25.1	32.9 $\pm$ 0.2	33.4 $\pm$ 0.2

Table 1. Comparison between our LBM methods and previous non-LLM-based methods in AuctionNet.

	Method	Budget Utilization ( $\uparrow$ )		CPA Ratio ( $\downarrow$ )		Conversions ( $\uparrow$ )		Score ( $\uparrow$ )
	Method	Dense	Sparse	Dense	Sparse	Dense	Sparse	Dense	Sparse
LLM	Prompting	0.743	0.722	0.878	0.903	289	24.2	286	23.4
	SFT	0.765	0.741	0.882	0.804	297	27.0	286	26.9
	GRPO	0.860	0.773	0.945	0.816	327	29.9	292	29.7
	LLM-DT	0.902	0.861	0.936	0.847	366	36.0	328	32.5
	Prompt-LLM-DT	0.914	0.970	0.931	0.965	355	35.8	322	30.3

Table 2. Comprehensive evaluation of LLM-based auto-bidding methods in AuctionNet.

	DT-score					LBM
Reweight $w$	0	0.1	0.2	0.5	1.0
Budget Utlization ( $\uparrow$ )	0.884	0.839	0.818	0.780	0.632	0.938
CPA Ratio ( $\downarrow$ )	0.895	0.846	0.826	0.831	0.725	0.960
Conversions ( $\uparrow$ )	364	354	352	329	278	382
Score ( $\uparrow$ )	329	340	343	323	276	348

Table 3. Performance with various reward setting.

Panel of Figure 2 — Figure 2. Relationship between the CPA ratio and model behavior by 1000 random samples. When the CPA ratio exceeds 1, it indicates that the CPA constraint is not satisfied, suggesting a high likelihood of decreasing the bidding parameter, i.e. , $\Delta a<0$ . When the CPA ratio is smaller than 1, it will be more likely to increase the bidding parameter $\Delta a>0$ . However, this relationship is not clearly reflected in DT and pretrained LLM, but is more evident in LLM finetuned with GQPO.

Instruction	Budget Utilization( $\uparrow$ )	CPA Ratio( $\downarrow$ )	Conversions( $\uparrow$ )	Score( $\uparrow$ )
Increase	0.969	1.028	368	315
Decrease	0.894	0.924	373	325
base	0.938	0.960	382	348

Table 4. Language instruction’s impact on performance.

	Budget Ratio	0.5	0.75	1.0	1.25	1.5
Dense	DT	186	234	364	393	445
Dense	LBM(P)	198	285	376	462	520
Sparse	DT	18.2	27.5	31.3	43.5	50.0
Sparse	LBM(P)	21.3	28.5	37.4	43.6	50.6

Table 5. Generalization to various budget settings. Performance under conversions.

Figure 3. Training loss curve of LLM-DT and LBM-Act, supporting the efficiency of language-guided learning.

	Dense		Sparse
Model Size	Conversions	Score	Conversions	Score
3B	376	320	37.4	32.9
7B	375	320	37.3	34.6
32B	374	318	37.1	33.5

Table 6. Impact of LBM-Think’s model size.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4. Illustration of the inference procedure of LBM with a detailed case of the prompt and CoT. For a timestep $t$ , the LBM-Think first generates a CoT at timestep $t-1$ within the interval $\Delta t$ . Then, when timestep $t$ arrives, the LBM-Act receives the pre-generated CoT along with the detailed numerical sequence information to generate an action, which involves adjusting the bidding parameter.

Panel of Figure 5 — Figure 5. Visualization of attention scores varying across different positions and heads. For simplicity, we let positions from #0 to #6 as the language part, while the remaining positions are the numerical part. The two modalities are effectively fused using different attention heads. For instance, heads such as #4, #5, #6, and #10 in the final layer predominantly focus on the language part, whereas other heads are more attuned to the numerical parts. Interestingly, we observed that the attention in the middle layers distinctly separates the language parts (positions 0-6) from the numerical parts, with position 7 marking the beginning of the numerical part.

Params	AuctionNet	AuctionNet-Sparse
Trajectories	479,376	479,376
Delivery Periods	9,987	9,987
Time steps in a trajectory	48	48
State dimension	16	16
Action dimension	1	1
Action range	[0, 493]	[0, 8178]

Appendix

Table 7. The parameters of AuctionNet and AuctionNet-sparse.

	LBM-Think			LBM-Act
#params	3B	7B	32B	0.5B
Inference Latency	2.5s	3.6s	8.9s	63ms

Appendix

Table 8. Inference Latency of different modules.

Figure 6. Finetune the LLM using GRPO so that it first performs step-by-step reasoning and then outputs an action. However, after around 150 training steps, the model only outputs the action without performing additional reasoning.

Hyperparameters	Value
Batch size	128
Number of steps	400000
Sequence length	10
Learning rate	1e-4
Number of attention layers	6
Number of heads	8

Appendix

Table 9. The detailed hyperparameters of Q-value networks

Automated Bidding

Automated bidding studies algorithms that choose auction bids over time while balancing value, budgets, and performance constraints.

See all 36 papers

Jul 19, 2026

AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization

Auto-bidding plays an essential role in online advertising, automatically adjusting bids for advertisers to optimize their commercial goals. The emerging AI-Generated Bidding (AIGB) paradigm widely adopts generative modeling to optimize bidding strategies, yet suffers from the limited mode coverage of offline datasets and inadequate task-state understanding, hindering effective exploration of optimal strategies. Large Language Models (LLMs), with prior world knowledge and reasoning capabilities, offer a promising approach to overcome these limitations. However, directly applying LLMs to auto-bidding tasks faces inherent challenges in limited numerical precision, hallucinations, and inference latency. To address these limitations, we propose AIGB-R1, a hierarchical self-evolving auto-bidding framework aiming to enhance AI-Generated Bidding via LLMs' Reasoning capabilities, comprising a high-level Planner module for macro-level strategy planning and a low-level Executor module for fine-grained decision-making. Building upon this, we design an experience-driven self-evolving loop, enabling autonomous strategy exploration and optimization from accumulated experience. We adopt a two-stage pipeline of offline pre-training and post-training alignment, and build an interactive bidding simulation environment for strategy rollout. Furthermore, we propose Decoupled Group Relative Policy Optimization (D-GRPO) to achieve end-to-end optimization via advantage decoupling. Experimental results on a large-scale public dataset demonstrate the effectiveness of AIGB-R1.

Yuejia Dou, Hesong Wang, Xinyu Zhang +6

Gaoling School of Artificial Intelligence, Renmin University of China Beijing, China · Alibaba Group Beijing, China

Automated Bidding Large Language Model-Guided Optimization Online Advertising Self-Improving Agents RL for LLM Agents

Ask PDF

May 7, 2026

On the Role of Language Representations in Auto-Bidding: Findings and Implications

Auto-bidding is a crucial task in real-time advertising markets, where policies must optimize long-horizon value under delivery constraints (e.g., budget and CPA). Existing methods for auto-bidding rely on compact numerical state representations: while they can implicitly capture delivery dynamics, they offer limited support for explicitly representing and controlling high-level intent, evolving feedback, and operator-style strategic guidance in real campaigns. Meanwhile, Large Language Models (LLMs) offer a powerful method for encoding semantic information, it remains unclear when LLMs help and how to integrate them without sacrificing numerical precision. Through systematic preliminary studies, we find that (1) LLM embeddings contain bidding-relevant cues yet cannot replace numerical features, and (2) gains emerge only with careful semantic--numeric integration rather than naive concatenation. Motivated by these findings, we propose SemBid, a novel auto-bidding framework that injects LLM-encoded semantics into offline bidding trajectories at the token level. SemBid introduces three semantic inputs: Task, History, and Strategy. It injects these semantics as tokens alongside numerical trajectory tokens and uses self-attention to integrate them, improving controllability and generalization across objectives. Across diverse scenarios and budget regimes, SemBid outperforms competitive baselines from offline RL and generative sequence modeling, with more consistent gains in overall performance, constraint satisfaction, and robustness. Our code is available at: blue{here}.

Guanyu Zhu, Jining Luan, Hanwen Du +11

City University of Hong Kong · South China Agricultural University · University of Electronic Science and Technology of China +11

Automated Bidding Online Advertising

Ask PDF

May 19, 2026

Generative Auto-Bidding with Unified Modeling and Exploration

Automated bidding is central to modern digital advertising. Early rule-based methods lacked adaptability, while subsequent Reinforcement Learning approaches modeled bidding as a Markov Decision Process but struggled with long-term dependencies. Recent generative models show promise, yet they lack explicit mechanisms to balance exploration and safety, relying solely on action perturbations or trajectory guidance without a safety fallback. This results in inefficient exploration and elevated financial risk for advertising platforms. To address this gap, we propose GUIDE (Generative Auto-Bidding with Unified Modeling and Exploration), a framework that synergistically integrates directed exploration with a safe fallback mechanism. GUIDE employs a Decision Transformer (DT) to jointly model historical bidding actions and environmental state transitions. A Q-value module guides the DT's exploration via regularization constraints, while an Inverse Dynamics Module (IDM) leverages DT-predicted future states to infer robust, behaviorally consistent actions as a safe policy fallback. The Q-value module then adaptively selects the final action between these two options, balancing exploration and safety. Together, these components form an integrated "explore-safeguard-select" pipeline that unifies efficiency and safety. We conduct extensive experiments on public datasets, in simulated auction environments, and through large-scale online deployment on Taobao, a leading Chinese advertising platform. Results show GUIDE consistently outperforms state-of-the-art baselines across all scenarios. In real-world deployment, GUIDE achieves notable gains: +4.10% ad GMV, +1.40% ad clicks, +1.66% ad cost, and +3.52% ad ROI, demonstrating its effectiveness and strong industrial applicability.

Mingming Zhang, Feiqing Zhuang, Na Li +7

Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University · Taobao & Tmall Group of Alibaba

Automated Bidding Online Advertising Decision Transformer RL Exploration RL for Generative Models Safe RL

Ask PDF

Abstract

Automated Bidding

Explore similar work

AIGB-R1: Self-Evolving Generative Auto-Bidding via Hierarchical Planner-Executor Optimization

On the Role of Language Representations in Auto-Bidding: Findings and Implications

Generative Auto-Bidding with Unified Modeling and Exploration