We study coupon allocation on an online marketplace offering multiple service categories. Issuing coupons requires choosing a vector of category-specific discounts, accounting for both their direct effects and cross-category demand shifts. We present TaQFormer, a treatment-as-query Transformer for this multidimensional treatment setting. Each treatment coordinate is represented as a query token, allowing cross-attention to estimate the response to the full treatment vector. To combine large but policy-biased observational logs with a smaller stream of randomized controlled trial data, we use a two-stage pipeline: observational pretraining learns general outcome patterns, followed by LoRA fine-tuning on RCT data with a request-specific fusion gate. We evaluate the resulting system through the checks required for production, including offline uplift metrics, treatment-parallel serving, latency monitoring, and a 14-day online A/B test. At essentially the same subsidy cost rate, \model increases business value per user by 1.26% over the production baseline and significantly outperforms both the production baseline and the RCT-only model, while meeting the latency budget. It was subsequently rolled out to all marketplace traffic and has operated at full scale for several months as of October 2026. The deployment illustrates how multidimensional treatment modeling and observational pretraining, combined with adaptation using RCT data, can support effective joint discount decisions in production.
Figures & tables
Figure 1. TaQFormer architecture: (a) focal-category response prediction with a treatment decoder followed by a separate focal-response readout, and an auxiliary branch; (b) pre-norm self-attention with token-wise FFNs; (c) a Treatment Decoder Block in which decoded treatment states query request memory. Three-panel diagram of the \modelarchitecture: the overall data flow from input features through the Encoder Block stack to the treatment decoder, focal-response readout, and auxiliary head; the internal structure of an encoder block; and the internal structure of one treatment decoder block.
Figure 2. TaQFormer training: (a) OBS pretraining of all modules; (b) RCT adaptation with a frozen OBS reference, a shared frozen Encoder Block prefix, and a LoRA-adapted suffix. Only LoRA factors, a newly initialized RCT prediction stack (including its treatment encoder), and gates are updated. Gate inputs use stop-gradient; focal logits are fused, while the direct auxiliary output comes from the RCT stack. Two-panel diagram of the two-stage training pipeline: observational pretraining of the full model, followed by randomized adaptation in which a frozen observational route and a newly initialized RCT prediction stack receive frozen-base and LoRA-adapted representations, respectively. An instance-wise gate fuses focal logits; the RCT stack provides the direct auxiliary output.
Group
Model
Qini
AUC
PCOC
Baseline
PEPNet
0.1612
0.6776
1.3737
External
EFIN (adapted)
0.2129
0.6938
0.9802
External
TransTEE (adapted)
0.2612
0.7362
1.0223
Ablation
flat MLP + Treatment Decoder
0.2592
0.7184
1.1395
Ablation
Encoder Block + gated towers
0.2536
0.7391
0.9446
Encoder
Wukong + Treatment Decoder
0.2391
0.7051
0.9608
Table 1. Single-stage comparison after one epoch on the randomized stream. Runs share data, seed, batch size, and GPU class and use the same held-out RCT week; learning-rate schedules and auxiliary losses differ. Higher is better for Qini and AUC; PCOC is best at 1.0. Bold marks the best value per column; alternative-encoder rows probe encoder choice and are excluded from bolding.
Figure 3. Single-stage TaQFormer capacity comparison after one RCT epoch. Width: dmodel∈{16,32,64,128} at setting L2 ; depth settings: L1 – L4 use 2, 4, 6, and 8 Encoder Blocks and 1, 2, 3, and 4 Treatment Decoder Blocks, respectively, at dmodel=64 . Open marker: deployed L2 configuration (four backbone blocks, two blocks per treatment decoder; 34.5M parameters). The validation loss is the primary BCE. Single seed, one epoch. Three panels compare primary BCE, Qini, and AUC across model widths and depth settings. Open markers identify the deployed configuration with four backbone blocks and two blocks per treatment decoder.
Training variant
Qini
AUC
PCOC
Single-branch training
OBS only
0.2548
0.7382
1.0315
RCT only
0.2697
0.7393
1.0012
OBS + RCT (shuffled)
0.2665
0.7398
1.0136
OBS → RCT (full fine-tuning)
0.2673
0.7421
1.0114
Frozen-reference adaptation
Table 2. Training-strategy comparison on the same held-out RCT week as Table 1 . Single-branch rows share the response-model architecture; the lower group retains a frozen OBS reference. In the lower group, encoder full fine-tuning updates the RCT Encoder Block suffix directly, whereas direct residual uses s=sobs+srct . Higher is better for Qini and AUC; PCOC is best at 1.0. Bold marks the best value in each column across all rows.
Model
Business value index
Subsidy rate
PEPNet
100.00
20.43%
RCT-only
101.08∗
20.45%
Two-stage (TaQFormer)
101.26∗†
20.42%
Table 3. Online A/B results. Business value per user is indexed to PEPNet =100 . Superscripts ∗ and † indicate significant gains over PEPNet and RCT-only, respectively ( p<0.05 ).
In online advertising, marketing interventions such as coupons introduce significant confounding bias into Click-Through Rate (CTR) prediction. Observed clicks reflect a mixture of users' intrinsic preferences and the uplift induced by these interventions. This causes conventional models to miscalibrate base CTRs, which distorts downstream ranking and billing decisions. Furthermore, marketing interventions often operate as multi-valued treatments with varying magnitudes, introducing additional complexity to CTR prediction. To address these issues, we propose the Unified Multi-Valued Treatment Network (UniMVT). Specifically, UniMVT disentangles confounding factors from treatment-sensitive representations, enabling a full-space counterfactual inference module to jointly reconstruct the debiased base CTR and intensity-response curves. To handle the complexity of multi-valued treatments, UniMVT employs an auxiliary intensity estimation task to capture treatment propensities and devise a unit uplift objective that normalizes the intervention effect. This ensures comparable estimation across the continuous coupon-value spectrum. UniMVT simultaneously achieves debiased CTR prediction for accurate system calibration and precise uplift estimation for incentive allocation. Extensive experiments on synthetic and industrial datasets demonstrate UniMVT's superiority in both predictive accuracy and calibration. Furthermore, real-world A/B tests confirm that UniMVT significantly improves business metrics through more effective coupon distribution.
Siyun Yang, Shixiao Yang, Jian Wang +6
Kuaishou Technology Beijing, China · Beijing Institute of Technology Beijing, China · Independent Researcher Beijing, China
Large-scale targeting and recommendation systems are typically built around predictive scores fed into heuristic or local allocation. When the business goal is incremental impact, as in marketing campaigns, incentives, and notifications, this paradigm systematically misallocates resources toward users who would have acted anyway. We present a decision-centric framework that instead optimizes causal effects under global constraints, aligning three components under a single objective: a causal neural network with a Transformer backbone for individual treatment-effect estimation, a Bayesian neural-bandit layer for uncertainty-aware exploration, and a dual-based large-scale linear-programming layer for constrained allocation. The framework also supports sequential context and multi-outcome, attribute-conditioned scoring through a Transformer encoder and outcome embeddings. We evaluate it with offline simulations on a public bandit dataset, targeted architectural ablations, and an online A/B test on LinkedIn Feed marketing traffic. We also distill production lessons on causal training-data construction and cost and delivery control, which were critical to successful deployment. The end-to-end treatment policy delivered a statistically significant +7.20% lift in the primary long-term-value metric, demonstrating the feasibility of production-scale causal optimization under business constraints.
Post-click conversion rate (pCVR) prediction requires jointly modeling feature interactions and sequential user behaviors. The KDD Cup 2026 Tencent UniRec Challenge calls for a unified architecture addressing both. We observe that existing unified architectures often generate query tokens---the central information hub---with projection-based multi-layer perceptrons (MLPs), without explicit token-to-query attention for refining the query side. We propose QueryFormer, centered on a stackable unified field--sequence block that bridges non-sequential multi-field features and behavioral sequences, and provide a latency-aware scaling study over view width H, model width, depth, data, and compute. The block generates queries through cross-attention and packs sequence queries into shared-parameter attention. QueryFormer secured 1st place in the Industrial Track, achieving an official test area under the ROC curve (AUC) of 0.83254; a modest post-competition scale-up reached 0.832713. Within our grid, H-scaling improves validation AUC from 0.84540 to 0.84615 and beats HyFormer at comparable budgets. Ablation identifies query generation as the largest contributor. Packed shared-parameter cross-attention keeps H=8 inference latency to only 1.89x that of H=1, positioning the bridge as an efficient stackable unified block.