Let the Agent Steer: Closed-Loop Ranking Optimization via Influence Exchange

Yin Cheng; Liao Zhou; Xiyu Liang; Dihao Luo; Tewei Lee; Kailun Zheng; Weiwei Zhang; Mingchen Cai

cs.AIMar 29, 2026

Let the Agent Steer: Closed-Loop Ranking Optimization via Influence Exchange

Authors: Yin Cheng, Liao Zhou, Xiyu Liang, Dihao Luo, Tewei Lee, Kailun Zheng, Weiwei Zhang, Mingchen Cai,

Organizations: Shopee

Abstract

Recommendation ranking is fundamentally an influence allocation problem: a sorting formula distributes ranking influence among competing factors, and the business outcome depends on finding the optimal "exchange rates" among them. However, offline proxy metrics systematically misjudge how influence reallocation translates to online impact, with asymmetric bias across metrics that a single calibration factor cannot correct. We present Sortify, the first fully autonomous LLM-driven ranking optimization agent deployed in a large-scale production recommendation system. The agent reframes ranking optimization as continuous influence exchange, closing the full loop from diagnosis to parameter deployment without human intervention. It addresses structural problems through three mechanisms: (1) a dual-channel framework grounded in Savage's Subjective Expected Utility (SEU) that decouples offline-online transfer correction (Belief channel) from constraint penalty adjustment (Preference channel); (2) an LLM meta-controller operating on framework-level parameters rather than low-level search variables; (3) a persistent Memory DB with 7 relational tables for cross-round learning. Its core metric, Influence Share, provides a decomposable measure where all factor contributions sum to exactly 100%. Sortify has been deployed across two markets. In Country A, the agent pushed GMV from -3.6% to +9.2% within 7 rounds with peak orders reaching +12.5%. In Country B, a cold-start deployment achieved +4.15% GMV/UU and +3.58% Ads Revenue in a 7-day A/B test, leading to full production rollout.

Figures & tables

Figure 1: Three-panel overview: (a) dual-channel architecture diagram, (b) online GMV/Orders uplift trend across rounds, (c) LLM correction convergence curve

Figure 2: Old vs. New Paradigm: Left—manual optimization loop (offline search, deploy, human judgment, discard state, repeat). Right—Sortify closed loop (offline search, deploy, auto-calibrate via dual channels, accumulate in Memory DB, repeat with improved priors). Highlights the structural differences: stateless vs. persistent, single-axis vs. dual-channel, manual vs. LLM-orchestrated.

Figure 3: Three-layer architecture diagram. Layer 1 (Human/Config) feeds objectives and constraints to Layer 2 (LLM + Algorithm), which contains the Belief channel, Preference channel, and LLM meta-controller. Layer 2 outputs calibrated target_range and penalty_weight to Layer 3 (Optuna TPE Search, 5000 trials $\times$ 25 workers). Layer 3 produces best parameters $\to$ Redis $\to$ online A/B $\to$ Memory DB $\to$ back to Layer 2.

Figure 4: Flow diagram showing: item pairs $\to$ pairwise score differences $\to$ per-factor share (sum=1) $\to$ rank-weighted aggregation $\to$ $I_{\text{gmv}}$ , $I_{\text{order}}$ , $I_{\text{ecpm\_term}}$ . Below: 7 parameters map into the sorting formula, changing factor contributions.

Figure 5: Two-axis diagram. Horizontal axis: Belief (target_range = position of constraint boundary). Vertical axis: Preference (penalty_weight = hardness of constraint boundary). Arrows show LMS moving along Belief axis continuously, LLM making discrete jumps along Belief axis, and violation pressure moving along Preference axis. The two axes are orthogonal — corrections on one do not affect the other.

Figure 6: Input assembly: 20-round episode history (offline/online metric pairs) + 30 prior update records (LMS and LLM correction history) + current penalty weights $\to$ Context JSON $\to$ LLM reasoning $\to$ Output: {delta_intercept, penalty_multiplier, reason, evidence_keys}. Two arrows from output: delta_intercept feeds into Belief channel (intercept update), penalty_multiplier feeds into Preference channel (weight scaling). Safety bounds shown as guard rails on both outputs.

Table	Role	Key Fields	Write Pattern
episodes	Fact table: one row per round with paired offline-online snapshots	episode_key, offline_metrics, online_uplifts, status	Append-only; status: offline $\to$ deployed $\to$ online
offline_candidates	Top-5 parameter candidates per round	episode_key, rank, params, metrics	Append per round
prior_relations	Current Belief channel state: slope & intercept per metric pair	relation_key, slope, intercept, eta	Upsert per round
prior_observations	Raw (offline, online, beta) triples per metric pair and episode	relation_key, episode_key, offline_uplift, online_uplift	Append with idempotency
prior_update_history	Belief channel audit trail	method (lms/llm), old/new slope, old/new intercept	Append-only
penalty_wt_upd_hist	Preference channel audit trail	constraint_metric, pressure, step, old/new weight	Append-only

Table 1: Memory DB schema overview.

Figure 7: Complete data flow across one YOLO cycle: (1) A/B data pulled $\to$ (2) Episode recorded in Memory DB $\to$ (3) LMS reads prior_observations, updates prior_relations $\to$ (4) LLM context assembled from episodes + update histories $\to$ (5) LLM proposal applied to intercepts and penalty weights $\to$ (6) Target ranges and penalty weights feed into Optuna objective $\to$ (7) 5000-trial search $\to$ (8) Best parameters published to Redis $\to$ (9) New A/B data generated $\to$ back to (1). Memory DB is the central state store connecting all steps.

Component	Specification
Search engine	Optuna TPE sampler Akiba et al. (2019)
Trials per round	5,000
Parallel workers	25 (Constant Liar strategy)
State database	SQLite ( agent_memory.db )
Parameter publishing	Redis key-value store
Search time per round	15–30 minutes

Table 2: System configuration.

Figure 8: YOLO Continuous Loop: State machine diagram with three states — Coldstart (bootstrap params, anchor time window), Initial (first full pipeline with manual confirmation), YOLO (infinite loop: wait 3.5h, pull A/B, pipeline, publish, update state, loop). Retry logic shown as a self-loop on the A/B pull step (up to 5 retries). Exit arrow from non-A/B failures. Recovery arrow from restart reading yolo-state.env back into YOLO state.

	Exp-401838	Exp-437160
Market	Country A PDP	Country B PDP
Rounds	7 (warm-start)	23 (V1: 11 + V3: 12)
Duration	25 hours	V1: 03-05–07; V3: 03-10–13; freeze 03-14; 7d AB 03-15–22; rollout 03-24
Start cond.	Warm-start from existing Memory DB state	Cold-start, default priors
Obs. window	3.5 h/round	6 h/round (V3); 7d static AB post-freeze
Proc. time	25–50 min/round	25–50 min/round

Table 3: Experiment overview.

Round	GMV	Orders	Ads Revenue	Advertiser Value
R2	$-$ 3.6%	0.0%	+0.7%	+7.9%
R3	$-$ 1.5%	+1.7%	$-$ 0.8%	$-$ 1.0%
R4	+0.2%	+1.7%	$-$ 8.7%	$-$ 8.1%
R5	+1.9%	+3.8%	$-$ 5.8%	$-$ 5.6%
R6	+0.9%	+2.7%	$-$ 9.4%	$-$ 10.8%
R7	+9.2%	+12.5%	+5.7%	$-$ 8.9%

Table 4: Exp-401838 online A/B results — round-by-round GMV and Orders progression.

Figure 9: Online KPI Trends: Single-panel line chart showing Exp-401838 GMV and Orders uplift across 7 rounds. Dashed horizontal line at 0% for reference. Highlights the R4–R7 consecutive positive region and the R7 peak results.

Round	$I_{\text{gmv}}$ Uplift	Online GMV	Observation
R2	+18.2%	$-$ 3.6%	Significantly optimistic offline; positive offline uplift did not translate to positive GMV
R7	+41.6%	+9.2%	Optimistic bias remains, but after calibration the offline gains begin to translate reliably into positive online GMV

Table 5: Representative offline-online gap in Exp-401838.

Figure 10: Offline-Online Transfer Gap Visualization: Scatter plot with Exp-401838’s per-round offline $I_{\text{gmv}}$ uplift on the x-axis and corresponding online GMV on the y-axis. Identity line ( $y=x$ ) shown for reference. Points fall well below the identity line, showing systematic optimistic bias; later rounds are closer to the positive quadrant.

Statistic	Exp-401838
ps_ads_wo range	1.96–8.91
Oscillation ratio (max/min)	4.5x
Complete swing cycles	2
Second cycle narrower than first	Yes

Table 6: Exp-401838 parameter oscillation statistics.

Figure 11: Parameter Evolution Heatmap: Heatmap with 7 parameters (rows) $\times$ 7 rounds (columns) from Exp-401838. Color scale from low to high values. The ps_ads_wo row shows the most pronounced back-and-forth oscillation, but the later half has a narrower swing range.

Phase	Exp-401838
Early (R2)	5 items, full recalibration
Middle (R3–R6)	2–3 items, decreasing magnitudes
Late (R7)	2 items, smallest magnitudes ( $-0.005$ )
Convergence trend	Clear convergence : 5 $\to$ 2 items, magnitude $\to$ $\sim$ 0

Table 7: Exp-401838 LLM correction convergence.

Figure 12: LLM Correction Convergence: Two-panel visualization. Top panel: bar chart showing the number of non-zero LLM correction items per round for Exp-401838, descending from 5 at R2 to 2 at R7. Bottom panel: line chart showing maximum correction magnitude per round, with an overall declining trend.

Metric	Value	Constraint / Target	Status
Offline objective	0.493	Maximize	Achieved
$I_{\text{gmv}}$ uplift	+4.9%	$\geq$ +4%	Met
$I_{\text{ecpm\_term}}$ change	$-4.5$ %	$\geq$ $-5$ %	Near boundary
ads_top10_rate	—	Constrained	Met
ps_org_wc change	+271.2% (1.0 $\to$ 3.71)	—	Largest adjustment
ratio_ads change	$-32.0$ %	—	Ad share compressed

Table 8: Exp-437160 cold-start results (Country B, V1 Round 1).

Parameter	Value	Interpretation
ps_org_wc	7.667	Organic Click weight (dominant)
ps_ads_wc	4.381	Ads Click weight (high)
ps_org_wg	0.568	Organic GMV weight (suppressed)
ps_org_wo	0.805	Organic Order weight (suppressed)
ps_ads_wg	0.641	Ads GMV weight (suppressed)
ps_ads_wo	0.957	Ads Order weight (moderate)

Table 9: Exp-437160 frozen parameters (R7, Country B).

Metric	Diff	Note
GMV/UU	+4.15%	Primary KPI
GMV	+4.10%	Consistent with GMV/UU
GMV/Order	+3.97%	Average order value drives GMV
Order/UU	+0.17%	Near-neutral
Ads Revenue	+3.58%	Ad revenue co-positive
CPC	+4.19%	Higher per-click value

Table 10: 7-day A/B core metrics (YMAL scene, yin_exchange vs Control).

Metric	Value	Notes
Total cycle time	$\sim$ 4 hours	3.5h accumulation + 25–50min processing
Daily optimization rounds	$\sim$ 6/day	4h/round cadence
Trials per round	5,000	25 workers; 15–30min search
Human intervention (post-coldstart)	Zero	Within the inner optimization loop
Consecutive unattended rounds	7; 23	Exp-401838 and Exp-437160; governance-limited, not failure-limited
A/B fetch retry success	5 $\times$ 5 min	No data loss observed

Table 11: Operational efficiency metrics.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition	Introduced
$\pi_{q}(\boldsymbol{\theta})$	Ranking permutation for request $q$ under parameters $\boldsymbol{\theta}$	Eq. 1
$S_{n_{q}}$	Symmetric group acting on $n_{q}$ items	Eq. 1
$\mathbf{S}_{q}(\boldsymbol{\theta})$	Total score vector for request $q$	Eq. 2
$H_{qij}$	Comparison hyperplane $x_{i}=x_{j}$ for items $i,j$	Eq. 2
$C_{\pi}$	Ranking chamber corresponding to permutation $\pi$	Eq. 2
$\boldsymbol{\theta}$	7-dimensional sorting parameter vector	Eq. 10

Appendix

Table 12: TAB-APP-02. Symbol definitions.

Abbreviation	Full Form
GMV	Gross Merchandise Value
PDP	Product Detail Page
TPE	Tree-structured Parzen Estimator
LMS	Least Mean Squares
YOLO	You Only Live Once (continuous pipeline mode)
KPI	Key Performance Indicator

Appendix

Table 13: Abbreviations.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 514 papers

SR-Agent: An Experience-Driven Agentic Framework for Post-Ranking Strategy Refinement in E-Commerce Recommendation
Jul 20, 2026Hanchen Yang, Kaiwen Yang, Junpeng Zhuang +7Recommender Systems Self-Improving Agents
Kuaishou Technology, Beijing, China
Decoding ML Decision: An Agentic Reasoning Framework for Large-Scale Ranking System
Feb 20, 2026Longfei Yun, Yihan Wu, Haoran Liu +13Learning to Rank
Meta, Menlo Park, United States
RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Jul 31, 2026Haoran Ling, Yuecheng Li, Zeyu Song +5Large Language Model-Guided Optimization Agent Harness Optimization
Georgia Institute of Technology · Kuaishou Technology, China

Jul 20, 2026J/K move · Enter open · S save

SR-Agent: An Experience-Driven Agentic Framework for Post-Ranking Strategy Refinement in E-Commerce Recommendation

Hanchen Yang, Kaiwen Yang, Junpeng Zhuang, Yang He+6

Kuaishou Technology, Beijing, China

Recommender Systems Self-Improving Agents

Open paper AskPDF arXiv

User experience is a first-class objective in industrial e-commerce recommender systems (RS). Post-ranking strategies, which govern diversity, similarity, and exposure over a ranked list, are widely deployed in industrial RS for their simplicity and low serving cost. However, as the online recommendation environment evolves continuously, these statically configured strategies gradually become stale, thereby degrading the user experience. Refining them typically relies on manual inspection, diagnosis, and updates, making it slow, costly, and difficult to scale or reuse. Although recent LLM-based agents (e.g., RecUserSim, SimUSER, and Self-EvolveRec) offer promising directions, none of them close the full loop of automated, self-evolving strategy refinement. To bridge this gap, we introduce SR-Agent, which, to the best of our knowledge, is the first agentic framework deployed to refine post-ranking strategies in industrial RS. SR-Agent unifies three components: (i) a UserSim agent that applies inspection skills to surface user-perceived bad cases; (ii) an Analysis agent that consolidates recurring bad cases into structured, reusable diagnoses; and (iii) a constrained Strategy Refinement Harness that maps diagnoses to typed and bounded actions, gated by a four-stage reward pipeline with reversible rollback. Deployed on the Kuaishou e-commerce platform, SR-Agent continuously runs this refinement loop and, in a one-month online A/B test, increases order volume by 0.71%, browsing depth by 0.34%, and clicked-category diversity by 0.48%, while markedly shortening the refinement cycle and lowering operational cost.

Abstract

Recommender Systems

Explore similar work

SR-Agent: An Experience-Driven Agentic Framework for Post-Ranking Strategy Refinement in E-Commerce Recommendation

Decoding ML Decision: An Agentic Reasoning Framework for Large-Scale Ranking System

RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems