cs.IRNov 19, 2026

Cost-Aware Agentic Feature Optimization System for Fleet-Scale Ranking Models

Authors: Chang Yang, Yao Xuan, Wendan Yan, Nhat Le, Jiang Liu, Longhao Jin, Zhen Peggy Yao, Alireza Vahdatpour, 

Organizations: Meta, Inc., Sunnyvale, USA · Meta, Inc., Bellevue, USA

Abstract

Large Web platforms operate large fleets of ranking models, each selecting features from a shared pool of tens of thousands. We present an end-to-end agentic system that optimizes features across the fleet under resource constraints shared across models and those set per model, such as feature pipeline cost and serving throughput (QPS), respectively. In this system, LLM agents propose hypotheses, and deterministic tools execute experiments and verify them. For each model, an LLM generates and shortlists cross features from a combinatorial candidate space, and all features are then ranked by several importance methods. Feature blending mixes these rankings into a single selection, using Pareto fronts across tasks for multi-task models. An LLM Researcher agent then tunes the feature selection in 10--25 asynchronous trials under resource caps, and a trimming step keeps the pipeline cost within budget. Across production models, the system improves normalized entropy by 0.31% on average within each model's budget, and the trimming step cuts pipeline cost by 2.3% with no regression beyond measurement noise. The optimized models were launched, and in online A/B tests they improved a primary online metric by 0.20%.

Figures & tables

Explore similar work

CardsList
  1. AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent

    Apr 7, 2026Wenyue Hua, Sripad Karne, Qian Xie +4LLM Agent Workflow OptimizationLLM Agent Efficiency

  2. Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

    Jul 29, 2026Yicheng Feng, Yan Zhang, Yan Cheng +1Decision-Focused LearningTool-Using Agents

  3. Which Model for Which Role? Cost-Efficient Pareto Recommendation for LLM Agentic Workflows

    Oct 19, 2026Qian Xie, Wenyue Hua, Yueli He +7LLM Agent Workflow OptimizationMulti-Objective Optimization