cs.IRNov 7, 2026

Slim Foundation Models for Latency-Critical Recommendation Serving

Authors: Danyal Aftab, Anas Zafar, Assadig Abakr, Steven Davy

Organizations: Technological University Dublin, Dublin, Ireland · National University of Computer and Emerging Science, Karachi, Pakistan

Abstract

Click-through rate (CTR) prediction is a core component of recommendation and advertising systems. Large language models (LLMs) offer rich semantic priors that improve prediction quality, particularly in cold-start and long-tail settings. Injecting a full-scale LLM into a CTR pipeline is, however, prohibitively expensive at inference time: existing LLM-enhanced CTR methods avoid this cost by caching LLM outputs offline rather than serving the LLM online. We propose a resource-efficient pipeline, SlimFM, that makes online LLM-based semantic modeling practical: (1) dependency-aware structured pruning compresses LLaMA3-8B by 44% to a 4.5B-parameter backbone; (2) LoRA-based field-aware fine-tuning trains the pruned model to produce discriminative field-level semantic embeddings from lightweight textual prompts; (3) a semantic-guided fusion module injects these embeddings into the feature interactions of a conventional CTR model. Across four datasets and six CTR backbones, SlimFM retains over 99% of uncompressed-model accuracy while reducing inference latency by up to 3.4x relative to existing LLM-enhanced baselines.

Figures & tables

Explore similar work

CardsList
  1. LLM-HYPER: Generative CTR Modeling for Cold-Start Ad Personalization via LLM-Based Hypernetworks

    Apr 13, 2026Luyi Ma, Wanjia Sherry Zhang, Zezhong Fan +10CTR PredictionHypernetworks

  2. Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement

    Dec 5, 2025Nils Strassenburg, Boris Glavic, Tilmann RablCost-Aware InferenceSmall Language Models