Language Model Cascading

Ask about this topic

Latest papers

CardsList

22 papers3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. TRIAGE: Reliability- and Load-Conditioned Escalation for Two-Tier LLM Serving with a Semantic Evidence Judge

    Oct 31, 2026Basavaraj R M, Aryan Bharat KumarLLM ServingLLM Routing

    Independent Researcher

  2. CascadeLedger: Counterfactual Auditing of Information and Resource Integrity in LLM Cascades

    Oct 14, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +3Language Model CascadingLLM Auditing

    School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

  3. Evaluating Escalation Signals for LLM Routing: Targets, Controls, and Five Ways to Fool Yourself

    Oct 5, 2026Ramin Pishehvar, Andrea Morandi, Mahesh ViswanathanLLM RoutingLanguage Model Cascading

    Cisco Systems, Inc., San Jose, CA, USA

  4. HARISSA: Inference-Time Self-Checks for Efficient and Safe Local Language Model Deployment

    Sep 29, 2026Kenan Alkiek, Moontae Lee, David Jurgens +1On-Device Language Model InferenceSelective Prediction

    School of Information University of Michigan Ann Arbor, MI 48109, USA · SNU-LG AI Research Center Seoul, South Korea

  5. Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference

    Sep 7, 2026Zheyuan Wang, Siyu Li, Peiqiao Song +3LLM Inference EfficiencyLLM Routing

    School of Artificial Intelligence, Beijing Normal University · School of Information Science and Engineering, Chongqing Jiaotong University · School of Computer Science and Technology, Dalian University of Technology +1

  6. Cheap Verifiers, Large Blind Spots: Measuring the Reliability Cost of Cost-Saving Cascades

    Sep 1, 2026Dushyant Rajput, Nirdesh Chauhan, Siddharth KosarajuLLM Inference EfficiencyLLM Answer Verification

    AltSlate Labs LLP

  7. FrugalSOT - Frugal Search Over the Models

    Aug 21, 2026Pradheep P, Yuvanesh S, Harish KB +4Language Model CascadingOn-Device Language Model Inference

    Computer Science and Engineering Vellore Institute of Technology Vellore, India · Electronics and Communication Engineering Vellore Institute of Technology Vellore, India · Department of Analytics Vellore Institute of Technology Vellore, India +2

  8. RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learning

    Aug 16, 2026Shihong Huang, Shengjie Wang, Hong Ma +1LLM RoutingLanguage Model Cascading

    Polytechnic Institute, Zhejiang University, Hangzhou, China · Department of Logistics and Maritime Studies, Faculty of Business, The Hong Kong Polytechnic University, Hong Kong, China

  9. Share the Judge, Learn the Deferral: Where Specialization Helps LLM Evaluation

    Jul 30, 2026Weining ZhangLLM EvaluationLLM-as-a-Judge

    Cheung Kong Graduate School of Business

  10. Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference

    Jul 27, 2026Yifan Dou, Shikan Lian, Shibo LiLLM Inference EfficiencyLLM Inference Scheduling

    Department of Computer Science, Florida State University

  11. Cluster, Route, Escalate: Cascaded Framework for Cost-Aware LLM Serving

    Jun 25, 2026Yasmin Moslem, Magdalena Kacmajor, Vasudevan Nedumpozhimana +11LLM ServingCost-Aware Inference

    ADAPT Centre, Trinity College Dublin · Huawei Research

  12. RLM-Cascade: Response-Level Speculative Decoding for Cost-Efficient LLM API Serving

    Jun 22, 2026Haifeng Wu, Srinivasan Manoharan, Fangbo Tu +2LLM Inference ServingSpeculative Decoding

    PayPal

  13. SAFE-Cascade: Cost-Adaptive Vision-Language Routing for Chart Question Answering

    Jun 17, 2026Ayush Dwivedi, Qixin Wang, Ashvi Soni +5Multimodal UnderstandingEfficient VLM Inference

    Walmart USA · University of Arkansas USA

  14. Forced Deferral: Manipulating Routing Decisions in Multimodal LLM Cascades

    Jun 13, 2026Zhongye Liu, Yaopei Zeng, Yurui Chang +1Adversarial PerturbationsMultimodal Robustness

    Pennsylvania State University

  15. Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method

    Jun 6, 2026Kyoungmin Kim, Martin Catheland, Anastasia AilamakiSemantic SearchLLM Inference Acceleration

    EPFL Lausanne, Switzerland

  16. Is Escalation Worth It? On the Depth of LLM Cascades

    May 7, 2026Dylan BouchardCost-Aware InferenceLLM Routing

  17. RPRA: Predicting an LLM-Judge for Efficient but Performant Inference

    Apr 14, 2026Dylan R. Ashley, Gaël Le Lan, Changsheng Zhao +7Language Model CascadingLanguage Model Performance Prediction

    Meta Platforms, Inc., Menlo Park, United States of America · The Swiss AI Lab IDSIA (USI-SUPSI), Lugano, Switzerland · Università della Svizzera italiana, Lugano, Switzerland +2

  18. Streaming Model Cascades for Semantic SQL

    Apr 1, 2026Paweł Liskowski, Kyle SchmausCost-Aware InferenceLLM Routing

    Snowflake Inc. · Poznań, Poland · San Francisco, CA, USA

  19. Do Small Language Models Know When They're Wrong? Confidence-Based Cascade Scoring for Educational Assessment

    Mar 29, 2026Tyler BurleighLanguage Model CascadingAutomated Grading

    Khan Academy

  20. Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning

    Mar 4, 2026Chuang Zhang, Zizhen Zhu, Yihao Wei +5Language Model CascadingEfficient Language Model Reasoning

    Amazon Web Services · Tsinghua University

  21. Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey

    Feb 23, 2026Yasmin Moslem, John D. KelleherLLM RoutingEfficient Language Model Inference

    ADAPT Centre School of Computer Science & Statistics Trinity College Dublin Dublin, Ireland

  22. A Unified Approach to Routing and Cascading for LLMs

    Oct 14, 2024Jasper Dekoninck, Maximilian Baader, Martin VechevLLM RoutingLanguage Model Cascading

    Department of Computer Science, ETH Zurich, Switzerland.