cs.PFNov 1, 2026

DIMWIT: Directly Invoking Models Without Intensive Tuning

Authors: Saarth Deshpande, Neeraja J. Yadwadkar, Venkat Arun

Organizations: The University of Texas at Austin

Abstract

Resource managers typically encode what to do in a task-specific policy---a heuristic, an optimization procedure, or a model trained for the task---that must be developed before use and may need revision as the workload or system changes. We propose bypassing policy development and instead asking pretrained models to make resource-management decisions directly, given the current system state, desired outcome, and available choices. We evaluate this approach on microservice autoscaling, DRAM sizing, and video encoding, using a frontier LLM and two small decision models. On two microservice applications, a small decision model meets latency targets while using within 10% of the replicas allocated by policy-based autoscalers and within 13% of the frontier LLM. In DRAM sizing, a small decision model reduces DRAM usage by up to 27.5% relative to static provisioning while maintaining at least 95% of baseline performance. In video encoding, it completes encodes up to 4.3×\times faster than the frontier LLM, with an imperceptible 1.1-point decrease in Video Multi-Method Assessment Fusion (VMAF), a metric of perceptual video quality. Making individual resource-management decisions may be considerably simpler than developing a general strategy for making them.

Figures & tables

Explore similar work

CardsList
  1. Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement

    Dec 5, 2025Nils Strassenburg, Boris Glavic, Tilmann RablCost-Aware InferenceSmall Language Models

  2. DWISE: Denoising Workload-Informed Scheduling Engine

    Oct 29, 2026Lie Li, Wen Li, Junxiao Shen +2Diffusion Model ServingDiffusion Language Model Inference

  3. End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inference

    Jun 26, 2026Yuhang Chen, Jinhao Duan, Ruichen Zhang +11LLM InferenceStructured Sparsity