Resource managers typically encode what to do in a task-specific policy---a heuristic, an optimization procedure, or a model trained for the task---that must be developed before use and may need revision as the workload or system changes. We propose bypassing policy development and instead asking pretrained models to make resource-management decisions directly, given the current system state, desired outcome, and available choices. We evaluate this approach on microservice autoscaling, DRAM sizing, and video encoding, using a frontier LLM and two small decision models. On two microservice applications, a small decision model meets latency targets while using within 10% of the replicas allocated by policy-based autoscalers and within 13% of the frontier LLM. In DRAM sizing, a small decision model reduces DRAM usage by up to 27.5% relative to static provisioning while maintaining at least 95% of baseline performance. In video encoding, it completes encodes up to 4.3× faster than the frontier LLM, with an imperceptible 1.1-point decrease in Video Multi-Method Assessment Fusion (VMAF), a metric of perceptual video quality. Making individual resource-management decisions may be considerably simpler than developing a general strategy for making them.
Figures & tables
Figure 1: Architecture common to the Dimwit autoscalers. At each tick, the autoscaler reads per-service utilization via the Kubernetes Metrics API and patches the service’s HPA object with the desired replica count R , causing HPA to adjust the service’s Deployment.
Parameter
Description
Choices
FFmpeg Default
subme
Precision of motion estimation and optimization of block encoding
2, 4, 7, 9
7
me
Motion search algorithm
dia, hex, umh
hex
ref
Number of reference frames used for prediction
1, 3, 5
3
trellis
Optimization of quantized values
0, 1, 2
1
bframes
Maximum number of consecutive B-frames
0, 3, 5
3
b_strategy
Algorithm for selecting B-frame positions
0, 1, 2
1
Table 1: x264 encoding parameters available to the LLMs, their allowed values, and the defaults under the medium preset.
Figure 2: Example frame from Elephants Dream Blender Foundation (2006) , shown uncompressed (left) and encoded at the same bitrate using two different x264 encoder configurations (center and right). The encoded versions achieve VMAF scores of 94.59 and 38.51, respectively, illustrating how encoder settings affect video quality.
Use case
Laya
Jev
Opus 5.5
Autoscaling (past decisions)
–
–
10
DRAM Sizing (1 s access samples)
3
3
10
Video Compression (past segments)
2
5
5
Table 2: Past observations in each model’s state. Opus 5.5 also sees its own past reasoning for those decisions.
Figure 3: Per-endpoint median p95 latency when horizontally scaling replicas using HPA, Erlang, Agar, Claude Opus 5.5 (low and high reasoning effort), Laya, and Jev on CloudLab. Jev-Single sends a separate request per service every 15 s, while Jev-Batched sends one batched request for all services every 15 s. The horizontal dashed line marks the 300 ms SLO . For each application, the boundary boxes show average running replicas, except for Jev-Single on Social Network and TeaStore and Laya on TeaStore, where they show requested replicas; excess pods remain Pending due to cluster capacity limits.
Workload
Inputs
RSS
Description
Liblinear Fan et al. (2008)
kddb
20.69
Logistic regression-based classifier.
GapBS-PR Beamer et al. (2015)
twitter
12.32
Compute PageRank score of a graph.
GapBS-CC Beamer et al. (2015)
kronecker
69.29
Compute connected components of graph
Silo Tu et al. (2013)
TPC-C
75.68
In-memory transactional database.
YCSB-C
71.40
Btree Achermann & Panwar (2020)
-
42.01
In-memory index lookup benchmark.
Table 3: DRAM sizing workload characteristics. Resident set size (RSS) is in GiB.
Figure 4: DRAM usage and performance achieved with TierSizer, Laya, Opus 5.5 (low and high reasoning effort), and Jev, normalized to static 2:1 DRAM provisioning. Bars show average DRAM usage and performance, while black markers indicate minimum DRAM usage (peak savings). The dashed red line marks the 95% performance target.
Figure 5: Video quality (mean VMAF) versus encoding time across video clips, with encoding parameters updated every 5 seconds. Laya achieves the lowest encoding time on all videos while maintaining quality close to FFmpeg medium . A difference of 6 VMAF points is commonly considered barely perceptible to human viewers Ozer (2017) . Smaller differences are generally imperceptible.
Video
Duration
Motion Description
Big Buck Bunny Blender Foundation (2008)
9:56
Colourful animation, smooth gradients, fine textures, moderate motion
Elephants Dream Blender Foundation (2006)
10:54
Dark, highly detailed mechanical scenes with complex camera movement
Sintel Trailer Blender Foundation (2010)
0:52
Fast-paced action, rapid cuts and dynamic camera movement
Table 4: Videos used in our encoding experiments and their motion characteristics. All source videos are 720p at 24 fps, obtained from Xiph.org Xiph.Org Foundation () .
Figure 6: Impact of decision interval, the time between encoding parameter updates, on encoding time and video quality for the three videos described in Table 4 . Lower encoding time and higher VMAF are better.
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.