AI clusters keep data in remote object stores and rely on local caches to stop GPUs from waiting for data. A cache performs well only when its prefetching and eviction policies match the access pattern, yet these patterns differ across AI workloads and change between phases of one workflow. Existing methods either need code changes, which compromises user transparency, or use low-end heuristics, which cannot follow such changes. Large language models (LLMs) can infer access behavior without user hints, but direct LLM control of cache actions suffers from long reasoning delay, high token cost, and poor decision quality. We present Chiron, a cache system based on advisory switching control. An LLM asynchronously diagnoses the access pattern and recommends a policy. A deterministic harness gates this advice, executes expert-designed policies locally, and returns cache feedback for review. We implement Chiron atop JuiceFS and evaluate it on nine AI workloads. It reduces job completion time by 14.3-41.2% over the best baseline on popular AI workloads.
Figures & tables
Figure 1: Caching across AI workloads and workflow phases.
Temporal patterns
Workloads
Example access sequence
Desired prefetching policy
Desired eviction policy
Sequential
LLM model loading Fu et al. (2024) , LLM tokenization Raffel et al. (2020) , ⋯
1.jpeg → 2.jpeg → 3.jpeg → 4.jpeg
Stride prefetching Chen and Baer (1995)
Eager eviction Li et al. (2008)
Random
Vision fine-tuning Dosovitskiy et al. (2021) , Speech fine-tuning Seo et al. (2021) , ⋯
10969.jpeg → 13625.jpeg → 19003.jpeg → 8077.jpeg
No prefetching Azure (2026)
No eviction Mohan et al. (2021)
Skewed
Recommendation training Naumov et al. (2019) , RAG retrieval Jin et al. (2025) , ⋯
A.csv → B.csv → A.csv → A.csv
Neighborhood prefetching Li et al. (2004)
LRU eviction Mattson et al. (1970)
Table 1: Representative file access patterns with example workloads, access sequences, and desired cache policies.
Figure 2: Caching performance (cache hit ratio) of low-end and code-intrusive adaptations on ResNet-50 workflow (W1 in Table 2 ). For code-intrusive, we manually select the most suitable prefetching or eviction policy for each workflow phase according to its access pattern.
Figure 3: Input/output interface of our end-to-end cache-control prototype.
Figure 4: Qwen3.8 Flash overhead over 128 calls per configuration. Left: latency CDF. Right: token cost including input and output tokens. Both include the two cases of thinking-on and thinking-off.
Figure 5: Cache-action quality. Prefetching accuracy is the fraction of prefetched blocks used before eviction or the end of the workload. Eviction accuracy is the fraction of evicted blocks next requested beyond the Belady boundary Song et al. (2020) or not requested again in the remaining trace.
Figure 6: Supervisory versus advisory switching control. (a) A trusted supervisor switches ( σ ) among controllers Ci based on output y . (b) In Chiron , the LLM only advises; a deterministic harness gates its advice, switches among policy tools, and returns feedback for review. Thick arrows carry switching authority; dashed arrows are asynchronous.
Figure 7: Chiron architecture.
Figure 8: The diagnosis interface combines fixed instructions and three groups of runtime input with a compact pattern-level response. Only key instructions and selected runtime prompt fields are shown; ellipses omit details given in Appendix A. The response is illustrative.
Figure 9: Patterns and confidence scores of diagnoses from Qwen3-Max (released in September 2025 and relatively dated compared with models we evaluate in Section 6 ) (left) and CHR with and without a confidence filter (right) during the training phase of the ResNet-50 workflow.
Figure 10: Diagnosis schedule tradeoff in the ResNet-50 workflow (W1 in Table 2 ): JCT and token cost under different diagnosis intervals.
ID
Workloads
Dataset
Dataset Size (Cache Budget)
Access Pattern
W1
ResNet-50 workflow He et al. (2016)
ImageNet subset ILSVRC ()
1.395 GiB (1 GiB)
Sequential, random, skewed
W2
Minimal-FNO workflow Takamoto et al. (2022)
PDEBench PDEBench ()
7.67 GiB (4 GiB)
Sequential, random, skewed
W3
Stable Diffusion 1.5 workflow Stable Diffusion ()
Mercity LAION subset Mercity ()
3.87 GiB (1 GiB)
Random, sequential
T1
Qwen tokenization task Qwen Team ()
C4 shards Allen Institute for AI ()
2.37 GiB (2 GiB)
Sequential
T2
Qwen2.5-3B weight loading task Qwen Team ()
Qwen2.5-3B weights Qwen Team ()
5.75 GiB (4 GiB)
Sequential
T3
ViT-Base fine-tuning task Dosovitskiy et al. (2021)
COCO 2014 subset NaiveDev ()
6.09 GiB (1 GiB)
Random
Table 2: Workloads named by model in the end-to-end experiment. IDs label the workloads in Figures 11 and 12 .
Figure 11: Five-policy comparison on multi-phase workflows W1–W3 in Table 2 . No-cache has zero CHR in all three workflows and is omitted from the CHR plot. JCT is normalized to JuiceFS within each workflow.
Figure 12: JCT on single-phase tasks T1–T6 in Table 2 , normalized to JuiceFS for each task.
Figure 13: Eviction and prefetch comparisons on the Stable Diffusion 1.5 workflow (W3 in Table 2 ).
Figure 14: Ablation studies. Structured statistics are evaluated on the Stable Diffusion 1.5 workflow (W3 in Table 2 ), normalized to full Chiron; confidence gating and feedback reflection are evaluated on the ResNet-50 workflow trace.
Figure 15: Diagnosis-model sensitivity on the ResNet-50 workflow replay.
Figure 16: Policy comparison in the large-scale dataset simulation. Cache capacity is expressed as a fraction of dataset bytes. JCT is normalized to JuiceFS at each capacity.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Field
Meaning
Definition
Scalar fields in both window and recent_tail
length
Number of accesses
n
unique_ratio
Distinct IDs per access
m/n
access_entropy
Diversity of ID frequencies
−∑u∈Upulog2pu/log2m if m>1 ; 0 otherwise
gini
Inequality of ID frequencies
mn∑j=1m(2j−m−1)c(j)
top_5_access
Share of accesses to the five most frequent IDs
n∑j=1min(5,m)c[j]
Appendix
Table 3: Structured statistics computed from the access sequence. Ratios with zero denominators are 0 unless an explicit fallback is shown. Ratios, entropy, and Gini are rounded to three decimal places; mean reuse gap is rounded to one. Lengths, step differences, and counts are integers.
Review-context field
Meaning
Value
review_trigger
Reason for requesting diagnosis
chr_anomaly if g=chr ; periodic_retry otherwise
previous_control_state
Earlier controller state
s
previous_committed_pattern
Earlier committed pattern
p∈P∪{null}
older_prefix_may_be_stale
Possible outdated access prefix
(g=chr)∨(s=transition)
recent_tail_length
Number of tail accesses
t
Appendix
Table 4: Review-context fields in Review context . The stale-prefix flag is a Boolean warning, not an observed phase boundary.
Figure 17: Sensitivity to the CHR reflection threshold on the ResNet-50 workflow. Both the drop and increase thresholds take the value shown. Recovery time is the mean time from a phase boundary until an accepted diagnosis commits the matching pattern.