cs.LGNov 15, 2026

Microfitting: Continual Learning via Cached Test-Time Training

Authors: Nick Ryan, Peter A. Devine, Ben Sirb, Hannah Sansford, Alex Chiocchi

Organizations: Interval

Abstract

Continual learning is a promising approach to specialize language models to an organization's private data. However, current methods exhibit a number of drawbacks including catastrophic forgetting, where new training updates overwrite prior knowledge. We present microfitting, which trains and caches a small adapter for each document. Each adapter is trained once, offline, on synthetic questions targeted at demonstrated base-model shortcomings and reused whenever that document is queried. This method borrows from test-time learning literature and sidesteps the problem of continual learning through parameter isolation, allowing the system to specialize on new data without overwriting prior knowledge. The process that generates adapter training data also improves over time, revising its recipe from previous outcomes and fresh examples of the target workload. On DeepSeek V4 Flash, microfitting substantially improves accuracy over the base model. In Enterprise Document Operations (EDO), an open-book Q&A environment over real-world enterprise documents, our method achieves a 38.2% relative improvement over base-model performance. Microfitting makes continual specialization practical and cost-efficient: a frozen model can adapt to an evolving private corpus overnight on local hardware, without modifying the base model or previously cached adapters.

Figures & tables

Explore similar work

CardsList
  1. Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

    Jul 26, 2026Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli +2Continual Learning for LLMsContinual LoRA

  2. Improving Sparse Memory Finetuning

    Apr 6, 2026Satyam Goyal, Anirudh Kanchi, Garv Shah +1Continual Learning for LLMsSparse Fine-Tuning

  3. Mashup Learning: Faster Finetuning by Remixing Past Checkpoints

    Mar 10, 2026Sofia Maria Lo Cicero Vaina, Artem Chumachenko, Max RyabininLLM Fine-TuningLLM Training