cs.AIOct 28, 2026

Decompose, Don't Expand: Capability-Independent AI Systems by Construction

Authors: Tejaswi Gowda

Organizations: Arizona State University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A recurring design choice in building AI systems is whether, when a task is too hard for a model, to expand the model (make it larger, better-prompted, fine-tuned), or to decompose the task (extract the parts that admit a deterministic solution to a host, leaving the model only the irreducibly fuzzy residue). We make three claims. First, that many tasks commonly delegated whole to a model are in fact deterministic tasks with a small generative core. Second, and our central empirical result, that decomposing such a task yields a measurable and desirable property we call parity by construction: on the extracted sub-tasks the model-size performance curve becomes flat, because no model performs them at all. We measure this across nine models spanning a 494M-parameter local checkpoint to a frontier Opus-class model, a thirty-fold parameter range on the local family alone: once the host resolves the context, operation selection scores identically for all nine models and selector resolution for eight of nine, while the one task that does not decompose, labelling, remains sloped across the same range. Because the task leaves the model, a stock checkpoint suffices: no task-specific training and no fine-tuning. Third, that this is a general design discipline, not a domain trick: we show the same move recurring across physics setup, motion retargeting, and object-model editing, and in a Web-CLI tool family (same author, so not an independent replication). We give a method for locating the cut, the construction that makes a cut exist (a deterministic surface and a closed grammar over it), and a way to make the residue safe (resolve the deterministic context; verify the generative output), position the work against neuro-symbolic AI, tool-augmented language models, and generate-and-verify approaches, and are explicit about what is measured (one domain) versus illustrative (the rest).

Figures & tables

Figure 1: The construction. A deterministic surface (bottom; a small labelled object graph) makes answers computable. A DSL over it (middle) makes the model’s output bounded. The three drivers (top) enter through the same DSL; they share one model of the scene and one history/undo, which runs beneath the surface.
Figure 2: The two-way gate. A natural-language request passes through a deterministic host, which resolves the selector and operation and hands them to the model as fixed context. The model supplies the residual arguments and ambiguous choices, and the result is executed as an undoable scene update. A manual path, $S(selector).op(args) , bypasses the model entirely.
task0.5B1.5B3B7B14BHaikuSonnetOpusFable
op-selection9%76%77%95%95%95%100%95%100%
selector-resolution5%55%36%41%45%59%59%50%68%
arg-extraction68%91%86%95%95%95%95%95%95%
multi-op0%54%46%38%62%77%77%69%85%
labelling67%78%67%78%89%89%89%100%100%
overall27%71%64%72%77%83%84%81%89%
Table 1: Scaffolded baseline, before host resolution: the model composes the whole task. Local checkpoints at Q4_K_M; frontier models through the Anthropic API. Where more than one repeat exists the mean is reported; the 1.5B column is the mean of three repeats and the spread is at most 5 points on any task.
Figure 3: Work removed from the model across the harness ladder: a bare request with the scene, a scaffolded prompt with real selectors and the op vocabulary, constrained JSON-schema output, and host-resolved selector and op. All four rungs send a prompt of comparable size (10–11.5k tokens for the local tokenizer); the rungs differ in what the prompt contains , not in how much of it there is.
rung1.5BHaikuSonnetFable
bare55%43%53%57%
scaffolded71%83%84%89%
constrained70%81%77%83%
host-resolved88%86%89%90%
Table 2: Overall score at each rung of the harness ladder (Figure 3 ), for the four models measured at all four rungs. The ladder is not monotonic: constrained output scores below scaffolded for every one of these models, and improves only on bare. Host resolution is the only rung that produces a flat curve across models.
task0.5B1.5B3B7B14BHaikuSonnetOpusFable
op-selection86%86%86%86%86%86%86%86%86%
selector-resolution91%91%91%91%91%86%91%91%91%
arg-extraction86%91%91%91%91%95%91%91%91%
multi-op85%85%85%85%85%77%85%77%85%
labelling †67%78%67%78%89%78%89%100%100%
overall85%88%86%88%89%86%89%89%90%
Table 3: After host resolution. Scores on each task, scored on resolved nodes. Labelling ( †\dagger ) is not decomposed; it is the residual the model still owns. Op-selection is identical for all nine models and selector-resolution for eight of nine.
modelop-sel.selectorarg-extr.multi-oplabelling
1.5B22/2222/2221/2213/138/9
3B22/2222/2221/2213/137/9
7B22/2222/2221/2213/138/9
14B22/2222/2221/2213/137/9
Haiku20/2221/2220/2212/138/9
Sonnet22/2222/2221/2213/137/9
Table 4: Fixture-level agreement with the 0.5B checkpoint under host resolution. Each cell counts fixtures on which the two models reach the same outcome.
Figure 4: Where the labels come from. Everything downstream of the surface is resolved by the host. The one task that does not decompose, labelling, is the step that would replace the author: the measured residue (Table 3 ).
Figure 5: The same construction in a second domain. Left: Strata. Right: the Web-CLI family. Both have a deterministic surface (bottom), a closed grammar over it (middle) and several drivers entering through one entry point (top). Same author and same project: this is not an independent replication.
More in this topic

LLM Tool Use

Large language model tool use studies how models select, call, and coordinate external tools to extend their reasoning and take actions.

See all 297 papers

Explore similar work

CardsList
Oct 19, 2026

On the Fragility of Tasks under Composition: Implicit Budgeting in Language Models

Modern language models are increasingly asked to handle several tasks within a single prompt. Doing so requires solving each task correctly while integrating the resulting answers into a coherent response. In this paper, we identify a fundamental limitation in this process, where models make substantially more errors when tasks are composed, even when they can solve every constituent task reliably in isolation. This degradation cannot be fully explained by the accumulation of independent errors. Across both open-weight models and the frontier model Claude Opus 5, our findings instead point to an implicit thinking budget underlying this effect. In particular, under composition, models use substantially fewer tokens per task than in isolation, even when the full sequence remains well within the context window. Tasks that require longer solution traces in isolation are consequently more fragile\textit{fragile} under composition, i.e., models make more errors on prompts containing these tasks. In isolation, non-fragile tasks receive more reasoning than they need and waste compute. However, by grouping tasks according to their fragility, the implicit budgeting can be manipulated to reduce token usage by 4343% across a range of math and linguistic tasks while also preserving accuracy.

Chenpan Li, Thies Brockmoeller, Stefanie Jegelka +2

TU Munich · MIT CSAIL

Compositional ReasoningEfficient Language Model ReasoningAdaptive Reasoning in Language Models
AskPDF
Oct 15, 2026

System1-Arena: Which Decisions Can Be Left to System 1?

Large language models (LLMs) often make discrete decisions, such as flagging phishing or escalating to a human. System-1 models, such as Jev and the OpenAI Decisions API, return probabilities over options in a single forward pass, while System-2 approaches generate reasoning steps. Which decisions can System-1 models handle, and which require generative reasoning? We introduce System1-Arena, a benchmark that gives all models the same decision requests and scores their probabilities. It contains 92k questions across 105 tasks from 268 public sources, in four groups: text classification and knowledge, applied decisions, evaluation and control, and agent and action. It also includes 138 closed-loop environments, from web shopping and SWE-bench coding control to robotics, games, and driving; in real-time mode, the world evolves while a model decides. We evaluate 30 models: 7 closed LLMs, 13 open LLMs read at the answer token, and 10 System-1 models. The largest System-1 models match or exceed fast closed LLMs in non-agentic accuracy, with better-calibrated probabilities at one-quarter of the latency. A reasoning LLM is the most accurate, at about 70x the cost of a fast LLM. In closed loops, large open LLMs succeed most often (up to 32.9%); small System-1 models rarely succeed.

Tianyu Chen, Xinyang Liu, Ruyi An +2

The University of Texas at Austin

LLM Decision-MakingLLM EvaluationEfficient Language Model Inference
AskPDF
Oct 14, 2026

Open-world AI: a Tale of Three Requirements

The closed world asks AI for specialized competence and lets it manufacture the iid data its theory assumes: a fixed criterion licenses collecting examples until the task is covered. The open world asks for versatility --- taking up tasks a machine was not built for, from few examples and with little prior from a human designer --- and offers data that is grouped, not shuffled. Its tasks are not known in advance, so a machine must do inference-time learning (1st requirement). Modeling tasks as sharing latent factors and decomposing the training objective, we derive two more: a rich representation, retaining more than iid generalization requires, and a predictive disentanglement that aligns inputs and outputs into matched low-dimensional groups. The analysis further predicts the need for a consistent inner learner, and the benefit of long contexts spanning related tasks. We test these claims on language models up to 8B, asking whether a model has a property (qualitative) rather than how much it wins (quantitative). Models that meet the requirements improve as data from related tasks accumulates; the rest degrade, and neither scale nor layer-type interleaving repairs this.

Jianyu Zhang, Léon Bottou

FAIR, Meta∗ · Flatiron Institute

Continual Learning for LLMs
AskPDF