cs.LGOct 21, 2026
SaveCircuits of LLM Classifiers
Organizations: Harvard School of Engineering and Applied Science · Kempner Institute for the Study of Natural & Artificial Intelligence · School of Electrical and Computer Engineering, Georgia Institute of Technology · Department of Electrical and Computer Engineering, University of California, Santa Barbara ·
Abstract
Language models make it easy to build classifiers by simply prompting a model with the sample to classify. However, their predictions are neither deterministic nor uniformly reliable across samples. How can we combine predictions from different models to improve reliability? We view LLM classifiers as noisy measurement instruments of an underlying target label. Our goal is to maximize average classification accuracy at a given average query cost. To this end, we introduce circuits that organize LLM classifiers into an adaptive sequence of queries and decisions. We then characterize the optimal circuit, obtain the fundamental limits of achievable accuracy, and provide practical insights on realizing the optimal circuit. Across seven classification tasks spanning safety, health, political science, human preference, agent evaluation, decision-making and reasoning, our method improves the cost-accuracy frontier over routing and cascading baselines.
Figures & tables
Figure 1: Classification circuits and their cost–accuracy tradeoffs. Majority voting, routing, and cascades are examples of circuits, differing in how they select and combine LLM classifiers. For any prompt, our method searches over a broad class of circuits and uses a dynamic program to determine which classifier to query next and when to stop. The right panel shows such circuits for the prompt : starting from its embedding, the circuit may predict immediately (as in A ) or sequentially acquire additional LLM measurements as needed (as in B and C ). Across prompts, these decisions produce the cost–accuracy frontier for the BeaverTails safety dataset.
Figure 2: Illustration of a few known circuits within the framework of Definition 2 .
Figure 3: Generating circuits. We first fit a probabilistic model from labeled prompts and instrument measurements ( Section 3.2 ). At inference time, a prompt embedding is passed through this model, and a dynamic program generates a circuit that adaptively queries instruments and decides when to stop ( Equation 7 ).
Figure 4: LMCircuit improves the cost-accuracy frontier over routing and cascading baselines. As decreases and additional queries become less costly, the circuits make more calls and also change which instruments they use (see lower panel). The allocations differ substantially across tasks, indicating that the value of an instrument depends on the classification problem and on other measurements available to the circuit.
Figure 5: LMCircuit is substantially faster to construct than most routing and cascading baselines. Across all seven tasks, LMCircuit is trained in 1–3 minutes, while some baselines require minutes to hours.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: The circuit’s beliefs are calibrated at every budget. For each task, left: Brier score on the test set ( lower is better ) against expected cost per item; right: reliability diagrams of the circuits at different budgets and of majority voting. The dashed line is perfect calibration. The circuit’s expected calibration error is at most at every budget on six tasks, and at most on BIG-Bench Hard, whereas majority voting’s is to and its miscalibration differs in direction by task.
Figure 7: Instrument answers are independent within a prompt but correlated across prompts. (a) coefficient of two instruments’ answers within a prompt, averaged over prompts on which both instruments’ answers vary. (b) The same coefficient across prompts, pooling all calls. (c) Total correlation of the eight answers on each prompt ( Equation 43 ) against the 95th percentile of its permutation null. Red marks a rejection at the 5% level. Within a prompt the coefficients lie between and , while across prompts they reach . Only one of the 20 prompts rejects joint independence, the rate expected by chance.
Figure 8: The fitted model predicts each instrument’s accuracy on unseen data without labels and recovers nearly all of the dependence between instruments. Correlations and accuracies implied by the fitted model on unlabeled test prompts ( Equation 44 ) against those measured from recorded calls and labels. Left: correlation of two instruments’ answers across prompts (one point per pair). Middle: correlation of their errors. Right: accuracy of each instrument. The dashed line is equality.
Figure 9: Additional queries improve accuracy when instrument errors vary across calls. We stratify unseen prompts by and report accuracy for four predictors: the embedding prediction , which makes no instrument calls; our circuit at normalized budgets of and ; and majority voting over the instruments the circuit can query. Numbers above the bars give the mean number of calls per prompt, and percentages below each group give its share of the test set. Relative to the embedding prediction, the circuit at budget improves accuracy on noisy prompts by 5–44 percentage points. On all right prompts, it comes within 1.3 percentage points of majority voting on four tasks while using 38–86% fewer calls.
Figure 10: The circuit allocates more queries when the prior prediction and instrument measurements disagree. We present the expected number of calls per test prompt for circuits at normalized budgets , , and on Arena, BeaverTails and PolitiCAUSE, grouped by prompt type (left), instrument disagreement (middle: , , , and ), and whether the embedding prediction agrees with the majority of the recorded calls (right). Percentages below each group give its share of the test set. Mean calls increase from all right to mostly wrong prompts on every task. The clearest separation occurs when conditioning on agreement with the embedding prediction: at budget , the circuit makes – as many calls when the embedding and instrument majority disagree on Arena ( vs. ), BeaverTails ( vs. ) and PolitiCAUSE ( vs. ).
Figure 11: Instruments with higher information per unit cost enter the circuit earlier. Expected calls per unseen prompt to each instrument as a function of the circuit’s expected cost per item. Instruments are colored by their rank under the estimated information rate in Equation 45 , with the corresponding value in bits per unit cost shown in parentheses. Across the three tasks, instruments with larger tend to begin receiving calls at lower expected cost. The highest-rate instrument enters first on every task, including Arena, where Gemma 26B is not the cheapest instrument.
Figure 12: Building the instrument pool. We build the instrument pool one instrument at a time according to different strategies: by estimated conditional information given the embedding and the pool (greedy algorithm), by accuracy on the training calls (most accurate first), cheapest first, or in random order (mean of three runs). Every point is our circuit restricted to that pool, using a single fitted model (an earlier version of the model without latent types). The circled point is the circuit with all instruments. Top: frontier area of test accuracy. Middle: test accuracy when every instrument in the pool is queried ( ). Bottom: the upper bound of Proposition 4 on the drop in Bayes error from adding each instrument ( its estimated conditional information, in nats), and the drop in the model’s Bayes error that the instrument’s answers actually produce, both on test.
Figure 13: Example of generated circuit and instrument’s error correlations on BeaverTails
Figure 14: Example of generated circuit and instrument’s error correlations on LMArena