Compositional Conformal Risk Control for Multi-Stage LLM Cascades
Organizations: College of Information Science and Technology, Jinan University, Guangzhou, 510632, Guangdong, China
Abstract
Multi-stage large language model (LLM) cascades balance accuracy and inference cost, but reliable deployment also requires controlling error risk and identifying its sources across stages. We propose Compositional Conformal Risk Control (CompCRC), a risk-control framework for pre-specified multi-stage LLM cascades. We decompose the risk of accepting incorrect answers into additive stage-wise contributions over the full query population, linking local risk budgets to a system-level guarantee. CompCRC constructs and freezes candidate budget allocations and acceptance thresholds, selects one candidate on independent data, and certifies its stage-wise risk contributions on separate held-out data. Under conditional exchangeability, we establish a finite-sample guarantee: the marginal expected accepted-error risk of the complete deployment procedure does not exceed the global risk budget. At the reported comparison budget of , all three stages of CompCRC pass their local checks on MMLU, GSM8K, and HellaSwag. Compared with Global-CRC, which passes system-level risk certification at the same budget, CompCRC reduces observed accepted-error risk by 1.33--2.00 percentage points, improves final accuracy by 0.67--2.00 percentage points, and reduces weighted inference cost by 30.37% on HellaSwag. The selected cost-based budget allocation also improves accuracy per unit cost by 10.6%--46.7% over uniform allocation. Further experiments show that stage-wise statistics identify stages that fail local certification checks, providing diagnostic information alongside system-level risk control.
Figures & tables
| Partition | Purpose | Usage |
| Candidate construction | Determine and fix local budgets and acceptance thresholds before independent selection. | |
| Candidate selection | Compare the accuracy and average invocation cost of fixed candidates. | |
| Stage-wise certification | Independently assess the selected configuration’s risk contribution at each stage. | |
| Baseline calibration | Determine the routing parameters of the Route-only baseline. | |
| Test set | Final evaluation | Report risk, accuracy, and inference cost. |
| Rule | Stage weight |
| Uniform | |
| Cost | |
| Flow | |
| Error | |
| Cost–flow | |
| Cost–error |
| Method | Formal guarantee | CRC status | Risk | Acc. | Cost | Acc./Cost |
| MMLU | ||||||
| L1 only | None | N/A | 0.2533 | 0.7467 | 0.6587 | 1.1336 |
| L2 only | None | N/A | 0.1500 | 0.8500 | 1.0978 | 0.7743 |
| L3 only | None | N/A | 0.1167 | 0.8833 | 3.1364 | 0.2816 |
| Route-only | None | N/A | 0.1733 | 0.8200 | 1.0000 | 0.8200 |
| FrugalGPT-style | None | N/A | 0.2067 | 0.7933 | 0.8636 | 0.9186 |
| Stage | CRC status | |||||
| 0.45 | L1 | 0.2632 | 0.2571 | 0.98 | 0.8036 | |
| L2 | 0.1316 | 0.1285 | 0.98 | 0.5331 | ||
| L3 | 0.0553 | 0.0208 | 0.38 | 0.3127 | ||
| System | 0.4500 | – | Pass | – | ||
| 0.30 | L1 | 0.1817 | 0.3542 | 1.95 | 0.7419 | |
| L2 | 0.0814 | 0.2132 | 2.62 | 0.3956 |
| Dataset | Rule | Highest-utilization stage | Risk | Acc. | Cost | Acc./Cost |
| MMLU | Uniform | L2 | 0.1400 | 0.8600 | 1.5489 | 0.5552 |
| Cost | L1 | 0.1667 | 0.8333 | 1.2854 | 0.6483 | |
| GSM8K | Uniform | L1 | 0.2067 | 0.7833 | 1.2775 | 0.6132 |
| Cost | L1 | 0.2367 | 0.7533 | 1.1105 | 0.6783 | |
| HellaSwag | Uniform | L2 | 0.3367 | 0.6633 | 1.6573 | 0.4002 |
| Cost | L1 | 0.3500 | 0.6500 | 1.1069 | 0.5872 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | CC-3 | CompCRC (Ours) |
| Final accuracy | 85.00% | 83.33% |
| Average weighted invocation cost | 34.5905 | 1.9514 |
| Fallback rate | 13.67% | 0.00% |
| Stage | CRC status | |||||
| 0.25 | L1 | 0.1462 | 0.0972 | 0.66 | 0.7143 | |
| L2 | 0.0731 | 0.0533 | 0.73 | 0.6429 | ||
| L3 | 0.0307 | 0.0312 | 1.02 | 0.4074 | ||
| System | 0.2500 | – | NC | – | ||
| 0.45 | L1 | 0.2632 | 0.1724 | 0.66 | 0.3571 | |
| L2 | 0.1316 | 0.0219 | 0.17 | 0.6429 |
| Stage | CRC status | |||||
| 0.35 | L1 | 0.2069 | 0.1724 | 0.83 | 0.4564 | |
| L2 | 0.0880 | 0.1097 | 1.25 | 0.6682 | ||
| L3 | 0.0551 | 0.0364 | 0.66 | 0.9152 | ||
| System | 0.3500 | – | NC | – | ||
| 0.45 | L1 | 0.2632 | 0.2257 | 0.86 | 0.4206 | |
| L2 | 0.1316 | 0.1066 | 0.81 | 0.5472 |
LLM Reliability
Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.
See all 845 papers