stat.MLMay 9, 2026

Core-Halo Decomposition: Decentralizing Large-Scale Fixed-Point Problems

Authors: HaixiangYang XuJiefu ZhangXudong WuZihan ZhouJun HeJiayu Chen

Organizations: Purdue University · The University of Hong Kong · Johns Hopkins University

Abstract

We study solving large-scale fixed-point equation x=Fˉ(x)x^\star=\bar F(x^\star) with decomposition. Standard strict decomposition assigns each agent a disjoint block and evaluates updates using only owned coordinates. For most operators, however, a block update may depend on variables outside the block. Truncating these dependencies by strict decomposition changes the mean operator and creates structural bias that cannot be removed by more samples, smaller stepsizes, or additional consensus. We therefore propose Core-Halo decomposition, which separates write ownership from read-only evaluation context: each agent updates its own core and reads from an overlapping halo. By aligning the Core-Halo decomposition with the block-dependence structure of Fˉ\bar F, the original fixed-point problem can be implemented faithfully in a decentralized multi-agent system. We further characterize the fundamental obstruction faced by strict decomposition through a Bellman closure condition and a blockwise bias lower bound, showing that local-only updates can alter the original fixed-point operator. Finally, we conduct extensive experiments across a range of application settings, and demonstrate that Core-Halo achieves near-centralized performance while retaining the parallelism benefits of decentralization.

Explore similar work

Jul 10, 2026math.OC

Solving Stochastic Fixed-Point Equations with High Probability

We study stochastic fixed-point equations T(x)=x\mathbf{T}(\mathbf{x}) = \mathbf{x} over normed spaces (E,)(\mathcal{E}, \|\cdot\|), where the operator T\mathbf{T} is nonexpansive or contractive and is accessed only through unbiased stochastic evaluations with bounded second central moment. Given ε>0,δ(0,1)ε> 0, δ\in (0, 1), the goal is to output xE\mathbf{x} \in \mathcal{E} such that T(x)xε\|\mathbf{T}(\mathbf{x}) - \mathbf{x}\| \leq ε with probability at least 1δ1-δ. We introduce VR-GHAL, a variance-reduced gradual Halpern method for quadratically smoothable Banach spaces. The key algorithmic ingredient is a recursive stochastic estimator based on clipped differences of oracle evaluations: instead of clipping τ(x;ξ)τ(\mathbf{x}; ξ) itself, we clip stochastic differences at the Lipschitz scale γxyγ\|\mathbf{x} - \mathbf{y}\|. This makes the estimator pathwise Lipschitz along the algorithmic trajectory while permitting martingale concentration under finite second moments in the native norm. Our main theorem gives an anytime high-probability residual bound: on a single event of probability at least 1δ1 - δ, the residual decreases nearly geometrically across epochs, up to lower-order logarithmic factors. Under only bounded variance, displaying only the dependence on the target error εε and Lipschitz constant γ(0,1]γ\in (0, 1] of T\mathbf{T}, the resulting oracle complexity is min{ε5,(1γ)3ε2}\min\{ε^{-5}, (1-γ)^{-3}ε^{-2}\}. Under a Lipschitz-in-expectation oracle, the dependence improves to the corresponding ε3ε^{-3} nonexpansive rate (i.e., for γ=1γ= 1), and under samplewise nonexpansiveness to ε2ε^{-2}.
Jelena Diakonikolas
May 29, 2026math.OC

S^3LDBO: A Snapshot Single-Loop Algorithm for Decentralized Bilevel Optimization

Networked AI systems increasingly rely on multiple agents that collaboratively learn and adapt models over communication networks. In such systems, bilevel formulations naturally arise in hyperparameter optimization, data cleaning, and meta-learning, but the repeated evaluation of gradients, Jacobians, and Hessians can impose a substantial computational burden on individual agents. To address this challenge, we propose Snapshot-SLDBO (S3^3LDBO), an efficient single-loop decentralized bilevel optimization algorithm that enables agents to intermittently skip expensive derivative evaluations through a snapshot mechanism. This mechanism can be interpreted as an autonomous computation-adaptation strategy for networked AI, where agents selectively perform costly local updates while maintaining global collaborative learning. We establish the ergodic iteration complexity and the high probability nonergodic iteration complexity of the proposed algorithm within a deterministic setting. Experimental results on hyperparameter optimization with synthetic and MNIST datasets, data hyper-cleaning on Fashion-MNIST, and decentralized meta-learning on miniImageNet demonstrate that the proposed algorithm improves computational efficiency while maintaining competitive learning performance.
Chao Yin, Youran Dong, Shiqian Ma +2
Jul 30, 2026cs.AI

HALO: Heterogeneous Admission through Localized Obligations for Safe Agentic Execution

Recent agentic AI systems may return a heterogeneous response containing notices, requests, handoffs, and actions. Conditions can change before external use, so components from the same response need not remain supported together. Rejecting the whole response discards useful components, whereas checking components independently can leave a dependent without its prerequisite. We present Heterogeneous Admission with Localized Obligations (HALO), a runtime protocol that preserves supported components whose declared prerequisites also remain supported, rechecks each exact action before dispatch, and allows blocked actions to be replaced only by fresh candidates. HALO matched all 96 admission expectations and passed all 20 protocol tests. In structured-response replay, it retained 248/248 supported components, including 128/128 unaffected by unrelated changes, while a whole-response policy retained 0/248. Across ten cold-start PX4/Gazebo sessions, HALO blocked every tested stale route, observed no matching stale setpoint, and completed all fresh recoveries.
Taewoo Park, Kyeonghyun Yoo, Kiseok Kim +2