Task-Sufficient Contraction: Source Selection for Machine Information Interfaces

Joss Armstrong

cs.ITOct 6, 2026

Task-Sufficient Contraction: Source Selection for Machine Information Interfaces

Authors: Joss Armstrong

Organizations: Ericsson Ireland

Abstract

A declared task can sometimes certify a reduced source before a downstream encoder, codebook, rate, distortion target, or optimizer is chosen. This paper studies when one such reduction preserves the complete downstream problem family, a property termed Task-Sufficient Contraction. The reduced source is fixed by the task before the later operating point is selected. An exact contraction allows the later problem to be solved on that source with the same result as if the full source had been retained. For a machine with a fixed set of possible actions and a fixed loss, the paper identifies a consumer-specific source by merging states only when every available action has the same regret in both. For finite action sets, replacing the richer source by this reduced source preserves the complete one-step rate-regret curve, even though the reduction is fixed before the distortion target is chosen. A second result gives an exact characterization for quadratic loss on affine feasible-action sets: the canonical reduced source is the projection onto the directions in which feasible actions can differ. Under a fixed energy budget, this becomes centered load, while retaining only the optimal water-filled action is too coarse. Earlier Information Bottleneck, semantic rate-distortion, and goal-oriented quantization results are then used to distinguish exact, architecture-conditioned, approximate, failed, and corrected contractions. The framework suggests a way for heterogeneous machines to exchange what a receiving task needs without first aligning their full internal representations.

Explore similar work

May 28, 2026cs.IT

Support sufficiency as action-sufficient compression: a single-cycle rate-regret formulation

Robust decision-making requires compression. A system that forms a rich support state cannot usually preserve its full structure at the point of action. It must retain only those distinctions needed to act, verify, abstain, or defer under the current consequence geometry. This paper formalizes support sufficiency as action-sufficient compression. Let

H

denote a full support state,

\mathcal{A}

a finite action set, and

Z

a consequence geometry specifying payoff structure. For fixed

Z

, the coarsest exactly action-sufficient compression is the quotient of support space by policy equivalence. Two support states may be merged exactly when they require the same optimal action. This clarifies why content-only and scalar-confidence-only arbitration fail whenever their induced partitions cross action boundaries. Approximate sufficiency is then defined by bounded expected policy regret. In the finite single-cycle setting, this yields a rate-regret problem with source

H

, reproduction alphabet

\mathcal{A}

, and distortion given by consequence-sensitive regret. The optimal stochastic action channel inherits the standard rate-distortion Gibbs form, applied here to support states with regret distortion. The contribution is interpretive: action adequacy is distinguished from reconstruction fidelity, information-bottleneck prediction, and rational inattention. Robust single-cycle arbitration does not require preserving all support, but it does require preserving the distinctions that consequence geometry makes action-relevant.

Mark Walsh

Sufficiency Arbitration Data Compression Methods

Ask PDF

Oct 5, 2026cs.IT

On the Cardinality of Optimal Representations in the Binary-Source Information Bottleneck

The information bottleneck (IB) seeks a representation

U

of a source

X

that retains as much information as possible about a target

Y

, subject to a constraint on

I(U;X)

. A classical argument shows that it suffices to consider representations with at most

|\mathcal{X}|+1

symbols, and this bound is known to be tight whenever

|\mathcal{X}| \geq 3

. We show that the binary case behaves differently: if

X

is binary and

Y

is finite, then for every joint distribution of

(X,Y)

and every rate constraint, the IB optimum is attained by a binary

U

. Hence the bound

|\mathcal{U}| \leq |\mathcal{X}|+1

sharpens to

|\mathcal{U}| \leq |\mathcal{X}|

for binary sources. The proof combines a separating hyperplane argument with the observation that, for a binary source, the ratio of the second derivatives of the two entropy functions involved is concave.

Dier Tang, Jun Chen

Department of Mathematics, The University of Hong Kong · Electrical and Computer Engineering, McMaster University

Information-Theoretic Limits

Ask PDF

May 11, 2026cs.IT

Cross-Domain Lossy Compression via Constrained Minimum Entropy Coupling

This paper studies cross-domain lossy compression through the lens of minimum entropy coupling (MEC) with rate and classification constraints. In this setting, an encoder observes samples from a degraded source domain, while the decoder is required to generate outputs following a prescribed target distribution and to preserve information relevant to a downstream classification task. Motivated by logarithmic-loss distortion, we adopt an information-based objective that maximizes the coupling strength between the source and reconstruction, rather than minimizing a sample-wise distortion. Under common randomness, we formulate a rate-constrained MEC problem (MEC-B) and show that the intermediate representation can be removed without loss of optimality, yielding an equivalent deterministic coupling formulation. For Bernoulli sources, closed-form expressions are derived with and without classification constraints. In addition, we implement a neural restoration framework using quantization, entropy modeling, distribution matching, and classification regularization. Experiments on MNIST super-resolution and SVHN denoising show that increasing the available rate improves classification accuracy and yields more informative reconstructions.

Nam Nguyen, Hassan Tavakoli, An Vuong +2

School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, 97331

Cross-Entropy Losses Data Compression Methods Distortion

Ask PDF