A Mathematical Theory of Interpretation: Rational Entropy, Spectral Readout, and Confusability as a Resource

Blake Reynolds

cs.ITAug 24, 2026

A Mathematical Theory of Interpretation: Rational Entropy, Spectral Readout, and Confusability as a Resource

Authors: Blake Reynolds

Organizations: Conjecture Labs

Abstract

This article presents the abridged core of A Mathematical Theory of Interpretation (MTI), which treats interpretation as observer-relative spectral measurement under an access structure. MTI makes interpretation a method-design problem: access, query, utility, and medium determine what an observer can select, identify, communicate, or refuse. On a learning-invariant Hilbert realization, Rational Entropy measures residual uncertainty across knowledge, utility, and medium. In the finite-effective regime, we classify its zero set. Pairwise confusability is equivalent to uniform atomic collapse, while a unique utility maximum can select one atom even when other zero-cost states remain non-atomic. This reverses the usual zero-error role of confusability: agreement in at least one observer direction excludes unresolved multi-atom readings, while the joint label preserves identification. The corresponding free-design capacity is the product of all but the smallest direction budget. A four-condition certificate characterizes sharp, decodable, medium-faithful, and order-independent readout on a finite commuting code sector and returns typed obstructions when those guarantees fail. Together, these results establish MTI as a theoretical basis for constructing interpretation methods with explicit access assumptions, guarantees, and failure modes.

Figures & tables

Figure 1: The Meeting toy model. The same knowledge–utility–medium structure appears first inside one presenter and then as a distributed reconciliation problem. The green throat is the access/readout boundary. The diagram is a dictionary for the construction and is pedagogical, not a claim that every organization is literally one observer or is structured in an observer-complementary manner.

Result family	Conclusion	Regime
Access-structured method design	Compiles a target–readout pair into its fibers, added sources of resolution, validation obligations, and strongest licensed output.	General set-theoretic
Invariant-sector construction	The mean-ergodic projector selects the learning-invariant sector; $L$ -covariance makes query restriction measurement-consistent.	Unitary/covariant
Finite collapse classification	Classifies every zero minimizer; pairwise confusability is equivalent to uniform atomicity; unique utility maximization gives selected atomicity.	Finite pure point
Static observation law	Returns selected spectral support; eigenvector language is reserved for a selected pure-point atom.	Finite derived or general conditional
Compatibility boundary	Commuting selected projectors remove order effects; isolated-projector order error is bounded by the query commutator.	Finite dimensional
Interpretive coding capacity	The free-design capacity is the product of all but the smallest direction budget.	Finite labels

Table 1: Result spine of the abridged article.

Figure 2: Exact spectral outcome law. A self-adjoint query induces a projection-valued spectral measure and hence a probability law. The observer maps that one outcome law into knowledge, utility, and medium coarse-grainings.

Figure 3: Static observation law. The finite route derives the selected support from the Rational Entropy zero set and utility. The general route is conditional on a theorem-local support bridge. Output is projection-valued, with eigenvector shorthand only in the singleton pure-point case.

Figure 4: Interpretive-code inversion. Collapse admissibility requires every pair to agree in at least one direction; readout identification requires that no pair agree in all directions.

Figure 5: Free-design cylinder capacity. A smallest direction is held constant to supply pairwise confusability, while the other two directions carry the identifiable grid. Cylinders are canonical optimizers but not the only optimizers.

Figure 6: The zero-error readout boundary. The static code sector is certified when commutation, atomic separation, unique selection, and medium faithfulness hold. Spectral, selection, and medium failures appear at the observer aperture; noncommutativity is a property of the protocol chain.

Figure 7: Realized capacity pipeline. The free-design envelope is an outer bound. A realized certificate additionally depends on actual spectral atoms, observer labels, separating gaps, and empirical perturbation margins.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Group	Content controlled
A1	Statistical geometry	$\ell(M)$ is a regular statistical manifold on the admissible region; the Fisher–Rao metric exists and is positive on the relevant tangent bundle; Hessian-potential notation is chart-local.
A2	Hilbert realization	$d\mu_{g}$ is the measure used for $L^{2}(\ell(M),d\mu_{g})$ ; Riesz, compactness, and density-operator language are invoked only under their stated hypotheses.
A3	Access structure	$P_{O}$ is an orthogonal hard-support projector; $W_{O}$ is a positive multiplication weight on the hard domain; soft weighting cannot enlarge hard support; aggregation-mediated observation factors through its readout map.
A4	Learning representation	The declared learning flow preserves the observer domain and induces a strongly continuous unitary representation when the mean-ergodic theorem is used. Semigroup or non-preserving regimes require separate hypotheses.
A5	Invariant projection	$\Pi_{H}^{L}$ exists as the mean-ergodic projector onto $\mathcal{H}_{V}=\operatorname{Fix}(U)$ . Stable observation is posed on $\mathcal{H}_{V}$ by Axiom 2.8 ; this is a domain restriction, not an entropy consequence.
A6	Self-adjoint query	Every spectral statement uses a specified self-adjoint realization. Unless stated otherwise, the query is $L$ -covariant, so $\Pi_{H}^{L}$ reduces it and the invariant-sector spectral measure is the restriction of the ambient one.

Appendix

Table 2: Assumption ledger for the abridged theory.

Result	Assumptions	Dependency note
Fiber criterion	Set-theoretic map data	No probabilistic or spectral hypothesis.
Mean-ergodic sector	A2, A4–A5	Provides the stable observation domain.
Safe query restriction	A5–A6	Reduction preserves the spectral outcome law.
Rational Entropy zero set	A7–A8	Finite conditional entropy yields exact injectivity criterion.
Finite collapse classification	A5–A9	Uses full finite spectral-law realization, zero criterion, and utility selection.
Static observation law	A5–A9	Finite derived route or conditional general support route.

Appendix

Table 3: Principal theorem dependencies.

Class	Use	Boundary
Definitions, axioms, and assumptions	Constructs the object, primitive rule, or local hypothesis.	Does not prove a downstream consequence without a stated result.
Formal results	Formal statement with explicit regime and proof route.	Carries only the hypotheses written locally or imported through a cited assumption.
Finite realization	Specializes the projection-valued law to atoms and finite labels.	Does not imply a global finite spectrum or a universal discretization.
Dictionary/diagram	Translates the construction into communication, measurement, or meeting language.	Cannot carry proof burden.
Free-design capacity	Optimizes over abstract finite labels with fixed budgets.	Is an outer envelope for realized observers, not an automatic achievability theorem.
Numerical certificate	Transfers exact finite structure through operator and label margins.	Conditional on the estimator, gap, label stability, and concentration assumptions.

Appendix

Table 4: Claim classes used in the abridged article.

Claim	Class	Boundary
Static observation law	Theorem family	Finite support is derived; general Borel support is conditional on a full-mass bridge.
Finite collapse classification	Theorem	Pairwise confusability characterizes uniform atomicity; unique utility maximization characterizes selected atomicity.
Finite coding capacity	Theorem family	Exact on abstract finite label products; realized codes obey the converse but need not attain it.
Zero-error interpretive readout	Finite iff certificate	Complete only on the declared attained finite atom-separated code window.
Compatibility boundary	Proposition family	Commuting selected projectors remove order effects; noncommuting protocols need additional dynamic structure.
Access-structured method design	Proposition/protocol	Separates readout information from assumptions and decoder-supplied resolution; it is not a universal estimator.

Appendix

Table 5: Major claim ledger.

Symbol	Meaning
$M$	Ambient space of informational configurations.
$L_{t}$ , $\Phi_{t}$	Learning mechanism and the induced flow on the learned realization.
$\ell(M)$	Learned information manifold fixed at the observation regime.
$g$ , $d\mu_{g}$	Fisher–Rao metric and its induced measure.
$\mathcal{H}=L^{2}(\ell(M),d\mu_{g})$	Information Hilbert realization.
$O=(\ell(O),m_{O},U_{O},c_{O})$	Observer: learned state, medium, utility, and active context.

Appendix

Table 6: Principal notation.

The Query Channel: Information-Theoretic Limits of Masking-Based Explanations
Apr 17, 2026Erciyes Karakaya, Ozgur ErcetinFeature Attribution Explainable Artificial Intelligence
*Department of Electrical and Computer Engineering, University of Maryland, College Park, USA · †Faculty of Engineering and Natural Sciences, Sabanci University, Turkiye
Structural Limits of the Information-Theoretic Uncertainty Decomposition
Sep 30, 2026Jakob Lønborg Christensen, Christian F. Baumgartner, Morten Rieger Hannemose +2Aleatoric Uncertainty Information-Theoretic Lower Bounds
DTU Compute Technical University of Denmark · Faculty of Health Sciences and Medicine University of Lucerne
Prospective Interpretation Risk: Principled Communication Control Between LLMs
Sep 27, 2026Wanrong Yang, Rehan Deen, Julian Ma +7Multi-Agent LLM Systems Pragmatic Reasoning in Language Models
University of Liverpool · Independent Researcher · University College London +1

Apr 17, 2026J/K move · Enter open · S save

The Query Channel: Information-Theoretic Limits of Masking-Based Explanations

Erciyes Karakaya, Ozgur Ercetin

*Department of Electrical and Computer Engineering, University of Maryland, College Park, USA · †Faculty of Engineering and Natural Sciences, Sabanci University, Turkiye

Feature Attribution Explainable Artificial Intelligence Neural Network Interpretability Explainability Evaluation Information-Theoretic Lower Bounds

Open paper AskPDF arXiv

Masking-based post-hoc explanation methods, such as KernelSHAP and LIME, estimate local feature importance by querying a black-box model under randomized perturbations. This paper formulates this procedure as communication over a query channel, where the latent explanation acts as a message and each masked evaluation is a channel use. Within this framework, the complexity of the explanation is captured by the entropy of the hypothesis class, while the query interface supplies information at a rate determined by an identification capacity per query. We derive a strong converse showing that, if the explanation rate exceeds this capacity, the probability of exact recovery necessarily converges to one in error for any sequence of explainers and decoders. We also prove an achievability result establishing that a sparse maximum-likelihood decoder attains reliable recovery when the rate lies below capacity. A Monte Carlo estimator of mutual information yields a non-asymptotic query benchmark that we use to compare optimal decoding with Lasso- and OLS-based procedures that mirror LIME and KernelSHAP. Experiments reveal a range of query budgets where information theory permits reliable explanations but standard convex surrogates still fail. Finally, we interpret super-pixel resolution and tokenization for neural language models as a source-coding choice that sets the entropy of the explanation and show how Gaussian noise and nonlinear curvature degrade the query channel, induce waterfall and error-floor behavior, and render high-resolution explanations unattainable.

Abstract

Explore similar work

The Query Channel: Information-Theoretic Limits of Masking-Based Explanations

Structural Limits of the Information-Theoretic Uncertainty Decomposition

Prospective Interpretation Risk: Principled Communication Control Between LLMs