cs.LGNov 2, 2026

Structured Priors: Towards an algebra of component priors for prior-data fitted networks

Authors: Thomas Bartz-Beielstein

Abstract

Prior-data fitted networks are typically trained using synthetic data generated by a single, manually constructed generator. In cases where such a generator combines multiple components, e.g., in TabPFN, Mitra, or TabICLv2, this is achieved through a mixture using manually specified weights, the impact of which on the network only becomes apparent after pre-training. Instead of such a hand-built generator, this work proposes a structured prior: a small set of named component priors, linked by operators whose algebraic properties are formally proven and whose influence on the trained network has been validated. Within a measure-theoretic framework, the operators of mixture, convolution, amplitude scaling, local transform, and projection are defined. Their governing laws are proven. These operators are implemented in the macl2l software package. Experiments demonstrate that the implemented generators reproduce the identities within the resolution of a permutation test. Furthermore, it is shown that trained networks obey the distributive law; that a mixed prior composed of small components performs as well as or better than any individual component in seven out of eight open regression benchmarks; and that on two of these tables a structured prior reaches, with a ninth of the data budget, an error that a generic random-function prior does not reach even with the full budget. A theorem on model averaging demonstrates that networks trained on individual components constitute the Bayes predictor for the mixture, provided that the component networks are themselves the Bayes predictors for their respective priors. However, the pre-registered test of this averaged predictor with trained component networks failed, because the networks identify the generating component too rarely. The paper proposes a research program focused on the algebra of structured priors for prior-data fitted networks.

Figures & tables

Explore similar work

CardsList
  1. How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?

    Oct 9, 2026Manuel Hülskamp, Julius Kobialka, Emanuel Sommer +2Bayesian Model AveragingPrior-Data Fitted Networks

  2. Characterizing the Separability Gap in Prior-Data Fitted Networks

    Oct 10, 2026Julius Kobialka, Manuel Hülskamp, Thomas Nagler +1Prior-Data Fitted NetworksBayesian Inference

  3. Mechanistic Evidence for Spectral Structures in Prior-Data Fitted Networks

    Jan 29, 2026Kaustubh Sharma, Srijan Tiwari, Ojasva Nema +1Prior-Data Fitted NetworksNeural Network Interpretability