Many problems across the sciences compare two stochastic processes through the sequences they generate. Reinforcement learning asks what a new policy would return from the trajectories of an old one; molecular simulation asks how much more a ligand binds from samples of the unbound state; climate attribution asks how a heat wave shifted under warming; and language-model post-training asks what a deployed agent will do from logged interactions. In every case, one quantity would answer all these questions at once: the ratio of the two probability laws over their generated sequences, the trajectory density ratio. This ratio exists only under strong conditions. When one process reaches states the other cannot, the ratio does not exist; when their overlap is thin, its variance grows without control with the sequence length; and when they concentrate on different lower-dimensional sets, the ratio is undefined altogether. Weak support of this kind is the norm for sequence data, not a sampling accident, and its price is paid in identifiability rather than in noise alone. This survey organizes the field around a three-level diagnosis of weak support and a map of the nine retreats that methods take when the full ratio is out of reach, from direct estimation and reweighting through staged decomposition, stationarization, and partial-identification bounds to smoothing, transport couplings, operator spectra, and direct estimation of a single functional. Each retreat keeps part of what the ratio offers and pays for the rest, and each application domain has standardized the retreat its data force. The survey closes with open problems: testable diagnostics at every level, partial identification at the trajectory level, a rate theory of time-amplified error, and benchmarks with ground truth.
Figures & tables
Figure 1: The trajectory-density-ratio problem. Top: we observe trajectories of a stochastic process Q but need expectations under a target process P ; the ratio wT=dP/dQ , a product of one-step ratios, converts one into the other. Bottom: weak support breaks this bridge in three ways—no overlap leaves wT undefined, thin overlap makes its variance grow with the horizon, and singular support leaves no density.
Level
Condition
supports overlap
low-dim. support
ratio exists
identifiable
consistent
worsens with horizon
Standard remedies
L1 support non-coverage
P≪Q
×
×
×
×
×
✓
restrict the space, report bounds
L2 thin overlap
P≪Q , esssupwT=∞
✓
×
✓
✓
✓
✓
truncation, stationarization
L3 trajectory singularity
P⊥Q or low-dimensional support
×
✓
×
×
×
×
smoothing, bridges, spectra
Table 1: The three levels of weak support.
Figure 2: The map of retreats, the organizing claim of this survey. The trajectory ratio wT (center) is the only estimand that one estimate reuses across all trajectory functionals (Eqs. ( 1 )–( 2 )). Each arrow is a retreat to a surrogate estimand, labeled by the weak-support level that forces it. Each box states what the surrogate keeps, what it pays, and where it is developed. Table 6 states the same map on one page with the standardizing domains.
Figure 3: Two research lineages converge on trajectory density-ratio estimation. Top lane: process measures and weak support, from coupling free energies and sequential analysis to overlap diagnostics, entropic transport, and dequantified bridges. Bottom lane: sequence-level learning, from sequential analysis and direct risk minimization to prefix objectives, marginal corrections, and simulation-calibrated testing. Labels name representative milestones. The map that organizes both lanes is stated in Table 6 .
Tension
direct risk
prefix objectives
signatures
conditional computation
reuse & DR
occupancy
coverage reweighting
diagnostics & testing
Representation vs. accumulated error
✓
✓
✓
✓
×
×
×
×
Whole trajectory vs. statistical cost
×
×
×
×
✓
✓
×
×
Estimation vs. valid inference
×
×
×
×
×
×
✓
✓
Table 2: The three tensions of the sequential literature.
When exact probabilities cannot be computed, density ratio estimation (DRE) is a key subroutine in the wide range of applications where density ratios appear, from experimental design to generative modelling and reinforcement learning. In this paper, we introduce the concept of an expected log density ratio (ELDR): log density ratios that are integrated under some third, possibly distinct measure. We show many problems with implicit densities where DRE is naïvely used are actually ELDR estimation tasks, and prove fundamental limitations on finite-sample ELDR estimation. Based on these results, we argue that the typical DRE-based approaches are unsuitable for ELDR estimation, and provide some foundational motivation towards bespoke ELDR estimators. To demonstrate, we propose a simple modification to some DRE methods that allows samples convergent to the integral distribution to appear in training data. We show through benchmarks that this can improve ELDR estimation error.
Avi Amalanshu, Chandler Squires, Yizhou Lu +1
Machine Learning Department, Carnegie Mellon University
We construct a binary mixed-regime process with one deterministic textual regime and one random regime governed by an unobserved latent state. Even an ideal infinite-capacity sequence predictor that exactly recovers the text-only marginal law can become overconfident when the observed prefix is compatible with the wrong latent regime. The resulting entropy difference is not an ordinary optimization error; it is a sufficiency gap caused by marginalization over an unobserved state. We then formalize retrieval, tool use, and external grounding through an auxiliary binary signal with fidelity γ∈[1/2,1]. The resulting Bayesian update yields a contextual dominance threshold: a corrective signal reverses the posterior odds induced by the textual history exactly when its fidelity exceeds the text-only posterior weight assigned to the misleading regime. This threshold reduces, but does not generally eliminate, the sufficiency gap; complete closure requires perfect revelation of the relevant latent state or an equivalent verification mechanism. The analysis clarifies why temperature scaling cannot restore missing context, why grounding mechanisms must be both informative and learnably usable by the model, and why autonomous sequence models require structurally decoupled observers or verifiers in high-stakes domains.
Estimating density ratios between pairs of intractable data distributions is a core problem in probabilistic modeling, enabling principled comparisons of sample likelihoods under different data-generating processes across conditions. While exact-likelihood models such as normalizing flows offer a promising approach to density ratio estimation, naive evaluations are computationally expensive and prone to discretization errors because they require simulating each distribution's likelihood independently. In this work, we leverage condition-aware flow matching to derive a single dynamical formulation for tracking density ratios along generative trajectories. We demonstrate competitive performance on simulated benchmarks for closed-form ratio estimation, and show that our method supports versatile tasks in single-cell genomics data analysis, where likelihood-based comparisons of cellular states across experimental conditions enable treatment effect estimation and batch correction evaluation.
Egor Antipov, Alessandro Palma, Lorenzo Consoli +3
Institute of Computational Biology, Helmholtz Munich · School of Computation Information and Technology, Technical University of Munich · Munich Center for Machine Learning (MCML) +2