Solving an inverse problem consists in finding the unknown parameters of a system from observations. This task is particularly challenging when observations have a strong stochastic component, producing diverse yet equivalent patterns. Self-organizing systems are a notable case: simple local rules can generate complex stochastic patterns. Many natural systems rely on such processes, making self-organization central to understanding natural phenomena. Inverse methods based on direct comparison of observed values struggle in this setting, as such comparisons cannot capture feature similarities between variable outcomes; as a result, the loss fails to meaningfully inform the optimizer. In this work, we show that the difficulty posed by stochasticity in observable patterns can be overcome by reformulating the inverse problem in an embedding space. By mapping stochastic patterns onto an invariant embedding space, where equivalent patterns have similar representations, we can effectively recover unknown causal parameters without the need for handcrafted objective functions or heuristics. We show that embedding models trained on large image datasets produce robust representations that capture these invariances. We evaluate the method on three self-organizing systems: a physical, a biological, and a social one; namely, a reaction-diffusion system, a model of embryonic development, and an agent-based model of social segregation. We show that the method reliably recovers parameters despite stochasticity in the pattern outcomes. We further apply the method to real biological patterns, highlighting its potential as a tool for both theorists and experimentalists to investigate the dynamics underlying complex stochastic pattern formation.
Figures & tables
Figure 1: Inverse problems and stochasticity in self-organizing systems. ( a ): An inverse problem consists in finding the mapping from observations to their model space (also referred to as parameter space , solution space or domain space [ 1 , 2 ] ). It presents three challenges: (i) existence: solutions may not exist for all observations; (ii) uniqueness: multiple causes may produce identical observations; and (iii) stochasticity: observations may differ for the same model parameters. ( b ): Three simulation instances of two self-organizing models run with the same parameters. (Top) : Reaction-diffusion model; the stochasticity results from varying initial conditions s0∼S . (Bottom) : Schelling’s segregation model; the stochasticity results from asynchronous updates in the model F . Both are examples of stochasticity in the observable space, the challenge our method addresses.
Figure 2: Method overview and invariant representations. ( a ): A target pattern is mapped onto an embedding space ztarget∈RD using the embedding model E , where D is the embedding space dimension. A forward model, parametrised by θ∈RM , where M is the degrees of freedom of the model, generates a pattern which is mapped to the same embedding space resulting in zsolution . The loss L between the target and the candidate solution embedding guides the optimizer, iteratively updating θ until convergence. By operating in embedding space, the method offers robustness against the inherent stochasticity of emergent patterns. ( b ): Diagram representing models parameters θi , generating observable patterns yi , and the visual patterns being mapped to an embedding zi . ( c ): Distance matrix between embedding representations z of four patterns (two reaction-diffusion and two ABM segregation patterns). Notice that each of the patterns is a unique instantiation of the model. Similarity clusters appear, suggesting that the embedding model represents patterns with similar features as similar vectors. The values displayed are pairwise cosine distances 1−cos(zi,zj) .
Type
Reaction-Diffusion
Blastocyst
Schelling
z-score
Direct pattern MSE
Baseline
0.158±0.082
0.507±0.099
0.200±0.129
0.74
CLIP
Embedding
0.038±0.021
0.321±0.097
0.044±0.033
−0.99
DINO
Embedding
0.017±0.010
0.423±0.044
0.044±0.032
−0.75
Nomic
Embedding
0.042±0.022
0.408±0.028
0.054±0.041
−0.64
Fréchet Wavelet
Wavelet embedding
0.050±0.032
0.333±0.087
0.040±0.031
−0.90
Dissimilarity index
Domain-specific
0.222±0.060
0.483±0.107
0.040±0.028
0.47
Table 1: Mean absolute error (MAE) between the recovered parameters θ′ and the ground-truth parameters θ (median ± MAD across ten runs) for ten runs per target pattern under three embedding metrics (CLIP, Nomic, DINO), Fréchet Wavelet, direct pattern (pixel-to-pixel) MSE and the domain-specific metrics. Direct pattern MSE is the baseline: the same optimization without an embedding model. Every row reports parameter-recovery error with respect to ground truth values, while the corresponding similarity metric guides optimization. Z-scores are computed by first standardizing each metric within each system zi=(xi−μi)/σi where μi and σi are the mean and population standard deviation of the eight reported median MAEs within system i , then averaging these standardized values across the three systems. Baseline shown first; other rows grouped by type and ranked by unrounded z-score within each group. The lowest median MAE in each system and the lowest z-score are shown in bold.
Figure 3: Results for Gray-Scott reaction-diffusion system. Twelve target patterns yi ( top row ) and one reconstruction yi′ per target ( bottom row ), generated from the parameters recovered by the method. Reconstructions for all 10 independent training runs, each with a different random seed, are shown in appendix Fig. 7 .
Figure 4: Results for blastocyst embryonic developmental model. A single target pattern y (left column) , simulated based on a Cellular Potts model [ 49 ] . The developmental process is implemented as a set of stochastic differential equations [ 47 , 49 ] . The reconstructions columns show the patterns yn recovered by our method using CLIP embeddings for 10 independent training runs. Non-uniqueness in the forward system allows distinct parameter sets to produce successful developmental patterns, so large parameter-recovery MAE can coexist with successful pattern reconstruction.
Figure 5: Results for Schelling segregation model. Eight target patterns yi ( top row ) for tolerance parameter T∈{0.1i∣i∈{1,2,…,8}} ( left to right ), and one reconstruction yi′ per target ( bottom row ), generated from the parameters recovered by the method. For small values of T , agents converge with minimal migration as tolerance threshold is quickly satisfied, in middle range of T (3-6) segregation patterns become more pronounced, while for large values of T the system does not converge as threshold is unattainable. Reconstructions for all 10 independent training runs, each with a different random seed, are shown in appendix Fig. 8 .
Figure 6: Reconstruction of natural patterns. ( a ): Ocellated lizard pattern. Left: Photograph of an ocellated lizard from Manukyan et al. [51] , white bar indicates 5mm scale. Right: Reconstructed pattern by the set of reaction-diffusion parameters found by the method. ( b ): Zebra pattern. Left: Photograph of a zebra skin pattern [ 54 ] . Right: Pattern generated by the anisotropic reaction-diffusion model for the parameters found by the method.
Figure 7: Results for Gray-Scott model. Twelve target patterns yi (left column) , each produced by simulating Gray-Scott partial differential equations 4.1 for different parameters θi and random initial states s0i . The reconstructions columns show the patterns yin recovered by our method for 10 independent training runs.
Figure 8: Results for Schelling’s segregation model. Eight target patterns yi (left column) , each produced by simulating Schelling ABM model of segregation 4.3 for different parameters θi and random initial states s0i . The reconstructions columns show the patterns yin recovered by our method for 10 independent training runs.
Inverse problems in physical or biological sciences often involve recovering an unknown parameter that is random. The sought-after quantity is a probability distribution of the unknown parameter, that produces data that aligns with measurements. Consequently, these problems are naturally framed as stochastic inverse problems. In this paper, we explore three aspects of this problem: direct inversion, variational formulation with regularization, and optimization via gradient flows, drawing parallels with deterministic inverse problems. A key difference from the deterministic case is the space in which we operate. Here, we work within a probability space rather than Euclidean or Sobolev spaces, making tools from measure transport theory necessary for the study. Our findings reveal that the choice of metric---both in the design of the loss function and in the optimization process---significantly impacts the stability and properties of the optimizer.
Qin Li, Maria Oprea, Li Wang +1
Department of Mathematics, University of Wisconsin-Madison, Madison, WI · Center for Applied Mathematics, Cornell University, Ithaca, NY · School of Mathematics, University of Minnesota Twin Cities, Minneapolis, MN +1
Can we design a model such that its stochastic training favours a desired class of solutions without enforcing an explicit penalty? Under suitable conditions, the interplay between symmetries of a model's weight parametrization and stochastic training favours particular solutions, inducing an implicit bias. Building on this mechanism, we develop a framework for inverse-designing such biases by constructing novel parametrizations and their associated symmetries. We show how holomorphic functions make this construction and calculation simple and explicit. Specifically, we introduce a new parametrization that biases learned weights toward the binary values {−1,+1}. Numerical experiments confirm the theoretical predictions. They also show that our parametrization reproduces the same preference induced by an explicitly regularized model without adding a penalty to the training loss.
Department of Mathematics, Informatics and Geoscience, University of Trieste, Via Valerio 12/1, 34127 Trieste, Italy · Area Science Park, Padriciano, 34149 Trieste, Italy · McGovern Institute, MIT, Main Street, Cambridge, MA 02139, USA
Self-supervised learning (SSL) by predicting in latent space, without generating the input data itself, learns highly abstract, useful representations. Intuitively, this success is often attributed to its ability to discard nuisance information that is irrelevant to prediction. However, this poses a conundrum: both stochastic variation in a prediction-relevant latent signal and true nuisance make observations partly unpredictable; how could they be distinguished? Surprisingly, we prove that common SSL methods can achieve exactly this, by implicitly instantiating a latent-variable model with stochastic dynamics and observation-private nuisance. We trace their ability to recover the stochastic signal to two complementary principles: Predictive mutual information maximization ensures that representations retain the information needed for prediction, while latent distribution matching constrains how this information is encoded, thereby making the retained signal identifiable. We confirm this identifiability result in simulations for Gaussian predictors, which recover the true signal up to an affine transformation even in dynamic, nuisance-laden environments.
Fabian A. Mikulasch, Friedemann Zenke
Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland · Faculty of Science, University of Basel, Switzerland