PathFinder: Joint Decompositions of Linked Multimodal Datasets

Ying-Qiu Zheng; Alex Fung; Stephen M Smith; Rogier B Mars; Saad Jbabdi

cs.LGAug 15, 2026

PathFinder: Joint Decompositions of Linked Multimodal Datasets

Authors: Ying-Qiu Zheng, Alex Fung, Stephen M Smith, Rogier B Mars, Saad Jbabdi

Organizations: Oxford Centre for Integrative Neuroimaging University of Oxford Oxford, UK

Abstract

Low-rank matrix decompositions can uncover patterns and structure in data and have a number of different applications across many disciplines. Extensions to "joint" low-rank decompositions have been proposed to link datasets from different modalities. While these methods enable the discovery of common patterns across modalities, they require that all the multimodal data share one or more dimensions. We propose a new analysis method, PathFinder, that enables co-analysis of datasets that do not necessarily all share a dimension. The key insight is that as long as pairs or subgroups of matrices do share some dimension, and that there are one or more paths that link across the data matrices, a global joint decomposition can be sought out. This enables the joint estimation of common patterns across different modalities, species, or scales, where a one-to-one mapping across all data along some dimension is not necessarily available. We show that PathFinder is a general umbrella under which many matrix decomposition methods fall as special cases. It can be used to discover common patterns across disparate datasets and to make predictions for missing data or modalities.

Figures & tables

Figure 1: PathFinder concept. (A) Cortex $\times$ white matter matrices of two species that have incompatible dimensions. (B) Introducing subgroups of matrices that do have a dimension in common links all matrices, acting as a "path". (C) This can generalise to multiple domains and modalities with some modalities potentially missing for some domains.

Algorithm: Alternating Least Square in PathFinder
1:	Initialise $\mathbf{A_{\text{d}}}$ and $\mathbf{S_{\text{m}}}$ for $d=1,2,...D$ and $m=1,2,...M$
2:	while not converged do
	for $d=1,2,...D$
3a:	horizontally concatenate $\mathbf{X}_{\text{d,m}}$ as $\mathbf{\tilde{X}}_{\text{d}}$ and $\mathbf{S}_{\text{m}}$ as $\mathbf{\tilde{S}}$ , for $m$ if $\mathbf{X}_{\text{dm}}$ exists
3b:	set $\mathbf{A_{\text{d}}}\leftarrow\mathbf{\tilde{X}_{\text{d}}}\mathbf{\tilde{S}}(\mathbf{\tilde{S}}^{T}\mathbf{\tilde{S}}+\alpha\mathbf{I})^{-1}$
	for $m=1,2,...M$

Table 1: Pseudo code for the PathFinder alternating least squares algorithm.

Figure 2: Simulation results of a 2 x 3 grid of matrices spanning 2 modalities and 3 domains. Four of the six matrices (blue) are provided to PathFinder for fitting, while two (orange) were marked as missing. (A) Illustration of matrix configuration. (B) Predicted values are plotted against ground truth at noise level $\sigma=0.1$ and $r=20$ . All six matrices gave high reconstruction accuracy $R^{2}$ . (C) The effect of rank $r$ (number of components) at noise level 0.01. Shaded regions indicate the interquartile range over 10 runs. (D) Impact of the regularisation parameter.

Figure 3: Simulation results of cyclic data decomposition. (A) Example of cyclic data and its PathFinder configuration. (B) True and predicted activity. (C) True and reconstructed factors (with or without ICA post-processing).

Figure 4: Example of PathFinder reconstructing missing retinotopy task stimuli. Brain grayordinates and visual field pixels served as domains. The six different retinotopy tasks served as modalities. the RETCCW stimulus (Counter-clockwise rotating wedge) was marked as missing and held out from PathFinder training. The predicted RETCCW matches the actual stimulus.

Figure 5: PathFinder for Dual-Regression. (A) Example matrix configuration for PathFinder as Dual-Regression (B) PathFinder components are similar to dual regression spatial maps, both at the level of the group and individuals. (C) Visualised PathFinder and dual regression maps for the group and an example subject.

Figure 6: PathFinder decomposition of the cortico-basal direct loop. A: PathFinder configuration. B: running a winner-takes-all across components reveals functional circuits across the loop. C: connectivity matrices and their low-rank reconstructed counterparts.

Figure 7: PathFinder Extensions. (A) PathFinder can accommodate not only 2D matrices but also higher-order tensors. (B) An example of non-linear PathFinder using the encoder-decoder architecture. (C) PathFinder can drop the bipartite formulation, i.e., arranging the factors on one side instead of on left and right. This expands the applications for PathFinder, allowing it to run natively on biological networks.

Algorithm: Minibatch Alternate Least Square in PATHFINDER
1:	Initialise $\mathbf{A_{\text{d}}}$ and $\mathbf{S_{\text{m}}}$ for $d=1,2,...D$ and $m=1,2,...M$
2:	while not converged do
	for $d=1,2,...D$
3a:	initialise accumulators for updating $\mathbf{A}_{\text{d}}$ , $\mathbf{C}_{\text{num}}\leftarrow 0$ , $\mathbf{C}_{\text{denom}}\leftarrow 0$
3b:	for $m\in\mathcal{M}_{d}$ , $b\in\{1,2,...B(m)\}$
	set $\mathbf{C}_{\text{num}}\leftarrow\mathbf{C}_{\text{num}}+\mathbf{x^{\text{b}}_{\text{d,m}}}\mathbf{s^{\text{b}}_{\text{m}}}$

Table S1: Pseudo code of Minibatch ALS algorithm to solve PATHFINDER.

Algorithm: Joint SVD
1:	Initialise $\mathbf{U_{\text{d}}}$ , $\mathbf{D}_{d,m}$ , and $\mathbf{V_{\text{m}}}$ for $d=1,2,...D$ and $m=1,2,...M$
	while not converged do
2:	for $(d,m)\in\mathcal{M}$
	set $\mathbf{D}_{d,m}\leftarrow\mathbf{U}_{d}^{T}\mathbf{X}_{d,m}\mathbf{V}_{m}$
3:	for each domain $d=1,2,...D$
	for each component $n=1,2,...r$

Table S2: Pseudo code of the algorithm to solve joint SVD.

Algorithm: Mini-batch Joint SVD
1:	Initialise $\mathbf{U}_{d}$ , $\mathbf{V}_{m}$ for $d=1,\dots,D$ and $m=1,\dots,M$ ;
	set $\mathbf{D}_{d,m}\leftarrow\mathrm{diag}\bigl(\mathbf{U}_{d}^{T}\mathbf{X}_{d,m}\mathbf{V}_{m}\bigr)$ for $(d,m)\in\mathcal{M}$
	while not converged do
2:	for $(d,m)\in\mathcal{M}$
	set $\mathbf{D}_{d,m}\leftarrow\mathrm{diag}\bigl(\mathbf{U}_{d}^{T}\mathbf{X}_{d,m}\mathbf{V}_{m}\bigr)$
3:	for each domain $d=1,\dots,D$

Table S3: Pseudocode for the mini-batch Joint SVD algorithm.

T_{\text{pow}}

is the number of power-iteration steps per component (e.g. 2 in our implementation), and batch_size controls both the number of sampled matrices and the chunk size for matrix-vector products.

\mathcal{D}_{m}

denotes the set of domains for which modality

m

is observed.

Figure S1: Robustness to noise of PathFinder and independent SVD . PathFinder gave higher accuracy when the noise level is high, suggesting a denoising effect by sharing factors across domains and modalities.

Figure S2: (A) Convergence curve of various matrix sizes. Larger matrix size gives faster convergence. (B) Time required for 20 iterations as a function of matrix size.

Explore similar work

Knowledge-guided Pattern Discovery via Coupled Tensor Factorizations
Aug 13, 2026Gaute Johannessen, Geert Roelof van der Ploeg, Evrim AcarTensor Decomposition Scientific ML
Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo · Department of Data Science and Knowledge Discovery, Simula Metropolitan Center for Digital Engineering, Oslo
MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning
Jul 18, 2026Sana Tonekaboni, Viktoria Schuster, Caroline UhlerCross-Modal Representation Learning Multimodal Disentangled Representation Learning
Massachusetts Institute of Technology, Cambridge, MA, USA · Eric and Wendy Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, MA, USA · Vector Institute, Toronto, Canada +1
FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data
Aug 11, 2026Viktoria Schuster, Sana Tonekaboni, Caroline UhlerIntrinsic Dimensionality Representation Learning
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USA · Eric and Wendy Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, MA, USA · Technical University of Denmark, Lyngby, Denmark +1

Aug 13, 2026J/K move · Enter open · S save

Knowledge-guided Pattern Discovery via Coupled Tensor Factorizations

Gaute Johannessen, Geert Roelof van der Ploeg, Evrim Acar

Faculty of Mathematics and Natural Sciences, University of Oslo, Oslo · Department of Data Science and Knowledge Discovery, Simula Metropolitan Center for Digital Engineering, Oslo

Tensor Decomposition Scientific ML

Open paper AskPDF arXiv

In order to understand complex systems such as the human metabolome or human brain, different sensing technologies are used, generating complex data. These datasets are often multiway, i.e., with more than two axes of variation such as a subjects by metabolites by time array. While tensor factorizations have successfully revealed interpretable patterns from such complex data, they have so far been mainly data-driven. On the other hand, there is more to data -- there are computational models (of these systems), which are rich sources of prior information. In this paper, we introduce a knowledge-guided approach that brings together data and computational models by jointly analyzing real data and simulated data (generated using a computational model) using coupled tensor factorizations with linear coupling. Our experiments on real metabolomics measurements demonstrate that guiding the analysis of such noisy data with simulated data improves the pattern discovery performance while also revealing potential discrepancies between data and computational models.

Abstract

Explore similar work

Knowledge-guided Pattern Discovery via Coupled Tensor Factorizations

MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning

FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data