stat.MLOct 7, 2026
SaveUnpaired Canonical Correlation Analysis
Organizations: Department of Computer Science and AI Bar-Ilan University Ramat-Gan, Israel · Electrical and Computer Engineering Technion Haifa, Israel
Abstract
Canonical Correlation Analysis (CCA) is a fundamental method for multiview shared space learning. However, its strict reliance on paired data poses a significant limitation, as such data is often difficult to obtain or entirely unavailable. In this paper, we present Unpaired CCA (UCCA), a novel method that learns linear projections to maximize the correlation of the true underlying pairing without access to any paired samples during training. We first establish theoretical results connecting the Quadratic Assignment Problem (QAP) to CCA. Leveraging these theoretical insights, we derive a practical method to maximize correlation exclusively from unpaired data. To the best of our knowledge, UCCA is the first approach to learn maximally correlated projections in a strictly unpaired setting. We validate UCCA on real-world multi-modal datasets, demonstrating that it significantly outperforms recent unpaired alignment baselines in recovering the underlying true correlation. This work fills a critical gap between traditional statistical multiview learning and the growing field of unpaired data learning.
Figures & tables
Figure 1: The theoretical and empirical alignment of Unpaired CCA and QAP. (a) QAP vs. CCA: On a 10-sample synthetic dataset, evaluating all 10! possible pairings reveals a direct positive relationship between the QAP value and total correlation (App. E.1 ). (b) UCCA Performance: On the Handwritten benchmark (Sec. 6 ), UCCA captures high correlation despite training fully unpaired, significantly outperforming unpaired alignment baselines and approaching the paired CCA upper bound.
Figure 2: MCP and demonstration. (a) MCP successfully reconstructs the true pairing on aligned data. (b) MCP depends on input rotation, which leads to poor performance on unaligned data. (c) Conversely, optimizes over all orthogonal alignments, choosing the pairing with the highest correlation after the projection. In this example, it chooses the pairing from (a), enabling it to retrieve the true pairing, invariant to input (orthogonal) projections. A dynamic animation of this figure is available on our project page .
Figure 3: (a) Validation of Assum. 1 : Partial cumulative sums of singular values for the true permutation (#shuffles = 0) and various random shuffles on the Flickr dataset (8,000 samples; see Sec. 6 ). As the permutation diverges from , the partial sums strictly decrease, confirming the majorization property. (b) Validation of Thm. 2 . A comparison of (Nuclear) and (Frobenius) objectives using the setup from (a). Both objectives decay as the permutation moves away from , and both are locally maximized by the same ground-truth permutation.
Figure 5: Total Correlation results. Total correlation achieved across six distinct dataset configurations. UCCA consistently extracts highly correlated components, significantly outperforming all baselines, and often approaching the paired CCA upper bound.
Figure 6: in practice. Kendall Tau Distances between and approximated , computed on the anchors of the Handwritten benchmark (PC-PA) across 100 initializations. The strict concentration at zero empirically validates Thm. 2 .
Figure 7: Runtime and Performance. Total correlation versus runtime for UCCA and baselines on the handwritten (PC-PA) benchmark. UCCA is improved in robustness and accuracy with more clustering repetitions, yet outperforms all baselines even with fewer repetitions and shorter runtimes.
Figure 8: Total correlation performance on the Handwritten benchmark (PC-PA) across (a) data imbalances and (b) varying amounts of unpaired data . UCCA consistently outperforms baselines, achieving its peak performance when the maximum amount of data is accessible. This demonstrates that abundant unpaired data alone provides a strong alignment signal, improving performance without requiring any paired correspondences.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Extended validation of . Distance between and the approximated (denoted as ), computed on the anchors of the each dataset across 100 initializations. This figure complements Fig. 6 in the main text by demonstrating that the strict concentration at zero holds consistently across all evaluated domains, further validating Thm. 2 .
Figure 10: Extended runtime and performance analysis. Total correlation versus runtime (in seconds, log scale) for UCCA and baseline methods across the various datasets. The green numerical annotations on the UCCA curve denote the number of clustering repetitions. UCCA rapidly surpasses baseline performance even at a lower repetition setting and improves as repetitions increase, while maintaining a practical total execution time.
Figure 11: Effect of the QAP solver on performance. Total Correlation achieved across multiple initializations on the PC-PA Handwritten benchmark, using the 2-Opt and FAQ approximate QAP solvers. Both solvers yield very similar overall performance and alignment quality. Because FAQ failed to converge in two instances (points near zero), 2-Opt was selected as the default solver for UCCA simply to ensure consistent execution across all runs.
Figure 12: Effect of the number of anchors. Total Correlation versus the number of anchors used in the matching step. Increasing the number of anchors allows UCCA to capture the manifold geometry more accurately, leading to a sharp performance improvement that approaches the paired CCA upper bound (represented by the dashed horizontal line). The anchor count 20 is chosen to balance this high accuracy with computational efficiency.
Figure 13: Effect of the number of PCA dimensions. Total Correlation versus the number of PCA components used in the preprocessing step.
Figure 14: Embeddings visualizations. Visualizations of the generated embeddings of Paired-CCA and UCCA.
Figure 15: Synthetic toy dataset.
Figure 16: Extended empirical validation of Assum. 1 . Partial cumulative sums of singular values for the true permutation ( , denoted by 0 shuffles) and various randomly shuffled permutations across all datasets: PC-KL, PC-PA, KL-PA, SNARE, Flickr, and COCO. Consistent with the main text results (Fig. 3 a), the partial sums strictly decrease as the permutation diverges from , verifying the weak majorization property across all tasks.
Figure 17: Extended empirical validation of Theorem 2. Comparison of the (Nuclear Norm) and (Squared Frobenius Norm) objectives as the permutation is randomized away from the true alignment across all datasets: PC-KL, PC-PA, KL-PA, SNARE, Flickr, and COCO. In alignment with the main text results (Fig. 3 b), both objectives decay synchronously as the number of random shuffles increases, empirically confirming that both formulations are locally maximized by the same ground-truth permutation.
Figure 18: Data preparation: While the test is kept paired for evaluation purposes, we split the train such that no correspondence is available during training. This exactly replicates the unpaired settings discussed in the paper.