Parallax to Align Them All: An OmniParallax Attention Mechanism for Distributed Multi-View Image Compression

Haotian Zhang; Feiyue Long; Yixin Yu; Jian Xue; Haocheng Tang; Tongda Xu; Zhenning Shi; Yan Wang

cs.CVMar 4, 2026

Parallax to Align Them All: An OmniParallax Attention Mechanism for Distributed Multi-View Image Compression

Authors: Haotian Zhang, Feiyue Long, Yixin Yu, Jian Xue, Haocheng Tang, Tongda Xu, Zhenning Shi, Yan Wang,

Organizations: The National Engineering Laboratory for Video Technology, School of Computer Science, Peking University · Institute for AI Industry Research (AIR), Tsinghua University · Tsinghua Shenzhen International Graduate School, Tsinghua University

Abstract

Multi-view image compression (MIC) aims to achieve high compression efficiency by exploiting inter-image correlations, playing a crucial role in 3D applications. As a subfield of MIC, distributed multi-view image compression (DMIC) offers performance comparable to MIC while eliminating the need for inter-view information at the encoder side. However, existing methods in DMIC typically treat all images equally, overlooking the varying degrees of correlation between different views during decoding, which leads to suboptimal coding performance. To address this limitation, we propose a novel $\textbf{OmniParallax Attention Mechanism}$ (OPAM), which is a general mechanism for explicitly modeling correlations and aligned features between arbitrary pairs of information sources. Building upon OPAM, we propose a Parallax Multi Information Fusion Module (PMIFM) to adaptively integrate information from different sources. PMIFM is incorporated into both the joint decoder and the entropy model to construct our end-to-end DMIC framework, $\textbf{ParaHydra}$ . Extensive experiments demonstrate that $\textbf{ParaHydra}$ is $\textbf{the first DMIC method}$ to significantly surpass state-of-the-art MIC codecs, while maintaining low computational overhead. Performance gains become more pronounced as the number of input views increases. Compared with LDMIC, $\textbf{ParaHydra}$ achieves bitrate savings of $\textbf{19.72%}$ on WildTrack(3) and up to $\textbf{24.18%}$ on WildTrack(6), while significantly improving coding efficiency (as much as $\textbf{65}\times$ in decoding and $\textbf{34}\times$ in encoding).

Figures & tables

Figure 1: Visualization of OPAM correlations. Row 1 : Input views. Row 2 : Difference maps. Row 3 : Correlations, where consistent regions ( green ) are prioritized over occlusions ( red ).

Panel of Figure 2 — Figure 2: (a) The proposed ParaHydra framework. $\hat{y}_{\mathcal{K}\setminus\{k\}}$ and ${f}_{\mathcal{K}\setminus\{k\}}$ denote the sets of all view features excluding the $k$ -th view feature ${\hat{y}}_{k}$ and ${f}_{k}$ , respectively. (b) Parallax Multi Information Fusion Module (PMIFM). PMIFM integrates the aligned feature $f_{k}^{t}$ based on the semantic relevance $C_{k}$ between each side source $f_{k}$ and the main source $f_{i}$ . (c) Parallax Entropy Model. The figure illustrates the decoding process for the slice $\hat{y}^{i}_{k}$ . Each latent slice $\hat{y}^{i}_{k}$ is partitioned into anchor $\hat{y}^{i}_{k,\text{ac}}$ and non-anchor $\hat{y}^{i}_{k,\text{na}}$ parts. Anchor $\hat{y}^{i}_{k,\text{ac}}$ is decoded first using Gaussian parameters $(\mu^{i}_{k,\text{ac}},\sigma^{i}_{k,\text{ac}})$ predicted from the channel context $\Phi^{i}_{k,\text{ch}}$ (provided by previous slices $\hat{y}^{<i}_{k}$ ) and the hyperprior $\Phi_{H_{k}}$ . Non-anchor $\hat{y}^{i}_{k,\text{na}}$ is decoded next using Gaussian parameters $(\mu^{i}_{k,\text{na}},\sigma^{i}_{k,\text{na}})$ predicted from the local context $\Phi^{i}_{k,\text{lc}}$ (derived from anchor $\hat{y}^{i}_{k,\text{ac}}$ ), the channel context $\Phi^{i}_{k,\text{ch}}$ , the global context $\Phi^{i}_{k,\text{gc}}$ , and the hyperprior $\Phi_{H_{k}}$ .

Figure 3: Overview of OmniParallax Attention Mechanism (OPAM). Left: Parallax attention. Middle: Two-stage parallax attention in OPAM. OPAM applies horizontal (red) and vertical (blue) parallax attention sequentially to capture the full 2D spatial context. Right: Receptive fields of the aligned features. Each position in $f_{l}^{{hor}}$ attends to one row of $f_{r}$ , and each position in $f_{l}^{{ver}}$ attends to one column of $f_{l}^{{hor}}$ , allowing each position in $f_{l}^{{ver}}$ to attend to the entire 2D spatial domain of $f_{r}$ .

Figure 4: Rate-distortion curves of ParaHydra compared against various baselines.

Methods	InStereo2K(2)		Cityscapes(2)		WildTrack(3)		WildTrack(6)		Mip-NeRF 360(3)
Methods	PSNR	MS-SSIM	PSNR	MS-SSIM	PSNR	MS-SSIM	PSNR	MS-SSIM	PSNR	MS-SSIM
LDMIC	0%	0%	0%	0%	0%	0%	0%	0%	0%	0%
VVC	48.68%	80.69%	54.33%	101.92%	49.47%	115.64%	25.16%	120.67%	7.14%	96.47%
MV-HEVC	84.84%	182.11%	106.27%	202.95%	31.84%	176.86%	10.01%	168.04%	41.15%	171.15%
ECSIC	25.65%	11.99%	50.51%	41.12%	–	–	–	–	–	–
BiSIC	-2.95%	-4.65%	11.40%	-7.11%	–	–	–	–	–	–

Table 1: BDBR comparison relative to LDMIC across different datasets. The best results are highlighted in bold . The number of input views is indicated in parentheses. A dash (–) denotes that the corresponding model cannot be evaluated on that dataset.

	Methods
	VVC	MV-HEVC	BiSIC-fast	CAMSIC	BiSIC	LDMIC	2D Attn	ParaHydra
Enc. Time (s)	349.47	24.67	0.35	0.87	0.30	9.27	0.62	0.27
Dec. Time (s)	0.37	0.17	0.41	1.79	0.49	21.43	0.67	0.33
Params (M)	–	–	327.66	602.05	305.71	214.98	108.81	105.25
FLOPs (T)	–	–	3.96	9.18	3.77	2.88	1.64	1.78

Table 2: Complexity comparison of various codecs evaluated on images with a resolution of 1024

\times

832 from the InStereo2K dataset.

Views	VVC	MV-HEVC	LDMIC-fast	LMVIC	ParaHydra
3	7.14%	41.15%	25.52%	-14.30%	-18.20%
4	53.06%	91.30%	14.31%	17.27%	-16.84%

Table 3: BDBR comparison relative to LDMIC on Mip-NeRF.

Figure 5: Ablation study on the WildTrack with 3 input views.

Panel of Figure 6 — Figure 6: Qualitative results on the InStereo2K. PSNR and bitrate (bpp) are denoted as PSNR / bpp. Zoom in for a better view.

Cross-View Attention

See all 9 papers

Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates
May 21, 2026Guojun Xu, Mingyang Zhang, Jianwen Xiang +3Learned Image Compression
School of Computer Science and Artificial Intelligence, Wuhan University of Technology
Multi-scale Image Representation Compression
Sep 2, 2026Tianhao Peng, Ho Man Kwan, Fan Zhang +2Learned Image Compression
Visual Information Lab, University of Bristol, UK · Tencent Media Lab, Palo Alto, USA
MixCompress: Mixture of Experts for Variable Rate Learned Image Compression
Jul 15, 2026Calvin-Khang Ta, Praneet Singh, Tong Shao +1Learned Image Compression Rate-Distortion Optimization
Dolby Laboratories, USA

May 21, 2026J/K move · Enter open · S save

Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates

Guojun Xu, Mingyang Zhang, Jianwen Xiang, Cheng Tan+2

School of Computer Science and Artificial Intelligence, Wuhan University of Technology

Learned Image Compression

Open paper AskPDF arXiv

Distributed Image Compression (DIC) is crucial for multi-view transmission, especially when operating at extremely low bitrates (< 0.1 bpp). Its core challenge is effectively utilizing side information to achieve high-quality reconstruction under strict bitrate budgets. However, existing DIC approaches struggle to exploit global context and object-level details from side information, leading to local blurring and the loss of fine details in the reconstruction. To address these limitations, we propose a Multimodal DIC framework (MDIC), which, for the first time, leverages side information in a multimodal manner into the DIC paradigm, effectively preserving fine-grained local details and enhancing global perceptual quality in reconstructed images. Specifically, we introduce a text-to-image diffusion-based decoder conditioned on textual side information extracted from correlated images to capture shared global semantics. Moreover, we design a feature-mask generator, supervised by a multimodal fine-grained alignment task, to strengthen the exploitation of visual side information. The generated mask serves two purposes: first, it guides the extraction of fine-grained details from losslessly transmitted side information to preserve the semantic consistency of reconstructed details; second, it regulates the extraction of clustered feature representations from the quantized VQ-VAE embeddings, compensating for category information lost under the extreme compression of the primary image. Extensive experiments on the widely used KITTI Stereo and Cityscapes datasets demonstrate that MDIC achieves state-of-the-art perceptual quality at extremely low bitrates.

Abstract

Cross-View Attention

Explore similar work

Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates

Multi-scale Image Representation Compression

MixCompress: Mixture of Experts for Variable Rate Learned Image Compression