Temporal Residual Bottleneck for Robust Asynchronous Collaborative Perception

Melih Yazgan; Ahmed Abouelazm; J. Marius Zöllner

cs.CVOct 7, 2026

Temporal Residual Bottleneck for Robust Asynchronous Collaborative Perception

Authors: Melih Yazgan, Ahmed Abouelazm, J. Marius Zöllner

Organizations: FZI Research Center for Information Technology · Karlsruhe Institute of Technology

Abstract

Collaborative perception extends the sensing range of autonomous vehicles, but its performance degrades when shared features arrive stale or incomplete. Most latency-robust methods compensate delayed collaborator features through flow-guided alignment or direct feature transport. In this work, we formulate asynchronous collaborative perception as temporal residual prediction. Our Temporal Residual Bottleneck keeps a deterministic pose-warped collaborator feature as a conservative anchor and uses a $Δt$ -conditioned xLSTM to extract residual temporal evidence from the available history. A detector-facing residual bottleneck then applies only gated, regularized corrections before ego-side fusion, reducing the risk of overwriting reliable static structure when temporal correspondence is uncertain. Experiments on DAIR-V2X and OPV2V show that our method is especially effective under severe fixed/irregular delays and packet drops. On DAIR-V2X, the reported checkpoint trades a small amount of synchronized peak accuracy for better robustness under stronger communication degradation. Controlled diagnostics further indicate that direct feature transport has oracle headroom but can become unreliable when deployed without accurate correspondence. These results support temporal residual fusion as a practical alternative for asynchronous and incomplete collaborative perception. Code will be publicly released at https://url.fzi.de/8dk38.

Figures & tables

Figure 1 : Overview of the proposed temporal residual bottleneck. Delayed collaborator BEV features are processed in the collaborator local frame. The latest stale feature $S_{i}^{t}$ is pose-warped into a geometric baseline $B_{i}^{t}$ , while a $\Delta t$ -conditioned xLSTM extracts temporal residual evidence $R_{i}^{t}$ from the available history. A detector-facing residual bottleneck predicts a correction $\delta F_{i}^{t}$ , which is applied through an object-focused gate $G_{i}^{t}$ as $\hat{F}_{i}^{t}=B_{i}^{t}+G_{i}^{t}\odot\delta F_{i}^{t}$ . The updated collaborator features are then warped to the ego frame and fused as multi-scale intermediate features for detection.

Figure 2 : Asynchronous collaborative-perception setting. At ego time $t$ , the latest collaborator feature $S_{i}^{t}$ has age $\Delta t_{i}$ . Earlier features form the irregular history $\mathcal{H}_{i}^{t}$ ; features may be unavailable ( $m_{i,k}=0$ ) or spatially incomplete.

Method	Sync	Irr. 300 ms	Irr. 500 ms
Where2Comm+SyncNet [ 11 ] (pub.)	0.792 / 0.692	0.732 / 0.625	0.710 / 0.612
CoBEVFlow [ 22 ] (pub.)	0.775 / 0.663	0.742 / 0.626	0.732 / 0.621
CoDynTrust [ 28 ] (pub.)	0.792 / 0.700	0.743 / 0.643	0.736 / 0.637
Where2Comm+SyncNet (rep.)	0.782 / 0.688	0.738 / 0.606	0.732 / 0.603
CoBEVFlow (rep.)	0.788 / 0.660	0.739 / 0.597	0.718 / 0.586
CoDynTrust (rep.)	0.798 / 0.668	0.726 / 0.586	0.719 / 0.582

Table 1 : DAIR-V2X baseline comparison. AP@0.5/AP@0.7 under synchronized and irregular-delay settings. “pub.” denotes published reference numbers from CoDynTrust [ 28 ] ; “rep.” denotes reproduced integrations in our framework. Bold and underline denote the best and second-best results, respectively.

Method	AP@0.7	Comm. Rate	Payload
		( $r$ )	(MB)
Where2Comm+SyncNet (rep.)	0.606	0.1225	0.79
CoBEVFlow (rep.)	0.597	0.1441	0.93
CoDynTrust (rep.)	0.586	0.2278	1.47
LRCP	0.639	1.0000	6.45
Ours	0.640	0.1572	1.01

Table 2 : Communication footprint under DAIR-V2X irregular 300 ms delay. Payload is estimated per collaborator BEV feature packet.

Method	Sync	Irr. 300 ms	Irr. 500 ms
Where2Comm [ 8 ]	0.870 / 0.786	0.801 / 0.664	0.768 / 0.639
Where2Comm+SyncNet [ 11 ]	0.870 / 0.786	0.811 / 0.706	0.789 / 0.691
CoBEVFlow [ 22 ]	0.837 / 0.783	0.822 / 0.740	0.819 / 0.732
CoDynTrust [ 28 ]	0.865 / 0.819	0.857 / 0.795	0.853 / 0.783
Ours	0.895 / 0.835	0.884 / 0.818	0.868 / 0.799

Table 3 : OPV2V Culver City comparison. AP@0.5/AP@0.7 under synchronized and irregular-delay settings.

Noise $(\sigma_{t},\sigma_{\theta})$	LRCP	Ours
0.1 m / 0.1 ∘	0.784 / 0.630	0.761 / 0.634
0.3 m / 0.3 ∘	0.722 / 0.560	0.738 / 0.602
0.5 m / 0.5 ∘	0.668 / 0.534	0.712 / 0.589

Table 5 : Robustness to pose/localization noise. DAIR-V2X irregular 300 ms delay with inference-time fusion-pose noise. The noise column gives matched

(\sigma_{t},\sigma_{\theta})

in meters/degrees. We report AP@0.5/AP@0.7.

Table 7

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Training Configuration
Component	Setting
Optimizer	Adam, learning rate $2\times 10^{-3}$ , weight decay $10^{-4}$ , $\epsilon=10^{-10}$
Schedule	Cosine decay with 7-epoch warmup (warmup LR $2\times 10^{-5}$ ); DAIR-V2X: 60 epochs \| OPV2V: 30 epochs
Batch size	DAIR-V2X: 2 \| OPV2V: 1
Async training	Random binomial delay with $p\in[0.05,0.30]$
Architecture & Temporal Branch Settings

Appendix

Table 8 : Supplementary hyperparameters for scratch training. Detailed training, temporal branch, residual bottleneck, object gate, and packet-drop settings.

Panel of Figure 3 — Figure 3 : Burst packet-drop visualization. Top row: First scenario. Bottom row: Second scenario. Left: BEV detection result. Right: detector confidence map with burst-dropped BEV cells highlighted in gray.

Panel of Figure 4 — Figure 4 : Qualitative BEV detections under delay and pose noise. Left: ours. Right: LRCP. Top: irregular 500 ms delay. Bottom: irregular 500 ms delay with pose noise (0.3 m / 0.3 ∘ ).

Figure 5 : Latency comparison. Runtime measured on RTX 3090.

Variant	AP@0.3	AP@0.5	AP@0.7
Released hard ROI mask	0.810	0.739	0.597
No ROI mask	0.822	0.760	0.611
Identity warp, no mask	0.822	0.760	0.611
Background-preserving ROI blend	0.822	0.760	0.611

Appendix

Table 9 : CoBEVFlow ROI-mask diagnostic. DAIR-V2X AP at

p=0.3

for reproduced CoBEVFlow. The background-preserving blend matches the method description more closely than the released hard-mask multiplication.

Explore similar work

AFFormer: Adaptive Feature Fusion Transformer for V2X Cooperative Perception under Channel Impairments
May 3, 2026Xi Zhou, Tao Huang, Qing-Long Han +2V2X Collaborative Perception
College of Science and Engineering, James Cook University, Smithfield, QLD 4878, Australia · School of Engineering, Swinburne University of Technology, Melbourne, VIC 3122, Australia · Transport for NSW, Sydney, NSW, Australia
INTACT: Ego-Guided Typed Sparse Evidence Retrieval for Heterogeneous Collaborative Perception
Jun 3, 2026Chen Li, Shengrong Yuan, Jialong Zuo +3Collaborative Perception Heterogeneous Robot Teams
National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology · Zhejiang Normal University
UECP: Uncertainty-Enhanced Collaborative Perception
Jun 22, 2026Kang Yang, Tianci Bu, Peng Wang +3Collaborative Perception Uncertainty-Aware Fusion
Renmin University of China, Beijing, China · College of Systems Engineering, National University of Defense Technology, Changsha, China · Intelligent Shipping Center, China Waterborne Transport Research Institute, Beijing, China

May 3, 2026, cs.CVJ/K move · Enter open · S save

AFFormer: Adaptive Feature Fusion Transformer for V2X Cooperative Perception under Channel Impairments

Xi Zhou, Tao Huang, Qing-Long Han, Rana Abbas+1

College of Science and Engineering, James Cook University, Smithfield, QLD 4878, Australia · School of Engineering, Swinburne University of Technology, Melbourne, VIC 3122, Australia · Transport for NSW, Sydney, NSW, Australia

V2X Collaborative Perception Channel Modeling

Open paper AskPDF arXiv

Accurate 3D object detection is essential for ensuring the safety of autonomous vehicles. Cooperative perception, which leverages vehicle-to-everything (V2X) communication to share perceptual data, enhances detection but is vulnerable to channel impairments, such as noise, fading, and interference. To strengthen the reliability of intelligent transportation systems, this work improves the robustness of V2X cooperative perception under communication conditions that reflect common channel impairments. This paper proposes an Adaptive Feature Fusion Transformer (AFFormer), a Transformer-based framework that mitigates the adverse effects of corrupted features by modeling temporal, inter-agent, and spatial correlations. AFFormer introduces three key modules: Multi-Agent and Temporal Aggregation for context-aware fusion across agents and over time, Dual Spatial Attention for efficient modeling of spatial dependencies, and Uncertainty-Guided Fusion for entropy-driven refinement of fused features. A teacher-student knowledge distillation strategy further enhances robustness by aligning fused features with reliable early-collaboration supervision. AFFormer is validated on the V2XSet and DAIR-V2X datasets, where it consistently outperforms existing methods under both ideal and impaired communication conditions, demonstrating improved robustness to communication-induced feature degradation while maintaining a competitive efficiency-accuracy trade-off.

Abstract

Explore similar work

AFFormer: Adaptive Feature Fusion Transformer for V2X Cooperative Perception under Channel Impairments

INTACT: Ego-Guided Typed Sparse Evidence Retrieval for Heterogeneous Collaborative Perception

UECP: Uncertainty-Enhanced Collaborative Perception