SIGMA-Lane: Scale-pyramId Gated MAmba for Temporally Consistent Video Lane Detection
Authors: Tiancheng Zhang, Mengmeng Wang, Yan Gao, Xiangjie Kong, Guojiang Shen, Jiaxin Du
Organizations: Zhejiang Key Laboratory of Visual Information Intelligent Processing, College of Computer Science and Technology, Zhejiang University of Technology
Video lane detection requires predictions that remain stable across frames, yet severe vehicle occlusions can break temporal cues. In streaming recurrent models, corrupted observations may enter the hidden state and produce errors that persist into later frames. Existing occlusion-aware refinements usually provide obstacle masks as auxiliary inputs, so the state-update path is only indirectly protected. We propose SIGMA-Lane, which treats this failure mode as state contamination in State Space Model (SSM)-based temporal modeling. SIGMA-Lane places occlusion-aware gates on the SSM write and residual-fusion paths, controlling how current observations enter temporal memory and are fused back after temporal propagation. After coordinate-consistent affine alignment, the model combines two complementary paths: SSM-consistent dual-gating for temporal filtering and Structural Spatial Retrieval (SSR) for recovering missing lane structure from aligned historical priors. Experiments on VIL-100 and OpenLane-V show improved temporal stability under heavy occlusion, with competitive F1 and mIoU scores.
Figures & tables
Figure 1: Comparison of Video Lane Detection Paradigms. The window-batched, joint-aggregation paradigm (a) extracts features Ft−T+1:t from a fixed temporal window and aggregates them jointly, incurring high latency without reusing past computation. The recurrent, state-propagation paradigm (b) propagates the previous state (Ft−1,Lt−1) frame by frame, enabling streaming inference, but offers no mechanism to prevent corrupted observations from contaminating the hidden state under occlusion. Our memory-maintained, contamination-aware paradigm (c) maintains a video memory bank of historical features F , obstacle masks M , and lane masks L . After lane-guided affine alignment, two paths process the aligned data: SSM-Consistent Dual-Gating for occlusion-aware temporal filtering and SSR for lane-guided spatial recovery.
Figure 2: Overview of the SIGMA-Lane Aggregation Module. Video Memory stores historical features Ft−T+1:t , obstacle masks Mt−T+1:t , and the previous lane mask Lt−1 . The Affine Warp module aligns historical data to current-frame coordinates. Warped features then pass through two paths: (1) SSM-Consistent Dual-Gating with a scale-pyramid design for multi-scale temporal propagation and occlusion-aware gating, (2) SSR where current features Ft retrieve spatial context from warped Ft−1 enhanced by Lt−1 embeddings. The outputs are aggregated with current-frame features Ft via Add&Norm and FFN blocks, producing enhanced features F~t .
Figure 3: SSM-Consistent Dual-Gating with a scale-pyramid design. Input features X pass through LN and are split into high- and low-resolution branches. Each branch applies Input Gate, concatenates the Geometry-Aware Start Token s0 , performs Mamba2 [ 4 ] temporal modeling, and applies Output Gate, with both gates modulated by the obstacle mask M . The low-resolution branch is upsampled and merged with the high-resolution branch through Add&Norm to produce X~ . Finally, the current-frame features x~t are extracted from X~ and sent to subsequent modules.
Figure 4: Lane-Guided Affine Warp and Structural Spatial Retrieval. (a) In Lane-Guided Affine Warp, features xt−1 and lane mask ℓt−1 are fed into Lane-Masked GAP to compute lane-region-weighted statistics. Combined with current-frame features xt , an MLP predicts affine parameters p , where tanh constrains the transformation magnitude. The Affine Warp operator synchronously transforms xt−1 , ℓt−1 , and mt−1 using the same grid, yielding aligned outputs x~t−1 , ℓ~t−1 , and m~t−1 . (b) In Structural Spatial Retrieval, current-frame features xt pass through LN to form the query Q . Warped features x~t−1 and lane-mask embedding e(ℓ~t−1) are normalized via LN and summed to form key K and value V . Multi-head cross-attention produces the enhanced feature output.
Figure 5: Visualization of the spatial prior learned by Geometry-Aware State Initialization. Each strip shows the input image, position-prior heatmap, and ground-truth lanes. The learned prior is concentrated around lane-like structures and provides a cold-start bias before reliable temporal history is available.
Recent work has applied Mamba style state space models (SSMs) to video anomaly detection, yet existing approaches still rely on buffering clips or windows internally, lack a theoretical account of how temporal memory relates to detection latency, and benchmark efficiency only through GPU throughput rather than the edge hardware these methods are intended to target. We introduce a strictly causal streaming anomaly detector whose fixed size state is updated in O(1) time and memory per incoming frame, with no lookahead and no clip buffering. Its temporal core is a diagonal linear state space recurrence with an input and state dependent decay gate, trained self supervised through causal next embedding prediction on a frozen visual backbone. We derive a closed form relationship between the recurrence decay spectrum and both detection delay and the shortest anomaly it can reliably capture, then validate empirically on UCSD Ped2 and CUHK Avenue. The settling delay bound predicted from the learned base decay (57 to 59 frames) sits far above the measured detection delay (1.6 and 18.4 frames), showing that the event boundary gate, not the base decay, governs responsiveness. We further report end to end latency and throughput measured directly on Apple M3 Pro hardware, 0.74 ms and 0.77 ms per frame (over 1300 FPS), rather than simulated GPU numbers. With an untuned initial configuration the method reaches 67.9 percent and 70.2 percent frame level AUC on Ped2 and Avenue, trailing prior non causal SSM baselines in accuracy. Ablations over decay rate, state size, and gating reveal that the gate contribution is dataset size dependent, hurting accuracy on the smaller Ped2 training set but helping on the larger Avenue one. Closing this accuracy gap and extending evaluation to a third, larger benchmark are immediate next steps.
Yogesh Kumar
Indian Institute of Technology Jodhpur, Jodhpur, India
We present SAM3Dual, our third-place solution to the MOSEv2 track of the 8th Large-scale Video Object Segmentation (LSVOS) Challenge at ECCV 2026. SAM3Dual is a training-free inference extension of pretrained SAM 3 that explicitly separates temporal memory into a short-term branch for recent observations and a long-term branch for interval-sampled historical representations. The two memory responses are combined using a deterministic sequence-relative fusion schedule and conservatively modulated by the previous-frame object confidence. All pretrained SAM 3 parameters remain frozen, requiring no task-specific training, fine-tuning, test-time training, or online parameter optimization. The complete system achieved an official J&F score of 64.37 and ranked third in the MOSEv2 track. This result highlights the potential of reorganizing temporal memory entirely at inference time to obtain competitive long-term VOS performance while preserving the pretrained model.
Unsupervised video object-centric learning aims to decompose dynamic scenes into temporally persistent entity representations. Existing recurrent video slot-attention methods propagate a fixed set of slots across frames, but typically assume unconditional slot propagation: every slot is updated and decoded at every frame, regardless of whether its corresponding object is visible. We show that this design violates a basic lifecycle requirement for persistent slots: when an object is absent or fully occluded, its slot should preserve its previous state and avoid explaining unrelated visible content. Instead, unconditional propagation creates update-induced state drift, where current-frame evidence overwrites the absent object's representation, and decoder-induced reconstruction interference, where the inactive slot remains coupled to reconstruction through decoder attention. We propose Temporal Slot Activation (TSA), a lightweight mechanism that learns a per-slot, per-frame activation score αk,t∈(0,1) without visibility supervision. TSA uses this activation as a shared latent control variable for slot lifecycle modeling. When a slot is inactive, TSA anchors its state to the previous slot via activation-gated updating and suppresses its decoder participation through an activation-dependent additive bias on attention logits before softmax normalization. This jointly reduces state drift and reconstruction-driven interference. To improve decisions under partial occlusion and gradual reappearance, TSA further conditions activation prediction on a per-slot temporal memory produced by a Temporal Context Encoder. We evaluate TSA on MOVi-C, MOVi-E, YTVIS-HQ, and the occlusion-heavy OVIS benchmarks, using standard (FG-ARI, mBO) and tracking-based (IDF1, HOTA) metrics. TSA improves object decomposition and temporal identity preservation, with large gains on long, heavily occluded videos.
Duc Nguyen, Sieu Tran, Hao Vo +6
University of Arkansas, USA · Max Planck Research School for Intelligent Systems · Google Research, Google +2