cs.CVOct 22, 2026

StateBridge: Phase- and Track-Grounded Reasoning for Synthetic-to-Real Traffic Video Captioning and VQA

Authors: Blessing Agyei Kyem, Richard Dzinyela, Joshua Kofi Asamoah, Eugene Denteh, Andrews Danyo, Solomon Owusu-Ansah, Dickson Gyimah Asiedu, Armstrong Aboah

Organizations: North Dakota State University, Fargo ND 58102, USA · Sustainable Mobility and Advanced Research in Transportation (SMART) Lab · Texas A&M University, College Station 77840, TX, USA · Ghana Institute of Management and Public Administration

Abstract

Traffic safety videos provide rich evidence for understanding interactions between pedestrians and vehicles, but synthetic-to-real evaluation makes this setting challenging because models must learn from simulated scenes and generalize to real-world traffic. In this work, we present StateBridge, a traffic video-language framework designed to operate under this synthetic-only training protocol. Our approach uses phase-aware clip construction, pedestrian and vehicle track summaries, and structured prompts to ground vision-language models in stable traffic evidence rather than appearance alone. We fine-tune public vision-language models on synthetic captioning and visual question answering examples, then combine their outputs through fact-locked caption fusion and evidence-diverse VQA consensus to improve consistency across actor roles, behavioral phases, and question types. StateBridge achieves an official S2 score of 55.4679 on AI City Challenge 2026 Track 2 using only synthetic training data. These results demonstrate the effectiveness of structured evidence transfer for traffic safety captioning and visual question answering.

Figures & tables

Explore similar work

CardsList
  1. Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA

    Sep 16, 2026Nguyen Hoai Thuong Bui, Thanh Nguyen Vo, Trinh Tra Giang Nguyen +1VLMs for Autonomous DrivingVisual Question Answering

  2. UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations

    Aug 13, 2026Peng Li, Qianqian Xu, Shilong Bao +2Pedestrian Intention PredictionIntelligent Transportation Systems