cs.CVNov 16, 2026

VISTA: A Framework for Vision-Informed Semantic Traffic Awareness

Authors: Murat Arda Onsu, Poonam Lohan, Burak Kantarci, Aisha Syed, Matthew Andrews, Sean Kennedy

Organizations: University of Ottawa, Ottawa, ON, Canada · Nokia Bell Labs, 600 March Road, Kanata, ON K2K 2E6, Canada

Abstract

The Internet of Things (IoT) is transforming intelligent transportation systems by connecting roadside cameras, edge computing units, vehicles, and traffic infrastructure into a shared sensing and decision-making ecosystem. However, transmitting raw traffic video over vehicle-to-everything (V2X) links is bandwidth-intensive, while deploying large multimodal models directly on resource-constrained vehicles is often impractical. To address these limitations, this article proposes VISTA (Vision-Informed Semantic Traffic Awareness), a multimodal vision-language edge-intelligence framework that combines predictive video representation learning with lightweight language-based reasoning to provide communication-efficient traffic situational awareness. In the proposed architecture, roadside surveillance units employ the Video Joint Embedding Predictive Architecture (V-JEPA) and an attentive-pooler module to convert traffic video clips into compact future-scene embeddings. These embeddings are transmitted through V2X links to vehicle-mounted systems, where a TinyLlama small language model, enhanced with a custom visual adapter, interprets the embeddings alongside natural-language (NL) traffic queries. VISTA enables vehicles to obtain interpretable traffic information, reason about upcoming scene dynamics, and identify potential collision risks without receiving raw video streams. Experimental results in a digital-twin urban traffic environment show that VISTA achieves 97.77% accuracy and an 89% F1 score for future collision prediction while reducing the communication payload by several orders of magnitude compared with raw RGB video transmission. These results demonstrate the feasibility of embedding-based multimodal intelligence for low-latency, communication-efficient, and safety-oriented IoT-enabled transportation systems.

Figures & tables

Explore similar work

CardsList
  1. Spatiotemporal Semantic V2X Framework for Cooperative Collision Prediction

    Jan 23, 2026Murat Arda Onsu, Poonam Lohan, Burak Kantarci +3Intelligent Transportation SystemsSemantic Communication

  2. A Lightweight Digital-Twin-Based Framework for Edge-Assisted Vehicle Tracking and Collision Prediction

    Mar 7, 2026Murat Arda Onsu, Poonam Lohan, Burak Kantarci +3Traffic Accident AnticipationIntelligent Transportation Systems