Composed Video Retrieval

Follow this topic Ask about this topic

A composed query pairs a reference video with an instruction, such as changing an action or describing its consequence, to retrieve a matching target video. Systems commonly use video-text embeddings and feature interpolation to find candidates, while multimodal language models can reason about implied edits and rerank a shortlist. Research also explores temporal localization and consistency, since a good match may need to preserve identities and show coherent state changes across a long video. Evaluation uses Recall@K on composed-retrieval benchmarks, and a central tension is balancing scalable embedding search with the finer reasoning of more costly models.

Latest papers

13 papers0 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Beyond Similarity: Foundation Models as an Efficient Backbone for Training-Free Composed Video Retrieval
Sep 9, 2026Dmitry Demidov, Muhammad Zaigham Zaheer, Omkar Thawakar +2Multimodal Foundation Models Composed Video Retrieval
Mohamed bin Zayed University of Artificial Intelligence, UAE
Learning Sample-wise Rank-aware Interpolation Weights for Composed Visual Data Retrieval
Aug 23, 2026Boseung Jeong, Taegyu Park, Donghyeon Kwon +2Composed Image Retrieval Composed Video Retrieval
AI Center, Samsung Electronics, Republic of Korea · Dept. of CSE, POSTECH, Republic of Korea · Dept. of AI, Ajou University, Republic of Korea +1
IMAGINE: Adaptive Schema-Imagery Enhanced Composition for Composed Video Retrieval
Jun 6, 2026Jiale Huang, Zixu Li, Zhiwei Chen +3Cross-Modal Alignment Cross-Modal Retrieval
Shandong University · Jinan, Shandong, China · Qilu University of Technology +1
Reason-Then-Retrieve for CoVR-R with Structured Edit Prompts and Dense-Sparse Fusion
Jun 1, 2026DongQing Liu, MengShi Qi, HongWei JiCross-Modal Retrieval Video Reasoning
Beijing University of Posts and Telecommunications
Training-Free Composed Video Retrieval via Visual Representation-Guided Video-LLM Reasoning
Jun 1, 2026Yang Liu, Qianqian Xu, Peisong Wen +2Cross-Modal Retrieval Composed Video Retrieval
School of Computer Science and Technology, University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence +2
R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking
May 31, 2026Zixu Li, Yupeng Hu, Zhiheng Fu +3Cross-Modal Retrieval Composed Video Retrieval
1Shandong University · 2Harbin Institute of Technology (Shenzhen)
Dual-Route Top-K Retrieval with 1v1 VLM Reranking for the CoVR-R
May 31, 2026Yuyang Sun, Yongliang Wu, Xingyu Zhu +8Multimodal Reranking Cross-Modal Retrieval
Southeast University · National University of Singapore · Independent Researcher +2
Reason, Retrieve, Re-rank: A Zero-Shot Reasoning-Aware Framework for Composed Video Retrieval
May 30, 2026Ali AlaviMultimodal Reranking Cross-Modal Retrieval
The Ohio State University
UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrieval
Apr 22, 2026Haokun Wen, Xuemeng Song, Haoyu Zhang +3Cross-Modal Learning Learning to Rank
Harbin Institute of Technology (Shenzhen) Shenzhen, China · Southern University of Science and Technology Shenzhen, China · Pengcheng Laboratory Shenzhen, China +1
ReTrack: Evidence-Driven Dual-Stream Directional Anchor Calibration Network for Composed Video Retrieval
Apr 20, 2026Zixu Li, Yupeng Hu, Zhiwei Chen +4Cross-Modal Learning Cross-Modal Representation Learning
School of Software, Shandong University · School of Computer Science and Technology, Shandong Jianzhu University
ViLL-E: Video LLM Embeddings for Retrieval
Apr 13, 2026Rohit Gupta, Jayakrishnan Unnikrishnan, Fan Fei +3Video-Language Models Vision-Language Models
CoVR-R:Reason-Aware Composed Video Retrieval
Mar 20, 2026Omkar Thawakar, Dmitry Demidov, Vaishnav Potlapalli +5Composed Video Retrieval VLM Reasoning
Mohamed bin Zayed University of AI · University of Chicago · University of Wisconsin-Madison +1
CAST: Modeling Visual State Transitions for Consistent Video Retrieval
Mar 9, 2026Yanqing Liu, Yingcheng Liu, Fanghong Dong +3Video Retrieval Composed Video Retrieval
Google · University of California, Santa Cruz · MIT

Sep 9, 2026J/K move · Enter open · S save

Beyond Similarity: Foundation Models as an Efficient Backbone for Training-Free Composed Video Retrieval

Dmitry Demidov, Muhammad Zaigham Zaheer, Omkar Thawakar, Abdelrahman Mohamed Shaker+1

Mohamed bin Zayed University of Artificial Intelligence, UAE

Multimodal Foundation Models Composed Video Retrieval

Open paper AskPDF arXiv

Composed video retrieval (CoVR) searches a gallery for the target video that realizes a natural-language modification of a source clip. However, at gallery scale, this creates a fundamental tension: compact embeddings enable efficient, reusable search but can miss the transient actions, state changes, and subtle constraints that demand fine-grained video reasoning, whereas applying large multimodal models uniformly sacrifices scalability. To address these limitations, we propose that frozen foundation models should instead occupy complementary roles, with inference depth adapted to query difficulty. Based on this premise, we introduce \methodname{}, a framework for training-free \methodexpansion{}. Specifically, a composed-query embedding first searches reusable video-only gallery representations; uncertain queries undergo bounded reranking and candidate expansion; ambiguous edits trigger target-description generation; and only close leading candidates reach multimodal verification. To support these roles, frame selection, spatial resolution, and time cues are adapted to each stage. Across complete target-gallery evaluations, our method reaches state-of-the-art performance among training-free approaches, with 89.55 and 93.43 R@1 on Dense-WebVid-CoVR and CoVR-R, respectively (with more than +35% and +25% absolute margins to the closest counterpart). These results show that adaptively orchestrating foundation-model capabilities can combine scalable retrieval with fine-grained reasoning without task-specific training. The source code and all relevant guidelines are available on https://github.com/demidovd98/CoVRAGE.