cs.CVNov 9, 2026

TRIAD: Multimodal Retrieval across Abstraction, Constraints, and State

Authors: Xiangyang Wu, Chen Ju, Xu Chen, Likang Peng, Jinsong Lan, Haihong Tang, Yang Hu

Organizations: Alibaba Group · Hangzhou International Innovation Institute, Beihang University · Sino-Italian Joint Research Laboratory on Prognostics and Health Management for Intelligent Maintenance, Beihang University

Abstract

Multimodal retrieval increasingly supports agentic systems, where user requests can involve cross-level relations, combine multiple constraints, and evolve across dialogue turns. In these settings, a highly similar result may still violate the user's requirements, while retrieved content may directly inform downstream actions without human screening. We introduce TRIAD, a multimodal retrieval benchmark for evaluating how well retrieval systems satisfy these complex, evolving requests. TRIAD comprises three tracks: abstraction, testing bidirectional retrieval between visual instances and concepts at different hierarchical levels; constraints, testing the joint satisfaction of positive and negative conditions; and state, testing the integration of requirements across dialogue turns and the updating of earlier conditions when users revise their requests. The benchmark contains six subsets, each with 1,000 queries and a fixed pool of 1,000 candidates per query, evaluated using a common multi-positive mean average precision (mAP) protocol. Simpler reference tasks and hard negatives grouped by the requirements they violate help characterize specific retrieval difficulties. We evaluate 20 configurations of 17 systems and find that no system leads across all six subsets. Performance declines with hierarchical distance, and negative-only queries generally underperform positive-only queries with the same number of constraints. Across paired tasks expressing the same final intent, multi-turn retrieval yields lower mAP in 39 of 40 configuration-domain comparisons. The benefits of explicit reasoning also vary across models and tasks. These findings highlight the gap between retrieving similar content and satisfying a complete request.

Figures & tables

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval

    Mar 1, 2026Xuan Lu, Kangle Li, Haohang Huang +3Multimodal IRCross-Modal Retrieval

  2. ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge

    Feb 10, 2026Yijie Lin, Guofeng Ding, Haochen Zhou +3Multimodal IRVisual Spatial Reasoning

  3. MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents

    Mar 5, 2026Dannong Xu, Zhongyu Yang, Jun Chen +6Multimodal IRMultimodal Model Evaluation