Multimodal IR

IR: Information Retrieval

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 81

All topics

AiSearch: Interactive Multi-Modal Search with VLMs
Oct 1, 2026Ali Koksal, Mei Chee Leong, Vicky Sintunata +2Multimodal IR Vision-Language Models
Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore
ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression
Sep 29, 2026Zijing Cai, Yuzhe Wang, Jingxian Zhu +2Multimodal IR Multimodal Embedding
University of Science and Technology of China · Hefei University of Technology · National University of Singapore
VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings
Sep 27, 2026Peixi Wu, Mingzhou Jiang, Feipeng Ma +11Multimodal IR Efficient Multimodal Inference
University of Science and Technology of China · Tsinghua University · Kuaishou +1
UniK: Universal Knowledge Perception for Digital and Physical AI
Sep 21, 2026Nirmit Desai, Kunal Sawarkar, Aditya Mahakali +3Multimodal RAG Multimodal IR
AIntropy AI
Integrating knowledge from case reports: a medical ontology based multimodal information system with structured summary
Sep 17, 2026Shuyu Guo, Lan Huang, Yichen Liu +2Multimodal IR Rare Disease Diagnosis
College of Computer Science and Technology Jilin University Changchun · College of Software Jilin University Changchun · Department of Laboratory Medicine Sichuan Provincial People’s Hospital, School of Medicine, University of Electronic Science and Technology of China Chengdu
Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
Sep 16, 2026Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj +1Multimodal RAG Multimodal IR
University of Tübingen, Tübingen, Germany · Corporate Research, Robert Bosch GmbH, Renningen, Germany · University of Stuttgart, Stuttgart, Germany
RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models
Sep 15, 2026Xun Liang, Honghui Yang, Weihang Pan +6Multimodal IR Multimodal Large Language Models
State Key Lab of CAD&CG, Zhejiang University · Xiaohongshu Inc. · School of Software Technology, Zhejiang University
Hypergraph-Regularized Gramian Volumes for Multimodal Retrieval
Sep 14, 2026Anindya Nag, Ambuj Mehrish, Sebastiano VasconMultimodal IR Cross-Modal Alignment
Department of Environmental Science, Informatics and Statistics Ca’ Foscari University of Venice, Italy
Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings
Sep 14, 2026Mingzhou Jiang, Peixi Wu, Hang Cheng +7Multimodal IR Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Artificial Intelligence and Data Science, USTC · Kuaishou Technology +1
ReGround: Grounding Reviewer Comments in Multimodal Evidence
Sep 11, 2026Serwar Basch, Lizhen Qu, Iryna GurevychMultimodal IR Scholarly Peer Review
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), TU Darmstadt · Department of Data Science & AI, Monash University, Australia
MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval
Sep 1, 2026Debanjan Mahata, Atharva Tendle, Daniel Preotiuc-Pietro +2Multimodal IR Visual Document Retrieval
Bloomberg, NYC, USA
WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents
Aug 28, 2026Zongkai Liu, Hui Zhang, Liqiang Niu +7Multimodal IR Multimodal Search Agents
Weixin AI, Tencent · Sun Yat-sen University
When Images Look Right and Retrieve Wrong: Coverage-Guided Cross-Scale Re-Indexing for Knowledge-Faithful Generative Perception
Aug 21, 2026Guangyuan Dong, Chuang Liu, Haoyu Wang +10Multimodal IR Remote Sensing
1National University of Singapore · 2Wuhan University · 4Nankai University +6
Generative Universal Multimodal Retrieval with Dual-role Identifiers
Aug 13, 2026Kaipeng Li, Haitao Yu, Xuanchen ZhouMultimodal IR Generative Retrieval
Independent Researcher, Tsukuba, Ibaraki, Japan · Institute of Library, Information and Media Science, University of Tsukuba, Tsukuba, Ibaraki, Japan · College of Knowledge and Library Sciences, University of Tsukuba, Tsukuba, Japan
Attribute-Conditioned Multimodal Slot Factorization for Controllable Fashion Retrieval
Aug 12, 2026Najmeh Forouzandehmehr, Topojoy Biswas, Evren Korpeoglu +1Multimodal IR Multimodal Disentangled Representation Learning
Walmart Global Tech Sunnyvale, California, USA
Learning from Failures: Retrieval-Centric CoT via Hard Negatives for Unified Multimodal Retrieval
Aug 6, 2026Zelong Sun, Jun Wang, Kaicheng Yang +3Multimodal IR Multimodal CoT Reasoning
Glint Lab
Bayesian Data Reweighting Improves Multimodal Retrieval for Knowledge-Based Visual Question Answering
Aug 3, 2026Jingchen Sun, Shaobo Han, Ruiyi Zhang +5Multimodal IR Contrastive Learning
University at Buffalo · NEC Laboratories America · Adobe Research +2
UEmbed: Unified Sparse and Dense Multimodal Embeddings
Aug 3, 2026Tingyu Song, Mingxin Li, Yanzhao Zhang +5Multimodal IR Learned Sparse Retrieval
CASIA · University of Chinese Academy of Sciences · Alibaba Group +1
Douyin Multimodal Embedding Model Technical Report
Aug 3, 2026Haonan Chen, Chu Li, Zhicheng Wang +4Multimodal IR Multimodal Pretraining
ByteDance Douyin Search Multimodal Team · Renmin University of China GSAI
DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents
Aug 3, 2026Huanyao Zhang, Jiepeng Zhou, Runhao Zhao +12Multimodal IR Multimodal Large Language Models
1PKU · 2HKUST(GZ) · 3NUDT +3
V-Mem: Modality-Routed Retrieval for Long-Term Multimodal Agentic Memory
Aug 2, 2026Dingyi Kang, Dongming Jiang, Yi Li +2Multimodal IR Multimodal Memory
Department of Computer Science, The University of Texas at Dallas · Department of Electrical and Computer Engineering, University of Florida
UniHEAR: Unified Heterogeneous-Source Attentive Retrieval for Knowledge-Based Visual Question Answering
Aug 2, 2026Ganzhong Luo, Yang Ren, Hanyong Wang +2Multimodal IR Multimodal Reranking
School of Aeronautics and Astronautics, Sichuan University
MMShopBench: A Real-Log Benchmark for Multimodal, Multi-Turn Shopping Agents
Jul 31, 2026Zeying Hao, Hao Guo, Mengtao Xu +5Multimodal IR Multimodal Search Agents
Alibaba Group
When Does Explicit View Routing Work? A Controlled Study of Multi-View Graph-Text Alignment
Jul 29, 2026Xiao Yue, Guangzhi QuMultimodal IR Multi-View Learning
Xiao Yue Guangzhi Qu Oakland University
Diverse-Intent Multi-Turn Fashion Image Retrieval
Jul 22, 2026Mingqiang Tang, Haokun Wen, Meng Liu +3Multimodal IR Cross-Modal Alignment
1Southern University of Science and Technology, Shenzhen, China · 2Harbin Institute of Technology (Shenzhen), Shenzhen, China · 3Shandong University, Jinan, China +1
Pailitao-MMSearch: Building Native E-Commerce Multimodal Search Foundation
Jul 20, 2026Xiaohan Ye, Xu Chen, Zihan Gong +15Multimodal IR Multimodal Pretraining
Taobao & Tmall Group of Alibaba
From Modalities to Propositions: A Language-Centric Framework for Multimodal Intelligence
Jul 18, 2026Nadine Chang, Maying Shen, Shizhe Diao +6Multimodal IR Cross-Modal Representation Learning
NVIDIA · University of Ottawa
Multimodal Scenario Similarity Search for Autonomous Driving
Jul 10, 2026Tamás Matuszka, András Tamásy, Balázs SzolárAutonomous Driving Datasets Multimodal IR
aiMotive
Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation
Jul 7, 2026Shenxi Liu, Kan Li, Mingyang Zhao +2Multimodal IR Temporal Video Grounding
School of Computer Science and Technology, Beijing Institute of Technology · Department of Computing, The Hong Kong Polytechnic University · Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
CMDR: Contextual Multimodal Document Retrieval
Jul 7, 2026Ryota Tanaka, Taku Hasegawa, Kyosuke NishidaMultimodal IR Multimodal Embedding
Human Informatics Labs., NTT, Inc., Tokyo, Japan
Vortex: Multi-Modal Fusion System for Intelligent Video Retrieval
Jun 18, 2026Duc-Tho Nguyen, Hieu-Hoc Tran-Minh, Khanh-Hoa Lam +4Multimodal IR Information Retrieval
University of Science, VNU-HCM, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
VCG: A Multimodal Retrieval Framework for E-Commerce Video Feeds under Extreme Cold-Start Conditions
Jun 17, 2026Katya Mirylenka, Egor Malykh, Mahdyar Ravanbakhsh +10Multimodal IR Cold-Start Recommendation
TU Wien, Vienna, Austria · Zalando Switzerland AG, Zurich, Switzerland · Zalando SE, Berlin, Germany
Hierarchical Multi-Modal Retrieval for Knowledge-Grounded News Image Captioning
Jun 17, 2026Minh-Loi Nguyen, Xuan-Vu Le, Long-Bao Nguyen +2Multimodal RAG Multimodal IR
University of Science, VNU-HCM, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
Multimodal and Multiscale Spatial-Temporal Semantic Search and Recommendation with AI Foundation Models
Jun 15, 2026Yuanyuan Tian, Wenwen Li, Xiao Chen +4Multimodal IR Multimodal Embedding
School of Geographical Sciences and Urban Planning, Arizona State University Tempe, Arizona, USA · Alaska Native Tribal Health Consortium Anchorage, Alaska, USA · Woodwell Climate Research Center Falmouth, Massachusetts, USA +1
Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning
Jun 13, 2026Zhengbo Zhang, Changtao Miao, Jinbo Su +10Multimodal IR Multimodal Grounding
School of Artificial Intelligence, UCAS · Institute of Automation, CAS · Ant Digital Technologies, Ant Group +2
PRISMR: Overcoming Parse Collapse in Multimodal Listwise Ranking via Parameterized Representation Internalization
Jun 11, 2026Hao Jiang, Xin Li, Annan Wang +4Multimodal IR Learning to Rank
1Nanyang Technological University · 2Peking University · 3Independent Researcher
Bridging the Modality Gap in Forensic Image Retrieval
Jun 10, 2026Ricardo González-Gazapo, Annette Morales-González, Yoanna Martínez-Díaz +2Multimodal IR Digital Forensics
Advanced Technologies Application Center (CENATAV), Havana, Cuba · Centro de Sistemas Complejos, Facultad de Física, Universidad de La Habana, Havana, Cuba
Struct-Searcher: Agentic Structural Thinking Advances Multimodal Deep Information Seeking
Jun 5, 2026Fan Zhang, Vireo Zhang, Shengju Qian +7Multimodal IR Belief Revision
1CUHK · 2LIGHTSPEED · 3PKU +3
MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarios
Jun 5, 2026Haitian Wang, Ruoxi Sun, Quantong Qiu +5Multimodal IR Multimodal Embedding
Soochow University, Suzhou, China · Ant Group, Beijing, China
TAPO: Tool-Aware Policy Optimization via Credit Transfer for Multimodal Search Agents
Jun 4, 2026Chengqi Dong, Chuhuai Yue, Hang He +6Multimodal IR Token-Level Credit Assignment
University of Science and Technology of China · 2Meituan
UniCA: Bi-directional Cross-Attention with Positive Similarity Loss for Robust Multi-Modal Retrieval
Jun 3, 2026Yini Huang, Wenlong ZhangMultimodal IR Cross-Modal Alignment
The Hong Kong University of Science and Technology (Guangzhou), Guangdong, China · Southern Medical University, Guangdong, China
Fine-grained Fragment Retrieval in Multi-modal Long-form Dialogues
Jun 3, 2026Hanbo Bi, Zhiqiang Yuan, Chongyang Li +7Multimodal IR Conversational Search
Pattern Recognition Center, WeChat AI, Tencent Inc, China · Aerospace Information Research Institute, Chinese Academy of Sciences
Overview of the EReL@MIR 2025 Multimodal Document Retrieval Challenge (Track 1)
Jun 2, 2026Jingbiao MeiMultimodal IR Multimodal Embedding
University of Cambridge · Cambridge, United Kingdom
Beyond Classification: Dynamic Adapter Routing for Continual Multimodal Retrieval
May 29, 2026Alicja Dobrzeniecka, Filip Szatkowski, Sebastian Cygert +2Multimodal IR VLM Adaptation
NASK National Research Institute · IDEAS Research Institute Warsaw University of Technology · Warsaw University of Technology +2
Inference-Free Multimodal Learned Sparse Retrieval for Production-Scale Visual Document Search
May 29, 2026Gyu-Hwung Cho, Youngjune Lee, Kiyoon Jeong +5Multimodal IR Learned Sparse Retrieval
NAVER Corp., Republic of Korea · Seoul National University, Republic of Korea · NAVER Corp. +6
DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark
May 28, 2026Ruofan Hu, Menghui Zhu, Jieming Zhu +8Multimodal IR LLM Reranking
Zhejiang University · Hangzhou, Zhejiang, China · Huawei Technologies Co., Ltd +2
HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering
May 28, 2026Joongmin Shin, Gyuho Shim, Jeongbae Park +2Multimodal RAG Multimodal IR
Human-inspired AI Research, Korea University · Department of Computer Science and Engineering, Korea University · Computer Science and Engineering, Konkuk University
UniNote: A Unified Embedding Model for Multimodal Representation and Ranking
May 28, 2026Jinghan Zhao, Wenwei Jin, Anqi Li +5Multimodal IR Multimodal Embedding
Xiaohongshu Beijing, China · Shanghai Jiao Tong University Shanghai, China · Huazhong University of Science and Technology Wuhan, Hubei, China +1
DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agents
May 27, 2026Jiamian Wang, Ruiyi Zhang, Tong Yu +5Multimodal IR Multimodal QA
Rochester Institute of Technology · ⋆Work done during an internship at Adobe Research. · Adobe Research
Your Embedding Model is SMARTer Than You Think
May 24, 2026Jianrui Zhang, Hyun Jung Lee, Sukanta Ganguly +3Multimodal IR Efficient Multimodal Inference
UW-Madison · Korea University · NetApp, Inc.
U-CESE: Unified Clip-based Event Search Engine for AI Challenge HCMC 2025
May 22, 2026Duc-Nhuan Le, Hoang-Phuc Nguyen, Thanh-Duy Lam +2Multimodal IR Information Retrieval
Faculty of Information Technology, University of Science, VNU-HCM · Vietnam National University, Ho Chi Minh City, Vietnam
TIGER-FG: Text-Guided Implicit Fine-Grained Grounding for E-commerce Retrieval
May 18, 2026Xinyu Sun, Huangyu Dai, Lingtao Mao +5Multimodal IR Multimodal Grounding
Kuaishou Technology · Beijing, China
SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain
May 18, 2026Lingtao Mao, Huangyu Dai, Xinyu Sun +4Multimodal IR Agentic Search
Kuaishou Technology Beijing, China
SOLAR: Self-supervised Joint Learning for Symmetric Multimodal Retrieval
May 15, 2026Wenjie Yang, Hang Yu, Yuyu Guo +1Cross-Modal Learning Multimodal IR
Ant Group
GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective
May 15, 2026Xu Wang, Xunkai Li, Yinlin Zhu +2Cross-Modal Learning Multimodal IR
School of Airspace Science and Engineering, Shandong University, WeiHai, China · Department of Computer Science, Beijing Institute of Technology, Beijing, China · School of Computer Science and Engineering, Sun Yat-sen University, GuangZhou, China
Very Efficient Listwise Multimodal Reranking for Long Documents
May 12, 2026Yiqun Sun, Pengfei Wei, Lawrence B. HsiehMultimodal IR Efficient Multimodal Inference
Magellan Technology Research Institute (MTRI), Japan.
Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents
May 11, 2026Shijue Huang, Hangyu Guo, Guanting Dong +8Multimodal IR Tool-Using Vision-Language Agents
Hong Kong University of Science and Technology · Renmin University of China · The Chinese University of Hong Kong +3
Hystar: Hypernetwork-driven Style-adaptive Retrieval via Dynamic SVD Modulation
May 11, 2026Yujia Cai, Boxuan Li, Chenghao Xu +1Multimodal IR VLM Adaptation
School of Computer Science and Technology, Xidian University, Xi’an, Shaanxi, China · School of Electronic Engineering, Xidian University, Xi’an, Shaanxi, China
Urban-ImageNet: A Large-Scale Multi-Modal Dataset and Evaluation Framework for Urban Space Perception
May 11, 2026Yiwei Ou, Chung Ching Cheung, Jun Yang Ang +5Multimodal IR Instance Segmentation
University of Auckland · University of Pennsylvania · Stanford University +1
Omni-DeepSearch: A Benchmark for Audio-Driven Omni-Modal Deep Search
May 9, 2026Tao Yu, yiming ding, Shenghua Chai +16Multi-Hop QA Multimodal IR
CASIA · UCAS · BAAI +2

Oct 1, 2026, cs.AIJ/K move · Enter open · S save

AiSearch: Interactive Multi-Modal Search with VLMs

Ali Koksal, Mei Chee Leong, Vicky Sintunata, Ching Ling Chin+1

Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore

Multimodal IR Vision-Language Models Information Retrieval Vision-Language Retrieval

Open paper AskPDF arXiv

Modern retrieval systems must both be automated and interactive, allowing users to search and refine results in real time. We present AiSearch, a flexible multimodal retrieval framework that leverages the zero shot capabilities of Vision Language Models (VLMs) for natural language search over images and videos. AiSearch supports interactive search refinement through user feedback to tailor results to the user's intent, and allows visual benchmarking across multiple VLMs, enabling users to select the most suitable model for their task.

Momentum

Related topics

Latest papers 81