Image Captioning

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, down 12% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 59

All topics

Hierarchical Multi-Modal Retrieval for Knowledge-Grounded News Image Captioning
Jun 17, 2026Minh-Loi Nguyen, Xuan-Vu Le, Long-Bao Nguyen +2Multimodal RAG Multimodal IR
University of Science, VNU-HCM, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
CIAN: Multi-Stage Framework for Event-Enriched Image Captioning via Retrieval-Augmented Generation
Jun 16, 2026Trinh Thi Thu Hien, Trung-Nghia LeMultimodal RAG Image Captioning
Faculty of Information Technology, University of Science, Ho Chi Minh City · 2Vietnam National University, Ho Chi Minh City, Vietnam
Gaze Heads: How VLMs Look at What They Describe
Jun 12, 2026Rohit Gandikota, David BauImage Captioning Visual Attention
Northeastern University
Zero-Shot Captioning for Cultural Heritage: Automated Image Analysis of Traditional Indonesian Clothing
Jun 11, 2026Anugrah Aidin Yotolembah, Novanto Yudistira, Gembong Edhi SetyawanRetrieval-Augmented Generation Image Captioning
Department of Informatics Engineering Faculty of Computer Science Brawijaya University Malang, Indonesia
CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning
Jun 8, 2026Penghui Yang, Long Xing, Xiaoyi Dong +10Image Captioning Multimodal Pretraining
Tsinghua University · University of Science and Technology of China · Microsoft +4
Beyond Self-Attention: Sub-Quadratic Vision Transformers for Fast Image Captioning
Jun 7, 2026Chiradeep Ghosh, Dakshina Ranjan KiskuImage Captioning Vision Transformer
Department of Computer Science and Engineering · National Institute of Technology Durgapur · Durgapur, India
Vision-Language Asymmetry in Bistable Image Captioning
Jun 6, 2026Arohan AgateImage Captioning Vision-Language Models
University of Washington.
T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining
May 30, 2026Tayeba Qazi, Ayush Maheshwari, Prerana Mukherjee +1Thermal Imaging Image Captioning
Indian Institute of Technology Delhi, India · NVIDIA AI Technology Center, India · Jawaharlal Nehru University, India
A Pilot Study on Curator-Guided Multilingual Art Description for Blind and Low-Vision Audiences with Small Vision-Language Models
May 29, 2026Iosif Tsangko, Andreas Triantafyllopoulos, George Margetis +2Image Captioning VLM Adaptation
Technical University of Munich Munich, Germany · Foundation for Research and Technology – Hellas Heraklion, Greece · National University of Science and Technology Politehnica Bucharest Bucharest, Romania
VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning
May 27, 2026Xingyu Lu, Jinpeng Wang, Yi-Fan Zhang +13Image Captioning Object Hallucination in VLMs
1Tsinghua Shenzhen International Graduate School · 2Harbin Institute of Technology, Shenzhen · 3Chinese Academy of Sciences +1
BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model
May 20, 2026Gonçalo Gomes, Bruno Martins, Chrysoula ZervaVLM Evaluation Image Captioning
Instituto Superior Técnico, University of Lisbon · INESC-ID · Instituto de Telecomunicações
Machine Intelligence that Understands Visual and Linguistic Information and Interacts with Humans and Environments
May 20, 2026Van Quang NguyenImage Captioning Efficient ViTs
TOHOKU UNIVERSITY · Graduate School of Information Sciences · Department of System Information Sciences
Retrieval-Augmented Long-Context Translation for Cultural Image Captioning: Gators submission for AmericasNLP 2026 shared task
May 20, 2026Aashish Dhawan, Christopher Driggers-Ellis, Dzmitry Kasinets +2Retrieval-Augmented Generation Image Captioning
University of Florida
FullFlow: Upgrading Text-to-Image Flow Matching Models for Bidirectional Vision--Language Generation
May 19, 2026Eric Tillmann Bill, Enis Simsar, Alessio Tonioni +1Flow Matching Image Captioning
ETH Zurich · Google
ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison
May 19, 2026Tianle Li, Xuyang Shen, Yan Ma +7VLM Evaluation Reward Modeling
1The Chinese University of Hong Kong · 2MiniMax
EmoMind: Decoding Affective Captions from Human Brain fMRI
May 16, 2026Bilal A. Mohammed, Lin Gu, Ruogu FangImage Captioning Neural Decoding
Department of Biomedical Engineering Vanderbilt University Nashville, TN · Research Institute of Electrical Communication Tohoku University Sendai, Japan · Department of Biomedical Engineering University of Florida Gainesville, FL
HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning
May 14, 2026Man Wang, Chenyang Liu, Wenjun Li +5Remote Sensing Image Understanding Image Captioning
College of Computer Science, Inner Mongolia University, Hohhot 010021, China · Research Center for Spatiotemporal Intelligence, Inner Mongolia University, Hohhot 010021, China · Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University, Beijing 100191, China +3
BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
May 8, 2026Shaokai Ye, Vasileios Saveris, Yihao Qian +3Multi-Objective Reinforcement Learning Image Captioning
Apple
Weather-Robust Scene Semantics with Vision-Aligned 4D Radar
May 8, 2026Kali Hamilton, Christoffer HeckmanCross-Modal Learning VLM Robustness
Autonomous Robotics and Perception Group at the University of Colorado Boulder, Boulder, CO 80309, USA.
DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioning
May 6, 2026Yuancheng Wei, Haojie Zhang, Linli Yao +7Image Captioning Multimodal Model Evaluation
South China University of Technology · Peking University · The University of Hong Kong +2
Sentinel2Cap: A Human-Annotated Benchmark Dataset for Multimodal Remote Sensing Image Captioning
May 4, 2026Lucrezia Tosato, Gianluca Lombardi, Ronny HanschRemote Sensing Image Understanding Image Captioning
LIPADE, Université Paris Cité, 75006 Paris, France. · SAPIA, ONERA, Palaiseau, France. · LCQB, Sorbonne Université, CNRS, IBPS, 75005 Paris, France. +1
Linguistically Informed Multimodal Fusion for Vietnamese Scene-Text Image Captioning: Dataset, Graph Framework, and Phonological Attention
Apr 30, 2026Nhi Ngoc-Yen Nguyen, Anh-Duc Nguyen, Nghia Hieu Nguyen +2Image Captioning Low-Resource Language Processing
Faculty of Information Science and Engineering · University of Information Technology · Vietnam National University, Ho Chi Minh, Viet Nam
CANVAS: Captioning Art with Narrative Visual-Audio AI Systems
Apr 30, 2026Vignesh NagarajanImage Captioning
BASIS Phoenix High School, Phoenix, Arizona
Retrieval-Guided Generation for Safer Histopathology Image Captioning
Apr 27, 2026Md. Enamul Hoq, Wataru Uegami, Saghir Alfasly +6Multimodal RAG Computational Pathology
Kimia Lab, Department of Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA · Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USA · Department of Computer Science and Engineering, Princeton University, Princeton, NJ, USA +2
JSSFF: A Joint Structural-Semantic Fusion Framework for Remote Sensing Image Captioning
Apr 27, 2026Swadhin Das, Vivek YadavRemote Sensing Image Understanding Image Captioning
Computer Science and Engineering · University of Petroleum and Energy Studies · Dehradun, India +1
STAND: Semantic Anchoring Constraint with Dual-Granularity Disambiguation for Remote Sensing Image Change Captioning
Apr 25, 2026Yanpei Gong, Beichen Zhang, Hao Wang +5Remote Sensing Image Understanding Image Captioning
Harbin Institute of Technology, Weihai, China
Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contexts
Apr 20, 2026Run Xu, Lu Li, Rongzhao Zhang +1Image Captioning Multimodal Generation
Nanyang Technological University · Tongji University · Shanghai Artificial Intelligence Laboratory
Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
Dec 16, 2025George-Andrei Dima, Răzvan-Alexandru Smădu, Dumitru-Clementin CercelVisual Question Answering Image Captioning
National University of Science and Technology POLITEHNICA Bucharest, Bucharest, Romania
Redemption Score: A Multi-Modal Evaluation Framework for Image Captioning via Distributional, Perceptual, and Linguistic Signal Triangulation
May 22, 2025Ashim Dahal, Ankit Ghimire, Saydul Akbar Murad +1Image Captioning Multimodal Model Evaluation
University of Southern Mississippi, Hattiesburg, MS, USA · School of Business & Technology, Emporia State University, Emporia, KS, USA

Jun 17, 2026, cs.CVJ/K move · Enter open · S save

Hierarchical Multi-Modal Retrieval for Knowledge-Grounded News Image Captioning

Minh-Loi Nguyen, Xuan-Vu Le, Long-Bao Nguyen, Hoang-Bach Ngo+1

University of Science, VNU-HCM, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam

Multimodal RAG Multimodal IR Image Captioning

Open paper AskPDF arXiv

Traditional image captioning methods often struggle to generate comprehensive, context-rich descriptions, especially for details not directly observable from visual cues. To overcome this, we propose a novel retrieval-augmented image captioning framework that generates captions with deeper insights, such as object attributes, event context, and underlying significance, by leveraging external knowledge. Our approach features a hierarchical multi-modal article retrieval mechanism that moves beyond monolithic text entities. This retrieval considers article structure-aware features, including weighted textual components (e.g., headlines, body sections) and visual placement patterns, alongside multi-faceted similarity computations (content--visual, visual--visual, and discourse positioning). A subsequent contextual relevance refinement stage further enhances the retrieved information. The retrieved articles then serve as the knowledge base for caption generation: first, a VLM generates a concise image description; second, we segment relevant information from the retrieved articles based on this description; and finally, an LLM utilizes both the description and extracted knowledge to generate a comprehensive, contextually detailed caption. We participated in the ACM Multimedia EVENTA 2025 Challenge and achieved 5th place with an overall score of 0.2824 on the private test set of the OpenEvent-V1 dataset. Source code is publicly released at https://github.com/mf0212/EVENTA-Challange.

Momentum

Related topics

Latest papers 59