Vision-Language Model-as-a-Judge

Momentum

6 papers in the last four weeks, with none the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 23

All topics
CardsList
  1. Visual Jev Rewards: Reference-Bound Verification for Multi-Subject Image Generation

    Oct 7, 2026Baoteng Li, Wenzhuo Wu, Kongming Liang +1Reward ModelingImage Generation Evaluation

  2. Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation

    Oct 1, 2026Yuan Huang, Zirui Song, Xiuying ChenVLM RobustnessLLM-as-a-Judge

  3. VisionQ: VLM-as-a-Judge Taxonomy, Dataset and Benchmark for Qualitative Analysis in Computer Vision

    Sep 30, 2026Vu Dinh Xuan, Duc-Hai Nguyen, Minh-Dung Dao +6VLM EvaluationVision-Language Model-as-a-Judge

  4. It's Not What the Image Shows: Irrelevant Context Destabilises VLM Judges Without Informing Them

    Sep 29, 2026Nagham Omar, Mahmoud Jabarin, Kinan Ibraheem +1VLM EvaluationFigurative Language Understanding

  5. ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation

    Sep 21, 2026Lijian Wu, Henry Hengyuan Zhao, Zijian Zhang +3VLM EvaluationLLM Evaluation

  6. Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation

    Jun 22, 2026Yu Cao, Ziquan Liu, Zhensong Zhang +3VLM EvaluationPhysical Consistency in Video Generation

  7. TimeVista: Exploring and Exploiting Vision-Language Models as Judges for Time Series Forecasting

    Jun 15, 2026Zhi Chen, Yuxuan Wang, Jialong Wu +5VLM EvaluationForecasting Benchmarks

  8. TECCI: Tricky Edits of Collected and Curated Images

    May 31, 2026Aishwarya Agrawal, Roy Hirsch, Yasumasa Onoe +2VLM EvaluationImage Editing Evaluation

  9. A Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluation

    May 29, 2026Yi Zhao, Siqi Wang, Zhe Hu +2VLM EvaluationLLM-as-a-Judge

  10. DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification

    May 10, 2026Rui Liu, Dian Yu, Zhenwen Liang +6Multimodal Large Language ModelsMultimodal Reward Modeling

  11. VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation

    Apr 28, 2026Divake Kumar, Sina Tayebati, Devashri Naik +2VLM EvaluationVision-Language Models

  12. SycoPhantasy: Quantifying Sycophancy and Hallucination in Small Open Weight VLMs for Vision-Language Scoring of Fantasy Characters

    Apr 27, 2026Arya Shah, Deepali Mishra, Chaklam SilpasuwanchaiVLM EvaluationVision-Language Models

  13. Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models

    Apr 23, 2026Mohammed Safi Ur Rahman Khan, Sanjay Suryanarayanan, Tushar Anand +1VLM EvaluationVLM Robustness

  14. Lost in Translation: Do LVLM Judges Generalize Across Languages?

    Apr 21, 2026Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Mir Tafseer Nayeem +5VLM EvaluationMultilingual VLM Evaluation

  15. MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge

    Apr 20, 2026Sua Lee, Sanghee Park, Jinbae ImLLM-as-a-JudgeMultimodal Large Language Models

  16. When Vision-Language Models Judge Without Seeing: Exposing Informativeness Bias

    Apr 20, 2026Xiaohan Zou, Roshan Sridhar, Mohammadtaher Safarzadeh +1VLM EvaluationVision-Language Alignment

  17. TIIF-Bench: How Does Your T2I Model Follow Your Instructions?

    Jun 2, 2025Xinyu Wei, Jinrui Zhang, Zeqing Wang +4Vision-Language Model-as-a-JudgeT2I Generation Evaluation

  18. Multimodal Language Models as Text-to-Image Model Evaluators

    May 1, 2025Jiahui Chen, Candace Ross, Reyhane Askari-Hemmat +5LLM-as-a-JudgeBenchmark Design