cs.CVNov 10, 2026

AITHER: Authority-Informed Evidence Routing for Mixed-Authority Vision-Language Reasoning

Authors: Xinjin Li, Haoyue Zhang, Xinyi Gou, Yuruo Gong, Yuanjian He, Muzi Li, Yujian Long, Yu Ma, 

Organizations: Columbia University, United States of America · UC Berkeley, United States of America · Texas A&M University, United States of America · Rice University, United States of America · 

Abstract

Reliable vision-language reasoning requires a control decision that conventional grounding methods leave implicit: which input is entitled to determine the answer? Auxiliary text may be a defeasible claim about an image, a binding task instruction, or compatible context. Correcting every image-text conflict toward the image therefore repairs false claims but overwrites valid task rules. We formalize this setting as mixed-authority vision-language reasoning and introduce AITHER, an Authority-Informed Text-Image Hierarchy for Evidence Routing. Its final policy, the Semantic-Frame Router, is a training-free black-box control layer that (i) compiles explicit task directives, (ii) recovers standardized spatial frames from metadata-scrubbed pixels, (iii) resolves the semantic polarity of remaining notes, and (iv) routes among frozen observable candidates with fail-closed semantics. This ordered decomposition converts authority arbitration from an unconstrained prompt preference into a reproducible inference program. Across 1,620 MATRIX predictions, AITHER reaches 95.1% accuracy, exceeding the strongest task-and-frame baseline by 9.1 points with a 95% record-cluster interval of 6.8-11.5 points and no harmed records. It further reaches 100.0% on 1,800 controlled FRAME predictions and 93.3% on GQA-MATRIX. The results show that explicit authority routing can simultaneously improve false-claim correction, preserve task-defined answers, and eliminate clean-context overcorrection.

Figures & tables

Explore similar work

CardsList
  1. CaVe-VLM-CoT: An Interpretable Vision-Language Model Framework

    Jun 16, 2026Sneha Rao, Shaina Raza, Dhanesh RamachandramVLM EvaluationVision-Language Grounding

  2. When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning

    Aug 5, 2026Yongxin Wang, Ruizhe Zhou, Yueling Tang +4Multimodal QAMultimodal Reasoning