cs.SDNov 13, 2026

CoTTAGE: An Explainable Judge for Text-to-Audio Generation

Authors: Yu-Xiang Lin, Yun-Man Hsu, Bo-Ru Lu, Kuan-Po Huang, Wenze Ren, Ming-To Chuang, Chun-Yi Kuan, Jiawei Du, 

Organizations: National Taiwan University · Amazon AGI · NTU AI-CoRE

Abstract

Rapid progress in text-to-audio (TTA) generation demands reliable and explainable automatic evaluation. However, existing metrics fail to simultaneously deliver high correlation with human judgments and meaningful, audio-grounded explanations, forcing reliance on labor-intensive human evaluation that bottlenecks model development iteration. To address this, we propose CoTTAGE, a reasoning-based evaluator for text relevance (REL) and audio quality (OVL). Following a reason-then-predict paradigm, CoTTAGE first generates an audio-grounded rationale to diagnose errors before predicting scores. Supported by over 10K newly collected human OVL annotations, CoTTAGE is first trained via supervised fine-tuning and then refined with reinforcement learning using a Thurstone-based preference objective that converts sampled score distributions into pairwise preference probabilities and rewards agreement with rankings derived from human judgments. Across six benchmarks spanning REL and OVL, CoTTAGE achieves state-of-the-art correlation with human judgments while producing human-preferred explanations to accelerate TTA development.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. AudioScape-TTA: A Structured Soundscape Benchmark for Fine-Grained Text-to-Audio Evaluation

    Aug 5, 2026Jinting Wang, Yuguang Yang, Shengyu Li +4Text-to-Audio GenerationRubric-Based Evaluation

  2. Resonate: Reinforcing Text-to-Audio Generation via Online Feedback from Large Audio Language Models

    Mar 12, 2026Xiquan Li, Junxi Liu, Wenxi Chen +3Text-to-Audio GenerationGroup Relative Policy Optimization