Generative AI Evaluation

Follow this topic Ask about this topic

Momentum

29 papers in the last four weeks, up 142% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 211

All topics

SenseJudge: Human-Centric Preference-Driven Judgment Framework
Jun 2, 2026Rui Li, Junfeng Liu, Xiangwen Kong +2LLM Evaluation LLM-as-a-Judge
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · StepFun · Xi’an Jiaotong University
AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following
Jun 2, 2026Haitao Li, Tian Tan, Yuguang Yang +2Audio-Language Model Evaluation LLM-as-a-Judge
Zhejiang University · Shanghai Innovation Institute · Shanghai Jiao Tong University +1
LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models
Jun 1, 2026Lu Liu, Huiyu Duan, Chenxin Zhu +6Image Restoration Image Quality Assessment
Shanghai Jiao Tong University, Shanghai, China
AI as a Tool for Simulation-Based Experiments in Literary Studies
Jun 1, 2026Matthew WilkensComputational Literary Studies Controllable Text Generation
Department of Information Science Cornell University
"I've Seen How This Goes": Characterizing Diversity via Progressive Conditional Surprise
Jun 1, 2026Matthew Khoriaty, David Williams-King, Shi FengDiverse Text Generation Generative AI Evaluation
ERA Fellowship · George Washington University
Child-directed speech facilitates production, not comprehension, in BabyLMs
May 31, 2026Bastian Bunzeck, Sina ZarrießLanguage Model Pretraining LLM Evaluation
Computational Linguistics, Department of Linguistics Bielefeld University, Germany
Temporally-Aligned Evaluation for Audio-Driven Talking Head Generation
May 31, 2026Zhicheng Zhang, Lei Wang, Yu Zhang +1Talking Head Generation Audio-Driven Facial Animation
School of Business, University of New South Wales (UNSW), Australia. · School of Engineering and Built Environment, Griffith University, Australia. · Data61/CSIRO, Australia.
Can AI Review Improve Paper Drafting? An Empirical Study on 20 Computer Architecture Submissions
May 31, 2026Di WuScholarly Peer Review Automated Peer Review
University of Central Florida Department of ECE Orlando, FL, USA
RWGBench: Evaluating Scholarly Positioning in Related Work Generation
May 30, 2026Anzhe Xie, Weihang Su, Jiaxin Mao +4LLM Evaluation Automated Evaluation
Tsinghua University Beijing, China · Renmin University of China Beijing, China
Industrializing Prediction-Powered Inference: The GLIDE Library for Reliable GenAI and Agentic Systems Evaluation
May 29, 2026Grégoire Martinon, Ibrahim Merad, Mohammed RakiLLM-Assisted Annotation LLM Agent Evaluation
Emerton Data, 16 Avenue Hoche, 75008, Paris, France.
A Persona-Based Evaluation Framework for Pluralistic Alignment in Generative AI
May 29, 2026Atahan KaragozLLM Alignment Persona Consistency
Department of Computer Science University of Basel Basel, Switzerland
Conformal Reliability: A New Evaluation Metric for Conditional Generation
May 29, 2026Yachen Gao, Xinwei Sun, Yikai Wang +4Uncertainty Quantification Conformal Prediction
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · School of Data Science, Fudan University, Shanghai, China +2
Generative Models and Statistical Validation
May 28, 2026Sascha Diefenbacher, Sofia Palacios Schweitzer, Gregor KasieczkaGenerative Modeling Density Estimation
Institut für Theoretische Physik, Universität Heidelberg, Germany · Physics Division, Lawrence Berkeley National Laboratory, Berkeley, USA · NHETC, Department of Physics & Astronomy, Rutgers University, Piscataway, NJ, USA +1
Temporal Stability and Few-Shot Prompting in Math Task Assessment
May 28, 2026Danielle S. Fox, Brenda L. Robles, Elizabeth DiPietro Brovey +1Generative AI in Education Few-Shot Prompting
Learning Research and Development Center, University of Pittsburgh · Institute for Learning, University of Pittsburgh
Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation
May 28, 2026Haoyue Yang, Zhangxiao Shen, Fan Ding +8Code Generation Evaluation Automated Evaluation
Baidu Inc., Beijing, China
BEAMS: Benchmarking and Evaluating AI for Modeling and Simulation
May 27, 2026Sara Metcalf, William SchoenbergAI for Science Human-in-the-Loop AI
ISEE Systems
The Cases LJP Never Sees: Prosecution Decision Prediction for More Complete Criminal Liability Assessment
May 27, 2026Junyu Lu, Qi Wei, Peishuo Zheng +6AI-Assisted Decision Making Legal Reasoning
1Beijing Institute of Technology, Zhuhai · 2Osaka University · 3Xi’an University of Technology +2
GRADE: Generalizable Reasoning-Aware Dialogue Evaluation for AI Tutors
May 27, 2026Parth Bhalerao, Jeromy Chang, David Chou +1Intelligent Tutoring Systems Low-Rank Adaptation
Santa Clara University
GrowLoop: Self-Evolving Conversation Evaluation Seeded by Human
May 26, 2026Yihang Lin, Yunze Gao, Zeyang Lin +3Benchmark Design Human-in-the-Loop Evaluation
Amap, Alibaba Group · The Chinese University of Hong Kong, Shenzhen
AI evaluation may bias perceptions: The importance of context in interpreting academic writing
May 26, 2026Shang Wu, Randol YaoAI-Generated Text Detection Generative AI Evaluation
UC Irvine · MIT
Plans for Evaluating Structured Generative Search Summaries
May 26, 2026Tetsuya Sakai, Jina Lee, Hanpei Fang +1Language Model Generation Evaluation Summarization Evaluation
Waseda University/Naver Corporation · Tokyo, Japan · Waseda University +2
LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV
May 25, 2026Tengfei Liu, Yang Shi, Xuanyu Zhu +17Audio-Video Generation Long-Horizon Video Generation
Peking University · Kling Team · Nanjing University +6
AI-Assisted Systematization for Evaluating GenAI Systems
May 25, 2026Dhruv Agarwal, Emily Sheng, Chad Atalla +6Generative AI Evaluation
Cornell University · Microsoft Research
Divergent Recommendations, Convergent Diagnoses: Cross-Provider Failure-Mode Convergence in AI Commercial Recommendation
May 22, 2026Will Jack, Noah Lehman, Keller Maloney +1Generative Engine Optimization Large Language Model-Based Recommendation
Unusual
NLG Evaluation: Past, Present, Future
May 22, 2026Ehud ReiterLLM Evaluation Language Model Generation Evaluation
Dept of Computing Science University of Aberdeen Aberdeen, UK
EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation
May 22, 2026Songlin Yang, Haobin Zhong, Ruilin Zhang +23VLM Evaluation Video Quality Assessment
The Hong Kong University of Science and Technology · Tencent · Tsinghua University +5
UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment
May 22, 2026Yuanyuan Wang, Dongchao Yang, Yayue Deng +4Reward Modeling Speech Generation Evaluation
The Chinese University of Hong Kong · Tsinghua University · Independent Researcher
Hidden Human-Like Nature of Machine-Generated Texts: Theory and Detection Enhancement
May 22, 2026Chenwang Wu, Yiu-ming Cheung, Bo Han +1AI-Generated Text Detection Generative AI Evaluation
Department of Computer Science, Hong Kong Baptist University, Hong Kong, China · School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui 230000, China
Verifiable Provenance and Watermarking for Generative AI: An Evidentiary Framework for International Operational Law and Domestic Courts
May 20, 2026Gustav Olaf Yunus Laitinen-Fredriksson Lundström-Imanov, Nurana AbdullayevaData Provenance Digital Forensics
Department of Military Studies, Försvarshögskolan (Swedish Defence University), Drottning Kristinas väg 37, 115 93 Stockholm, Sweden · School of Law, ADA University, Ahmadbey Aghaoghlu Street 61, Baku AZ1008, Azerbaijan
On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists
May 20, 2026Seungone Kim, Dongkeun Yoon, Kiril Gashteovski +55Scholarly Peer Review Automated Peer Review

Jun 2, 2026, cs.CLJ/K move · Enter open · S save

SenseJudge: Human-Centric Preference-Driven Judgment Framework

Rui Li, Junfeng Liu, Xiangwen Kong, Linhai Xu+1

State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · StepFun · Xi’an Jiaotong University

LLM Evaluation LLM-as-a-Judge Generative AI Evaluation

Open paper AskPDF arXiv

Large Language Models (LLMs) as judges across various scenarios such as assessing model responses is becoming an increasingly accepted paradigm. However, existing judgment approaches often rely on trained judgers using fixed preference data, which tend to overlook diverse user preferences and struggle to adapt to real-world human-AI dialogue scenarios. To address these limitations, we propose SenseJudge, a customizable judgment framework driven by human preferences and SenseBench, a diverse and challenging instruction-following benchmark derived from real-world multi-turn interactions. We applied the automatic judgment framework and benchmark to two tasks: (1) LLMs as personalized judges, and (2) model ranking. We conducted extensive experiments, and the results demonstrate that the SenseJudge framework surpasses other judgment methods and models in the LLMs-as-personalized-judges task and achieves model ranking that aligns with real human sense. Additionally, we conducted analyses on position bias and consistency, alongside ablation studies, which affirmed the robustness of SenseJudge.

Momentum

Related topics

Latest papers 211