cs.CLJun 8, 2026

MAVEN: A Macro-Societal Value Evaluation Framework of Multimodal Content with Compact Aligned Evaluators

Authors: Zijuan Zhao, Zheren Fu, Hou Xia, Licheng Zhang, Yi Liu, Zhendong Mao

Organizations: University of Science and Technology of China · State Key Laboratory of Communication Content Cognition, People’s Daily Online

Abstract

Assessing whether multimodal content aligns with macro-societal values, such as peace, justice, and freedom, has become an increasingly urgent challenge. Existing frameworks are largely confined to safety-oriented taxonomies, text-only psychometric probes, or single-label classification. Therefore, we propose MAVEN, a hierarchical framework for macro-societal value evaluation of multimodal content, grounded in international human-rights instruments and cultural value theory. MAVEN organizes values into 6 primary dimensions and 72 secondary indicators, supporting multi-level quantitative scoring. Building on MAVEN, we construct a human-verified multimodal benchmark and a soft-match metric to evaluate VLMs' assessments across value dimensions. For evaluator optimization, we propose a span-adaptive variant of multi-level preference optimization for evaluator distillation, together with a training-free multi-role consensus strategy at inference time. We evaluate existing open- and closed-source VLMs on our benchmark, revealing shared tendencies and clear differences in macro-societal value judgments. Experiments show that our compact 2B evaluator matches its 8B counterpart in the same family and approaches frontier closed-source VLMs, offering a practical path toward scalable macro-societal value evaluation. Our SA-MDPO implementation and MacroValue-Bench are available at https://github.com/zzzzzzzzjj/MAVEN.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Towards Cross-lingual Values Judgment: A Consensus-Pluralism Perspective

    Feb 19, 2026Yukun Chen, Xinyu Zhang, Boyi Deng +6Cross-Cultural Language Model EvaluationMultilingual Language Model Evaluation

  2. ValueGround: Evaluating Culture-Conditioned Visual Value Grounding in MLLMs

    Apr 7, 2026Zhipin Wang, Christoph Leiter, Christian Frey +3Cultural Knowledge in Language ModelsVLM Evaluation

  3. MM-SCALE: Evaluating Evidence-Grounded Moral Judgment in Vision-Language Models

    Feb 3, 2026Eunkyu Park, Wesley Hanwen Deng, Cheyon Jin +8VLM EvaluationVLM Reasoning