Evidence-based Distributional Alignment for Large Language Models

Viet-Thanh Pham; Lizhen Qu; Zhuang Li; Gholamreza Haffari

cs.LGMar 3, 2026

Evidence-based Distributional Alignment for Large Language Models

Authors: Viet-Thanh Pham, Lizhen Qu, Zhuang Li, Gholamreza Haffari

Organizations: Department of Data Science & AI, Monash University, Australia · Royal Melbourne Institute of Technology, Australia

Abstract

Distributional alignment enables large language models (LLMs) to predict how a target population distributes its responses across answer options, rather than collapsing disagreement into a single consensus answer. However, existing LLM-based distribution prediction is often unstable and degrades under cultural and domain shift. Token score-based estimates can change with minor option wording or formatting, response sampling-based estimates are expensive and sensitive to prompts and decoding settings, and directly generated distributions are frequently miscalibrated. We propose Evi-DA, an evidence-based alignment technique that improves the fidelity and robustness of LLM-based distribution estimation under domain and cultural shift. Given a target country and a multiple-choice question, Evi-DA retrieves related World Values Survey items and their answer distributions, predicts a coarse Welzel value signature for each option, and infers the country-conditioned answer distribution in a structured format. We train the LLMs using a two-stage pipeline, where reinforcement learning optimizes survey-derived rewards that encourage accurate intermediate value predictions, faithful final distributions, well-formed structured outputs, and reduced cultural bias. Across in-domain and out-of-domain benchmarks and multiple open-source backbones, Evi-DA reduces Jensen-Shannon divergence between predicted and gold distributions relative to strong baselines, with average relative improvements of up to 44%.

Figures & tables

Figure 1: Illustration of Evi-DA - our proposed method for distributional alignment with LLMs.

Method	Qwen3-8B	Qwen3-14B	Qwen3-32B	Ministral-8B	Ministral-14B	Llama3-8B	Average
Baselines
Model Log-Probabilities	0.41	0.42	0.22	0.24	0.24	0.21	0.29
Model Log-Probabilities + KL	0.37	0.32	0.16	0.23	0.19	0.15	0.24
Opinion Sampling	0.30	0.32	0.21	0.25	0.12	0.30	0.25
Verbalized Distribution	0.26	0.21	0.20	0.26	0.31	0.22	0.24
Modular Pluralism	0.20	0.21	0.15	0.18	0.16	0.19	0.18

Table 1: Results of different distributional alignment methods on the in-domain test set. Results are calculated on the Jensen-Shannon Divergence metric (lower is better).

Method	Qwen3-8B	Qwen3-14B	Qwen3-32B	Ministral-8B	Ministral-14B	Llama3-8B	Average
Baselines
Model Log-Probabilities	0.47	0.43	0.36	0.33	0.39	0.37	0.39
Model Log-Probabilities + KL	0.44	0.32	0.29	0.26	0.29	0.18	0.30
Opinion Sampling	0.44	0.31	0.34	0.41	0.36	0.30	0.36
Verbalized Distribution	0.43	0.39	0.34	0.29	0.32	0.38	0.36
Modular Pluralism	0.32	0.29	0.22	0.34	0.34	0.28	0.29

Table 2: Results of different distributional alignment methods on the out-of-domain test set. Results are calculated on the Jensen-Shannon Divergence metric (lower is better).

Method	Qwen3-8B	Qwen3-14B	Qwen3-32B	Ministral-8B	Ministral-14B	Llama3-8B	Average
Evi-DA - No training	0.22	0.30	0.23	0.20	0.25	0.23	0.24
Evi-DA - No Welzel’s values	0.29	0.45	0.38	0.21	0.36	0.33	0.40
Evi-DA - No Evidence	0.48	0.38	0.46	0.39	0.55	0.38	0.44
Evi-DA (Default)	0.25	0.27	0.16	0.17	0.22	0.28	0.23

Table 3: Performance of Evi-DA when removing core components, measured in Jensen-Shannon Divergence (lower is better). Evaluation results calculated on the out-of-domain test set.

Method	America	Germany	France	China	Vietnam
Task: LMH Signature Prediction (Accuracy)
Evi-DA - Without training	0.87	0.89	0.82	0.78	0.72
Evi-DA - With training	0.92	0.91	0.88	0.85	0.83
Task: Distribution Prediction (Jensen-Shannon Divergence)
Evi-DA - Without training	0.23	0.29	0.32	0.37	0.43
Evi-DA - With training	0.12	0.18	0.18	0.22	0.31

Table 4: Performance of Evi-DA with and without GRPO on the tasks of LMH signature prediction and distribution prediction on different countries. Evaluation is carried out on the out-of-domain test set.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Reward Weights
$\lambda_{1}$ for $R_{\text{LMH}}$	0.25
$\lambda_{2}$ for $R_{\text{dist}}$	0.45
$\lambda_{3}$ for $R_{\text{schemaA}}$	0.15
$\lambda_{4}$ for $R_{\text{schemaB}}$	0.15
Core GRPO Params

Appendix

Table 5: Hyperparameters of GRPO for Evi-DA.

Figure 2: Illustration of Evi-DA with different LLM backbones on the out-of-domain benchmark when changing the value $K$ for number of retrieved samples.

Figure 3: Illustration of Evi-DA with different Embedding Models for Retrieval on the out-of-domain benchmark.

LLM Alignment

Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.

See all 611 papers

Speak vs. Know: Balancing Behavior and Knowledge Distributions of Large Language Models
Oct 20, 2026Yang Liu, Masahiro Kaneko, Chenhui ChuLLM Evaluation Language Model Generation Evaluation
Kyoto University · MBZUAI
Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook
Mar 16, 2026Jaehyeok Lee, Xiaoyuan Yi, Jing Yao +4LLM Evaluation LLM Alignment
Sungkyunkwan University · Microsoft Research Asia · Singapore University of Technology and Design
Controlling Distributional Bias in Multi-Round LLM Generation via KL-Optimized Fine-Tuning
Apr 7, 2026Yanbei Jiang, Amr Keleg, Ryandito Diandaru +4LLM Alignment LLM Fine-Tuning
University of Melbourne · Mohamed bin Zayed University of Artificial Intelligence · Oracle

Oct 20, 2026J/K move · Enter open · S save

Speak vs. Know: Balancing Behavior and Knowledge Distributions of Large Language Models

Yang Liu, Masahiro Kaneko, Chenhui Chu

Kyoto University · MBZUAI

LLM Evaluation Language Model Generation Evaluation

Open paper AskPDF arXiv

Recent work reports that the distribution verbalized directly in the output text aligns far better with human opinion distributions than the next-token log-probability distribution of LLMs. However, the verbalized distribution reflects what LLMs know about a group's opinions (knowledge distribution), while what they actually output at deployment is governed by their log-probability distribution (behavior distribution). We find that when prompting LLMs to predict the opinion of a specific group, the most likely answer under the verbalized distribution differs from that under the log-probability distribution for 37% of World Values Survey questions, averaged over three LLMs. Thus, LLMs' knowledge does not match their behavior. This matters for evaluation: a verbalized distribution can score well against a group while its most likely answer is one the model never outputs, so the score no longer describes the model. The alignment scores for these two distributions reflect only the model's behavior or its knowledge, separately. We therefore propose a learnable combination of the two distributions that follows the most likely answer of the behavior distribution while preserving the shape of the knowledge distribution, with a single hyperparameter trading off behavioral fidelity against distribution quality. Our experiments on three opinion survey datasets and six models show that the balanced distribution improves behavioral consistency over the verbalized distribution from 71.6% to 95.6% on average, while the total variation distance to the human opinion distributions rises only slightly from 28.8% to 33.8%, and it remains effective under temperature scaling and cross-dataset settings.

Abstract

LLM Alignment

Explore similar work

Speak vs. Know: Balancing Behavior and Knowledge Distributions of Large Language Models

Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook

Controlling Distributional Bias in Multi-Round LLM Generation via KL-Optimized Fine-Tuning