cs.CLNov 2, 2026

A Conditional Mutual Information Analysis of Response Variety in Language Models

Authors: R. I. Cukier

Organizations: Department of Chemistry, Michigan State University, East Lansing, Michigan 48824-1322, USA

Abstract

Large language models provide a variety of responses at inference (generation) that conventionally depend on a temperature parameter. Using a conditional mutual information approach, a quantitative, entropy based analysis of response variety is given. Three limiting behaviors are defined: arbiter, independent (conditionally) and neutral (deterministic) that span no to maximal mutual information. We give examples of each behavior and suggest their application to: temperature analysis, language analysis, prompt compression, and knowledge graphs.

Explore similar work

CardsList
  1. Probabilistic Attribution For Large Language Models

    May 20, 2026Shilpika Shilpika, Carlo Graziani, Bethany Lusch +2LLM InterpretabilityLLM Uncertainty Estimation

  2. One Temperature to Rule Them All?

    Nov 9, 2026Aviv Orly, Ori Shem-Ur, Yaron OzLLM InferenceDiversity-Aware Sampling