Total Correlation
Total correlation, a measure of the collective dependence among multiple variables, is a key concept in various fields seeking to understand complex relationships beyond simple pairwise correlations. Current research focuses on improving the interpretability and reliability of correlation measures, particularly in the context of machine learning models and their susceptibility to spurious correlations, employing techniques like mutual information, Jensen-Shannon divergence, and various information-theoretic approaches. Understanding and mitigating the effects of spurious correlations is crucial for building robust and generalizable models across diverse applications, from natural language processing and image analysis to financial modeling and ecological systems analysis.
Papers
Correlation and Navigation in the Vocabulary Key Representation Space of Language Models
Letian Peng, Chenyang An, Jingbo Shang
Beyond correlation: The impact of human uncertainty in measuring the effectiveness of automatic evaluation and LLM-as-a-judge
Aparna Elangovan, Jongwoo Ko, Lei Xu, Mahsa Elyasi, Ling Liu, Sravan Bodapati, Dan Roth