Information Decomposition

Information decomposition aims to disentangle the interwoven contributions of multiple variables to a target variable, moving beyond simple measures of mutual information to reveal unique, redundant, and synergistic information components. Current research focuses on applying this framework to diverse areas, including fairness in machine learning, interpretability of deep learning models (like VAEs and diffusion models), and analysis of complex systems, often employing partial information decomposition (PID) and related information-theoretic methods. These analyses offer valuable insights into data structure, model behavior, and the relationships between variables, leading to improved model design, fairer algorithms, and a deeper understanding of complex systems.