Causal Feature Distillation

Causal feature distillation aims to improve the efficiency, interpretability, and robustness of machine learning models by focusing on the causal relationships between features and outcomes. Current research explores this through various techniques, including distilling simpler models (e.g., causal trees) from complex ensembles (e.g., causal forests), leveraging causal inference to address confounding factors in knowledge distillation, and incorporating causal reasoning into reinforcement learning and other tasks for enhanced explainability. This approach holds significant promise for building more trustworthy and reliable AI systems, particularly in high-stakes applications like risk prediction and speech enhancement, by improving model accuracy while providing insights into their decision-making processes.

Papers

August 2, 2024

Distilling interpretable causal trees from causal forests
Patrick Rehill
Conditional Average Treatment Effect Causal Forest Causal Feature Distillation

June 11, 2024

RaD-Net 2: A causal two-stage repairing and denoising speech enhancement network with knowledge distillation and complex axial self-attention
Mingshuai Liu, Zhuangqi Chen, Xiaopeng Yan, Yuanjun Lv, Xianjun Xia, Chuanzeng Huang, Yijian Xiao, Lei Xie
Knowledge Distillation Speech Enhancement Speech Signal Enhanced Speech Speech Enhancement Network Causal Feature Distillation

March 28, 2024

De-confounded Data-free Knowledge Distillation for Handling Distribution Shifts
Yuzheng Wang, Dingkang Yang, Zhaoyu Chen, Yang Liu, Siao Liu, Wenqiang Zhang, Lihua Zhang, Lizhe Qi
Causal Inference Distribution Shift Causal Graph Data Free Knowledge Distillation Causal Feature Distillation

December 30, 2023

Causal State Distillation for Explainable Reinforcement Learning
Wenhao Lu, Xufeng Zhao, Thilo Fryen, Jae Hee Lee, Mengdi Li, Sven Magg, Stefan Wermter
Reinforcement Learning Reward Function Causal Learning Explainable Reinforcement Learning rEward Decomposition Causal Feature Distillation

December 20, 2023

Task-Driven Causal Feature Distillation: Towards Trustworthy Risk Prediction
Zhixuan Chu, Mengxuan Hu, Qing Cui, Longfei Li, Sheng Li
Interpretable Causal Causal Feature Distillation

October 8, 2022

Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition
Junhao Zheng, Zhanxian Liang, Haibin Chen, Qianli Ma
Continual LEArning Entity Recognition Catastrophic Forgetting Causal Inference General Class Causal Feature Distillation

December 5, 2021

Causal Distillation for Language Models
Zhengxuan Wu, Atticus Geiger, Josh Rozner, Elisa Kreiss, Hanson Lu, Thomas Icard, Christopher Potts, Noah D. Goodman
Language Model Mutual Distillation Bridging Text Causal Feature Distillation Parallel Self Distillation