Information Redundancy
Information redundancy, the presence of duplicated or overlapping information, is a significant challenge across diverse fields, hindering efficiency and performance in machine learning, natural language processing, and data analysis. Current research focuses on identifying and mitigating redundancy through techniques like contrastive learning, attention mechanism optimization in large language models (LLMs), and data selection strategies based on information entropy and compression ratios. Addressing information redundancy is crucial for improving model efficiency, reducing computational costs, enhancing generalization, and ultimately leading to more robust and reliable systems in various applications.
Papers
Word Boundary Information Isn't Useful for Encoder Language Models
Edward Gow-Smith, Dylan Phelps, Harish Tayyar Madabushi, Carolina Scarton, Aline Villavicencio
FedRFQ: Prototype-Based Federated Learning with Reduced Redundancy, Minimal Failure, and Enhanced Quality
Biwei Yan, Hongliang Zhang, Minghui Xu, Dongxiao Yu, Xiuzhen Cheng