Memory and Communication

Research on memory and communication in artificial intelligence focuses on improving the efficiency and robustness of information processing and storage within complex systems, particularly large language models and federated learning. Current efforts concentrate on developing novel memory architectures, such as structured memory representations and content-addressable memory systems, alongside optimized communication strategies like hierarchical communication topologies and communication-efficient optimization algorithms. These advancements aim to overcome limitations in existing models, such as catastrophic forgetting and information overload, leading to more efficient and accurate AI systems with broader applications in various fields.

Papers

July 21, 2024

Enhancing Incremental Summarization with Structured Representations
EunJeong Hwang, Yichao Zhou, James Bradley Wendt, Beliz Gunel, Nguyen Vo, Jing Xie, Sandeep Tata
Large Language Model Text Summarization Unstructured Data Structured Representation Structured Knowledge Memory and Communication

June 20, 2024

Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization
Afonso de Sá Delgado Neto, Maximilian Egger, Mayank Bakshi, Rawad Bitar
Federated Learning Byzantine Robust Federated Learning Zeroth Order Optimization Byzantine Fault Memory and Communication

October 9, 2023

Rethinking Memory and Communication Cost for Efficient Large Language Model Training
Chan Wu, Hanxiao Zhang, Lin Ju, Jinjing Huang, Youshao Xiao, Zhaoxin Huan, Siyuan Li, Fanzhuang Meng, Lei Liang, Xiaolu Zhang, Jun Zhou
Large Language Model Large Language Model Training Communication Cost Whale Optimization Algorithm Memory and Communication

October 16, 2022

Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learning
Yejia Liu, Wang Zhu, Shaolei Ren
Continual LEArning Catastrophic Forgetting Continual Learning Experience Replay Task Distribution Memory and Communication

February 1, 2022

Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold
Sugandha Sharma, Sarthak Chandra, Ila R. Fiete
Catastrophic Forgetting Content Addressable Memory Molecular Scaffold Knowledge Recall Lack Interpretability Attractor Dynamic Entorhinal Hippocampal Memory and Communication

Memory and Communication

Papers

Enhancing Incremental Summarization with Structured Representations

Communication-Efficient Byzantine-Resilient Federated Zero-Order Optimization

Rethinking Memory and Communication Cost for Efficient Large Language Model Training

Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learning

Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold