Scalable Model
Scalable models aim to create machine learning systems that maintain or improve performance as their size and complexity increase, addressing limitations in current approaches. Research focuses on mitigating issues like catastrophic forgetting in continual learning, improving efficiency in handling high-dimensional data (e.g., through techniques like Mixture-of-Experts and hierarchical coarse-graining), and enhancing generalization across diverse datasets. These advancements are crucial for deploying effective AI solutions in resource-constrained environments and for tackling complex real-world problems across domains like robotics, recommendation systems, and medical AI, where data volume and model complexity are significant challenges.
Papers
Nonparametric Control-Koopman Operator Learning: Flexible and Scalable Models for Prediction and Control
Petar Bevanda, Bas Driessen, Lucian Cristian Iacob, Roland Toth, Stefan Sosnowski, Sandra Hirche
Context Neural Networks: A Scalable Multivariate Model for Time Series Forecasting
Abishek Sriramulu, Christoph Bergmeir, Slawek Smyl