Design Space
"Design space" refers to the range of possible configurations or parameters within a system, often explored to optimize performance or achieve specific goals. Current research focuses on efficiently navigating these spaces using various techniques, including gradient-based optimization, diffusion models, and evolutionary algorithms, often applied within specific model architectures like transformers, graph neural networks, and autoencoders. This exploration is crucial for advancing fields ranging from materials science and drug discovery (e.g., designing optimal protein structures or high-Tc superconductors) to artificial intelligence (e.g., improving the performance and explainability of LLMs and other AI models) and engineering design (e.g., optimizing the design of robots or aerospace vehicles). The ultimate aim is to develop more efficient and effective methods for exploring and exploiting design spaces across diverse scientific and engineering disciplines.
Papers
Elucidating the design space of language models for image generation
Xuantong Liu, Shaozhe Hao, Xianbiao Qi, Tianyang Hu, Jun Wang, Rong Xiao, Yuan Yao
Pre-training Distillation for Large Language Models: A Design Space Exploration
Hao Peng, Xin Lv, Yushi Bai, Zijun Yao, Jiajie Zhang, Lei Hou, Juanzi Li
Exploring the design space of deep-learning-based weather forecasting systems
Shoaib Ahmed Siddiqui, Jean Kossaifi, Boris Bonev, Christopher Choy, Jan Kautz, David Krueger, Kamyar Azizzadenesheli
VehicleSDF: A 3D generative model for constrained engineering design via surrogate modeling
Hayata Morita, Kohei Shintani, Chenyang Yuan, Frank Permenter
On the design space between molecular mechanics and machine learning force fields
Yuanqing Wang, Kenichiro Takaba, Michael S. Chen, Marcus Wieder, Yuzhi Xu, John Z. H. Zhang, Kuang Yu, Xinyan Wang, Linfeng Zhang, Daniel J. Cole, Joshua A. Rackers, Joe G. Greener, Peter Eastman, Stefano Martiniani, Mark E. Tuckerman
On the Design Space Between Transformers and Recursive Neural Nets
Jishnu Ray Chowdhury, Cornelia Caragea