First Order
First-order logic (FOL) is a foundational framework for representing knowledge and reasoning, currently experiencing renewed interest due to its integration with machine learning. Research focuses on leveraging FOL's expressive power within various applications, including improving large language model reasoning, optimizing neural network training, and enhancing knowledge graph querying through techniques like neuro-symbolic architectures and novel optimization algorithms (e.g., Adam variants, zeroth-order methods). These advancements are significant because they enable more robust, explainable, and efficient solutions to complex problems across diverse fields, from automated theorem proving to reinforcement learning and beyond.
Papers
Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates
Kento Imaizumi, Hideaki Iiduka
Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer
Yanjun Zhao, Sizhe Dang, Haishan Ye, Guang Dai, Yi Qian, Ivor W.Tsang