Training Time

Training time in machine learning focuses on optimizing the efficiency and effectiveness of model development, aiming to reduce computational costs and improve generalization performance without sacrificing accuracy. Current research explores diverse strategies, including hybrid training methods combining online and offline learning, hardware-aware optimization for multi-accelerator systems, and adaptive training frameworks that dynamically adjust computational resources. These advancements are crucial for deploying large-scale models on resource-constrained devices and accelerating the development of complex AI systems across various applications.

Papers

October 31, 2024

GPT or BERT: why not both?
Lucas Georges Gabriel Charpentier, David Samuel
Language Model Ticket BERT Causal Model GPT Neo Masked Language Training Time

October 19, 2024

On Designing Effective RL Reward at Training Time for LLM Reasoning
Jiaxuan Gao, Shusheng Xu, Wenjie Ye, Weilin Liu, Chuyi He, Wei Fu, Zhiyu Mei, Guangju Wang, Yi Wu
Reward Model Sparse Reward LLM Reasoning Training Time

October 8, 2024

Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning
Alexis Allemann, Àlex R. Atrio, Andrei Popescu-Belis
Reinforcement Learning High Resource Language Large Batch Translation Datasets Training Time

October 4, 2024

Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approach
Parikshit Pareek, Kaarthik Sundar, Deepjyoti Deka, Sidhant Misra
Bayesian Neural Network Constrained Optimization Non Convex Optimization Problem Optimization Proxy Training Time Limited Labeled Data Feasibility Constraint

September 27, 2024

Optimizing DNN Inference on Multi-Accelerator SoCs at Training-time
Matteo Risso, Alessio Burrello, Daniele Jahier Pagliari
Energy Efficient Inference Training Time Heterogeneous System on Chip

August 24, 2024

Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning
Mingliang Zhang, Sichang Su, Chengyang He, Guillaume Sartoretti
Multi Agent Reinforcement Learning Multi Task Multi Agent Challenge Cross Task Generalization Training Time

July 9, 2024

A Hybrid Training-time and Run-time Defense Against Adversarial Attacks in Modulation Classification
Lu Zhang, Sangarapillai Lambotharan, Gan Zheng, Guisheng Liao, Ambra Demontis, Fabio Roli
Adversarial Attack Adversarial Example Adversarial Training Based Rejection Training Time Modulation Classification

May 25, 2024

HETHUB: A Distributed Training System with Heterogeneous Cluster for Large-Scale Models
Si Xu, Zixiao Huang, Yan Zeng, Shengen Yan, Xuefei Ning, Quanlu Zhang, Haolin Ye, Sipei Gu, Chunsheng Shui, Zhezheng Lin, Hao Zhang, Sheng Wang, Guohao Dai, Yu Wang
Large Scale Model Training Time Heterogeneous GPU

February 20, 2024

Hybrid Training of Denoising Networks to Improve the Texture Acutance of Digital Cameras
Raphaël Achddou, Yann Gousseau, Saïd Ladjal
Synthetic Image Image Restoration Denoising Process Camera Model Denoising Network Raw Image Training Time

February 14, 2024

Enhancing ID and Text Fusion via Alternative Training in Session-based Recommendation
Juanhui Li, Haoyu Han, Zhikai Chen, Harry Shomer, Wei Jin, Amin Javari, Jiliang Tang
Text Modality Session Based Recommendation Novel Training Fusion Strategy Training Time ID Problem Text Fusion

January 27, 2024

Distilling Privileged Multimodal Information for Expression Recognition using Optimal Transport
Muhammad Haseeb Aslam, Muhammad Osama Zeeshan, Soufiane Belharbi, Marco Pedersoli, Alessandro Koerich, Simon Bacon, Eric Granger
Knowledge Distillation Optimal Transport Expression Recognition Training Time Arousal Prediction Multimodal Distillation

January 20, 2024

InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
Pengyu Wang, Dong Zhang, Linyang Li, Chenkun Tan, Xinghao Wang, Ke Ren, Botian Jiang, Xipeng Qiu
Large Language Model Alignment Problem Temporal Alignment Model Safety Training Time Harmlessness Alignment Model Guidance

December 26, 2023

Alternate Training of Shared and Task-Specific Parameters for Multi-Task Neural Networks
Stefania Bellavia, Francesco Della Santa, Alessandra Papini
Multi Task Sharing Matter Parameter Sharing Novel Training Training Time Task Specific Weight Loss Gradient

December 19, 2023

SCoTTi: Save Computation at Training Time with an adaptive framework
Ziyu Lin, Enzo Tartaglione, Van-Tam Nguyen
Device Training Computational Power Training Time

September 30, 2023

Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement
Kai Xu, Rongyu Chen, Gianni Franchi, Angela Yao
Distribution Detection Multiplicative Size Scaling Training Time Modern Deep Learning

May 26, 2023

On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss
Yihong Liu, Alexandra Chronopoulou, Hinrich Schütze, Alexander Fraser
Neural Machine Translation Language Pair Back Translation Training Time Language Modeling Loss Data Copying Unsupervised NMT

July 20, 2022

Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topology
Tuong Do, Binh X. Nguyen, Vuong Pham, Toan Tran, Erman Tjiputra, Quang D. Tran, Anh Nguyen
Arbitrary Topology Cross Silo Federated Learning Training Time

June 7, 2022

On the balance between the training time and interpretability of neural ODE for time series modelling
Yakov Golovanev, Alexander Hvatov
Time Series Inherent Interpretability Neural ODE Differential Equation Balancing Strategy Data Driven Modeling Training Time

June 6, 2022

A computational psycholinguistic evaluation of the syntactic abilities of Galician BERT models at the interface of dependency resolution and training time
Iria de-Dios-Flores, Marcos Garcia
BERT Model Multilingual Model Training Time Agreement Metric Word Prediction Monolingual BERT Model Dependency Analysis

June 2, 2022

Fast Benchmarking of Accuracy vs. Training Time with Cyclic Learning Rates
Jacob Portes, Davis Blalock, Cory Stephenson, Jonathan Frankle
Optimal Trade Training Time Cyclic Learning Training Network

Training Time

Papers

GPT or BERT: why not both?

On Designing Effective RL Reward at Training Time for LLM Reasoning

Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning

Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approach

Optimizing DNN Inference on Multi-Accelerator SoCs at Training-time

Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learning

A Hybrid Training-time and Run-time Defense Against Adversarial Attacks in Modulation Classification

HETHUB: A Distributed Training System with Heterogeneous Cluster for Large-Scale Models

Hybrid Training of Denoising Networks to Improve the Texture Acutance of Digital Cameras

Enhancing ID and Text Fusion via Alternative Training in Session-based Recommendation

Distilling Privileged Multimodal Information for Expression Recognition using Optimal Transport

InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance

Alternate Training of Shared and Task-Specific Parameters for Multi-Task Neural Networks

SCoTTi: Save Computation at Training Time with an adaptive framework

Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement

On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss

Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topology

On the balance between the training time and interpretability of neural ODE for time series modelling

A computational psycholinguistic evaluation of the syntactic abilities of Galician BERT models at the interface of dependency resolution and training time

Fast Benchmarking of Accuracy vs. Training Time with Cyclic Learning Rates