Accuracy Loss

Accuracy loss in machine learning models, particularly large language models and deep neural networks, is a significant challenge hindering broader deployment and application. Current research focuses on mitigating this loss through techniques like model compression (e.g., pruning, quantization, sparse representations), efficient training strategies (e.g., knowledge distillation, adaptive learning), and addressing specific sources of inaccuracy (e.g., uncertain positive learning, readout misalignment). Overcoming accuracy loss is crucial for realizing the full potential of advanced AI systems in resource-constrained environments and improving the reliability of AI-driven decision-making across various domains.

Papers

November 6, 2024

Neural Precision Polarization: Simplifying Neural Network Inference with Dual-Level Precision
Dinithi Jayasuriya, Nastaran Darabi, Maeesha Binte Hashem, Amit Ranjan Trivedi
Low Precision Neural Network Inference DNN Inference High Precision Accuracy Loss Universal Polarization Neural Polarization

September 30, 2024

Aggressive Post-Training Compression on Extremely Large Language Models
Zining Zhang, Yao Chen, Bingsheng He, Zhenjie Zhang
Neural Network Network Pruning Natural Language Processing Application Accuracy Loss Functional Compression

June 5, 2024

EdgeSync: Faster Edge-model Updating via Adaptive Continuous Learning for Video Data Drift
Peng Zhao, Runchu Dong, Guiqin Wang, Cong Zhao
Video Analytics Data Drift Edge Model Video Feature Accuracy Loss Model Update Adaptive Continual Learning

February 7, 2024

EfficientViT-SAM: Accelerated Segment Anything Model Without Accuracy Loss
Zhuoyang Zhang, Han Cai, Song Han
Segment Anything Model Accuracy Loss ViT Encoder EfficientViT SAM

January 8, 2024

Why does Prediction Accuracy Decrease over Time? Uncertain Positive Learning for Cloud Failure Prediction
Haozhe Li, Minghua Ma, Yudong Liu, Pu Zhao, Lingling Zheng, Ze Li, Yingnong Dang, Murali Chintalapati, Saravan Rajmohan, Qingwei Lin, Dongmei Zhang
Time Matter Failure Prediction Accuracy Loss Uncertainty Learning Disk Failure

November 29, 2023

Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation
Xiaoyue Mi, Fan Tang, Yepeng Weng, Danding Wang, Juan Cao, Sheng Tang, Peng Li, Yang Liu
Adversarial Training Topological Metric Accuracy Loss Adversarial Training Algorithm

October 9, 2023

Diagnosing Catastrophe: Large parts of accuracy loss in continual learning can be accounted for by readout misalignment
Daniel Anthes, Sushrut Thorat, Peter König, Tim C. Kietzmann
Continual LEArning Large Scale Catastrophic Forgetting Internal Representation Natural Disaster Hidden Representation Accuracy Loss Representation Shift Readout Head

June 15, 2023

Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML
Giang Nguyen, Sumon Biswas, Hridesh Rajan
Machine Learning Bias Mitigation Online autoML Fair Machine Learning Fairness Improvement Accuracy Loss Bias Mitigation Algorithm

June 5, 2023

SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, Dan Alistarh
Large Language Model Quantization Error Accuracy Loss

November 17, 2022

Zero-Shot Dynamic Quantization for Transformer Inference
Yousef El-Kurdi, Jerry Quinn, Avirup Sil
NLP Task Multiplier Free Quantization Model Quantization Transformer Inference Dynamic Quantization Accuracy Loss Holdout Set

November 1, 2022

TrimTail: Low-Latency Streaming ASR with Simple but Effective Spectrogram-Level Length Penalty
Xingchen Song, Di Wu, Zhiyong Wu, Binbin Zhang, Yuekai Zhang, Zhendong Peng, Wenpeng Li, Fuping Pan, Changbao Zhu
Automatic Speech Recognition ASR Model Thin Tail Effective Regularization Improved Algorithm Accuracy Loss

July 31, 2022

Ultra-low Latency Adaptive Local Binary Spiking Neural Network with Accuracy Loss Estimator
Changqing Xu, Yijian Pei, Zili Wu, Yi Liu, Yintang Yang
Neural Network Spiking Neural Network Brain Inspired Accuracy Loss Low Latency Spiking Neural Network Spiking Neural Network Conversion

March 17, 2022

Confidence Calibration for Intent Detection via Hyperspherical Space and Rebalanced Accuracy-Uncertainty Loss
Yantao Gong, Cao Liu, Fan Yang, Xunliang Cai, Guanglu Wan, Jiansong Chen, Weipeng Zhang, Houfeng Wang
Intent Detection Confidence Calibration Overconfident Prediction Accuracy Loss HyperSpherical Energy

March 16, 2022

Hyperbolic Uncertainty Aware Semantic Segmentation
Bike Chen, Wei Peng, Xiaofeng Cao, Juha Röning
Semantic Segmentation Segmentation Performance Accuracy Loss Classified Pixel

January 20, 2022

HEAM: High-Efficiency Approximate Multiplier Optimization for Deep Neural Networks
Su Zheng, Zhen Li, Yao Lu, Jingbo Gao, Jide Zhang, Lingli Wang
Deep Neural Network DNN Framework DNN Accelerator Class Wise Multiplier Accuracy Loss Approximate Multiplier

December 11, 2021

CHAMP: Coherent Hardware-Aware Magnitude Pruning of Integrated Photonic Neural Networks
Sanmitra Banerjee, Mahdi Nikdast, Sudeep Pasricha, Krishnendu Chakrabarty
Power Consumption Magnitude Pruning Photonic Neural Network Accuracy Loss

December 7, 2021

BDFA: A Blind Data Adversarial Bit-flip Attack on Deep Neural Networks
Behnam Ghavami, Mani Sadati, Mohammad Shahidzadeh, Zhenman Fang, Lesley Shannon
Deep Neural Network Feedback Alignment Accuracy Loss Bit Flip Attack Bit Flip Flip Attack

Accuracy Loss

Papers

Neural Precision Polarization: Simplifying Neural Network Inference with Dual-Level Precision

Aggressive Post-Training Compression on Extremely Large Language Models

EdgeSync: Faster Edge-model Updating via Adaptive Continuous Learning for Video Data Drift

EfficientViT-SAM: Accelerated Segment Anything Model Without Accuracy Loss

Why does Prediction Accuracy Decrease over Time? Uncertain Positive Learning for Cloud Failure Prediction

Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation

Diagnosing Catastrophe: Large parts of accuracy loss in continual learning can be accounted for by readout misalignment

Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML

SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression

Zero-Shot Dynamic Quantization for Transformer Inference

TrimTail: Low-Latency Streaming ASR with Simple but Effective Spectrogram-Level Length Penalty

Ultra-low Latency Adaptive Local Binary Spiking Neural Network with Accuracy Loss Estimator

Confidence Calibration for Intent Detection via Hyperspherical Space and Rebalanced Accuracy-Uncertainty Loss

Hyperbolic Uncertainty Aware Semantic Segmentation

HEAM: High-Efficiency Approximate Multiplier Optimization for Deep Neural Networks

CHAMP: Coherent Hardware-Aware Magnitude Pruning of Integrated Photonic Neural Networks

BDFA: A Blind Data Adversarial Bit-flip Attack on Deep Neural Networks