Different Pre Trained

Research on pre-trained models focuses on leveraging the knowledge embedded in large, general-purpose models to improve efficiency and performance on diverse downstream tasks, ranging from image classification and medical image analysis to natural language processing and defect detection. Current efforts concentrate on optimizing model selection for specific applications, developing techniques for efficient fine-tuning and knowledge transfer (e.g., parameter-efficient fine-tuning, masked fine-tuning), and mitigating biases inherent in pre-trained models. This work is significant because it addresses the limitations of training deep learning models from scratch, particularly with limited data, leading to advancements in various fields and improved resource utilization.

Papers

August 23, 2024

Deep Learning for Lung Disease Classification Using Transfer Learning and a Customized CNN Architecture with Attention
Xiaoyi Liu, Zhou Yu, Lianghao Tan
Deep Learning Transfer Learning Human Attention Chest X Ray X Ray Different Pre Trained

March 14, 2024

Impact of Synthetic Images on Morphing Attack Detection Using a Siamese Network
Juan Tapia, Christoph Busch
Global Impact Synthetic Image Siamese Network Morphing Attack Detection Synthetic Training Image Different Pre Trained

November 1, 2023

Latent Space Translation via Semantic Alignment
Valentino Maiorca, Luca Moschella, Antonio Norelli, Marco Fumero, Francesco Locatello, Emanuele Rodolà
Latent Space Neural Network Model Encoder Side Semantic Alignment Neural Module Different Pre Trained

October 26, 2023

Bridging The Gaps Between Token Pruning and Full Pre-training via Masked Fine-tuning
Fengyuan Shi, Limin Wang
Vision Transformer Fine Tuning Bridging Text Filling Gap Token Pruning Different Pre Trained

October 24, 2023

Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models
Raymond Li, Gabriel Murray, Giuseppe Carenini
Pre Trained Language Model Parameter Efficient Fine Tuning Large Relevance Improvement Machine Translated Linguistic Structure Different Pre Trained Mixture of Expert Architecture

October 3, 2023

An evaluation of pre-trained models for feature extraction in image classification
Erick da Silva Puls, Matheus V. Todescato, Joel L. Carbonera
Deep Learning Global Evaluation Pre Trained Model Image Classification Feature Extraction Image Classification Task Different Pre Trained Transfer Learning Strategy

September 5, 2023

A study on the impact of pre-trained model on Just-In-Time defect prediction
Yuxiang Guo, Xiaopeng Gao, Zhenyu Zhang, W. K. Chan, Bo Jiang
Global Impact Pre Trained Model Study Feature Pre Training Model Different Pre Trained Defect Prediction

July 20, 2023

Transfer Learning and Bias Correction with Pre-trained Audio Embeddings
Changhong Wang, Gaël Richard, Brian McFee
Transfer Learning Music Information Retrieval Bias Correction Different Pre Trained

July 19, 2023

DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis
Along He, Kai Wang, Zhihong Wang, Tao Li, Huazhu Fu
Pre Trained Model Parameter Efficient Fine Tuning Medical Image Analysis Large Pre Trained Model Visual Prompt Tuning Different Pre Trained

April 5, 2023

Towards Efficient Task-Driven Model Reprogramming with Foundation Models
Shoukai Xu, Jiangchao Yao, Ran Luo, Shuhai Zhang, Zihao Lian, Mingkui Tan, Bo Han, Yaowei Wang
Transfer Learning Foundation Model Vision Foundation Model Different Pre Trained Model Reprogramming

February 26, 2023

TransferD2: Automated Defect Detection Approach in Smart Manufacturing using Transfer Learning Techniques
Atah Nuh Mih, Hung Cao, Joshua Pickard, Monica Wachowicz, Rickey Dubay
Transfer Learning Pre Trained Model Defect Detection Smart Manufacturing Transfer Learning Approach Different Pre Trained Data Enhancement

February 25, 2023

An Analysis of Abstractive Text Summarization Using Pre-trained Models
Tohida Rehman, Suchandan Das, Debarshi Kumar Sanyal, Samiran Chattopadhyay
General Analysis Pre Trained Model Text Summarization Abstractive Text Summarization Different Datasets Different Pre Trained

December 13, 2022

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Guo, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan
Full Model Cross Modal Pre Training Different Modality Flexible Toolkit Pre Training Model Different Pre Trained

October 20, 2022

Composing Ensembles of Pre-trained Models via Iterative Consensus
Shuang Li, Yilun Du, Joshua B. Tenenbaum, Antonio Torralba, Igor Mordatch
Zero Shot Pre Trained Model Diverse Ensemble Large Pre Trained Model Different Pre Trained Probabilistic Consensus

September 6, 2022

Monkeypox virus detection using pre-trained deep learning-based approaches
Chiranjibi Sitaula, Tej Bahadur Shahi
Different Pre Trained Monkeypox Disease Large Scale Screening Monkeypox Disease Detection

July 7, 2022

Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space
Wenqi Shao, Xun Zhao, Yixiao Ge, Zhaoyang Zhang, Lei Yang, Xiaogang Wang, Ying Shan, Ping Luo
Full Model Pre Trained Model Different Pre Trained Pre Trained Deep Neural Network Fisher Information Metric Model Transferability Transferability Metric

January 26, 2022

Team Yao at Factify 2022: Utilizing Pre-trained Models and Co-attention Networks for Multi-Modal Fact Verification
Wei-Yao Wang, Wen-Chih Peng
Full Model Pre Trained Model Different Pre Trained Team AcieLee Co Attention

Different Pre Trained

Papers

Deep Learning for Lung Disease Classification Using Transfer Learning and a Customized CNN Architecture with Attention

Impact of Synthetic Images on Morphing Attack Detection Using a Siamese Network

Latent Space Translation via Semantic Alignment

Bridging The Gaps Between Token Pruning and Full Pre-training via Masked Fine-tuning

Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models

An evaluation of pre-trained models for feature extraction in image classification

A study on the impact of pre-trained model on Just-In-Time defect prediction

Transfer Learning and Bias Correction with Pre-trained Audio Embeddings

DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis

Towards Efficient Task-Driven Model Reprogramming with Foundation Models

TransferD2: Automated Defect Detection Approach in Smart Manufacturing using Transfer Learning Techniques

An Analysis of Abstractive Text Summarization Using Pre-trained Models

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities

Composing Ensembles of Pre-trained Models via Iterative Consensus

Monkeypox virus detection using pre-trained deep learning-based approaches

Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space

Team Yao at Factify 2022: Utilizing Pre-trained Models and Co-attention Networks for Multi-Modal Fact Verification