Large Pre Trained Model

Large pre-trained models (LPMs) are massive neural networks trained on enormous datasets, aiming to achieve strong generalization across diverse downstream tasks with minimal further training. Current research emphasizes efficient fine-tuning techniques, such as prompt engineering, low-rank adaptation (e.g., LoRA, SVFit), and sparse parameter updates, to reduce computational costs and improve model adaptability while addressing issues like overfitting and catastrophic forgetting. This field is significant due to LPMs' transformative impact on various applications, from natural language processing and computer vision to robotics and education, driving advancements in both theoretical understanding and practical deployment of AI systems.

Papers

October 24, 2023

Confounder Balancing in Adversarial Domain Adaptation for Pre-Trained Large Models Fine-Tuning
Shuoran Jiang, Qingcai Chen, Yang Xiang, Youcheng Pan, Xiangping Wu
Large Model Large Pre Trained Model Domain Invariant Feature Adversarial Loss Unobserved Confounders Adversarial Domain Adaptation Domain Classifier

October 19, 2023

An Emulator for Fine-Tuning Large Language Models using Small Language Models
Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, Christopher D. Manning
Language Model Fine Tuning Large Pre Trained Model Basic Emulator

October 17, 2023

October 16, 2023

October 10, 2023

Zero-Shot Open-Vocabulary Tracking with Large Pre-Trained Models
Wen-Hsuan Chu, Adam W. Harley, Pavel Tokmakov, Achal Dave, Leonidas Guibas, Katerina Fragkiadaki
Large Pre Trained Model Tracking by Detection Open World Object Detection Track Object Zero Shot Open Vocabulary

October 9, 2023

TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models
Zuxin Liu, Jesse Zhang, Kavosh Asadi, Yao Liu, Ding Zhao, Shoham Sabach, Rasool Fakoor
Imitation Learning Parameter Efficient Fine Tuning Large Pre Trained Model Task Adaptation Continual Adaptation Efficient Adaptation Terrain Aware Task Specific Adapter

October 4, 2023

October 2, 2023

Equivariant Adaptation of Large Pretrained Models
Arnab Kumar Mondal, Siba Smarak Panigrahi, Sékou-Oumar Kaba, Sai Rajeswar, Siamak Ravanbakhsh
Inter Part Equivariance Large Pre Trained Model Equivariant Network Canonicalization Network

September 2, 2023

Knowledge Graph Embeddings for Multi-Lingual Structured Representations of Radiology Reports
Tom van Sonsbeek, Xiantong Zhen, Marcel Worring
Language Model Knowledge Transfer Large Pre Trained Model Clinical Text Radiology Report Knowledge Graph Embeddings Multilingual Representation Snomed Ct

July 20, 2023

Pre-train, Adapt and Detect: Multi-Task Adapter Tuning for Camouflaged Object Detection
Yinghui Xing, Dexuan Kong, Shizhou Zhang, Geng Chen, Lingyan Ran, Peng Wang, Yanning Zhang
Multi Task Learning Pre Training Large Pre Trained Model Camouflaged Object Detection Task Specific Adapter Multi Task Adaptation Camouflage Object Detection

July 19, 2023

DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis
Along He, Kai Wang, Zhihong Wang, Tao Li, Huazhu Fu
Pre Trained Model Parameter Efficient Fine Tuning Medical Image Analysis Large Pre Trained Model Visual Prompt Tuning Different Pre Trained

July 12, 2023

Large Class Separation is not what you need for Relational Reasoning-based OOD Detection
Lorenzo Li Lu, Giulia D'Ascenzi, Francesco Cappio Borlino, Tatiana Tommasi
Object Recognition Large Pre Trained Model Distribution Accuracy Relation Information Class Separation Inter Class Feature

July 5, 2023

OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models
Shengding Hu, Ning Ding, Weilin Zhao, Xingtai Lv, Zhen Zhang, Zhiyuan Liu, Maosong Sun
Pre Trained Model Large Pre Trained Model Plug and Play Automatic Tuning Parameter Efficient Adaptation

July 3, 2023

Understanding the Transferability of Representations via Task-Relatedness
Akshay Mehra, Yunbei Zhang, Jihun Hamm
General Analysis Pre Trained Model Task Transferability Large Pre Trained Model Source Domain Bayesian Transfer

June 29, 2023

Integrating Large Pre-trained Models into Multimodal Named Entity Recognition with Evidential Fusion
Weide Liu, Xiaoyang Zhong, Jingwen Hou, Shaohua Li, Haozhe Huang, Yuming Fang
Uncertainty Estimation Large Pre Trained Model Multimodal Named Entity Recognition Trustworthy Prediction Evidential Fusion

June 21, 2023

Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift
Florian Seligmann, Philipp Becker, Michael Volpp, Gerhard Neumann
Distribution Shift Deep Ensemble Large Pre Trained Model Bayesian Deep Learning Approximate Inference Posterior Mode Large Scale Evaluation

June 20, 2023

LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching
Duy M. H. Nguyen, Hoang Nguyen, Nghiem T. Diep, Tan N. Pham, Tri Cao, Binh T. Nguyen, Paul Swoboda, Nhat Ho, Shadi Albarqouni, Pengtao Xie, Daniel Sonntag, Mathias Niepert
Supervised ImageNet Medical Imaging Large Pre Trained Model Graph Matching Large Scale Medical Large Scale Self Supervised

Large Pre Trained Model

Papers

Confounder Balancing in Adversarial Domain Adaptation for Pre-Trained Large Models Fine-Tuning

An Emulator for Fine-Tuning Large Language Models using Small Language Models

Rethinking Class-incremental Learning in the Era of Large Pre-trained Models via Test-Time Adaptation

Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters

Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference

PELA: Learning Parameter-Efficient Models with Low-Rank Approximation

Zero-Shot Open-Vocabulary Tracking with Large Pre-Trained Models

TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Models

Efficient Federated Prompt Tuning for Black-box Large Pre-trained Models

Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models

Equivariant Adaptation of Large Pretrained Models

Knowledge Graph Embeddings for Multi-Lingual Structured Representations of Radiology Reports

Pre-train, Adapt and Detect: Multi-Task Adapter Tuning for Camouflaged Object Detection

DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis

Large Class Separation is not what you need for Relational Reasoning-based OOD Detection

OpenDelta: A Plug-and-play Library for Parameter-efficient Adaptation of Pre-trained Models

Understanding the Transferability of Representations via Task-Relatedness

Integrating Large Pre-trained Models into Multimodal Named Entity Recognition with Evidential Fusion

Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift

LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching