Contrastive Language-Image Pretraining

Also known as CLIP

Follow this topic Ask about this topic

Latest papers

21 papers3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

PairCLIP-SWA: Similarity-Guided Balancing and Stochastic Weight Averaging for Generalized Deepfake Image Detection with OpenCLIP
Oct 27, 2026Md. Nakib, Sudipto Sarkar, A. J. A. Jubayer Talukder +1Contrastive Language-Image Pretraining Deepfake Detection
Department of Electrical and Electronic Engineering, BUET, Dhaka 1205, Bangladesh
Reversion, Retention, and Semantic Intervention: Measuring Functional Content in CLIP Fine-Tuning Deltas
Oct 25, 2026Chih-Ting Liao, Kezhen Bao, Xin CaoContrastive Language-Image Pretraining VLM Interpretability
University of New South Wales
Benchmarking CLIP for Zero-Shot Face and Periocular Gender Estimation
Oct 5, 2026Fernando Alonso-Fernandez, Kevin Hernandez-Diaz, Jose Maria Buades +1Contrastive Language-Image Pretraining Zero-Shot Image Classification
School of Information Technology, Halmstad University, Sweden · Computer Graphics and Vision and AI Group, University of Balearic Islands, Spain
TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval
Sep 2, 2026Uicheol Jung, Juyoung Hong, Hojung Kwon +1Video-Text Retrieval Contrastive Language-Image Pretraining
Sejong University, Seoul, Republic of Korea · Wisenut, Seongnam, Republic of Korea
What Does CLIP Learn for Regional Geolocalization? Probing Visual Cues and Scene Configuration After Adaptation
Aug 22, 2026Changyu Lee, Yeonsoo Park, Abdullah Alfarrarjeh +1Contrastive Language-Image Pretraining Geospatial Representation Learning
University of Southern California Los Angeles, USA · USC IMSC San Francisco, USA · German Jordanian University Amman, Jordan +1
A Paragraph is Worth a Thousand Captions: Rethinking Text Supervision for Vision-Language Retrieval
Aug 5, 2026Mahyar Ghazanfari, Amin Tabrizian, Arsyi Aziz +2Image-Text Retrieval Vision-Language Retrieval
The George Washington University
AspectCLIP: Optimizing CLIP Representation Space via Aspect-Guided Consistency Regularization
Jul 15, 2026Yiyang Yao, Shanglin Liu, Jianming Lv +4Vision-Language Alignment Multimodal Contrastive Learning
School of Computer Science and Technology, South China University of Technology, Guangzhou, China
Jolia: Concept-Level Vision-Language Alignment for 3D CT Contrastive Learning
Jun 23, 2026Julien Khlaut, Charles Corbière, Baptiste Callard +93D VLMs Medical Imaging Foundation Models
Raidium, Paris, 75014, France · Department of Vascular and Oncological Interventional Radiology,2026 Hôpital Européen Georges Pompidou, AP-HP, Paris, France · Faculté de Santé, Université Paris-Cité, Paris, France +3
What Does the Caption Really Say? Counterfactual Phrase Intervention for Compositional Data Selection in Vision-Language Pretraining
May 21, 2026Hyejin Go, Semi Lee, Hyesong ChoiTraining Data Selection Vision-Language Pretraining
Soongsil University Seoul, Republic of Korea
DINORANKCLIP: DINOv3 Distillation and Injection for Vision-Language Pretraining with High-Order Ranking Consistency
May 7, 2026Shuyang Jiang, Nan Yu, Yiming Zhang +2Contrastive Learning Vision-Language Alignment
University of California, Los Angeles · Aimaikj · HFIPS, Chinese Academy of Sciences +2
GazeCLIP: Gaze-Guided CLIP with Adaptive-Enhanced Fine-Grained Language Prompt for Deepfake Attribution and Detection
Mar 31, 2026Yaning Zhang, Linlin Shen, Zitong Yu +2Contrastive Language-Image Pretraining Deepfake Detection
Faculty of Computer Science and Technology, Qilu University of Technology (Shandong Academy of Sciences), Jinan, 250014, China · Computer Vision Institute, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China · National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University +5
No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive Models
Mar 26, 2026Hai X. Pham, David T. Hoffmann, Ricardo Guerrero +1Vision-Language Models Compositional Reasoning in VLMs
Samsung AI Center Cambridge, UK
GoldiCLIP: The Goldilocks Approach for Balancing Explicit Supervision for Language-Image Pretraining
Mar 25, 2026Deen Dayal Mohan, Hossein Souri, Vitali Petsiuk +4Vision-Language Pretraining Contrastive Language-Image Pretraining
AI Center – Mountain View, Samsung Electronics
IsoCLIP: Decomposing CLIP Projectors for Efficient Intra-modal Alignment
Mar 20, 2026Simone Magistri, Dipam Goswami, Marco Mistretta +3Contrastive Language-Image Pretraining Latent Space Alignment
Media Integration and Communication Center (MICC), University of Florence, Italy · Department of Computer Science, Universitat Aut`onoma de Barcelona, Spain · Computer Vision Center, Barcelona, Spain +1
Unifying Contrastive and Generative Objectives for Visual Understanding and Text-to-Image Generation
Mar 3, 2026Chao Li, Tianhong Li, Sai Vidyaranya Nuthalapati +9T2I Generation Visual Representation Learning
MIT Computer Science & Artificial Intelligence Laboratory · Meta AI
CLIP Is Shortsighted: Paying Attention Beyond the First Sentence
Feb 25, 2026Marc-Antoine Lavoie, Anas Mahmoud, Aldo Zaimi +2Position Bias in VLMs Vision-Language Alignment
University of Toronto Robotics Institute · Mila - Quebec AI Institute
Tune-Up Open-Weight CLIP: Optimization Framework for Self-Supervised Fine-tuning of CLIP
Jan 14, 2026Anant Mehta, Xingyu Chen, Xiyuan Wei +1VLM Adaptation Contrastive Language-Image Pretraining
Department of Computer Science & Engineering, Texas A&M University, USA
CLIMP: Contrastive Language-Image Mamba Pretraining
Jan 11, 2026Nimrod Shabtay, Itamar Zimerman, Eli Schwartz +1Vision-Language Pretraining VLM Robustness
Tel-Aviv University · IBM Research
Contrastive vision-language learning with paraphrasing and negation
Nov 20, 2025Kwun Ho Ngan, Saman Sadeghi Afgeh, Joe Townsend +1Vision-Language Retrieval Vision-Language Alignment
Fujitsu Research of Europe · United Kingdom · City St George’s, University of London
FoundDiff: Foundational Diffusion Model for Generalizable Low-Dose CT Denoising
Aug 24, 2025Zhihao Chen, Qi Gao, Zilong Li +4Diffusion Models for Image Restoration Medical Imaging Foundation Models
Institute of Science and Technology for Brain-inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University, Shanghai 200433, China · Shanghai Key Lab of Intelligent Information Processing, School of Computer Science, Fudan University, Shanghai 200433, China · School of Cyber Science and Engineering, Sichuan University, Chengdu, Sichuan 610065, China +1
CLIP Embeddings for AI-Generated Image Detection: A Few-Shot Study with Lightweight Classifier
May 15, 2025Ziyang OuContrastive Language-Image Pretraining AI-Generated Image Detection
Department of Electrical and Computer Engineering University of Rochester Rochester, NY 14627, USA

Oct 27, 2026J/K move · Enter open · S save

PairCLIP-SWA: Similarity-Guided Balancing and Stochastic Weight Averaging for Generalized Deepfake Image Detection with OpenCLIP

Md. Nakib, Sudipto Sarkar, A. J. A. Jubayer Talukder, Mohammad Ariful Haque

Department of Electrical and Electronic Engineering, BUET, Dhaka 1205, Bangladesh

Contrastive Language-Image Pretraining Deepfake Detection

Open paper AskPDF arXiv

Deepfake detectors are almost always deployed as the single checkpoint that scored best on a validation split, yet the reliability of that choice is rarely measured. We measure it with PairCLIP-SWA, a single-backbone detector that adapts an OpenCLIP ViT-L/14 visual tower by progressive block unfreezing on a class-balanced corpus built by CLIP-similarity-guided one-to-one subsampling, and averages the fine-tuning trajectory in weight space. Trained in-domain on the IEEE SP Cup 2025 (DF-Wild-Cup) benchmark under the same protocol as the strongest published three-backbone ensemble, the weight-averaged detector reaches 97.04% accuracy, 98.64% AU-ROC and a 4.07% equal error rate, against 94.46%, 97.60% and 5.00%. Its training log shows why averaging matters: over the averaged epochs the non-averaged checkpoint swings across 5.34 points of validation accuracy while its validation loss more than doubles, whereas the average is never worse and is up to 5.99 points better. On an OpenForensics-derived benchmark the same instability becomes a misranking: the checkpoint that validation accuracy selects (98.43%) scores 88.85% on a disjoint 10,905-image test partition, rejecting 21.1% of genuine photographs, while the terminal average, ranked last on validation, scores 94.76%. A study of twenty-two in-house configurations on the identical DF-Wild-Cup split, including preprocessing ablations, shows that CLIP pre-training rather than ensembling drives accuracy, that secondary face cropping costs about 34 points, and that frequency-domain inputs transfer poorly. We recommend per-class selection criteria measured off the early-stopping split.