cs.SEJun 3, 2025

Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findings

Authors: Alexandra González, Xavier Franch, David Lo, Silverio Martínez-Fernández

Organizations: Universitat Politècnica de Catalunya, Barcelona, Spain · Singapore Management University, Singapore

Abstract

Context: Open-source Pre-Trained Models (PTMs) provide extensive resources for various Machine Learning (ML) tasks, yet these resources lack a classification tailored to Software Engineering (SE). Objective: To address this gap, we derive a taxonomy encompassing 146 SE tasks and apply an SE-oriented classification to PTMs in a popular open-source ML repository, Hugging Face (HF). Method: Our repository mining study followed a five-phase pipeline: (i) identification of SE tasks from the literature; (ii) collection of PTM data from the HF API, including model card descriptions and metadata, and the abstracts of the associated arXiv papers; (iii) text processing to ensure consistency; (iv) a two-phase validation of SE relevance, involving humans and LLM assistance, supported by five pilot studies with human annotators and a generalization test; and (v) data analysis. This process yielded a curated catalogue of 5,027 SE PTMs. Results: We find that most SE PTMs target code generation and coding, emphasizing implementation over early or late development stages. In terms of ML tasks, text generation dominates within SE PTMs. Notably, the number of SE PTMs has increased markedly since 2023 Q2, while benchmarking remains limited: only 5.55% of all SE PTMs in our catalogue provide evaluation results. Conclusions: Our catalogue reveals documentation and transparency gaps, highlights SE activity imbalances, and supports automated SE scenarios, such as the selection of suitable PTMs.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures

    Oct 7, 2026Nicolas Lacroix, Frederic Precioso, Mireille Blay-Fornarino +1Small Language ModelsSoftware Engineering

  2. Software Engineering LLMs: Why We Need Them and How Software Engineering Helps Build Them

    Oct 24, 2026Amir M. Ebrahimi, Mohammed Mehedi Hasan, Seyed Shayan Daneshvar +3Code Language ModelsLLM Training