AI4EOSC: a Federated Cloud Platform for Artificial Intelligence in Scientific Research
Organizations: Instituto de Física de Cantabria (IFCA), CSIC-UC, Avda. los Castros s/n, Santander, 39006, Cantabria, Spain · Instituto de Instrumentación para Imagen Molecular (I3M), Centro Mixto CSIC - Universitat Politècnica de València (UPV), Camino de Vera s/n, Valencia, 46022, Valencia, Spain · Karlsruher Institut für Technologie, Kaiserstraße 12, Karlsruhe, 76131, Germany · Predictia Intelligent Data Solutions, Fernando de los Ríos 48, Santander, 39006, Cantabria, Spain · Istituto Nazionale di Fisica Nucleare (INFN), Via Enrico Fermi 40, Frascati, 00044, Roma, Italy · Laboratório de Instrumentação e Física Experimental de Partículas, Av. Prof. Gama Pinto 2, Lisboa, 1649-003, Portugal · Centro Nacional de Computação Avançada (CNCA), Avenida do Brasil, 101, Lisboa, 1700-066, Portugal · Institute of Informatics, Slovak Academy of Sciences (IISAS), Dúbravská cesta 9, Bratislava, 84507, Slovakia · Faculty of Informatics and Information Technologies, Slovak University of Technology, Ilkoviˇcova 2, Bratislava, 84216, Slovakia · Pozna´nskie Centrum Superkomputerowo Sieciowe, Jana Pawła II 10, Poznan, 61-139, Poland
Abstract
The rapid growth of Artificial Intelligence and Machine Learning in scientific research has highlighted a gap between industry-standard MLOps tools and platforms, and the unique requirements of modern and Open Science, particularly regarding the FAIR (Findable, Accessible, Interoperable, and Reusable) principles. This paper presents AI4EOSC, a federated, open-source platform designed to operationalize the full AI/ML lifecycle within the European Open Science Cloud (EOSC) ecosystem. Our methodology tackles the fragmentation of distributed research infrastructures by integrating a modular and distributed architecture comprising an AI development platform, a serverless AI-as-a-Service layer, and a federated orchestration model that is able to integrate heterogeneous compute and storage resources from distributed e-Infrastructures. AI4EOSC also introduces a ``FAIR-by-design'' approach that enforces metadata standardization (via MLDCAT-AP) and W3C PROV-compliant provenance tracking through a platform-integrated CI/CD pipeline. AI4EOSC added value is demonstrated through the delivery of a diverse set of community installations, showing consistent and seamless deployment across heterogeneous cloud providers. These installations are validated by a set of scientific cases, showing how our work reduces the manual burden on researchers while ensuring high levels of reproducibility and interoperability and providing an unified environment for development, training, and production of AI/ML models in the EOSC.