cs.LGNov 12, 2026

Sustainable Federated Fine-Tuning with LoRA: Environmental Impact and Performance Footprint

Authors: Pedro Augusto Londe Ribeiro, Rodrigo Moreira, Larissa Ferreira Rodrigues Moreira, Joberto S. B. Martins, Tereza C. Carvalho, Flávio de Oliveira Silva

Organizations: Federal University of Viçosa (UFV), Minas Gerais, Brazil · University of Salvador (UNIFACS), Salvador, Brazil · University of São Paulo (USP), São Paulo, Brazil · University of Minho (UMinho), Department of Informatics, ALGORITMI Centre, Braga, Portugal

Abstract

Federated Learning (FL) with Parameter-Efficient Fine-Tuning (PEFT) enables efficient adaptation of Large Language Models (LLMs), yet its energy and carbon costs across hardware remain poorly understood. We evaluated federated Low-Rank Adaptation (LoRA) fine-tuning on NVIDIA A40, RTX 6000 Ada, and A100 GPUs using a 222^2 full factorial design with 135M/360M models and 10/100 local steps. CodeCarbon tracked energy and estimated CO2_2-equivalent emissions. Descriptive allocation of variation assigned 84.8-91.2% of energy and emissions variation to local steps within these ranges. On A40 and RTX 6000 Ada, configurations with fewer steps achieved comparable accuracy with roughly 6-8-fold lower energy consumption and higher accuracy per kWh. These findings support smaller local budgets for energy-efficient federated fine-tuning within the evaluated setting.

Figures & tables

Explore similar work

CardsList
  1. FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models

    Nov 6, 2026Ensieh Khazaei, Baturalp Buyukates, Dimitrios HatzinakosLLM Fine-TuningFederated Fine-Tuning

  2. Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models

    Feb 23, 2026Jin Liu, Yinbin Miao, Ning Xi +1Federated Fine-TuningLow-Rank Adaptation