Sustainable Federated Fine-Tuning with LoRA: Environmental Impact and Performance Footprint
Authors: Pedro Augusto Londe Ribeiro, Rodrigo Moreira, Larissa Ferreira Rodrigues Moreira, Joberto S. B. Martins, Tereza C. Carvalho, Flávio de Oliveira Silva
Organizations: Federal University of Viçosa (UFV), Minas Gerais, Brazil · University of Salvador (UNIFACS), Salvador, Brazil · University of São Paulo (USP), São Paulo, Brazil · University of Minho (UMinho), Department of Informatics, ALGORITMI Centre, Braga, Portugal
Federated Learning (FL) with Parameter-Efficient Fine-Tuning (PEFT) enables efficient adaptation of Large Language Models (LLMs), yet its energy and carbon costs across hardware remain poorly understood. We evaluated federated Low-Rank Adaptation (LoRA) fine-tuning on NVIDIA A40, RTX 6000 Ada, and A100 GPUs using a 22 full factorial design with 135M/360M models and 10/100 local steps. CodeCarbon tracked energy and estimated CO2-equivalent emissions. Descriptive allocation of variation assigned 84.8-91.2% of energy and emissions variation to local steps within these ranges. On A40 and RTX 6000 Ada, configurations with fewer steps achieved comparable accuracy with roughly 6-8-fold lower energy consumption and higher accuracy per kWh. These findings support smaller local budgets for energy-efficient federated fine-tuning within the evaluated setting.
Figures & tables
Paper
Federated
PEFT
Direct energy
Carbon CO 2 e
GPU comparison
Factorial analysis
Iftikhar [ 7 ]
Li [ 12 ]
Hanindhito [ 20 ]
Wu [ 9 ]
Li [ 18 ]
Kuang [ 19 ]
TABLE I : Comparison with related work.
Fig. 1 : Experimental Methodology Workflow.
Factor
Low level
High level
A , model size
135M
360M
B , local steps per round
10
100
TABLE II : Experimental factors and levels.
Fig. 2 : Energy consumption and CO 2 -equivalent emissions by device and configuration.
Fig. 3 : Factor contributions to each recorded response.
Fig. 4 : Average GPU power and training duration by device and configuration.
Fig. 5 : Model performance profiles during training and evaluation.
Fig. 6 : Accuracy–energy trade-off and energy efficiency by device and configuration.