cs.SENov 13, 2026

Choosing Quantized Code LLMs by File Size and Energy per Solved Task

Authors: Aditi Patodiya

Organizations: Independent Researcher

Abstract

A developer running a code assistant on a laptop downloads one model file, and local inference moves the energy of every request onto that machine. A common rule says a larger model at lower precision tends to beat a smaller one at higher precision. We ran 24 GGUF files of Qwen2.5-Coder at four sizes on a fanless MacBook Air M2 and priced each in energy per solved task on 542 EvalPlus tasks. The result is a selection guide indexed by Ollama's listed file size, with an evidence label on each pick. At 1 and 2 GB it picks the q4_K_M files of the 1.5B and 3B models, both on the correctness and energy frontier under every accounting. At 3 GB as listed, the larger-model rule picks 7B-q2_K, the only measured 7B file that fits, and there it does not pay. That file brings no significant gain in solves over the 2 GB pick, 3B-q4_K_M, and the 90% CI limits any 3B-q4_K_M deficit to 2.4 points. Yet 7B-q2_K needs 2.03 times the energy per solved task on CPU, GPU and memory rails, or 1.66 to 1.81 times on whole-system power. On HumanEval+, five alternating session pairs give a ratio of 1.925 against 1.956 in the main grid. At 8 GB, 7B-q6_K buys a significant gain over the 2 GB pick at a higher energy price. From 1.5B up, q4_K_M trails the highest precision measured at its size by less than 3 points (90% CI). We release per-task results, power logs and the analysis script.

Figures & tables

Explore similar work

CardsList
  1. Quantize with Confidence? An Empirical Study of Quantization for Code Generation

    Jul 15, 2026Saima Afrin, Md. Zahidul Haque, Antonio MastropaoloLLM QuantizationCode Generation Evaluation

  2. PerfArena: Ranking LLMs by Code Performance

    Nov 13, 2026Rui Rua, Hailemariam Mersha, Joao Paulo Fernandes +1Code Generation EvaluationCode Generation Benchmarks

  3. Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM-Based Code Optimizations

    Oct 23, 2026Sadra Emami, Benjamin Weigell, Mohammad Reza Mousavi +3Large Language Model-Guided OptimizationCode Generation