cs.SEJul 3, 2026

Which Algorithm Specification Formats Help Language Models Implement Machine Learning Algorithms?

Authors: Masahiro KatoTaka Kato

Organizations: Data Analytics Department, Mizuho-DL Financial Technology, Co., Ltd. · NP-hard Inc.

Abstract

Large language models (LLMs) are increasingly used to implement algorithms from research manuscripts, but papers often leave implementation choices implicit. This study examines how the written format of an algorithm specification affects first-pass LLM implementation accuracy. We compare ordinary prose, LaTeX algorithm-style pseudocode, PDF-like extracted pseudocode, Markdown fields, YAML-like specifications, JSON-like specifications, and Python code stubs across five machine learning tasks, three models, and four experimental settings, yielding 4,020 generated implementations. Hidden tests evaluate details that often determine correctness, including tie-breaking, array shapes, numerical rules, return structures, and invalid-input behavior. Under the core-information setting, LaTeX algorithm-style pseudocode has the largest average format effect, with YAML-like specifications and ordinary prose close behind. Under complete information, GPT-5.4 mini shows no format differences in the matched comparisons, whereas Gemma 3 4B and Llama 3.2 3B still do. Code stubs do not consistently improve correctness despite specifying the function signature. The results support a writing recommendation: authors should state the interface, computation steps, numerical rules, and boundary-case behavior explicitly, instead of relying on a particular surface format to carry those details.

Explore similar work

CardsList
  1. The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation

    Jun 2, 2026Akanksha Narula, Mofasshara Binte Rafique, Laurent BindschaedlerLotteriesAblation

  2. Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language

    May 15, 2026Vinayshekhar Bannihatti Kumar, Disha Makhija, Manoj Ghuhan Arivazhagan +1Code Generation