cs.SENov 16, 2026

Can Coding Agents Write and Transpile Unit Tests

Authors: Gil Pasternak, AJ Langley, Tom Yotam, Ellie Kitanidis

Organizations: Code Metal

Abstract

Coding assistants built on Large Language Models (LLMs) have become a staple of modern software development, increasingly trusted to execute workflows and modernize code end-to-end. Much of this trust rests on passing tests, which are often written by the coding agents themselves. In this work, we examine the ability of these agents to both write reliable tests and translate them between programming languages. We benchmark three popular coding agents, each evaluated with a native frontier model and high reasoning effort. Verifying across a broad suite of metrics, we find that coding agents far exceed human written suites on traditional measures of coverage. This advantage, however, is uneven: agent test quality degrades sharply with project size, and much of the advantage lies in a greater number of written tests. We also show that agentic test translation---while seemingly strong in aggregate---remains inconsistent and unreliable.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents

    Feb 8, 2026Zhi Chen, Zhensu Sun, Yuling Shi +4Software Engineering AgentsAutomated Test Generation

  2. TestJack: Should you trust the results in coding benchmarks? Agentic Coding Benchmarks Auditing via Evaluator Evolution

    Oct 7, 2026Shuangjie Yao, Hao Wang, Koushik Sen +3LLM Agent EvaluationAI Agent Benchmarks

  3. ExecCritic: Learn to Test, Test to Improve for Coding Agents

    Sep 8, 2026Leitian Tao, Baolin Peng, Haorui Wang +7RL for Code GenerationSoftware Engineering Agents