Differential Testing

Ask about this topic

Overview

Differential testing checks multiple implementations, versions, or execution modes on the same inputs and treats disagreements as clues to bugs, even when no complete specification is available. Common approaches combine fuzzing or generated tests with reference executions, coverage guidance, and techniques such as symbolic execution or stateful sequence testing. The methods are used to assess generated and migrated code, compilers, decompilers, and software libraries. A key challenge is that agreement on tested inputs does not prove equivalence: input coverage, trustworthy reference behavior, and distinguishing real failures from harmless differences remain central concerns.

Latest papers

CardsList

17 papers4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. Certifying Repeatable Entry-Replay Witnesses for COBOL-to-Java Migration

    Oct 19, 2026Andras L. Ferenczi, Mariya Bessonov, Allwyn Ponnudurai +3Software EngineeringAutomated Test Generation

    American Express

  2. A Pitfall in the Evaluation of Gene Perturbation Models

    Oct 19, 2026George Panagopoulos, Saman Omidi, Tsz Pan Tong +1Single-Cell Perturbation ModelingDifferential Testing

    Department of Computer Science, University of Luxembourg, 2 Av. de l’Universite, 4365 Belval Esch-sur-Alzette, Luxembourg

  3. Stateful Testing for Deep Learning Compilers

    Oct 15, 2026Jiatao Ge, Ruixiang Qian, Yibo Hu +5Automated Software TestingDifferential Testing

    State Key Laboratory for Novel Software Technology, Nanjing University, China · UNIST, Republic of Korea

  4. Functionally Equivalent or Not? Graph-Grounded Differential Surrogate Execution for Code Equivalence

    Oct 3, 2026Amit Kachroo, Like Hui, Haitao Mao +2Differential Testing

    AWS AI Labs Santa Clara, California, USA · AWS AI Labs New York, USA

  5. The Output-Space Hypothesis: Enumerative Equivalence Checking for Tensor Programs

    Sep 17, 2026Paul Biberstein, Joseph Devietti, Mayur NaikDifferential TestingGPU Kernel Correctness Verification

    University of Pennsylvania Philadelphia, PA, USA

  6. When LLM Decompilers Recompile More and Preserve Less

    Sep 4, 2026Chang Liu, Edward Raff, Kristopher MicinskiSoftware Reverse EngineeringDifferential Testing

    Syracuse University · CrowdStrike

  7. Agentic Method for Deterministic Validation of Legacy Code Migration

    Jul 30, 2026Andras Ferenczi, Jordan Docherty, Mariya Bessonov +2Code TranslationAutomated Software Testing

    American Express

  8. SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Tests

    Jul 14, 2026Maha Ayub, Michael Konstantinou, Ahmed Khanfir +2Differential TestingSemantic Change Detection

    SnT, University of Luxembourg, Luxembourg · RIADI, ENSI, University of Manouba, Tunisia · Concordia University, Canada

  9. Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilities

    Jun 25, 2026Corban Villa, Sohee Kim, Austin Chu +2Differential TestingSoftware Vulnerability Detection

    UC Berkeley

  10. The Correctness Illusion in LLM-Generated GPU Kernels

    Jun 18, 2026Dipankar SarkarAutomated Software TestingDifferential Testing

    Arizona State University, USA

  11. Agentic Framework for Deep Learning workload migration via In-Context Learning

    Jun 14, 2026Qiyue Liang, Steven Ingram, George Vanica +4Code TranslationDifferential Testing

    Google, Sunnyvale, CA · 2Google, Kirkland, WA · 1Google, Sunnyvale, CA +2

  12. AIChilles: Automatically Uncovering Hidden Weaknesses in AI-Evolved Systems

    Jun 14, 2026Yajie Zhou, Ao Li, Ashwin Silla +2Automated Software TestingDifferential Testing

    University of Maryland, College Park · Carnegie Mellon University

  13. Jas: AI-Paired Engineering as a Revival of N-Version Programming

    Jun 5, 2026Jason HickeySoftware EngineeringDifferential Testing

    Independent

  14. SCDBench: A Benchmark for LLM-Based Smart Contract Decompilers

    May 27, 2026Kaihua Qin, Dawn Song, Arthur GervaisLLM EvaluationDifferential Testing

    University of Warwick · UC Berkeley · University College London

  15. By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode

    May 24, 2026Julius Vernie, Matthias GrabmairLLM EvaluationDifferential Testing

    Technical University of Munich (TUM) · Technical University of Munich

  16. Code Generation by Differential Test Time Scaling

    May 19, 2026Yifeng He, Ethan Wang, Jicheng Wang +2Differential TestingExecution-Guided Code Generation

    University of California, Davis · Wuhan University · University of Hong Kong

  17. EffiPair: Improving the Efficiency of LLM-generated Code with Differential Execution Feedback

    Apr 6, 2026Samira Hajizadeh, Suman JanaInference-Time OptimizationCode Optimization

    Department of Electrical Engineering, Columbia University, New York, NY, USA · Department of Computer Science, Columbia University, New York, NY, USA