Rethinking Code Similarity for Automated Algorithm Design with LLMs

Rui Zhang; Zhichao Lu

cs.AIMar 3, 2026

Rethinking Code Similarity for Automated Algorithm Design with LLMs

Authors: Rui Zhang, Zhichao Lu

Organizations: Department of Computer Science, City University of Hong Kong

Abstract

The rise of Large Language Model-based Automated Algorithm Design (LLM-AAD) has transformed algorithm development by autonomously generating code implementations of expert-level algorithms. Unlike traditional expert-driven algorithm development, in the LLM-AAD paradigm, the main design principle behind an algorithm is often implicitly embedded in the generated code. Therefore, assessing algorithmic similarity directly from code, distinguishing genuine algorithmic innovation from mere syntactic variation, becomes essential. While various code similarity metrics exist, they fail to capture algorithmic similarity, as they focus on surface-level syntax or output equivalence rather than the underlying algorithmic logic. We propose BehaveSim, a novel method to measure algorithmic similarity through the lens of problem-solving behavior as a sequence of intermediate solutions produced during execution, dubbed as problem-solving trajectories (PSTrajs). By quantifying the alignment between PSTrajs using dynamic time warping (DTW), BehaveSim distinguishes algorithms with divergent logic despite syntactic or output-level similarities. We demonstrate its utility in two key applications: (i) Enhancing LLM-AAD: Integrating BehaveSim into existing LLM-AAD frameworks (e.g., FunSearch, EoH) promotes behavioral diversity, significantly improving performance on three AAD tasks. (ii) Algorithm analysis: BehaveSim clusters generated algorithms by behavior, enabling systematic analysis of problem-solving strategies--a crucial tool for the growing ecosystem of AI-generated algorithms. Data and code of this work are open-sourced at https://github.com/RayZhhh/behavesim.

Figures & tables

Figure 1: Examples demonstrating existing code similarity metrics are insufficient for measuring algorithmic similarity. (a) Existing code similarity metrics, on the one hand, find the breadth-first search (BFS) and depth-first search (DFS) algorithms highly similar, despite the two algorithms being inherently different in their traversal logic. On the other hand, they also find that a re-implementation of DFS ( Code 3 ) based on recursion is a completely different algorithm from the original iteration-based implementation ( Code 2 ), despite the two codes essentially representing the same algorithm. (b) Merely checking the output of two algorithms cannot distinguish between two distinct algorithms. Both insertion sort and bubble sort algorithms yield identical sorted arrays, despite being inherently different algorithms.

Figure 2: Problem-solving behaviors on the traveling salesman problem (TSP) for two algorithms with highly similar codes. The only distinction in their implementations lies in the use of argmin() and argmax() , which leads to profoundly different behaviors: Algorithm 1 chooses the nearest neighbor node of the current node, while Algorithm 2 always steps to the farthest node away from the current node. Nevertheless, existing similarity metrics still assign them a high degree of similarity, failing to reveal their behavioral differences.

Panel of Table 1 — Table 1: Eight possible types of algorithmic similarity. “✓” indicates similarity in a given dimension (textual appearance, results, or behavior), while “✗” indicates dissimilarity. After excluding trivial cases, the dataset focuses on the four non-redundant types (Types 1–4).

Automated Algorithm Discovery

Automated algorithm discovery develops methods that generate and test executable algorithms for optimization, science, software, and machine learning.

See all 170 papers

Aug 7, 2026

PACE: Primitive-Aware Code Evolution for Automated Algorithm Design

Large Language Model (LLM)-based automated algorithm design typically evolves algorithms as complete, indivisible programs. While this whole-program perspective simplifies the search space, it fundamentally couples the useful local logic to its host program. Consequently, valuable code snippets vanish when the overall program is discarded, making it highly difficult to assess the contribution of individual algorithmic components.To address this, we propose Primitive-Aware Code Evolution (PACE), which decouples local logic from complete programs by representing it as persistent units called Executable Algorithmic Primitives (EAPs). To enable code-level transfer, PACE maintains a dynamic set of EAPs. Algorithm evolution is driven by primitive-aware operators that structurally guarantee the retention and cross-program transfer of these components. To evaluate them effectively, PACE leverages Thompson sampling based on parent-relative performance improvements, guiding primitive selection from the set without requiring extra evaluation datasets. Experiments on four tasks demonstrate that PACE effectively discovers competitive algorithms while structurally preserving valuable algorithmic components.

Zhuoliang Xie, Ruihao Zheng, Xiang Xu +2

1Southern University of Science and Technology · 2Shenzhen University

Automated Algorithm Discovery Automated Heuristic Design Large Language Model-Guided Optimization LLM-Based Program Synthesis

Ask PDF

May 11, 2026

Budget-Efficient Automatic Algorithm Design via Code Graph

Large language models (LLMs) have emerged as powerful tools for automatic algorithm design (AAD). However, existing pipelines remain inefficient. They operate at the granularity of full algorithms, redundantly rewriting recurring substructures and discarding low-fitness candidates that may contain valuable algorithmic features. We formalize budget-efficient automatic algorithm design, wherein the search policy maximizes realized fitness subject to limited computational cost. We propose a directed acyclic graph representation of algorithms and build a search framework that fully exploits the LLM's output. Instead of querying the LLM for full algorithms, we use it to obtain corrections: compact operators that add, replace, or remove code blocks. Each correction augments the graph, yielding new algorithms that compose with prior corrections. This graph structure decomposes algorithms into sets of corrections, enabling correction-level credit assignment that informs subsequent queries. We complement this framework with theoretical insights into the ideal balance between search depth and breadth at different budget levels. We validate our method empirically on three combinatorial optimization problems, demonstrating consistent superiority of our graph-based search over full-algorithm search at equal token budget. Finally, our experiments suggest that rich contexts help only when the LLM's prior knowledge is shallow, and can hinder performance otherwise.

Maxime Bouscary, Manxi Wu, Saurabh Amin

Operations Research Center Massachusetts Institute of Technology · Department of Civil and Environmental Engineering University of California, Berkeley · Laboratory for Information & Decision Systems Massachusetts Institute of Technology

Automated Algorithm Discovery Large Language Model-Guided Optimization LLM-Based Program Synthesis

Ask PDF

Jun 30, 2026

AlgoBench: Benchmarking Algorithmic Adaptation in Code Generation

High pass rates on established programming benchmarks such as HumanEval and LiveCodeBench do not always show whether a model can reason about algorithms. Many fixed benchmarks eventually become part of the public training ecosystem through released problem statements, editorials, and generated solutions, allowing later models to improve partly by exposure rather than by stronger algorithmic ability. We introduce ALGOBENCH, a framework that automatically builds novel algorithmic problems from known competitive-programming problems through structured constraint-shifting transformations. Each accepted ALGOBENCH variant is traceable to a source problem, but must make the original reference algorithm fail. Beyond pass@

k

, we introduce complexity-aware metrics -- including OPTT, OPTS, TRAPRATE, GAPT, and CONSENS -- to test whether a solution is not only functionally correct but also asymptotically suitable for the generated problem. Experiments across multiple LLMs and prompting strategies show that performance drops sharply on ALGOBENCH variants, retrieval can increase reuse of the old algorithm, and many correct-looking solutions fail to meet the required complexity. Error analysis shows that failures are mainly algorithmic rather than implementation-level, suggesting that ALGOBENCH evaluates adaptation beyond functional correctness.

Xinyuan Song, Zekun Cai, Liang Zhao

Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan

Synthetic Benchmark Generation Code Generation Evaluation Algorithmic Reasoning Code Generation Benchmarks Algorithmic Generalization

Ask PDF

Abstract

Automated Algorithm Discovery

Explore similar work

PACE: Primitive-Aware Code Evolution for Automated Algorithm Design

Budget-Efficient Automatic Algorithm Design via Code Graph

AlgoBench: Benchmarking Algorithmic Adaptation in Code Generation