Can Agentic LLMs Write Task-Based Parallel Code? A Study of OpenMP Tasking, HPX, StarPU and TTG
Authors: Linus Bantel, Anna-Lena Roth, Tim Thüring, Jonas Posner, Dirk Pflüger
Organizations: Institute for Parallel and Distributed Systems, University of Stuttgart, Stuttgart, Germany · Fulda University of Applied Sciences, Fulda, Germany
Task-based programming offers more expressive power than loop-level parallelism, but the runtimes providing it differ widely in API style, maturity, and the amount of public code to learn from. This paper asks how well LLM agents write C++ for four of them: OpenMP Tasking, HPX, StarPU, and TTG. Instead of single one-shot prompts, we use an agentic workflow in which the model edits files, compiles, and runs its kernels until it reproduces a reference output. Five locally hosted open-weight LLM implement seven HPC kernels, including matrix multiplication, Cholesky and LU factorization, in all four frameworks and a serial baseline to verify their understanding of the problem in general. Every kernel is evaluated independently for correctness at each concurrency level, strong scaling, and peak memory. Model capability dominates the outcome: three models produce correct, genuinely task-parallel kernels in 57-82% of the cases, while two models rarely compile their own code and almost never succeed. For the capable models, OpenMP Tasking and HPX are handled most reliably, StarPU is close behind, and TTG trails. The best generated GEMM and factorization kernels match or exceed the multithreaded OpenBLAS/LAPACK reference. However, correctness overstates parallel quality: several "parallel" kernels contain no parallelism at all; and serialized task graphs and unsuitable task granularity remain common.
Figures & tables
Figure 1: Pass@1 metric for the respective llm without any human correction (a, b), outcome of all generated kernels (c), and duration of the agent sessions (d). Kernels without task parallelism count as incorrect.
General C ++ errors, e.g., declaration order, mismatched types
7
Wrong use of an existing API, e.g., template arguments of ttg::make_tt
4
Incomplete code
3
Wrong results and timeouts
18
Table 1: Root causes of the 42 kernels that are wrong, time out, or do not build, determined by manual inspection and rebuilding.
Figure 2: Runtime of the correct and genuinely parallel kernels of the three strongest models (geometric mean).
Figure 3: Absolute runtime (top) and self-relative speedup (bottom) of the 18 selected kernels and the OpenBLAS/LAPACK reference with p threads pinned to the cores 0,…,p−1 . Colors denote the framework, line styles the model; dotted: ideal scaling of the fastest single-core run.
Large Language Models (LLM) show strong abilities in code generation, but their skill in creating efficient parallel programs is less studied. This paper explores how LLMs generate task-based parallel code from three kinds of input prompts: natural language problem descriptions, sequential reference implementations, and parallel pseudo code. We focus on three programming frameworks: OpenMP Tasking, C++ standard parallelism, and the asynchronous many-task runtime HPX. Each framework offers different levels of abstraction and control for task execution. We evaluate LLM-generated solutions for correctness and scalability. Our results reveal both strengths and weaknesses of LLMs with regard to problem complexity and framework. Finally, we discuss what these findings mean for future LLM-assisted development in high-performance and scientific computing.
Linus Bantel, Moritz Strack, Alexander Strack +1
Institute for Parallel and Distributed Systems, University of Stuttgart, Stuttgart, Germany
Multi-agent Large Language Model (LLM) systems offer a way to decompose complex tasks, such as coding, through parallelization and context isolation. However, adding agents in practice introduces inter-agent communication overhead, which incurs extra cost and can sometimes offset the efficiency gains. We formalize multi-agent orchestration as a graph partitioning problem that captures the communication-to-computation trade-off: task decomposition can shorten critical-path computation, but cross-agent dependencies require costly context transfer. We instantiate this view in repository-level software engineering and present Cohesion-aware Coder (Co-Coder), which builds dependency graphs from static analysis, isolates structural hub files, partitions the graph via community detection, and executes the partition with a dependency-aware scheduler. Across 28 real-world tasks on DevEval and CodeProjectEval, Co-Coder advances the Pareto-frontier over sequential and file-based parallel baselines as well as Claude Code with Agent Teams, lifting pass rate by up to 14.0%, achieving up to a 2.10x wall-clock speedup, and reducing API cost by up to 35%, with the largest gains on the most dependency-dense projects. Co-coder demonstrates how cohesion-aware orchestration can make parallel coding agents both theoretically grounded and practically efficient, suggesting a broader design principle for multi-agent systems.
Xu Yang, Lunyiu Nie, Ethan Chandra +3
1The University of Texas at Austin · University of Oxford
Translating parallel scientific codes between HPC programming frameworks is a critical but labor-intensive task. Existing LLM-based approaches rely on fixed sequential pipelines with undifferentiated repair strategies and correctness criteria that do not match scientific computing practice. We present an agent-skill framework that separates translation, compilation diagnosis, and execution validation into three autonomous agents coordinated by a verdict-driven orchestrator, routing each failure to a targeted repair skill rather than retrying blindly. An automated output check drives the self-correction loop, complemented by an offline LLM-based verification that exposes false positives escaping compiler and runtime checks. We evaluate the framework on 30 HeCBench kernels translated bidirectionally among CUDA, OpenMP, and SYCL with three open-source LLMs. Our framework attains the higher verified correctness rate in all 18 model and direction combinations over a state-of-the-art baseline while consuming up to 52.4% fewer tokens, remaining consistent across models and directions without architectural modification.
Jianuo Shen, Pengyi Xu, Bo Zhang
Department of Computer Science & Engineering, South China University of Technology, Guangzhou, Guangdong 510006 · Department of Computer Science, Michigan Technological University, Houghton, Michigan 49931