TeraAgent: A Distributed Agent-Based Simulation Engine for Simulating Half a Trillion Agents

Lukas Breitwieser; Ahmad Hesam; Abdullah Giray Yağlıkçı; Mohammad Sadrosadati; Fons Rademakers; Onur Mutlu

cs.DCSep 28, 2025

TeraAgent: A Distributed Agent-Based Simulation Engine for Simulating Half a Trillion Agents

Authors: Lukas Breitwieser, Ahmad Hesam, Abdullah Giray Yağlıkçı, Mohammad Sadrosadati, Fons Rademakers, Onur Mutlu

Organizations: ETH Zurich, Switzerland · Delft University of Technology, The Netherlands · CERN, Switzerland

Abstract

Agent-based simulation is a method for studying highly complex systems in disciplines as diverse as biology, medicine, sociology, economics, finance, and more. In this method, an agent represents a small part of the system, such as a human cell, and, through interactions among many agents, the system's behavior emerges, like the growth of cancer in the human body. Real-world complex systems, such as a human organ, comprise billions of agents. However, the state-of-the-art agent-based simulation platform, BioDynaMo, does not support the distributed processing of a simulation across servers. This limitation restricts the scalability of agent counts to the computational resources available within a single server, hindering users from conducting larger and more meaningful simulations. In this paper, we introduce TeraAgent, a new distributed agent-based simulation engine. To overcome BioDynaMo's scalability limitation, TeraAgent enables efficient exchange of agent information across servers by implementing two solutions: 1) a tailored serialization mechanism that allows agents to be accessed and mutated directly at the receive buffer, and 2) delta encoding that significantly reduces data transfer across servers by leveraging the iterative nature of agent-based simulations. Built on our solutions, TeraAgent 1) enables extreme-scale simulations with half a trillion agents (292x that of BioDynaMo) on 84,096 CPU cores, 2) reduces time-to-result leveraging parallelism across compute nodes, 3) improves interoperability with third-party tools, and 4) provides users with more hardware flexibility. TeraAgent serializes agents with a median speedup of 110x (37x for deserialization) over the state-of-the-art ROOT I/O baseline, delivers 8x the agent update rate per CPU core of Biocellion, and improves in-situ visualization performance by 39x.

Figures & tables

Fig. 1 : Overview of TeraAgent’s distributed execution, simplified to two dimensions and two ranks. (A) Space partitioning and the neighbor search grid. (B) The four cross-rank operations: 1) aura update, 2) agent migration where the moving agent lands within the aura region, 3) agent migration where the moving agents land outside any locally known aura region, and 4) load balancing.

Fig. 2 : The TeraAgent-IO serialization mechanism. Numbered nodes in (A) are contiguous memory blocks and edges are pointers; the arcs in (C) are pointer fields repointed in place, with no further allocation.

Fig. 3 : TeraAgent’s delta encoding scheme. The reference is identical on the sender and the receiver.

\mathrm{R}n

corresponds to node

n

in the message.

Characteristic	Cell proliferation	Epidemiology	Oncology
Create new agents	✓		✓
Delete agents			✓
Load imbalance		✓
Agents move randomly		✓	✓

TABLE I : Performance-relevant characteristics of the benchmark simulations (§ IV-B ).

Panel of Figure 5 — Fig. 4 : Result verification of TeraAgent. (a) Epidemiology and (b) oncology use cases. (c) Cell clustering, final state on eight MPI ranks illustrating the space partitioning grid colored by the rank that owns the volume ( ProcOwner )

Fig. 5 : Scalability of TeraAgent on Snellius: (A) strong scaling and (B) weak scaling. The gray dashed line marks ideal behavior in both panels. The error bars represent the minimum and maximum observed values.

Fig. 6 : Speedup (top) and normalized memory consumption (bottom) with respect to BioDynaMo. Each group shows three bars: OpenMP (the BioDynaMo baseline, normalized to 1), MPI-only, and MPI-hybrid. The legend is shared between the plots.

Fig. 7 : In-situ visualization with ParaView on System B: visualization speedup (top) and normalized memory consumption (bottom) for TeraAgent in MPI-only and MPI-hybrid configurations, both with respect to BioDynaMo (OpenMP). The legend is shared between the plots.

Metric	TeraAgent	Biocellion
Runtime per iteration	15.8 s	4.46 s
CPU cores	144	4,096
Number of agents	1.72B	1.72B
Agent updates per sec. per core	$7.56\times 10^{5}$	$9.42\times 10^{4}$
Open source	Planned	No

TABLE II : Comparison of TeraAgent and Biocellion on the cell clustering simulation with 1.72 billion agents.

Panel of Figure 10 — Fig. 8 : Comparison of the TeraAgent-IO serialization mechanism (labeled TA IO in the plots) and ROOT IO on Snellius.

Panel of Figure 11 — Fig. 9 : Effect of compression on System B. Each group shows three bars: no compression (the TeraAgent-IO baseline, normalized to 1), LZ4 only, and LZ4 with delta encoding. The legend is shared among all plots.

Agent-Based Modeling

Agent-based modeling simulates how interacting, heterogeneous entities and their environments produce patterns in social, economic, biological, and engineered systems.

See all 160 papers

Agensh: Scaling Organizational Intelligence to 1,024 Agents
Sep 22, 2026Zhihao Zhan, Ting Song, Li Dong +4Multi-Agent Coordination Multi-Agent Orchestration
Microsoft Research
Population-Scalable Multi-Agent World Modeling
Aug 9, 2026Renjie Zhao, Yuxiang Wu, Mingyu Zhang +7World Model Learning Multi-Agent Systems
Shanghai Jiao Tong University
Modeling Earth-Scale Human-Like Societies with One Billion Agents
Jun 7, 2025Haoxiang Guan, Jiyan He, Liyang Fan +10Human Behavior Simulation Computational Social Science
University of Science and Technology of China, Hefei, China. · Zhongguancun Academy, Beijing, China. · Zhongguancun Institute of Artificial Intelligence, Beijing, China. +4

Sep 22, 2026J/K move · Enter open · S save

Agensh: Scaling Organizational Intelligence to 1,024 Agents

Zhihao Zhan, Ting Song, Li Dong, Shaohan Huang+3

Microsoft Research

Multi-Agent Coordination Multi-Agent Orchestration Multi-Agent Systems Multi-Agent Coding LLM Agent Harnesses

Open paper AskPDF arXiv

A multi-agent system can reduce latency on complex tasks by executing work concurrently. Several pioneering harness frameworks support multi-agent systems. However, the scalability of current multi-agent harnesses is often constrained by a central orchestrator's capacity to allocate tasks and coordinate workers. To address this limitation, we introduce Agensh, a scalable self-organized multi-agent harness without a central orchestrator: concurrent workers execute a multi-agent cooperation loop, continuously gathering context, claiming and self-assigning sub-tasks, taking action and sharing findings, verifying results, and merging progress in an asynchronous manner. The loop is supported by the agentic organization infrastructure comprising three components: a shared workspace holds proposed, ongoing, and completed work; a message interface lets workers communicate; and shared context retains reusable findings and work intentions. To test the scalability of Agensh, we evaluate it on the five hardest ProgramBench tasks with GPT-5.6-sol (high). Scaling from 1 to 128 agents raises the mean final test-pass rate from 19.31% to 28.78%, an approximately 49% relative improvement. Larger organizations reach comparable test-pass rates earlier. On pandoc, scaling from 1 to 1,024 agents raises the final test-pass rate from 33.89% to 55.06%. Worker trajectories further show that different forms of self-organized cooperation gradually emerges and standardizes as the organization grows. These results reveal the number of agents as a new scaling dimension for multi-agent organizations to expand the frontier of general intelligence, offering a practical solution for complex tasks under hard latency constraints or time budgets.

Abstract

Agent-Based Modeling

Explore similar work

Agensh: Scaling Organizational Intelligence to 1,024 Agents

Population-Scalable Multi-Agent World Modeling

Modeling Earth-Scale Human-Like Societies with One Billion Agents