GetBatch: Distributed Multi-Object Retrieval for ML Data Loading

Alex Aizman; Abhishek Gaikwad; Piotr Żelasko

cs.DCFeb 25, 2026

GetBatch: Distributed Multi-Object Retrieval for ML Data Loading

Authors: Alex Aizman, Abhishek Gaikwad, Piotr Żelasko

Organizations: NVIDIA, Santa Clara, USA.

Abstract

Machine learning training pipelines consume data in batches. A single training step may require thousands of samples drawn from shards distributed across a storage cluster. Issuing thousands of individual GET requests incurs per-request overhead that often dominates data transfer time. To solve this problem, we introduce GetBatch - a new object store API that elevates batch retrieval to a first-class storage operation, replacing independent GET operations with a single deterministic, fault-tolerant streaming execution. GetBatch achieves up to 15x throughput improvement for small objects and, in a production training workload, reduces P95 batch retrieval latency by 2x and P99 per-object tail latency by 3.7x compared to individual GET requests.

Figures & tables

Panel of Figure 1 — Figure 1: AIStore sequential and batched random access data loading patterns. Sequential I/O (a) reads entire shards and selects samples from a buffer, while GetBatch (b) retrieves only the sampled items in a single request.

Figure 2: GetBatch execution model. A client submits a batch request to a proxy, which selects a Designated Target (DT). The proxy activates all other targets as senders. Senders stream locally owned data to the DT over peer-to-peer paths, and the DT emits a single output stream in strict request order.

Figure 3: Sustained throughput comparison between individual GET and GetBatch across object sizes and batch configurations. GetBatch yields the largest gains for small objects, where per-request overhead dominates.

		GetBatch
Object Size	GET	Batch 32	Batch 64	Batch 128
10 KiB	0.5	4.5 (9 $\times$ )	6.0 (12 $\times$ )	7.3 (15 $\times$ )
100 KiB	4.2	20.7 (4.9 $\times$ )	24.1 (5.7 $\times$ )	26.1 (6.2 $\times$ )
1 MiB	22.3	32.4 (1.5 $\times$ )	35.2 (1.6 $\times$ )	37.0 (1.7 $\times$ )

Table 1: Throughput (GiB/s) for individual GET and GetBatch. Speedup over GET in parentheses.

Method	P50	P95	P99	Avg
Batch Latency (ms)
Sequential I/O	243.7	431.2	638.9	261.4
Random GET	934.7	3668.7	4814.3	1320.0
GetBatch	427.5	1808.6	2744.7	624.7
Per-Object Latency (ms)
Sequential I/O	1.2	5.2	6.8	2.0

Table 2: Latency comparison during training. Values in milliseconds (ms).

BatchWeave: A Consistent Object-Store-Native Data Plane for Large Foundation Model Training
May 11, 2026Ting Sun, Junjie Zhang, Xiao Yan +7Fault-Tolerant Distributed Training Distributed Training
Lionrock AI Lab, China Merchants Group, Hong Kong, China · Wuhan University, Wuhan, China · The Chinese University of Hong Kong, Shenzhen, China
LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data
Jul 30, 2026Debopam Sanyal, Hongjie Chen, Alexey Tumanov +1Benchmark Design Cloud Computing
Georgia Tech · Dolby Labs
ObjectCache: Layerwise Object-Storage Retrieval for KV Cache Reuse
May 16, 2026Yu Zhu, Aditya Dhakal, Yunming Xiao +2LLM Inference Efficiency KV-Cache Offloading
ETH Zurich Switzerland · HPE Labs United States of America · The Chinese University of Hong Kong, Shenzhen China

May 11, 2026J/K move · Enter open · S save

BatchWeave: A Consistent Object-Store-Native Data Plane for Large Foundation Model Training

Ting Sun, Junjie Zhang, Xiao Yan, Songxin Zhang+6

Lionrock AI Lab, China Merchants Group, Hong Kong, China · Wuhan University, Wuhan, China · The Chinese University of Hong Kong, Shenzhen, China

Fault-Tolerant Distributed Training Distributed Training LLM Training

Open paper AskPDF arXiv

Modern Large Foundation Model (LFM) training has transformed the data pipeline from a static ingestion layer into a dynamic component that must co-evolve with the training process. Existing systems are ill-equipped: colocated dataloaders offer no failure isolation, while message queue-based disaggregated dataloaders operate on a record/offset abstraction that cannot express the batch-level semantics required by distributed training. We present BatchWeave, an object-store-native training data plane for distributed LFM training. BatchWeave uses versioned manifests and conditional object writes to coordinate batch publication, recovery, and lifecycle management. First, it introduces the Transactional Global Batch (TGB), which builds on versioned-manifest ACID storage semantics and extends them with training-specific consistency, including atomic all-rank batch visibility, a globally ordered step sequence, checkpoint-aligned lifecycle management, and end-to-end exactly-once recovery. Second, it realizes recovery and retention directly in the storage layer, by durably persisting producer state through the commit protocol and tying reclamation to distributed checkpoint state. Third, its Decentralized Adaptive Commit (DAC) algorithm sustains stable ingestion throughput as the manifest grows, without any inter-producer communication. Evaluations on large-scale multimodal pre-training and SFT workloads using 64 GPUs show that BatchWeave outperforms colocated dataloader throughput while providing full failure isolation, outperforms Apache Kafka in ingestion throughput, and achieves lower consumer read latency than Kafka.

Abstract

Explore similar work

BatchWeave: A Consistent Object-Store-Native Data Plane for Large Foundation Model Training

LayoutBench: Performance Benchmarking of Cloud Storage Layouts for Multimedia Data

ObjectCache: Layerwise Object-Storage Retrieval for KV Cache Reuse