Machine learning systems specialize computation to the workload and hardware, but communication remains in generic libraries whose device code and transport decisions are not visible to high-level compilers. We present Obaleth, a compiler that makes GPU-initiated communication part of a kernel's intermediate representation, co-optimizes it alongside computation, and lowers it to specialized transport code. Obaleth plans completions around buffer lifetimes and optimizes memory ordering, synchronization, and batching across NVIDIA and AMD GPUs. On H100 GPUs, Obaleth sends small messages up to 3.3× faster, and on 256 GPUs runs all-to-all 1.25× and 2.3× faster than NVSHMEM and NCCL GIN, with much smaller memory and register use. In vLLM, it serves Qwen3-235B on 16 B200 GPUs and answers requests up to 1.29× faster than with DeepEP.
Figures & tables
Figure 1: Posting a chain of commands. While a library pays a reservation, release, doorbell, and completion per call, Obaleth posts the chain with one of each.
Figure 2: Obaleth versus NCCL and NVSHMEM: all-to-all on H100 GPUs.
System
In IR
Specialized
Inter-node
Libraries
*SHMEM
✗
✗
✓
NCCL/RCCL GIN
✗
✗
✓
PyTorch SymmMem
∼
∼
✓
Workload-specific
DeepEP
✗
H
✓
FLUX
✗
H
∼
Comet
✗
H
✗
Table 1: GPU communication methods. In IR : in a compiler IR. Specialized by a compiler (✓) or by hand (H). ∼ : partially.
Figure 3: Obaleth compilation pipeline. Optimization passes rewrite each IR level using the facts it exposes.
Wait until \tl_replace_all:Nen flag_˙ \cd@pill flag ≥v
Table 2: Core operations in \tl_replace_all:Nen obaleth.language_˙ \cd@pill obaleth.language ( \tl_replace_all:Nen e_˙ \cd@pill e : endpoint, \tl_replace_all:Nen t_˙ \cd@pill t : token). Transfers also take a \tl_replace_all:Nen predicate_˙ \cd@pill predicate , transfers and signals an \tl_replace_all:Nen after_˙ \cd@pill after token, and \tl_replace_all:Nen wait_˙ \cd@pill [colback=cdGrey, colframe=cdGrey] wait may declare buffer reuse.
Figure 4: Obaleth ’s optimization passes, described in § 5.3 , applied cumulatively to two independent chains on one endpoint, removing per-call submissions, NIC holds, and atomics.
Figure 5: Comparison with NCCL and NVSHMEM on H100 GPUs. (a–c) Point-to-point between two nodes: (b) each library at its best issue width and transport, (c) one issuing thread per CTA. (d, e) All-to-all bus bandwidth on 8 and 256 GPUs (4 GPUs per node). (f) Least device memory per rank of any configuration while connecting, without the benchmark’s buffers but with NVSHMEM’s symmetric heap (dashed: without it). (g) Registers per thread of the all-to-all’s RDMA kernel and the ping-pong kernel. (h) Obaleth from CuTe DSL and Triton over each DSL’s NVSHMEM (§ 6.5 ). Tier-1 uses a GPU-scope doorbell in (b) at 16 and 64 CTAs, (d), and (h), and a system-scope one elsewhere; (a) shows Tier-2 only.
Figure 6: Passes in Tier-2 kernels. (a) Streams of 1, 4, and 16 chains of a 64 B put and a signal, passes added in pipeline order. (b) Defaults over all switchable passes off. S n : n -chain stream; PP: ping-pong of eight 64 B puts, answered by a put or a signal each. A2A: All-to-all on 8 GPUs.
Requests per second
1
2
4
Answer (s)
5.66 / 7.31
6.50 / 8.29
8.10 / 9.29
Speedup
1.29 ×
1.28 ×
1.15 ×
TTFT (ms)
125 / 134
127 / 140
131 / 134
TTFT p95 (ms)
376 / 192
312 / 218
311 / 201
TPOT (ms)
21.9 / 28.7
25.9 / 33.2
33.4 / 37.3
Correct (of 128)
123 / 124
124 / 120
120 / 122
Table 3: Qwen3-235B-A22B in vLLM on 16 B200 GPUs (EP16), 128 GSM8K Cobbe et al. (2021) requests per rate. Obaleth / DeepEP, medians unless noted. Answer: request latency; TTFT: time to first token; TPOT: time per output token.
Figure 7: Compiled MoE dispatch and combine over DeepEP on H100, one FP8 layer, median of five runs (times: Appendix Figure 13 ).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Slowdown of Tier-2’s defaults with one pass removed, for the workloads of Figure 6 .
Optimization
Removes
Requires
Tier
Where
Transport specialization
Run-time transport dispatch, dormant code
Peer transports (R)
1, 2
—
Doorbell scope
System-scope release
Queue placement (R)
1, 2
—
Publication planning
Ordering point of payload-free signals
Tokens (D)
1, 2
—
Inline forwarding
Staging store, source key, NIC DMA read
Small constant payload staged by scalar stores (D)
2
App. A
Submission batching
Per-command reservation, doorbell, CQE
Token chain (D)
1, 2
§ 5.3
Submission coalescing
Per-chain reservation, doorbell, CQE
Independent chains on one endpoint (D)
2
§ 5.3
Appendix
Table 4: Obaleth ’s optimization passes. Facts are derived from the IR (D), declared by the kernel or framework (K), or reported by the runtime before compilation (R). F marks Tier-2 kernels compiled from a framework pattern. ∗ Off by default.
Figure 9: Completion planning and epoch elision on one chain with wait(reuse=2) . Symbols as in Figure 4 .
Backend
Submission
Strong signal
Completion
mlx5
GPU writes WQEs; GPU or host rings
Fenced write or remote atomic
One CQE per planned interval
EFA
GPU writes WQEs to the NIC or a ring the host copies
Retire payload, then write
Count CQEs
Proxy
GPU writes commands to a host ring
Host-posted, per provider
Host-published sequence
Peer memory
Peer loads and stores
Release exchange
None
Appendix
Table 5: How each backend realizes submission, strong publication, and completion.
H100 (§ 6.2 –§ 6.5 , App. D , E )
B200 (§ 6.2 , § 6.4 , App. D )
MI355X (§ 6.3 , App. C )
GPUs
4 H100 per node, 1–64 nodes
8 B200 per node, 1–4 nodes
8 MI355X per node, 2 nodes
Intra-node
NVLink
NVLink
xGMI
Network
ConnectX (mlx5), 200 Gb/s InfiniBand
ConnectX-7 (mlx5), 400 Gb/s InfiniBand
AMD Pollara 400, 400 Gb/s RoCEv2
Submission
CPU-assisted
GPU-direct
Host proxy or GPU-direct (network), peer memory (xGMI)
Driver, runtime
Driver 595.91.07, CUDA 12.6
Driver 610.57.04, CUDA 13.0
ROCm 7.14.0, HIP 7.14
Triton
3.8.0
3.8.0
3.8.0
Appendix
Table 6: Evaluation platforms and software versions.
Figure 10: Obaleth on AMD MI355X GPUs in two nodes, one NIC each. (a) All-to-all with host-proxy submission. (b) A stream of independent 64 B chains whose buffers are reused every 16 rounds. Medians of five interleaved runs.
Figure 11: All-to-all on B200 GPUs with GPU doorbells, 8 GPUs per node. One configuration per library (NCCL GIN with warp-issued fused puts; NVSHMEM’s collective). Medians of three runs.
Figure 12: Obaleth called from CuTe DSL and Triton kernels, against NVSHMEM as each language ships it (NVSHMEM4Py; Triton-distributed) and, in gray, CUDA libraries. Tier-1 uses a GPU-scope doorbell.
Figure 13: Dispatch and combine for one MoE layer on H100, Qwen3-235B-A22B (top) and Qwen3-30B-A3B (bottom). Median per-call time at the slowest rank, then the median of five interleaved runs. cuTile variants pack and reduce in cuTile kernels around Obaleth ’s Tier-2 send kernel or NCCL’s \tl_replace_all:Nen all_to_all_˙ \cd@pill all_to_all . Triton-distributed runs over NVSHMEM’s IBGDA transport (median of three runs).
Moreh Inc., Seoul, South Korea · Dept. of Computer Science and Engineering, Seoul National University, Seoul, South Korea · Samsung Research, Seoul, South Korea +1