KOSMOS: Self-scheduling Persistent Kernels for Communication-Compute Overlap
Organizations: Advanced Micro Devices Inc., Austin, TX, USA.
Abstract
Tensor and sequence parallelism (TP+SP) and expert parallelism (EP) put collectives on the critical path of transformer layers. State-of-the-art fused kernels mitigate this with fixed roles or schedules, but this rigidity leaves large idle windows. Predicting a communication-computation split does not remove them either; a predetermined split cannot follow data readiness. We present KOSMOS, a library for self-scheduling persistent kernels whose workgroups use queues ordered by consumer demand to switch between communication and computation roles as data become ready, with no scheduler, per-call cost model, or tuned split. KOSMOS fuses every TP+SP collective of a transformer layer with its GEMM, and runs an entire expert layer, from router to fold, in one launch per direction. Evaluated on AMD Instinct MI350X and AMD Instinct MI355X GPUs, KOSMOS speeds up TP+SP end-to-end training in Megatron-LM by 1.10-1.28x in BF16 and 1.10-1.25x in MXFP8. KOSMOS accelerates end-to-end training of DeepSeek-V3 and Qwen3-235B proxies by 1.29-1.83x over DeepEP.
Figures & tables
| Mechanism | Alternative | Alternative vs KOSMOS |
|---|---|---|
| AG queue order | no local-first, chunk-major | 1.17–1.40 slower |
| RS production order | row-major | 1.41–1.45 slower |
| A2A push order | destination-major | 1.12–2.03 slower |
| 8-wave movers | 4 waves per WG (7–56 WGs) | 1.23–1.96 slower |
| Router fusion | separate router kernels | 1.01–1.02 slower |
| Shared-expert fusion | separate kernels | 1.02–1.04 slower |
| MI350X | MI355X | |||||||
| One-way | Two-way | Dedicated | Planned | One-way | Two-way | Dedicated | Planned | |
| switch | switch | combine | split | switch | switch | combine | split | |
| TP+SP AG+TN | 1.032 ∗ | 0.965 | – | – | 1.033 ∗ | 0.932 | – | – |
| TP+SP TN+RS | 1.031 ∗ | 1.024 | – | – | 1.018 ∗ | 1.012 | – | – |
| TP+SP AG+NN | 1.018 ∗ | 0.951 | – | – | 1.017 ∗ | 0.918 | – | – |
| MoE fwd, balanced | – | 1.029 ∗ | 1.018 | 0.985 | – | 1.027 ∗ | 1.008 | 0.982 |
| MI350X | MI355X | ||||
|---|---|---|---|---|---|
| Model | Prec. | Speedup | TFLOPS | Speedup | TFLOPS |
| 8B | BF16 | 1.20–1.27 | 474 596 | 1.24–1.28 | 478 612 |
| MXFP8 | 1.21–1.25 | 580 718 | 1.22–1.24 | 597 730 | |
| 70B | BF16 | 1.15–1.21 | 664 769 | 1.22–1.28 | 654 834 |
| MXFP8 | 1.19–1.22 | 893 1067 | 1.21–1.23 | 907 1119 | |
| 405B | BF16 | 1.10–1.16 | 570 r 645 r | 1.17–1.22 | 624 r 757 r |
| MI350X | MI355X | ||||
| Routing | Prec. | Speedup | TFLOPS | Speedup | TFLOPS |
| Qwen3-235B | |||||
| balanced | BF16 | 1.70 | 359 610 | 1.74 | 387 673 |
| MXFP8 | 1.82 | 372 677 | 1.83 | 400 731 | |
| skewed | BF16 | 1.31 | 249 327 | 1.29 | 275 355 |
| MXFP8 | 1.57 | 245 384 | 1.54 | 268 411 | |
Appendix figures & tables30 assets
Supplementary material from the paper’s appendix.
Appendix
| Component | Model | Pass | Fused (ms) | Separate (ms) | Without (ms) | Speedup |
|---|---|---|---|---|---|---|
| Router | dsv3 | forward | 7.557 | 7.612 | 7.168 | 1.007 |
| Router | dsv3 | backward | 13.773 | 14.005 | 13.711 | 1.017 |
| Router | qwen235 | forward | 3.622 | 3.679 | 3.398 | 1.016 |
| Router | qwen235 | backward | 5.808 | 5.918 | 5.752 | 1.019 |
| Shared expert | dsv3 | forward | 7.655 | 7.990 | 7.218 | 1.044 |
| Shared expert | dsv3 | backward | 15.533 | 15.797 | 13.865 | 1.017 |
| Config | Shape | Shipped (ms) | Chunk-major | Naive |
|---|---|---|---|---|
| TN | 70B-FC1-MBS8 | 7.579 | 8.040 (+6.1%) | 8.873 (+17.1%) |
| TN | 405B-QKV-MBS8 | 5.552 | 7.314 (+31.7%) | 7.742 (+39.4%) |
| NN | 70B-Down-MBS8 | 4.038 | 4.821 (+19.4%) | 5.109 (+26.5%) |
| NN | 405B-OProj-MBS8 | 5.184 | 7.022 (+35.5%) | 7.240 (+39.7%) |
| Shape | Band-major (ms) | Row-major (ms) | Row / band |
|---|---|---|---|
| 8B-OProj-MBS8 | 1.366 | 1.930 | 1.41 |
| 70B-Down-MBS8 | 4.156 | 6.038 | 1.45 |
| Op | WGs | 8 waves (shipped) | 4 waves | 4 waves, depth 2 | 4 waves, pipelined |
|---|---|---|---|---|---|
| AG | 7 | 9.772 | 19.131 | 10.525 | 19.460 |
| AG | 14 | 5.056 | 9.791 | 5.451 | 9.920 |
| AG | 28 | 2.734 | 5.044 | 2.899 | 5.063 |
| AG | 56 | 2.217 | 2.727 | 2.220 | 2.729 |
| AG | 112 | 2.278 | 2.230 | 2.273 | 2.237 |
| RS | 7 | 15.900 | 30.378 | 18.490 | – |
| Routing | Phase | Destination-minor (ms) | Destination-major (ms) | Slowdown |
|---|---|---|---|---|
| Balanced | Forward | 8.148 | 16.532 | 2.03 |
| Balanced | Backward | 13.789 | 20.567 | 1.49 |
| Skewed | Forward | 18.686 | 24.841 | 1.33 |
| Skewed | Backward | 30.077 | 33.653 | 1.12 |
| Reduction | end-to-end | |||||
|---|---|---|---|---|---|---|
| Shape | KOSMOS | RCCL | Ratio | KOSMOS | RCCL | Ratio |
| 8B-OProj-MBS8 | 2.21 | 1.73 | ||||
| 70B-Down-MBS8 | 2.21 | 1.71 | ||||
| Model | Routing | Phase | Push (ms) | Pull (ms) | Speedup |
|---|---|---|---|---|---|
| dsv3 | balanced | forward | 8.117 | 8.158 | 1.005 |
| dsv3 | balanced | backward | 13.783 | 13.725 | 0.996 |
| dsv3 | balanced | step | 21.043 | 21.025 | 0.999 |
| dsv3 | skewed | forward | 18.628 | 19.004 | 1.020 |
| dsv3 | skewed | backward | 30.055 | 30.052 | 1.000 |
| dsv3 | skewed | step | 45.144 | 45.506 | 1.008 |
| System | Scope | Train | MoE launches F/B | Crew sized by | Roles | Switch trigger | Switches /unit | Rtr | wgrad | TP+SP |
|---|---|---|---|---|---|---|---|---|---|---|
| FLUX Chang et al. (2024) | pair | ✓ | – | – | own tile | – | 0 | – | – | ✓ |
| TileLink Zheng et al. (2025a) | pair | ✗ | – | launch config | fixed | – | 0 | – | – | ✓ |
| ParallelKittens Sul et al. (2026a) | pair | ✗ | – | launch config | fixed | – | 0 | ✗ | ✗ | ✓ |
| Iris Awad et al. (2025) | pair | ✗ | – | launch config | fixed | – | 0 | – | – | |
| COMET Zhang et al. (2025) | pair | 2 / ? | offline, per shape | fixed | – | 0 | ✗ | ? | ✗ | |
| Primus MegaMoE AMD (n.d.b) | pair | ✓ | 3 / 5 a | autotuned per shape | fixed | – | 0 | ✗ | part. a | ✗ |
| Pass | Collective | GEMM | Kernel |
|---|---|---|---|
| fwd | AG | col-parallel | fused AG+TN |
| fwd | RS | row-parallel | fused TN+RS |
| bwd | AG | row-parallel dgrad | fused AG+NN |
| bwd | AG , AG | row-parallel dgrad + wgrad | fused AG+NN(+NT) |
| bwd | AG | col-parallel dgrad | bulk AG NN |
| bwd | RS | col-parallel wgrad | bulk RS NT |
| Parameter | Kernel | Input | Rule |
| Movers per peer | AG+GEMM, col. fprop | gathered width N | BF16: 6 if N is at least 3584, else 8. MXFP8: 8. |
| AG+GEMM, row dgrad | gathered width N | BF16: 6 if N is at least 4608, else 8. MXFP8: 8. | |
| bulk AG | contraction K | BF16: 2, 4, 6 if K is at least 7168, 3584, 2304; else 8. MXFP8: 2, 4 if K is at least 3584, 1792; else 8. | |
| Reducers per peer | GEMM+RS, row fprop | local K, waves w (tiles per cap) | K at least 4096: 4 (MXFP8 5). K at least 3072: 6 (MXFP8 8 if w is at least 16, else 7). K at most 768: 14, 10, 8 if w is at least 16, at least 8, or less. Otherwise 8 (MXFP8 10 if w is at least 16). |
| bulk RS | output width n | BF16: 2, 4 if n is at least 8192, 3584; else 8. MXFP8: 8 if n is 5120 to 8191, else 4. | |
| Dispatch crew | MoE forward | expert width I | BF16: 32, 24, 16 if I is below 1024, below 3072, or larger. MXFP8: 24576/I rounded to a multiple of 8 between 16 and 48, doubled if the forward also sends the column-wise input. Minimum over ranks. |
| BF16 | MXFP8 | |||||||||
| Model | MBS | Layers | TE baseline | KOSMOS | KOSMOS | Speedup | TE baseline | KOSMOS | KOSMOS | Speedup |
| ms/iter | ms/iter | TFLOPS | ms/iter | ms/iter | TFLOPS | |||||
| MI350X | ||||||||||
| 8B | 1 | 32 | 10056.2 | 7913.1 | 426.6 | 1.271 | 8708.3 | 6967.7 | 484.9 | 1.250 |
| 8B | 2 | 32 | 7727.6 | 6435.7 | 524.4 | 1.201 | 6729.2 | 5454.8 | 619.1 | 1.234 |
| 8B | 4 | 32 | 7119.4 | 5811.7 | 580.7 | 1.225 | 5939.6 | 4902.3 | 688.5 | 1.212 |
| BF16 | MXFP8 | |||||||
| Model | Routing | Backend | ms/iter | TFLOPS | KOSMOS speedup | ms/iter | TFLOPS | KOSMOS speedup |
| MI350X | ||||||||
| Qwen3-235B | balanced | KOSMOS | 545.1 | 610 | – | 491.4 | 677 | – |
| DeepEP | 925.6 | 359 | 1.698 | 894.7 | 372 | 1.821 | ||
| Qwen3-235B | skewed | KOSMOS | 1022.6 | 327 | – | 868.1 | 384 | – |
| DeepEP | 1337.7 | 249 | 1.308 | 1359.2 | 245 | 1.566 | ||
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-FC1-MBS1 | 8192 | 13312 | 16384 | 3.994 | 3.320 | 1.203 | 1.188 | 1.903 | 1.651 | 1.152 | 1.100 |
| 405B-QKV-MBS1 | 8192 | 2304 | 16384 | 1.128 | 0.926 | 1.218 | 1.366 | 0.740 | 0.521 | 1.420 | 1.343 |
| 70B-FC1-MBS1 | 8192 | 7168 | 8192 | 1.091 | 0.995 | 1.097 | 1.072 | 0.613 | 0.539 | 1.137 | 1.134 |
| 70B-QKV-MBS1 | 8192 | 1280 | 8192 | 0.516 | 0.469 | 1.101 | 1.028 | 0.308 | 0.281 | 1.097 | 1.044 |
| 8B-FC1-MBS1 | 8192 | 3584 | 4096 | 0.502 | 0.310 | 1.619 | 1.241 | 0.259 | 0.209 | 1.242 | 1.180 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-FC1-MBS1 | 8192 | 13312 | 16384 | 3.705 | 2.929 | 1.265 | 1.239 | 1.958 | 1.625 | 1.205 | 1.202 |
| 405B-QKV-MBS1 | 8192 | 2304 | 16384 | 1.200 | 0.959 | 1.252 | 1.431 | 0.754 | 0.524 | 1.438 | 1.419 |
| 70B-FC1-MBS1 | 8192 | 7168 | 8192 | 1.183 | 0.958 | 1.235 | 1.261 | 0.656 | 0.532 | 1.233 | 1.251 |
| 70B-QKV-MBS1 | 8192 | 1280 | 8192 | 0.540 | 0.486 | 1.113 | 1.031 | 0.308 | 0.291 | 1.059 | 1.051 |
| 8B-FC1-MBS1 | 8192 | 3584 | 4096 | 0.499 | 0.308 | 1.621 | 1.328 | 0.258 | 0.210 | 1.229 | 1.192 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-Down-MBS1 | 8192 | 16384 | 6656 | 2.015 | 1.703 | 1.183 | 1.252 | 1.382 | 1.253 | 1.103 | 1.396 |
| 405B-OProj-MBS1 | 8192 | 16384 | 2048 | 1.140 | 0.819 | 1.392 | 1.460 | 0.967 | 0.766 | 1.261 | 1.367 |
| 70B-Down-MBS1 | 8192 | 8192 | 3584 | 0.752 | 0.528 | 1.424 | 1.316 | 0.576 | 0.441 | 1.307 | 1.306 |
| 70B-OProj-MBS1 | 8192 | 8192 | 1024 | 0.541 | 0.375 | 1.441 | 1.297 | 0.462 | 0.381 | 1.215 | 1.220 |
| 8B-Down-MBS1 | 8192 | 4096 | 1792 | 0.354 | 0.234 | 1.509 | 1.317 | 0.271 | 0.222 | 1.217 | 1.238 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-Down-MBS1 | 8192 | 16384 | 6656 | 2.164 | 1.795 | 1.206 | 1.271 | 1.514 | 1.184 | 1.279 | 1.525 |
| 405B-OProj-MBS1 | 8192 | 16384 | 2048 | 1.194 | 0.840 | 1.421 | 1.433 | 1.016 | 0.770 | 1.320 | 1.338 |
| 70B-Down-MBS1 | 8192 | 8192 | 3584 | 0.807 | 0.559 | 1.445 | 1.378 | 0.609 | 0.442 | 1.377 | 1.383 |
| 70B-OProj-MBS1 | 8192 | 8192 | 1024 | 0.539 | 0.393 | 1.371 | 1.307 | 0.476 | 0.383 | 1.242 | 1.212 |
| 8B-Down-MBS1 | 8192 | 4096 | 1792 | 0.353 | 0.246 | 1.432 | 1.308 | 0.275 | 0.229 | 1.200 | 1.218 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-Down-MBS1 | 8192 | 6656 | 16384 | 2.172 | 1.976 | 1.099 | 1.127 | 2.314 | 1.840 | 1.258 | 1.163 |
| 405B-OProj-MBS1 | 8192 | 2048 | 16384 | 1.069 | 0.940 | 1.137 | 1.070 | 1.171 | 0.871 | 1.345 | 1.269 |
| 70B-Down-MBS1 | 8192 | 3584 | 8192 | 0.743 | 0.666 | 1.115 | 1.074 | 0.806 | 0.582 | 1.384 | 1.377 |
| 70B-OProj-MBS1 | 8192 | 1024 | 8192 | 0.511 | 0.421 | 1.214 | 1.056 | 0.660 | 0.382 | 1.728 | 1.240 |
| 8B-Down-MBS1 | 8192 | 1792 | 4096 | 0.411 | 0.273 | 1.506 | 1.025 | 0.465 | 0.269 | 1.730 | 1.386 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-Down-MBS1 | 8192 | 6656 | 16384 | 2.229 | 1.919 | 1.162 | 1.220 | 2.438 | 1.782 | 1.368 | 1.267 |
| 405B-OProj-MBS1 | 8192 | 2048 | 16384 | 1.152 | 0.966 | 1.192 | 1.124 | 1.233 | 0.907 | 1.360 | 1.310 |
| 70B-Down-MBS1 | 8192 | 3584 | 8192 | 0.782 | 0.630 | 1.241 | 1.209 | 0.827 | 0.569 | 1.453 | 1.462 |
| 70B-OProj-MBS1 | 8192 | 1024 | 8192 | 0.516 | 0.435 | 1.184 | 1.096 | 0.667 | 0.396 | 1.684 | 1.260 |
| 8B-Down-MBS1 | 8192 | 1792 | 4096 | 0.396 | 0.279 | 1.416 | 1.086 | 0.457 | 0.262 | 1.746 | 1.457 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-FC1-MBS1 | 8192 | 16384 | 13312 | 3.794 | 3.386 | 1.121 | 1.257 | 1.813 | 1.639 | 1.106 | 1.145 |
| 405B-QKV-MBS1 | 8192 | 16384 | 2304 | 1.178 | 0.669 | 1.761 | 1.743 | 0.665 | 0.391 | 1.701 | 1.722 |
| 70B-FC1-MBS1 | 8192 | 8192 | 7168 | 1.055 | 0.879 | 1.200 | 1.606 | 0.598 | 0.478 | 1.251 | 1.355 |
| 70B-QKV-MBS1 | 8192 | 8192 | 1280 | 0.548 | 0.324 | 1.694 | 1.582 | 0.319 | 0.197 | 1.621 | 1.631 |
| 8B-FC1-MBS1 | 8192 | 4096 | 3584 | 0.493 | 0.254 | 1.942 | 1.511 | 0.253 | 0.173 | 1.468 | 1.583 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-FC1-MBS1 | 8192 | 16384 | 13312 | 3.519 | 3.047 | 1.155 | 1.345 | 1.937 | 1.629 | 1.189 | 1.232 |
| 405B-QKV-MBS1 | 8192 | 16384 | 2304 | 1.246 | 0.673 | 1.850 | 1.776 | 0.689 | 0.401 | 1.718 | 1.748 |
| 70B-FC1-MBS1 | 8192 | 8192 | 7168 | 1.168 | 0.897 | 1.302 | 1.619 | 0.647 | 0.506 | 1.279 | 1.376 |
| 70B-QKV-MBS1 | 8192 | 8192 | 1280 | 0.558 | 0.339 | 1.645 | 1.511 | 0.320 | 0.203 | 1.577 | 1.581 |
| 8B-FC1-MBS1 | 8192 | 4096 | 3584 | 0.493 | 0.269 | 1.838 | 1.498 | 0.252 | 0.169 | 1.495 | 1.570 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-FC1-MBS1 | 13312 | 16384 | 8192 | 3.971 | 3.256 | 1.220 | 1.589 | 2.262 | 1.653 | 1.369 | 1.505 |
| 405B-QKV-MBS1 | 2304 | 16384 | 8192 | 1.145 | 0.660 | 1.735 | 1.793 | 1.011 | 0.625 | 1.619 | 1.516 |
| 70B-FC1-MBS1 | 7168 | 8192 | 8192 | 1.242 | 0.771 | 1.611 | 1.610 | 0.825 | 0.552 | 1.496 | 1.472 |
| 70B-QKV-MBS1 | 1280 | 8192 | 8192 | 0.559 | 0.336 | 1.663 | 1.558 | 0.479 | 0.333 | 1.436 | 1.328 |
| 8B-FC1-MBS1 | 3584 | 4096 | 8192 | 0.444 | 0.238 | 1.864 | 1.870 | 0.321 | 0.183 | 1.752 | 1.670 |
| BF16 | MXFP8 | ||||||||||
| Shape | M | N | K | Base ms | KOSMOS ms | Speedup | vs unf. | Base ms | KOSMOS ms | Speedup | vs unf. |
| 405B-FC1-MBS1 | 13312 | 16384 | 8192 | 3.620 | 3.015 | 1.201 | 1.586 | 2.323 | 1.612 | 1.441 | 1.602 |
| 405B-QKV-MBS1 | 2304 | 16384 | 8192 | 1.226 | 0.691 | 1.774 | 1.925 | 1.065 | 0.667 | 1.598 | 1.501 |
| 70B-FC1-MBS1 | 7168 | 8192 | 8192 | 1.228 | 0.841 | 1.460 | 1.637 | 0.852 | 0.533 | 1.598 | 1.581 |
| 70B-QKV-MBS1 | 1280 | 8192 | 8192 | 0.566 | 0.357 | 1.583 | 1.518 | 0.490 | 0.356 | 1.375 | 1.289 |
| 8B-FC1-MBS1 | 3584 | 4096 | 8192 | 0.446 | 0.234 | 1.907 | 1.926 | 0.323 | 0.194 | 1.664 | 1.594 |
| Routing | Model | Baseline ms | KOSMOS ms | KOSMOS TFLOPS | Speedup | vs KOSMOS serial |
|---|---|---|---|---|---|---|
| MI350X | ||||||
| balanced | DeepSeek-V3 | 17.041 | 7.670 | 752.6 | 2.222 | 1.463 |
| Qwen3.5-397B | 11.026 | 3.791 | 543.8 | 2.908 | 1.610 | |
| DeepSeek-V4-Pro | 17.443 | 8.531 | 761.2 | 2.045 | 1.314 | |
| Kimi-K2.7-Code | 18.219 | 8.117 | 711.1 | 2.244 | 1.438 | |
| GLM-5.2 | 14.753 | 6.562 | 754.0 | 2.248 | 1.476 | |
| Routing | Model | Baseline ms | KOSMOS ms | KOSMOS TFLOPS | Speedup | vs KOSMOS serial |
|---|---|---|---|---|---|---|
| MI350X | ||||||
| balanced | DeepSeek-V3 | 25.939 | 14.337 | 805.2 | 1.809 | 1.237 |
| Qwen3.5-397B | 16.245 | 5.918 | 696.7 | 2.745 | 1.479 | |
| DeepSeek-V4-Pro | 29.207 | 16.807 | 772.8 | 1.738 | 1.187 | |
| Kimi-K2.7-Code | 26.821 | 14.994 | 770.0 | 1.789 | 1.249 | |
| GLM-5.2 | 23.205 | 12.294 | 804.9 | 1.888 | 1.252 | |
| Routing | Model | TE MXFP8 ms | KOSMOS ms | KOSMOS TFLOPS | Speedup | vs KOSMOS serial |
|---|---|---|---|---|---|---|
| MI350X | ||||||
| balanced | DeepSeek-V3 | 15.554 | 5.326 | 1083.9 | 2.921 | 1.498 |
| Qwen3.5-397B | 10.323 | 3.202 | 643.9 | 3.224 | 1.464 | |
| DeepSeek-V4-Pro | 14.399 | 5.516 | 1177.4 | 2.611 | 1.424 | |
| Kimi-K2.7-Code | 15.916 | 5.514 | 1046.9 | 2.887 | 1.505 | |
| GLM-5.2 | 13.561 | 4.637 | 1067.1 | 2.925 | 1.483 | |
| Routing | Model | TE MXFP8 ms | KOSMOS ms | KOSMOS TFLOPS | Speedup | vs KOSMOS serial |
|---|---|---|---|---|---|---|
| MI350X | ||||||
| balanced | DeepSeek-V3 | 23.684 | 10.519 | 1097.5 | 2.251 | 1.430 |
| Qwen3.5-397B | 14.637 | 5.149 | 800.8 | 2.843 | 1.631 | |
| DeepSeek-V4-Pro | 25.560 | 12.512 | 1038.1 | 2.043 | 1.316 | |
| Kimi-K2.7-Code | 25.133 | 11.021 | 1047.6 | 2.281 | 1.476 | |
| GLM-5.2 | 20.706 | 8.911 | 1110.5 | 2.324 | 1.452 | |
GPU Kernel Optimization
GPU kernel optimization improves the speed and memory efficiency of computations by tailoring code to GPU hardware and workloads.
See all 238 papers