cs.MASep 29, 2026

RAVEN: Receiver-Conditioned Action-Value Encoding for Finite-Alphabet Multi-Agent Communication

Authors: Shuwei Sun, Chenxi Wang, Jian Huang, Weiyun Ru, Hui Cao

Organizations: Xi’an Jiaotong University Xi’an, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A message drawn from a small alphabet helps a teammate only if it keeps the distinctions that change that teammate's next decision. We show that scoring messages by action values averaged over the receiver's situation can erase exactly these distinctions, and we propose RAVEN (Receiver-conditioned Action-Value ENcoding), which trains a four-symbol, one-step-delayed channel to preserve each receiver's centered action-value profile within the receiver's own context. The sender never needs to know that context: the receiver decodes every symbol with its private information. We give two estimators of this target. With a teacher, offline RAVEN selects the codebook that exactly minimizes an empirical conditional distortion and distills it into a frozen sender; we bound the resulting codebook-selection error and one-step decision loss. Without a teacher, online RAVEN aligns, inside a QMIX learner, the deployed symbol pathway with a training-only continuous reference that shares its routing. Against five recent communication methods on eight navigation settings, offline RAVEN attains the highest return in seven, and removing receiver conditioning forfeits 83% of its communication gain. Online RAVEN raises predator-prey capture success from 53.2% to 96.0% over the same QMIX backbone without communication, and on SMAC and MPE it attains the best mean normalized score of 14 methods, including methods that exchange kilobit messages. Every RAVEN message costs 2 bits, 12-1,024x fewer than those of NDQ, CACOM and ExpoComm on navigation.

Figures & tables

Figure 1. Why the receiver’s context matters. Two sender types induce opposite action gaps in two receiver contexts. After averaging over contexts both profiles are zero and merging the types costs nothing ( Δ=0\Delta=0 , Eq. ( 6 )); compared within contexts they must be kept apart ( Δ=2\Delta=2 ), and the receiver decodes the same symbol differently in each context. Left: a two-by-two grid of bar pairs showing centered values of two receiver actions for two sender types in two contexts; the signs flip between contexts. Top right: after averaging, both types have flat profiles and would share one symbol. Bottom right: RAVEN assigns them different symbols, and the receiver maps symbol 1 to different actions in the two contexts.
Figure 2. RAVEN under a four-symbol, one-step-delayed interface. (a) Offline RAVEN builds a table of receiver-conditioned centered values from a teacher, selects the exactly optimal codebook, distills and freezes the sender, and trains the receivers on its symbols (table and grouping shown schematically). (b) Online RAVEN evaluates each receiver twice with one set of weights and one routing decision: once with the decoded symbols (deployed) and once with the senders’ continuous states (reference, training only). The alignment loss LCV\mathcal{L}_{\mathrm{CV}} between the two centered profiles updates only the encoder and the symbol embedding Φ\Phi ; TD losses train all networks. Two lanes. The offline lane goes from a Double-DQN teacher through sender types and receiver conditions to a type-by-condition table, an exact codebook mapping twelve types to four symbols, and a frozen sender with trained receivers. The online lane shows a recurrent sender producing a hard symbol delivered at the next step, a receiver whose shared routing feeds a deployed branch using symbol embeddings and a training-only reference branch using the sender's continuous state, a shared residual head, a QMIX mixer with fixed teammates, centered action profiles of both branches, and the two losses.
SettingRAVEN offRAVEN onSLIMNDQCACOMExpoCommQ^\widehat{Q} (ub)no msg.learn. ch.
N1-C2 (2 agents, pair)−- 9.59 ±0.60−- 12.35 ±1.68−- 18.81 ±0.76−- 21.34 ±1.87−- 17.25 ±0.97−- 14.98 ±1.44−- 8.52 ±0.16−- 19.27 ±0.19−- 14.49 ±0.65
N1-C3 (3 agents, ring)−- 11.47 ±0.39−- 15.21 ±2.78−- 18.79 ±0.71−- 19.98 ±0.95−- 17.84 ±0.15−- 17.35 ±0.92–––
N1-C4 (4 agents, ring)−- 13.74 ±0.64−- 17.86 ±0.30−- 20.02 ±0.85−- 20.06 ±0.67−- 17.89 ±0.44−- 17.92 ±0.30–––
N1-C6 (6 agents, ring)−- 18.15 ±0.12−- 17.96 ±0.23−- 20.13 ±0.72−- 24.32 ±3.55−- 17.78 ±0.28−- 17.94 ±0.26–––
N2-L1 (1 listener)−- 7.70 ±0.17−- 7.86 ±0.24−- 11.89 ±3.01−- 14.19 ±3.02−- 12.64 ±4.32−- 15.42 ±0.28–––
N2-L2 (2 listeners)−- 7.86 ±0.14‡{}_{\pm 0.14}{}^{\ddagger}−- 9.13 ±1.87−- 12.10 ±2.92‡{}_{\pm 2.92}{}^{\ddagger}−- 15.08 ±2.78−- 9.65 ±3.58−- 14.16 ±2.39–––
Table 1. Navigation: team return at the final checkpoint (mean ±s.d.{}_{\pm\text{s.d.}} over 5 seeds, 1,000 held-out episodes per model; higher is better). Best per row in bold, second underlined (reference columns excluded). ‡ Trained with one listener and deployed unchanged. The last row gives the bits per message. MACC, which diverges under the enforced one-step delay (Section 4.1 ), is reported in Table 6 . The reference columns hold the teacher fixed on N1-C2 (ten-seed controlled study of Section 4.3 ): the centralized teacher itself ( Q^\widehat{Q} , evaluated on its selection episodes, hence an optimistic upper bound), receivers supervised by the same teacher without messages (no msg.), and the same supervision with a learned four-symbol channel in place of the constructed codebook (learn. ch.).
Figure 3. Final return against team size (mean and 95% interval over 5 seeds). (a) Rings of two to six agents. (b) Broadcast to one to six listeners: filled markers are models trained with one listener and deployed unchanged (dashed: CACOM and ExpoComm transferred in the same way), hollow markers are retrained at every size. Left: in ring navigation every method has a lower return with six agents than with two; offline RAVEN is highest for two to four agents and all methods except NDQ and SLIM meet near minus 18 at six agents. Right: in broadcast, offline RAVEN deployed zero-shot stays near minus 7.8 for all listener counts, above online RAVEN, CACOM, SLIM, ExpoComm and NDQ, the last of which degrades to minus 24.6 at six listeners.
Figure 4. Mechanism of offline RAVEN on navigation. (a) Return lost by each variant relative to full RAVEN on N1-C2 (paired mean and 95% interval over 10 independently trained seeds; filled: 4 symbols, hollow: 2 symbols). (b) Held-out reconstruction risk R\mathcal{R} against return for RAVEN and three condition variants, both centered within seed (grey lines join the methods of one seed). (c) Return lost when incoming messages are removed at test time (fresh seeds; dots are seeds); SLIM uses the same 2-bit budget. Three panels. Left: a forest plot in which every variant loses return relative to full RAVEN, the largest losses coming from removing messages and from removing the receiver condition. Middle: a scatter plot with a negative trend between reconstruction risk and return, RAVEN points at the lowest risk and highest return. Right: bars showing that RAVEN loses much more return than SLIM when its messages are removed on the three- and four-agent rings.
Method3m8mMMMMMM23s5zSpreadTagCryptoBits/msgNorm.Rank
RAVEN on99.72 ±0.2299.95 ±0.06100.0 ±0.087.35 ±8.3097.80 ±1.12−- 29.00 ±0.59277.3 ±3.048.03 ±0.0420.981.75
QMIX ( Rashid et al., 2018 )99.8098.8098.6057.8086.40−- 43.4223.390.3800.747.3
QPLEX ( Wang et al., 2021b )99.7599.5030.7562.0096.50−- 31.57235.145.4400.805.0
TeamComm ( Yang et al., 2024 )99.2599.0098.0064.2591.00−- 43.7368.5648.00≥\geq 2,0480.845.6
TGCNet ( Zhang et al., 2025 )99.0099.2574.0066.5096.75−- 44.9555.4648.00≥\geq 2,0480.805.9
CACOM ( Li and Zhang, 2024 )96.2543.5098.7534.2597.25−- 48.47251.13.18≥\geq 160.678.1
Table 2. SMAC (win rate, %) and MPE (team return) at 2.05M environment steps. RAVEN on : mean ±s.d.{}_{\pm\text{s.d.}} with 1,000 evaluation episodes per model (SMAC: 4 seeds; MPE: 5 seeds). Reference methods: our runs of the authors’ official open-source code under the same budget; mean over runs. “Norm.” and “rank” average the per-task min–max normalized score and the rank over all 14 methods on the eight tasks (the remaining seven methods are listed in Table 15 ). † Single-neighbor variant. Best per column in bold, second underlined.
Figure 5. Online RAVEN against controls that change only the communication pathway (mean ±\pm s.d.; 100 monitoring episodes per point). (a) Predator–prey capture success, 5 seeds. (b) MPE Spread team return, 5 seeds (variant without the reference TD loss: 3 matched seeds). Left: capture success of online RAVEN rises to about 85 percent within 0.4 million steps and above 90 percent by 0.5 million steps, while the same-backbone QMIX falls to about 12 percent and recovers only to about 50 percent. Right: on Spread, online RAVEN is above the variant without the reference TD loss, which is above the same-backbone QMIX, during most of training.
vs.Bits/msgFLOPsParamsTrain time
RAVEN off , navigation
SLIMsame–95.6–98.039.6–55.4
NDQ97.982.2–96.8−- 1.4 – 85.1–
CACOM91.785.8–97.7−- 2.1 – 85.0–
ExpoComm99.991.5–98.348.4–92.3–
RAVEN on , navigation
Table 3. Resources saved by RAVEN, 100(1−RAVEN/other)100(1-\text{RAVEN}/\text{other}) in %, as the range over settings (navigation) or tasks (SMAC and MPE). Parameters and execution FLOPs per team step are counted on the networks used in the runs; training time is compared on the same machine. ⋆ Lower bound (only part of the message counted).
Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

SymbolMeaning
NN , K=4K=4number of agents; alphabet size (2 bits per message)
XjtX_{j}^{t} , MjtM_{j}^{t}permitted local input and symbol of sender jj at step tt
ZitZ_{i}^{t} , Mit\mathbf{M}_{i}^{t}local information and mailbox of receiver ii
Ai\mathcal{A}_{i} , PPlegal actions of receiver ii ; centering matrix I−11⊤/∣Ai∣I-\mathbf{1}\mathbf{1}^{\top}/|\mathcal{A}_{i}|
∥v∥A2\|v\|_{\mathcal{A}}^{2}∣A∣−1∑ava2|\mathcal{A}|^{-1}\sum_{a}v_{a}^{2}
Q^\widehat{Q} , ξn\xi_{n} , ρ\rhoteacher, its input at the arrival time, partner-action reference
Appendix
Table 4. Notation.
TaskAgentsDescriptionEdgesMetricRAVEN
N1-C2 reference navigation2MPE simple_reference : each agent knows only its partner’s goal color; 25 stepspair, both waysteam returnoffline, online
N1-C3/C4/C6 ring navigation3/4/6each agent’s goal is known only to its predecessorpredecessor →\to successorteam returnoffline, online
N2-L1/L2/L4/L6 broadcast1+L1{+}LMPE simple_speaker_listener : an immobile speaker sees the target; listeners see landmarksspeaker →\to listenerslistener returnoffline, online
Predator–prey PP135×55\times 5 grid, vision radius 1; all agents must reach the same target within 20 stepsall-to-allcapture successonline
SMAC 3m, 8m, MMM, MMM2, 3s5z3/8/10/10/8StarCraft II micromanagement ( Samvelyan et al., 2019 ) ; MMM2 super hard, 3s5z hardall-to-all, receiver keeps 2win rateonline
MPE Spread, Tag, Crypto3/4/3coverage, pursuit and encrypted signaling; in Crypto the eavesdropper Eve is controlled like the other agents and her reward is included in the summed team return; 25 stepsall-to-allteam returnonline
Appendix
Table 5. Tasks. “Edges” are the directed message edges that every method may use; all methods use the same one-step delay and the same execution-time information.
SettingRAVEN (offline)RAVEN (online)SLIMNDQCACOMExpoCommMACC
N1-C2−- 9.59 ±0.60−- 12.35 ±1.68−- 18.81 ±0.76−- 21.34 ±1.87−- 17.25 ±0.97−- 14.98 ±1.44−- 48.12 ±7.75
N1-C3−- 11.47 ±0.39−- 15.21 ±2.78−- 18.79 ±0.71−- 19.98 ±0.95−- 17.84 ±0.15−- 17.35 ±0.92−- 43.44 ±15.20
N1-C4−- 13.74 ±0.64−- 17.86 ±0.30−- 20.02 ±0.85−- 20.06 ±0.67−- 17.89 ±0.44−- 17.92 ±0.30−- 49.47 ±6.68
N1-C6−- 18.15 ±0.12−- 17.96 ±0.23−- 20.13 ±0.72−- 24.32 ±3.55−- 17.78 ±0.28−- 17.94 ±0.26−- 37.97 ±6.34
N2-L1−- 7.70 ±0.17−- 7.86 ±0.24−- 11.89 ±3.01−- 14.19 ±3.02−- 12.64 ±4.32−- 15.42 ±0.28−- 129.72 ±58.17
N2-L2−- 7.86 ±0.14 ‡−- 9.13 ±1.87−- 12.10 ±2.92 ‡−- 15.08 ±2.78−- 9.65 ±3.58−- 14.16 ±2.39−- 115.26 ±71.17
Appendix
Table 6. Navigation: team return at the fixed final checkpoint (mean ±s.d.{}_{\pm\text{s.d.}} over 5 training seeds, 1,000 held-out episodes per model; higher is better). Best per row in bold, second best underlined. ‡ RAVEN (offline) and SLIM deploy the L1 model zero-shot on L2/L4/L6; all other methods are retrained at every size. MACC is implemented from its paper; under the enforced one-step delay its communication critic assigns credit to a message within the step it is sent, this attribution no longer matches the action the message influenced, and training diverges, which is why Table 1 omits it.
RAVEN (offline) minus … 
SettingSLIMNDQCACOMExpoCommMACC 
N1-C2++ 9.22 ∗∗∗ (5/0)++ 11.75 ∗∗∗ (5/0)++ 7.66 ∗∗∗ (5/0)++ 5.39 ∗∗∗ (5/0)++ 38.54 ∗∗ (5/0) 
N1-C3++ 7.32 ∗∗∗ (5/0)++ 8.51 ∗∗∗ (5/0)++ 6.37 ∗∗∗ (5/0)++ 5.88 ∗∗∗ (5/0)++ 31.97 ∗ (5/0) 
N1-C4++ 6.28 ∗∗∗ (5/0)++ 6.32 ∗∗∗ (5/0)++ 4.15 ∗∗∗ (5/0)++ 4.18 ∗∗∗ (5/0)++ 35.73 ∗∗ (5/0) 
N1-C6++ 1.97 ∗∗ (5/0)++ 6.17 † (5/0)−- 0.38 † (1/4)−- 0.21 (1/4)++ 19.82 ∗ (5/0) 
N2-L1++ 4.19 † (4/1)++ 6.49 ∗∗ (5/0)++ 4.94 † (4/1)++ 7.72 ∗∗∗ (5/0)++ 122.02 ∗ (5/0) 
Appendix
Table 7. Paired seed-level comparisons on navigation: mean difference in team return, Holm-adjusted significance within each family of comparisons ( ∗∗∗p<0.001{}^{***}p<0.001 , ∗∗p<0.01{}^{**}p<0.01 , ∗p<0.05{}^{*}p<0.05 , †p<0.1{}^{\dagger}p<0.1 ) and the number of seeds with positive/negative difference. C3/C4/C6 comparisons with SLIM use fresh seeds that never took part in method development; L2/L4/L6 comparisons with SLIM compare two zero-shot transfers.
Figure 6. Final team return of every training seed (small dots), the seed mean (large marker) and its 95% interval (bar) in the eight navigation settings. MACC is omitted for scale.
Figure 7. Navigation learning curves (mean ±\pm s.d. over 5 seeds). Horizontal lines are final values of methods without recorded curves; the y-axis is clipped at −30-30 .
SettingRAVEN (online)CACOMExpoCommMACC
N1-C3−- 17.03−- 18.19−- 18.10−- 46.92
N1-C4−- 18.12−- 18.47−- 18.29−- 47.44
N1-C6−- 18.32−- 18.87−- 18.26−- 38.22
N2-L2−- 12.54−- 11.95−- 15.35−- 115.92
N2-L4−- 12.48−- 13.01−- 15.68−- 135.90
N2-L6−- 13.69−- 15.88−- 16.18−- 148.49
Appendix
Table 8. Sample efficiency on navigation: time-averaged team return over 0.05–1.2M training steps (area under the mean learning curve divided by its length; higher is better). Only methods that record intermediate evaluations are listed.
 development seedsfresh seedsgain from messages
SettingΔ\Deltaseeds >0>0Δ\Deltaseeds >0>0RAVENSLIM
N1-C3++ 7.045/5++ 7.325/5++ 6.44++ 1.17
N1-C4++ 5.525/5++ 6.285/5++ 4.64++ 0.49
N1-C6++ 2.495/5++ 1.975/5++ 0.41++ 0.28
Appendix
Table 9. Ring navigation, RAVEN (offline) minus SLIM on the development seeds and on fresh confirmation seeds, and the return each method loses when its incoming messages are removed at test time (message dependence, fresh seeds).
 RAVEN (offline)SLIMCACOMExpoComm
Settingzero-shotzero-shotzero-shotretrainedzero-shotretrained
N2-L2−- 7.86−- 12.10−- 12.96−- 9.65−- 15.78−- 14.16
N2-L4−- 7.92−- 12.19−- 12.94−- 10.68−- 15.75−- 13.66
N2-L6−- 7.82−- 12.26−- 12.95−- 13.81−- 15.78−- 13.98
Appendix
Table 10. Zero-shot scale transfer on the broadcast family: a model trained with one listener is deployed unchanged to 2/4/6 listeners (“zero-shot”) and compared with the same method retrained at each size.
Figure 8. Zero-shot transfer from one listener (filled markers) against retraining at every size (hollow markers); mean and 95% interval over 5 seeds.
Figure 9. Margin of each RAVEN implementation over the strongest external method of each setting (named under the axis).
  K = 4 (2 bits)K = 2 (1 bit)
VariantchangesreturnΔ\Deltaseeds ++ / −-pp (Holm)returnΔ\Delta
RAVEN (offline), full–−- 9.45 ±0.46–––−- 14.78–
local-residual target + geometric conditiontarget and condition−- 10.75 ±1.72++ 1.30 [0.10, 2.50]7/20.146−- 15.31++ 0.53
geometric receiver conditioncondition−- 13.54 ±1.79++ 4.09 [2.85, 5.32]10/02.6×10−42.6\times 10^{-4}−- 16.12++ 1.33
plain learnable 4-symbol channel (no codebook)codebook construction−- 14.49 ±0.65++ 5.04 [4.62, 5.46]10/09.7×10−99.7\times 10^{-9}−- 16.80++ 2.02
sender unfrozen after distillationsender freezing−- 14.57 ±0.99++ 5.12 [4.53, 5.72]10/03.5×10−83.5\times 10^{-8}−- 17.37++ 2.59 ∘
Appendix
Table 11. Mechanism ablation of RAVEN (offline) on N1-C2 (10 independently trained seeds, each with its own teacher; all variants share the teacher and construction data of their seed and match capacity and number of updates). Δ\Delta : full method minus variant (paired, 95% CI). pp : Holm-adjusted within the pre-registered K=4 family (the sender-freezing comparison belongs to the candidate-internal family). K=2 comparisons are descriptive ( ∘ : computed from the per-seed records, not part of a registered family).
 all four methodscontrols only
DiagnosticK=4K=2K=4K=2
total risk RR0.833 / 0.760.800 / 0.720.667 / 0.400.633 / 0.35
abstraction term AA0.850 / 0.780.817 / 0.720.700 / 0.450.667 / 0.35
compression term DD0.800 / 0.720.617 / 0.260.867 / 0.800.833 / 0.70
teacher-action regret0.783 / 0.720.767 / 0.660.767 / 0.650.767 / 0.60
Appendix
Table 12. Does the finite objective predict task return? For each seed, diagnostics are computed on held-out samples for RAVEN (offline) and three condition variants (“all four”) or for the three variants only; entries are the mean over 10 seeds of pairwise direction concordance / Spearman ρ\rho between lower diagnostic and higher return.
Methodevery stepevery 2every 4once10% loss30% loss
bytes / episode50261425050
RAVEN (offline), full−- 9.78−- 9.78−- 9.79−- 9.84−- 9.80−- 9.88
local-residual target + geometric condition−- 11.60−- 11.60−- 11.61−- 11.71−- 11.62−- 11.68
geometric receiver condition−- 14.20−- 14.22−- 14.24−- 14.37−- 14.22−- 14.25
plain learnable 4-symbol channel (no codebook)−- 15.05−- 15.10−- 15.15−- 15.73−- 15.07−- 15.10
Appendix
Table 13. Communication rate and loss on N1-C2 (8 training seeds, frozen models). The receiver keeps the last symbol that arrived; “every kk ” sends a fresh symbol every kk steps, “once” only at the first step, and “loss” drops each message independently. Entries are mean team returns.
Figure 10. Return under reduced sending rates and message loss (N1-C2, 8 seeds). Numbers under the axis give the serialized payload per episode.
Figure 11. Left: return lost when incoming messages are removed at test time (ring navigation, fresh seeds). Right: empirical symbol entropy of offline RAVEN on N1-C2 for each of 10 seeds; dashed lines mark the maximum.
TaskMetricControlRAVENcontrolΔ\Delta [95% CI]seeds ++ / −-pp
Predator–prey PP1capture success (%)same-backbone QMIX96.0453.20++ 42.84 [15.5, 70.2]5/00.024
MPE Spreadteam returnsame-backbone QMIX−- 29.00−- 31.87++ 2.87 [1.77, 3.97]5/00.002
MPE Spreadteam returnw/o reference TD loss−- 28.83−- 29.52++ 0.69 [0.06, 1.32]3/00.043
Appendix
Table 14. Online implementation against controls that differ only in the communication pathway. The same-backbone QMIX shares the GRU backbone, optimizer, replay, exploration schedule, budget and per-seed initialization and removes codec, receiver, reference branch and LCV\mathcal{L}_{\mathrm{CV}} ; the ablation removes only the reference-branch TD loss. pp : paired two-sided tt -test (PP1: Holm-adjusted).
Figure 12. Predator–prey. Left: capture success during training (mean ±\pm s.d., 5 seeds). Right: final success of each seed; lines connect runs with the same seed and initialization.
Figure 13. Spread: online RAVEN against the variant without the reference-branch TD loss (3 matched seeds). Left: learning curves; right: final return per seed.
Figure 14. Training losses (means over seeds, 50k-step bins). Top: TD loss of the deployed branch, of the reference branch and of the same-backbone QMIX. Bottom: the alignment loss LCV\mathcal{L}_{\mathrm{CV}} .
Method3m8mMMMMMM23s5zSpreadTagCryptonorm.rank
RAVEN (online)99.72 ±0.2299.95 ±0.06100.00 ±0.0087.35 ±8.3097.80 ±1.12−- 29.00 ±0.59277.3 ±3.048.03 ±0.040.981.75
QMIX99.8098.8098.6057.8086.40−- 43.4223.390.380.747.3
QMIX (large)97.5799.3196.8826.9194.91−- 44.17−- 46.0421.690.708.6
QPLEX99.7599.5030.7562.0096.50−- 31.57235.145.440.805.0
OW-QMIX100.0096.2098.0038.7589.60−- 51.50−- 51.098.700.698.6
CW-QMIX98.0096.5096.002.2580.50−- 49.39−- 22.8747.210.699.9
Appendix
Table 15. SMAC (win rate, %) and MPE (team return) at 2.05M environment steps. RAVEN (online): mean ±s.d.{}_{\pm\text{s.d.}} (SMAC: 4 seeds; MPE: 5 seeds), 1,000 final evaluation episodes per model. Reference methods: mean over our runs (4–9) of the authors’ official open-source code, at the last evaluation before 2.05M steps (about 100 test episodes). Best per column in bold, second underlined (ties marked alike). The last two columns give the mean per-task min–max normalized score and the mean rank over the eight tasks. § single-neighbor ExpoComm variant.
Method3m8mMMMMMM23s5z
RAVEN (online)92.5 / 0.2593.2 / 0.2592.8 / 0.3044.8 / 1.5061.4 / 1.20
QMIX90.8 / 0.4093.1 / 0.1586.3 / 0.4529.3 / –59.1 / 1.15
QPLEX97.8 / 0.1597.2 / 0.1086.9 / 0.3025.9 / –90.0 / 0.30
QMIX+TeamComm82.1 / 0.6091.9 / 0.2088.7 / 0.3534.8 / –64.8 / 1.00
QMIX+TGCNet78.6 / 0.6093.1 / 0.3084.0 / 0.5042.5 / –67.3 / 1.00
CACOM92.2 / 0.2038.9 / –82.9 / 0.4512.4 / –64.7 / 0.90
Appendix
Table 16. SMAC learning speed: time-averaged win rate over 0.05–2.05M steps (%) / first training step (M) at which the mean curve reaches 80% (“–”: never within budget). Reference curves are binned to 50k steps.
Figure 15. SMAC learning curves of online RAVEN (mean ±\pm s.d.) and six reference methods (mean over runs, binned to 50k steps). † Single-neighbor variant.
Figure 16. MPE learning curves. Online RAVEN and the same-backbone QMIX: mean ±\pm s.d. over 5 seeds; reference methods: mean over runs, binned to 50k steps.
Figure 17. Final SMAC win rate of each online RAVEN seed (squares) against the mean of every reference method (ticks; the best one in black).
Figure 18. Final score of each method on each task, colored by the per-task min–max normalized score (cell values are win rates or returns; the mean normalized score is in brackets).
Figure 19. Training time per model, compared within one machine: (a) offline RAVEN (teacher included) and SLIM; (b) online RAVEN, CACOM and MACC (log scale).
RAVENvs.paramsexec. FLOPsbits / messagemeasured traffictraining time
RAVEN (offline)SLIM95.6 / 98.0–0.00–39.6 / 55.4
RAVEN (offline)NDQ−- 1.4 / 85.182.2 / 96.897.92––
RAVEN (offline)CACOM−- 2.1 / 85.085.8 / 97.791.67––
RAVEN (offline)ExpoComm48.4 / 92.391.5 / 98.399.90––
RAVEN (offline)MACC−- 229.0 / 50.945.4 / 89.30.00––
RAVEN (online)SLIM81.7 / 81.8–0.00––
Appendix
Table 17. Resource savings of RAVEN against every external communication-learning method, 100(1−RAVEN/other)100(1-\text{RAVEN}/\text{other}) in %; “lo / hi” is the range over settings or maps; negative values mean RAVEN uses more. Parameters and FLOPs (one team decision step) are counted on networks rebuilt from the code and configurations of the runs. Measured traffic is the number of bits delivered during the whole training run. Training time is compared only between runs on the same machine (offline vs. SLIM on one GPU server; online vs. CACOM and MACC on one Mac). ⋆ lower bound: only the request round of CACOM and only the message vector of TeamComm/TGCNet are counted.
SettingRAVEN (offline)RAVEN (online)SLIMNDQCACOMExpoCommMACC
N1-C211.7k / 23k48.0k / 190k264.0k / –32.7k / 128k32.4k / 160k68.0k / 268k10.7k / 42k
N1-C635.2k / 68k48.3k / 578k– / –34.7k / 409k34.5k / 567k68.2k / 807k10.7k / 125k
N2-L15.3k / 10k48.0k / 190k264.0k / –32.7k / 128k32.4k / 160k68.0k / 268k10.7k / 42k
N2-L65.3k / 16k48.3k / 677k264.0k / –35.2k / 485k35.0k / 688k68.3k / 943k10.7k / 146k
Appendix
Table 18. Absolute cost. Top: navigation, deployed parameters / execution FLOPs per team decision step (SLIM: parameters recorded by its runs, no FLOP count). Bottom: SMAC and MPE, parameters / FLOPs / bits per agent message (lower bounds for CACOM, TeamComm and TGCNet).
Figure 20. (a) Bits per message. (b) Deployed parameters against execution FLOPs per team decision step on SMAC and MPE. (c) The same for the eight navigation settings; SLIM’s recorded parameter count is marked by the dashed line.

Explore similar work

CardsList

  1. Attention-based Hierarchical Variational Information Bottleneck for Robust Multi-Agent Communication under Variable Bandwidth

    Sep 28, 2026Lukas Koch Vindbjerg, Qi Zhang, Yury Brodskiy +1Information BottleneckBandwidth Extension

    Department of Electrical and Computer Engineering, Aarhus University, Aarhus, Denmark · EIVA a/s, Denmark · DIGIT, Aarhus University, Aarhus, Denmark

  2. Emergence, Not Bandwidth: Physical Coupling and the Limits of Learned Multi-Agent Communication

    Sep 28, 2026Mihir Chauhan, Aniket BeraMulti-Agent Reinforcement LearningMulti-Agent Communication

    IDEAS Lab, Department of Computer Science Purdue University

  3. Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications

    May 13, 2026Maxwell Standen, Junae Kim, Claudia SzaboMulti-Agent Reinforcement LearningMulti-Agent Communication

    The University of Adelaide · DST Group