Effective human-machine teaming requires agents that coordinate with unpredictable human partners. Reinforcement learning from live human feedback or large behavioural datasets is expensive, while purely simulated training overfits to artificial conventions and fails at Zero-Shot Coordination with real people. We propose a framework for training teaming agents against Cognitively Constrained Partners: simulated partners whose diversity emerges from procedural constraints grounded in human physiological literature, specifically reaction delays and perceptual limitations, rather than from reward shaping or historical training checkpoints. In a modified Lunar Lander environment, we report preliminary evidence that these partners play more like real people than unconstrained agents do, and that agents trained against them match the strongest cross-play baselines while being the only ones whose performance does not depend on how their training partners were generated. We also describe a human-participant study designed so that its rankings can be compared directly against cross-play, testing how well that standard evaluation predicts real human-agent teaming.
Figures & tables
Figure 1: Lunar Lander environment from Gymnasium [ 8 ] . Physical geometry (left) and the partitioned control pipeline (right). The main engine and side thrusters must be coordinated to reach the landing pad.
Figure 2: Individual controller distributions for mean press duration (side and main), score, and idle time (neither control) across human, constrained, and unconstrained groups.
Figure 3: Mean return of each agent–partner pairing, averaged over 10 independently trained replicates per agent type. Parenthesised figures are the standard deviation across the 15 partners comprising each pool. One of the fifteen MARL side agents is the co-adapted partner of each MARL replicate.
Teaming agent
Matched pool
Held out
Change
95% CI
Constrained
203.4
266.6
+63.1
[ +23.1,+125.3 ]
PBT
277.9
242.9
−34.9
[ −58.3,−15.8 ]
FCP
280.1
232.6
−47.6
[ −70.7,−28.9 ]
MARL
242.7
222.7
−19.9
[ −51.1,+5.5 ]
Table 1: Mean return when each agent is paired with partners from the pool generated by the procedure it trained against and from the three pools it did not train against. Intervals are percentiles of a two-way cluster bootstrap over 5,000 resamples of trained replicates and partners, paired by replicate so that the strength of a given training run cancels from the difference.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Ex-Gaussian delay distributions for a single constrained agent, shown at the midpoint of each parameter range. The mean of every phase is sampled independently for each constrained agent, so each member of the population carries a different set of curves. The dashed curve is the convolution of the processing and mechanical delays, giving the total latency between a change in the environment and the corresponding change in the partner’s output.
Parameter
Value
Notes
Discount factor ( γ )
0.99
Standard for continuous control
Learning rate
0.0005
Constant throughout training
Entropy coefficient
0.005
Balances exploration with destabilisation risk
Batch size
128
Empirically tuned on solo agents
Rollout length
1024
Empirically tuned on solo agents
Episode limit
750 steps
Same for all conditions
Appendix
Table 2: Training hyperparameters. All agents use RecurrentPPO with an MlpLstmPolicy .
Overlap
Wasserstein
Feature
Constrained
Unconstrained
Constrained
Unconstrained
Mean press duration (side)
0.73
0.00
0.32
2.01
Final speed
0.77
0.50
0.72
1.07
Final displacement
0.59
0.25
0.73
1.32
Final tilt
0.78
0.75
0.74
0.94
Idle (side)
0.36
0.71
0.86
0.42
Appendix
Table 3: Individual feature comparison of agents with the human distribution. Overlap is the shared probability mass (higher is closer); distance is the normalised Wasserstein distance (lower is closer). Bold indicates the closer of the two agent types under each of the two distributional measures.
Human
Constrained
Unconstrained
Feature
Mean
(SD)
Mean
(SD)
Mean
(SD)
Mean press duration, side (ms)
298.5
(80.1)
279.8
(98.2)
47.1
(9.3)
Mean press duration, main (ms)
561.8
(171.0)
353.2
(49.3)
98.5
(42.4)
Press duration CV, side
0.438
(0.173)
0.533
(0.088)
0.982
(0.196)
Press duration CV, main
0.537
(0.084)
0.435
(0.063)
0.858
(0.333)
Mean idle length (ms)
321.1
(104.7)
196.1
(192.5)
49.2
(13.0)
Appendix
Table 4: Mean value and standard deviation of each feature by group, averaged within and then across controllers. Bold indicates the agent type nearer the human mean. Reported alongside Table 3 , since normalisation via pooled standard deviation can distort rankings for features with long tails.
Figure 5: Distribution of behavioural features comparing human, constrained agent, and unconstrained agent groups.
Comparison
Shared families
Advantage
95% CI
Significance
vs PBT
FCP, MARL
−4.5
[−11.4,+1.9]
Not significant
vs FCP
PBT, MARL
+4.0
[−1.8,+10.0]
Not significant
vs MARL
PBT, FCP
+27.5
[+8.7,+52.7]
Significant
Appendix
Table 5: Held-out advantage of the constrained agent over each baseline, restricted to the partner families neither agent trained on so that both are averaged over the same partners. Intervals are percentiles of a two-way cluster bootstrap over 5,000 resamples, paired by partner so that per-partner difficulty cancels from the difference, with replicates resampled independently for each agent.
Figure 6: Instructions seen by participants at the start of the trial.