Reality and Perceptual Gaps in Expressive Swarm Robotics

Ali Imran; Abdalwhab Bakheet Mohammed Abdalwhab; Rafael Gomes Braga; Hélène Duval; Audrey Rochette; Vivek Shankar Varadhajan; Yann Bouteiller; Jean Mazerolle

cs.RONov 3, 2026

Reality and Perceptual Gaps in Expressive Swarm Robotics

Authors: Ali Imran, Abdalwhab Bakheet Mohammed Abdalwhab, Rafael Gomes Braga, Hélène Duval, Audrey Rochette, Vivek Shankar Varadhajan, Yann Bouteiller, Jean Mazerolle,

Organizations: INIT Robots Laboratory, Department of Mechanical Engineering, École de technologie supérieure, Montréal, Québec, Canada . · Department of Dance, Université du Québec à Montréal, Montréal, Québec, Canada . · MIST Lab, Department of Computer and Software Engineering, Polytechnique Montréal, Montréal, Québec, Canada .

Abstract

Moving robot swarms from simulation into the physical world can be difficult because small discrepancies in hardware, timing and communication can cascade across interacting robots and alter group behaviour. This challenge is known as the reality gap. When swarms operate in human environments, a second challenge emerges: a perceptual gap between what the swarm physically does and what human onlookers actually perceive. This study examines the same decentralised coordination code deployed across three robotic swarms-tabletop, aerial and human-scale mobile manipulators-during live artistic performances. We find that while the underlying algorithms executed reliably on all three platforms, physical embodiment drastically altered visible motion-distorting execution timescales by more than a factor of three and reshaping trajectory geometry. More importantly, audience surveys revealed that human interpretation does not mirror technical success: how observers attribute meaning (perceived purpose) and how they react emotionally (comfort versus threat) depend heavily on physical form. Meaning attribution and emotional responses did not vary together. Our results demonstrate that successfully transferring swarm software to physical hardware is insufficient for real-world deployment. Addressing the reality gap does not guarantee that the resulting collective motion will be readily interpreted by human observers. Swarms destined for human environments must be engineered not only for decentralised stability but also for the expressive physical dynamics through which humans interpret collective motion.

Figures & tables

Figure 1: From specified collective organisation to human interpretation: the sequential reality and perceptual gaps examined in this work. The physically realised collective forms the stimulus available to human perception.

Panel of Figure 2 — Figure 2: The three DESSAIM robotic swarms seen in their performing settings.

Figure 3: Example trajectories of the three behaviours on each platform: (a) Fireworks, (b) cyclic Pursuit, and (c) Lennard–Jones. Each behaviour runs from the same script on the Doodies, CrazyCognies and Sushis. Lines show individual robots, coloured by time.

Figure 4: Distribution of each robot’s direction of motion relative to the outward radial direction from the swarm centroid, $\alpha$ (0° outward, 180° inward, $\pm$ 90° tangential), for Fireworks ( a ), cyclic Pursuit ( b ), and Lennard–Jones ( c ) on the Doodies (motion capture), CrazyCognies (motion capture), and Sushis (overhead video). Radial motion produces a vertical bow-tie distribution, sustained circulation produces a lateral lobe, and motion without a preferred radial or tangential orientation produces an approximately circular distribution. Each outline represents one take; the dotted circle denotes a uniform angular distribution, and the polar-density scale is set independently for each panel. Column headers report the $\psi$ span between the most radial Fireworks take and the most tangential Pursuit take.

Figure 5: Median robot speed against the ratio of median tangential to median radial speed relative to the swarm centroid, on logarithmic axes. Above the dashed line, motion is predominantly tangential; below it, predominantly radial. Outlines group takes from the same platform–behaviour combination. Left of the axis break are the three Sushi Lennard–Jones takes; their very low speeds mean that the ratio is dominated by near-stationary periods rather than the brief corrective motions that maintained spacing.

Fireworks
	Doody	CrazyCogni	Sushi
Median $R_{g}$ (m)	$1.45\pm 0.24$	$1.53\pm 0.05$	$0.166\pm 0.004$
Median speed (mm s -1 )	$130\pm 40$	$166\pm 25$	$34.8\pm 1.0$
$\psi$	$+0.57\pm 0.07$	$+0.16\pm 0.07$	$+0.653\pm 0.015$
$P$	$0.33\pm 0.08$	$0.29\pm 0.07$	$0.157\pm 0.015$
$M$	$0.175\pm 0.029$	$0.17\pm 0.04$	$0.055\pm 0.008$

Table 1:

R_{g}

, radius of gyration;

\psi

, radial index;

P

, polarisation;

M

, milling order; modulation depth, peak-to-trough swing of the phase-folded

R_{g}

over its median. Measurements were obtained from motion capture for the Doodies and CrazyCognies and overhead video for the Sushis.

Figure 6: polarisation $P$ (shared heading; median over time of $\lVert\langle\hat{\mathbf{v}}_{i}\rangle\rVert$ ) against milling order $M$ (rotation about the centroid; median over time of $|\langle(\hat{\mathbf{r}}_{i}\times\hat{\mathbf{v}}_{i})_{z}\rangle|$ ), one point per take. Dashed boxes indicate the fish-school states of Tunstrøm et al. [ 33 ] : disordered ( $P,M<0.35$ ), milling ( $P<0.35$ , $M>0.65$ ), and polarised ( $P>0.65$ , $M<0.35$ ). These thresholds were defined for groups of 30–300 fish; with four to six robots, chance alone gives $P\approx 0.4$ . Grey regions show the union of the 95% null intervals across takes, each estimated from 200 randomised copies in which every robot’s velocity series was independently time-shifted and rotated while positions were preserved.

Figure 7: Local median position error relative to each behaviour’s own median, pooled across behaviours, for the aerial (left) and ground (right) platforms. Stippling marks locations where the local value cannot be distinguished from the platform’s typical error, covering 68% of the aerial field and 83% of the ground field. Circles indicate the smoothing bandwidth, which differs between platforms because of the number of samples (72,184 ground rows versus 26,120 aerial rows).

Platform	Behaviour	$n$	Horizontal error (m)	Samples $>0.5$ m
			median [Q1, Q3]	(%)
CrazyCognies	Fireworks	13	0.077 [0.049, 0.115]	1.1
	Cyclic pursuit	15	0.077 [0.050, 0.111]	0.2
	Lennard–Jones	13	0.078 [0.048, 0.111]	3.0
Doodies	Fireworks	14	0.128 [0.081, 0.224]	6.0
	Cyclic pursuit	14	0.107 [0.065, 0.155]	2.1

Table 2: Localisation uncertainty during expressive behaviours. Horizontal position error of the onboard estimate relative to motion capture, by platform and behaviour. Nine takes were recorded per platform, three per behaviour.

n

gives the number of robot-takes passing the correspondence criterion. Position error is reported as median [Q1, Q3]. The final column gives the fraction of synchronised samples with error exceeding 0.5 m.

Panel of Figure 8 — Figure 8: The three DESSAIM robotic platforms.

Figure 9: PyBuzz architecture used to preserve a common Buzz behavioural representation across simulation and heterogeneous physical platforms. Each logical robot executes an independent Buzz Virtual Machine (BVM), while platform adapters translate localisation, sensing, inter-robot communication, and motion commands between the behavioural layer and platform-specific interfaces.

Behaviour	Platforms	Designed collective organisation	Compositional function
Aggregation	S	Convergence toward the local centroid	Contraction; increasing density
Attraction point	S	Convergence toward an external reference	Directed collective displacement
Diffusion	S	Motion away from the local centroid	Expansion; decreasing density
Repulsion point	S	Motion away from an external reference	Directed collective avoidance
Fireworks	S, C, D	Cyclic aggregation–diffusion	Rhythmic contraction–expansion
Lennard–Jones	S, C, D	Regulation of inter-agent spacing	Cohesive, deformable spatial structure

Table 3: Expressive behaviour vocabulary used in DESSAIM. S: Sushis; C: CrazyCognies; D: Doodies.

Figure 10: Iterative research-creation process used in DESSAIM. Robotic experimentation, choreographic and movement exploration, performance, observation, public feedback, and interdisciplinary debriefing were linked through repeated cycles of reconfiguration. Technical, artistic, and perceptual observations could be reinvested in subsequent iterations, allowing changes to robot behaviour, localisation, dramaturgy, staging, embodiment, and mediation.

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 765 papers

Warmth and Competence in the Swarm: Designing Effective Human-Robot Teams
Apr 21, 2026Genki Miyauchi, Roderich Groß, Chaona ChenMulti-Robot Systems Swarm Robotics
School of Electrical and Electronic Engineering, University of Sheffield, Sheffield, UK · Department of Computer Science, Technical University of Darmstadt, Darmstadt, Germany · School of Computer Science, University of Sheffield, Sheffield, UK
Mediating Human-Swarm Interactions Through a Robot Helper: A Shepherding-Inspired Approach
Nov 2, 2026Arihant Yadav, Sophie Jörg, Eakta JainSwarm Robotics Robotic Control
Arihant Yadav & Eakta Jain are with Department of Computer & Information Science & Engineering , University of Florida, Gainesville, USA · Sophie Jörg is a Professor in the Information Systems and Applied Computer Sciences Department at University of Bamberg, Germany
Do Robots Need Body Language? Comparing Communication Modalities for Legible Motion Intent in Human-Shared Spaces
Apr 3, 2026Jonathan Albert Cohen, Kye Shimizu, Allen Song +3Human-Robot Interaction
MIT Media Lab Cambridge USA · New England Innovation Academy Cambridge USA

Apr 21, 2026J/K move · Enter open · S save

Warmth and Competence in the Swarm: Designing Effective Human-Robot Teams

Genki Miyauchi, Roderich Groß, Chaona Chen

School of Electrical and Electronic Engineering, University of Sheffield, Sheffield, UK · Department of Computer Science, Technical University of Darmstadt, Darmstadt, Germany · School of Computer Science, University of Sheffield, Sheffield, UK

Multi-Robot Systems Swarm Robotics Human-Robot Collaboration Human-Robot Interaction

Open paper AskPDF arXiv

As groups of robots increasingly collaborate with humans, understanding how humans perceive them is critical for designing effective human-robot teams. While prior research examined how humans interpret and evaluate the abilities and intentions of individual agents, social perception of robot teams remains relatively underexplored. Drawing on the competence-warmth framework, we conducted two studies manipulating swarm behaviors in completing a collective search task and measured the social perception of swarm behaviors when human participants are either observers (Study 1) and operators (Study 2). Across both studies, our results show that variations in swarm behaviors consistently influenced participants' perceptions of warmth and competence. Notably, longer broadcast durations increased perceived warmth; larger separation distances increased perceived competence. Interestingly, individual robot speed had no effect on either of the perceptions. Furthermore, our results show that these social perceptions predicted participants' team preferences more strongly than task performance. Participants preferred robot teams that were both warm and competent, not those that completed tasks most quickly. These findings demonstrate that human-robot interaction dynamically shapes social perception, underscoring the importance of integrating both technical and social considerations when designing robot swarms for effective human-robot collaboration.

Abstract

Robotics

Explore similar work

Warmth and Competence in the Swarm: Designing Effective Human-Robot Teams

Mediating Human-Swarm Interactions Through a Robot Helper: A Shepherding-Inspired Approach

Do Robots Need Body Language? Comparing Communication Modalities for Legible Motion Intent in Human-Shared Spaces