The Tower of Babel in artificial intelligence: who should speak, and when it pays to listen
Organizations: Department of Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany · Centre for the Advanced Study of Collective Behaviour, University of Konstanz, Konstanz, Germany · Department of Biology, University of Konstanz, Konstanz, Germany
Abstract
Two machines can each recognize an object well and still fail together. We study classifiers that see different parts of one object and send short messages to a receiver that identifies its class. Coordination asks how to design messages together; cooperation asks whether the team outperforms one sender given the whole budget. On three digit datasets, messages designed together beat messages designed apart by 18.07 to 22.68 percentage points on average with two-bit messages, with gains in all 24 configurations. A fixed agreement, you report the row and I the column, recovers about half this gain, with only the receiver learned. Senders that draw the same distinctions waste a narrow vocabulary, and senders whose distinctions cross multiply it. A team has no more possible messages than one sender with the same budget, so a partner must bring evidence the first lacks. A solvable model shows individual statistics alone cannot determine whether a partnership helps. On MNIST digits, six shared bits remove 49.54 percent of the strongest tested solo's errors on average. Paired measurements correctly predicted most sensor teams would reduce average accuracy on new participants. Who should speak is a property of the pair, and must be measured in pairs.
Figures & tables
Cooperative Perception
Cooperative perception lets connected vehicles, robots, and aerial agents combine sensor observations to understand scenes beyond a single viewpoint.
See all 121 papers