Multi-Agent Object Detection Framework Based on Raspberry Pi YOLO Detector and Slack-Ollama Natural Language Interface

Vladimir Kalušev; Branko Brkljač; Milan Brkljač

cs.CVApr 14, 2026

Multi-Agent Object Detection Framework Based on Raspberry Pi YOLO Detector and Slack-Ollama Natural Language Interface

Authors: Vladimir Kalušev, Branko Brkljač, Milan Brkljač

Organizations: The Institute for Artificial Intelligence Research and Development of Serbia Fruškogorska 1, 21000 Novi Sad, Republic of Serbia · Department of Power, Electronic and Telecommunication Engineering Faculty of Technical Sciences, University of Novi Sad Trg Dositeja Obradovića 6, 21000 Novi Sad, Republic of Serbia · Faculty of Finance, Banking and Auditing, Alfa BK University Bulevar maršala Tolbuhina 8, 11070 Belgrade, Republic of Serbia

Abstract

The paper presents design and prototype implementation of an edge based object detection system within the new paradigm of AI agents orchestration. It goes beyond traditional design approaches by leveraging on LLM based natural language interface for system control and communication and practically demonstrates integration of all system components into a single resource constrained hardware platform. The method is based on the proposed multi-agent object detection framework which tightly integrates different AI agents within the same task of providing object detection and tracking capabilities. The proposed design principles highlight the fast prototyping approach that is characteristic for transformational potential of generative AI systems, which are applied during both development and implementation stages. Instead of specialized communication and control interface, the system is made by using Slack channel chatbot agent and accompanying Ollama LLM reporting agent, which are both run locally on the same Raspberry Pi platform, alongside the dedicated YOLO based computer vision agent performing real time object detection and tracking. Agent orchestration is implemented through a specially designed event based message exchange subsystem, which represents an alternative to completely autonomous agent orchestration and control characteristic for contemporary LLM based frameworks like the recently proposed OpenClaw. Conducted experimental investigation provides valuable insights into limitations of the low cost testbed platforms in the design of completely centralized multi-agent AI systems. The paper also discusses comparative differences between presented approach and the solution that would require additional cloud based external resources.

Figures & tables

Panel of Figure 1 — Figure 1: Developed fast prototyping hardware testbed: (a) custom Raspberry Pi camera setup, (b) MIPI CSI camera board with Sony IMX219 color sensor.

Figure 2: Multi-agent system architecture: functional elements of the proposed object detection framework and implemented agentic components.

Component	Details, Raspberry Pi 4 model B
Processor	Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.8GHz
Memory	8 GB LPDDR4-3200 SDRAM
Wireless	2.4 GHz and 5.0 GHz IEEE 802.11ac wireless, Bluetooth 5.0
Ethernet	Gigabit Ethernet
USB ports	2 USB 3.0 ports; 2 USB 2.0 ports
GPIO	Raspberry Pi standard 40 pin GPIO header

Table 1: Hardware testbed specifications.

Specification	Details, Raspberry Pi module v2
Resolution	8 megapixels
Max Video	1080p47
Still Modes	1640 × 1232p41; 640 × 480p206
Sensor	Sony IMX219
Sensor Resolution	3280 × 2464 pixels

Table 2: Camera board specifications.

Panel of Figure 3 — Figure 3: Software components implementing proposed multi-agent architecture in Fig. 2 : (a) modules structure, (b) code snippet from "main.py" illustrating system initialization phase with reporting and communication agent subscription to agent specific router events.

Figure 4: Illustration of the implemented functionalities: (a) vision agent’s object tracking, and (b) report agent’s asynchronous LLM system call.

Figure 5: Characteristics of the messaging application: socket mode communication, message events subscription and the bot token scopes defining what functionality communication and control agents can ask for from the chatbot instance.

Figure 6: Multi-agent system operation with user interaction through created natural language interface based on the Slack channel and Ollama event report generation. Both notification and control commands are provided through familiar messaging application, while the object detection and tracking events are triggering combined image and text system responses.

Panel of Figure 7 — Figure 7: Illustration of system configuration without LLM based reporting agent: (a) object tracking and video encoding, (b) CPU load and memory usage.

Multi-Agent Orchestration

Multi-agent orchestration coordinates specialized AI agents, tools, and workflows to solve complex tasks efficiently and reliably.

See all 206 papers

An Intelligent-Cloud Edge Multimodal Interaction System for Robots
Jul 16, 2026Zihan Guo, Xiaoqi LiRobotics Gesture Recognition
Hainan University, China
Fetch.ai: An Architecture for Modern Multi-Agent Systems
Oct 21, 2025Michael J. Wooldridge, Attila Bagoly, Jonathan J. Ward +2Multi-Agent Orchestration Multi-Agent Systems
University of Oxford
ORICF -- Open Robotics Inference and Control Framework
May 10, 2026Andrés Meseguer Valenzuela, Luís Miguel Bartolín ArnauEfficient Multimodal Inference Edge Inference
Instituto Tecnológico de Informática (ITI), Paterna, Spain

Jul 16, 2026J/K move · Enter open · S save

An Intelligent-Cloud Edge Multimodal Interaction System for Robots

Zihan Guo, Xiaoqi Li

Hainan University, China

Robotics Gesture Recognition Human-Robot Interaction Multimodal Agents

Open paper AskPDF arXiv

Robust human-robot interaction in complex environments requires accurate gesture perception, semantic scene understanding, and reliable task planning under limited onboard computing resources. This paper presents a cloud-edge multimodal interaction framework that integrates an enhanced YOLO-based gesture detector with coordinated large language model (LLM) and vision-language model (VLM) agents. The proposed detector, incorporates the Convolutional Block Attention Module (CBAM) into the neck and replaces the baseline bounding-box regression objective with Distance-IoU (DIoU) loss. These modifications improve feature discrimination and localization for small or partially occluded gestures in complex backgrounds. The cloud layer performs gesture detection, scene understanding, multimodal fusion, and action planning, whereas the TonyPi robot locally handles data acquisition, communication, action execution, and feedback. Experiments on a public gesture dataset and a custom dataset show that YOLO-DC achieves precision values of 98.9% and 95.0%, with [email protected] values of 90.7% and 92.7%, respectively. System-level evaluation yields success rates of 95%, 88%, and 82% for single-action, composite-action, and vision-dependent tasks. A 30 participant evaluation yields an overall mean satisfaction score of 3.69 out of 5. These results demonstrate the feasibility of combining refined gesture detection with multimodal agents for resource-constrained robotic interaction.

Abstract

Multi-Agent Orchestration

Explore similar work

An Intelligent-Cloud Edge Multimodal Interaction System for Robots

Fetch.ai: An Architecture for Modern Multi-Agent Systems

ORICF -- Open Robotics Inference and Control Framework