cs.ROMar 20, 2026

Why Cognitive Robotics Matters: Lessons from OntoAgent and LLM Deployment in HARMONIC for Safety-Critical Robot Teaming

Authors: Sanjay Oruganti, Sergei Nirenburg, Marjorie McShane, Jesse English, Michael Roberts, Christian Arndt, Ramviyas Parasuraman, Luis Sentis

Organizations: Cognitive Science Department, Rensselaer Polytechnic Institute, Troy, NY 12180, USA. · University of Georgia, Athens, GA, 30605, USA. · Aerospace Engineering and Engineering Mechanics, University of Texas at Austin, TX, US.

Abstract

Deploying embodied AI agents in the physical world demands cognitive capabilities for long-horizon planning that execute reliably, deterministically, and transparently. We present HARMONIC, a cognitive-robotic architecture that pairs OntoAgent, a content-centric cognitive architecture providing metacognitive self-monitoring, domain-grounded diagnosis, and consequence-based action selection over ontologically structured knowledge, with a modular reactive tactical layer. HARMONIC's modular design enables a functional evaluation of whether LLMs can replicate OntoAgent's cognitive capabilities, evaluated within the same robotic system under identical conditions. Six LLMs spanning frontier and efficient tiers replace OntoAgent in a collaborative maintenance scenario under native and knowledge-equalized conditions. Results reveal that LLMs do not consistently assess their own knowledge state before acting, causing downstream failures in diagnostic reasoning and action selection. These deficits persist even with equivalent procedural knowledge, indicating the issues are architectural rather than knowledge-based. These findings support the design of physically embodied systems in which cognitive architectures retain primary authority for reasoning, owing to their deterministic and transparent characteristics.

Figures & tables

Explore similar work

CardsList
  1. From Language to Action: Can LLM-Based Agents Be Used for Embodied Robot Cognition?

    Mar 3, 2026Shinas Shaji, Fabian Huppertz, Alex Mitrevski +1Large Language Model-Based Robot PlanningLanguage Model-Based Control

  2. HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory

    Jun 22, 2026Xiaolin Zhou, Liu Liu, Tingyang Xiao +9Long-Horizon Robotic ManipulationMobile Robotics

  3. Agentic Cognitive Depth: Operational Criteria for Evaluating LLM Agents

    Oct 3, 2026Nijesh Upreti, Chris Sypherd, Vaishak BelleLLM Agent EvaluationLLM Agents