Implementation of a Metacognition Framework for Self-Awareness and Self-Regulation in Ensembles of LLMs

Charles Courchaine; Ricky J. Sethi; Hefei Qiu

cs.AIAug 15, 2026

Implementation of a Metacognition Framework for Self-Awareness and Self-Regulation in Ensembles of LLMs

Authors: Charles Courchaine, Ricky J. Sethi, Hefei Qiu

Organizations: Fitchburg State University, Worcester Polytechnic Institute Fitchburg, MA, USA · Fitchburg State University Fitchburg, MA, USA

Abstract

Large Language Models (LLMs) are notorious for struggling with assessing their own uncertainty, detecting knowledge conflicts, or recognizing when problems exceed their expertise; such limitations inevitably undermine reliability and trust in LLMs. In this paper, we present the first implementation of a metacognitive framework for ensembles of LLMs that addresses these challenges through explicit monitoring and control mechanisms. Our system computes a Metacognitive State Vector (MSV) quantifying self-awareness for monitoring across five dimensions derived from cognitive psychology: Emotional Response, Correctness Evaluation, Experiential Match, Conflicting Information, and Problem Importance. MSV values also provide self-regulation for control, automatically switching between System 1 (fast, single- or multi-node) and System 2 (deliberative, multi-node) processing based on query complexity. For System 2 execution, graph-theoretic algorithms control the assignment of specialized roles (Domain Expert, Critic, Evaluator, Synthesizer, and Generalist) to ensemble nodes according to their MSV-quantified metacognitive states. Our implementation allows users to explore how different query types trigger distinct processing modes. The Proof-of-Concept (PoC) demo showcases the framework with illustrative examples showing appropriate System 1/System 2 routing and helps visualize the metacognitive process via real-time radar charts and decision indicators. This PoC implementation demonstrates the feasibility of creating a framework for metacognitive self-awareness and self-regulation in LLM systems.

Figures & tables

Dimension	Formula	Meaning
Emotional Response	$ER=\bigl(ER_{v},\,ER_{a},\,\vec{\sigma}\bigr)$ , with $ER_{v}=\sum_{i=1}^{n}\epsilon_{i}v_{i}$ , with $\epsilon_{i}\geq 0$ , $\sum_{i=1}^{n}\epsilon_{i}=1$	Valence, affect, & stability
Correctness Evaluation	$CE=\alpha_{1}F_{1}(\text{logical\_consist})+\alpha_{2}F_{2}(\text{factual\_acc})+\alpha_{3}F_{3}(\text{contextual\_approp})$	Logical, factual validity
Experiential Matching	$EM=\omega_{1}K(\text{resp, know\_base})+\omega_{2}H(\text{resp, hist\_resp})+\omega_{3}C(\text{prompt, cue\_famil})$	Familiarity with past cases
Conflicting Information	$CI=\delta_{1}D(\text{internal\_consist})+\delta_{2}D(\text{source\_agree})+\delta_{3}D(\text{temporal\_stabil})$	Contradiction detection
Problem Importance	$PI=\beta_{1}C(\text{potential\_conseq})+\beta_{2}U(\text{temporal\_urg})+\beta_{3}I(\text{scope\_impact})$	Task priority & scope

Table 1. The Metacognitive State Vector (MSV)

Figure 1. 5-phase Ensemble Control Flow Process.

Figure 2. UI for Factual query with only System 1 response.

Panel of Figure 3 — Figure 3. Node contributions and synthesis.

Panel of Figure 4 — Figure 4. Radar charts for MSV for all five dimensions (left) and just the emotion vector breakdown (right).

Panel of Figure 5 — Figure 5. UI for complex queries with System 2 activation.

Metacognition in Language Models

Metacognition in language models studies how models assess their knowledge and reasoning, express uncertainty, and regulate their own actions.

See all 128 papers

Think $^{2}$ : Grounded Metacognitive Reasoning in Large Language Models
Feb 21, 2026Abraham Paul Elenjical, Vivek Hruday Kavuri, Vasudeva VarmaLanguage Model Self-Assessment Metacognition in Language Models
IIIT Hyderabad
Evidence for Limited Metacognition in LLMs
Sep 25, 2025Christopher AckermanLLM Evaluation LLM Uncertainty Estimation
Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs
Jun 30, 2026Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona +2RL for Language Models Reinforcement Learning
Yale University · Google Research

Feb 21, 2026J/K move · Enter open · S save

Think $^{2}$ : Grounded Metacognitive Reasoning in Large Language Models

Abraham Paul Elenjical, Vivek Hruday Kavuri, Vasudeva Varma

IIIT Hyderabad

Language Model Self-Assessment Metacognition in Language Models LLM Self-Correction LLM Reliability Adaptive Computation

Open paper AskPDF arXiv

Large Language Models (LLMs) demonstrate strong reasoning performance, yet their ability to reliably monitor, diagnose, and correct their own errors remains limited. We introduce a psychologically grounded metacognitive framework that operationalizes Ann Brown's regulatory cycle (Planning, Monitoring, and Evaluation) as a structured prompting architecture, and study its integration within a lightweight dual-process MetaController for adaptive effort allocation. Across diverse reasoning and diagnostic benchmarks (GSM8K, CRUXEval, MBPP, AIME, CorrectBench, and TruthfulQA) using Llama-3 and Qwen-3 (8B), explicit regulatory structuring substantially improves error diagnosis and yields a threefold increase in successful self-correction. Blinded human evaluations over 580 query pairs show an 84% aggregate preference for trustworthiness and metacognitive self-awareness over standard and Chain-of-Thought baselines. Grounding LLM reasoning in established cognitive theory offers a principled path toward more transparent and diagnostically robust AI systems.

Abstract

Metacognition in Language Models

Explore similar work

Think2^{2}2: Grounded Metacognitive Reasoning in Large Language Models

Evidence for Limited Metacognition in LLMs

Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs

Think $^{2}$ : Grounded Metacognitive Reasoning in Large Language Models