cs.AINov 5, 2026

Circe: Training-Free LLM Self-Evaluation via Component Interaction Consistency

Authors: Shuyi Miao, Zheng Li, Xu Xie, Linrui Xu, Fei Shen, Xiaoyu Du, Tat-Seng Chua

Organizations: Beihang University, Beijing, China · Nanjing University of Science and Technology, Nanjing, China · Independent Researcher, China · Central South University, Changsha, China · 

Abstract

As large language models (LLMs) are increasingly deployed in real-world applications, evaluating the correctness of their responses is crucial for reliable use. However, such evaluation is challenging because reference answers are rarely available in practice. Existing reference-free methods mainly rely on output signals, which may assign high scores to incorrect responses and thus do not necessarily reflect correctness. This motivates exploring the model's internal computation for complementary correctness signals. Recent studies exploit internal states, but they mainly analyze hidden-state trajectories across layers, which record only the combined updates of attention and feed-forward networks (FFNs) and overlook how these components interact. We therefore propose Circe, a training-free self-evaluation method that assesses a response using both internal and external signals, without reference answers or repeated sampling. For internal signals, we first introduce component interaction consistency (CIC), which measures the directional alignment between attention updates in earlier layers and attention-induced FFN output changes in later layers. We further incorporate representation geometry (RG) to characterize normalized displacement and angular changes along layerwise hidden-state trajectories. For the external signal, we convert the mean negative log-likelihood of generated tokens into a bounded likelihood confidence (LC) score. Finally, we fuse CIC, RG, and LC into a unified correctness score. Experiments on five LLMs and six benchmarks across four domains show that our method distinguishes correct from incorrect responses more effectively than existing baselines, and generalizes to multilingual, multimodal, and open-world settings.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Calibration-Constrained Diagnostics for Evaluating LLM Self-Evaluation

    Nov 1, 2026Jae Oh Woo, Mengdie Flora Wang, Rahul Ghosh +2Language Model Self-AssessmentConfidence Estimation in Language Models

  2. CELEUS: Certifiable and Efficient LLM Evaluation via E-Processes

    Jun 18, 2026Zhijian Zhou, Zesheng Ye, Zhaorun Chen +2LLM EvaluationAnytime-Valid Inference

  3. Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Data

    Jun 3, 2026XiuYu Zhang, Yi Shan, Junfeng Fang +1LLM EvaluationLLM-as-a-Judge