Agentic Self-Awareness: Aligning Knowing, Acting, and Holding

Qinglin Chen; Hiroaki Hayashi; Jiaxin Zhang; Chien-Sheng Wu

cs.AIOct 13, 2026

Agentic Self-Awareness: Aligning Knowing, Acting, and Holding

Authors: Qinglin Chen, Hiroaki Hayashi, Jiaxin Zhang, Chien-Sheng Wu

Organizations: Salesforce AI Research

Abstract

Frontier AI agents increasingly act in environments where seemingly solvable tasks may lack prerequisites, contain conflicting requirements, or reveal blockers only during execution. We formulate agentic self-awareness (ASA) as the alignment of task-grounded feasibility judgments (know), warranted behavior (act), and responses to reconsideration (hold), distinguishing what a task permits from what an agent can solve. We build paired feasible and blocked tasks in Terminal-Bench 2.1 and retail $τ^2$ -bench and introduce an authorship-aware M0-M3 rubric that separates recognizing a blocker from taking the certified deferral action for the correct reason. Separate probes assess stated feasibility beliefs and responses to a single challenge. In a Terminal-Bench protocol baseline, blocker recognition reaches 84.4%, but strict self-awareness (SA) reaches only 40.6%. Harness search without model-weight updates increases strict SA from 45.3% to 93.8% on reused Terminal-Bench search tasks; selection for abstention alone, however, can reduce success on feasible tasks. Independently initialized joint-objective searches improve both blocked-task SA and feasible-task solve on their own search sets. Retail held-out gains remain limited, and Terminal-Bench blocked-task transfer is not measured. This evaluation pipeline makes the recognition-action gap measurable and sets a two-sided target for improving agent decisions under changing task evidence.

Figures & tables

Figure 1: Overview of the proposed ASA decomposition. We characterize ASA through three related dimensions: belief quality (know), control (act), and stability (hold). Belief quality includes both feasibility discrimination and probability calibration; control tests whether the agent takes the warranted action; stability tests its response to challenge. Belief probes and scored trajectories are separate measurements. The example shows an augmented task whose requested records are unavailable. The agent identifies the blocker and defers rather than fabricating a result. We hold the model

M

fixed and optimize only its harness

H

Category	Decisive condition	Evidence appears	Required meta-action
Infeasible	A prerequisite is absent or the requested operation is unsupported	Task specification or initial state	Abstain; identify the blocker
Ambiguous	Requirements conflict or under-specify a warranted completion	Task specification	Abstain; surface the conflict or ambiguity
Trap	Apparently usable input contains a latent blocker	During execution	Abstain on discovery; identify the blocker

Table 1: Three types of blocked task. Examples illustrate the design rather than matched variations of one scenario. The feasible source tasks remain as should-act controls.

	$f=1$	$f=0$
$b=1$	Proceed ✓	Proceed $\times$
$b=0$	Defer $\times$	Defer ✓

Table 2: Belief

\times

feasibility at

t^{\star}

. Green actions match feasibility; red actions do not. For example, an agent with a conviction (

b=1

) would proceed on a task with no feasibility (

f=0

), which is an unwanted behavior.

Figure 2: The M0–M3 ladder. M0 records blocker recognition; M1 rejects fabricated completion; M2 requires the certified action; M3 requires the correct blocker. Strict SA requires M0, M1, agent authorship, and M3; relaxed SA substitutes M2 for M3. These gates apply to blocked tasks; feasible tasks are scored separately by native solve.

	TB 2.1	Retail $\tau^{2}$
Blocked (S/H)	$64/32$	$64/32$
Feasible (S/H)	$43/28$	$64/32$
Independent feasible	$17$	$17$

Table 3: Task splits. S/H denotes search/held-out; independent feasible tasks are evaluation-only.

Figure 3: Objective-conditioned search and feasible-task outcomes. Panel (a) shows search-set strict SA; panels (b,c) show feasible solve and false abstention on Terminal-Bench’s independent 17-task pool and retail’s 32 held-out counterparts. Gray ticks mark each independently initialized search’s own R0. Terminal-Bench bars average three reruns; retail bars are single evaluations. The panels use different task splits, and cross-search contrasts are descriptive.

Figure 4: Authorship gating removes scaffold-written credit. On the augmented search set (

n=64

), condition D yields 53 verified abstention artifacts, none agent-authored: naive M2 scores

82.8\%

, whereas authorship-gated relaxed SA scores

0\%

Benchmark	Objective	Strict SA (blocked)	Solve (feasible)
Terminal-Bench 2.1	SA-only	$45.3\rightarrow 93.8\%$	—
Terminal-Bench 2.1	Joint	$53.1\rightarrow 76.6\%$	$55.8\rightarrow 72.1\%$
Retail $\tau^{2}$	SA-only	$42.2\rightarrow 60.9\%$	—
Retail $\tau^{2}$	Joint	$40.6\rightarrow 51.6\%$	$40.6\rightarrow 62.5\%$

Table 4: Search-set outcomes (R0

\rightarrow

selected). Each row compares one independently initialized search with its own R0. SA-only selection evaluates blocked tasks only, hence no feasible search-set solve entry. Each task–configuration cell contains one stochastic trajectory; selection on these tasks makes the contrasts descriptive.

Figure 5: Late evidence is associated with lower relaxed SA. Rows are benchmarks and columns are independent search objectives. Open and filled markers show initial and selected harnesses on the search sets. Scores use relaxed, authorship-gated SA (Eq. 4 ); segments connect point estimates and are not confidence intervals.

Benchmark	Harness	$n_{\mathrm{A}}/n_{\mathrm{C}}$	AUROC $\uparrow$	Brier $\downarrow$	ECE ${}_{10}\downarrow$
Terminal-Bench 2.1	Initial	$71/46$	$0.890$	$0.186$	$0.192$
	Joint selected	$75/46$	$0.880$	$0.149$	$0.143$
Retail $\tau^{2}$	Initial	$64/64$	$0.631$	$0.318$	$0.308$
	SA-only selected	$64/64$	$0.858$	$0.227$	$0.220$
	Joint selected	$64/64$	$0.825$	$0.233$	$0.242$

Table 5: Held-out feasibility beliefs.

n_{\mathrm{A}}

is valid AUROC support and

n_{\mathrm{C}}

is valid Brier/ECE support. Terminal-Bench AUROC support varies by harness; its Brier/ECE entries use the same 46-task intersection. Retail uses the same 32 blocked and 32 feasible tasks for all three harnesses. Comparisons are within benchmark. Confidence intervals appear in Appendix Tables 15 and 16 .

Selected harness	$P(+\mid+)$	$P(+\mid-)$	$P(-\mid+)$	Contrast (pp)
Terminal-Bench	$86.9\pm 5.1$	$0.0\pm 0.0$	$13.1\pm 5.1$	$-13.1\pm 5.1$
Retail $\tau^{2}$ -Bench	$97.5\pm 3.1$	$57.9\pm 5.0$	$2.5\pm 3.1$	$+55.4\pm 4.4$

Table 6: Conditional transitions after one neutral challenge. Entries are percentages (mean

\pm

sample SD over six runs); in

P(x\mid y)

x

is post-challenge status and

y

is pre-challenge status. Contrast is the second transition minus the third, in percentage points. Retail uses the same predicate on both sides and reports

\mathrm{CRA}

; Terminal-Bench uses different predicates and reports

\Delta_{\chi}

. Rows are not a controlled cross-benchmark comparison.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Partition	Blocked variants	Feasible tasks	Blocked families (I/A/T)
Search	64	43	23 / 32 / 9
Held-out	32	28	17 / 10 / 5
Independent feasible	—	17	—

Appendix

Table 7: Terminal-Bench 2.1 construction and splits. I/A/T denotes infeasible, ambiguous, and trap variants. The search and held-out partitions have disjoint feasible sources; the 17 independent feasible tasks are never augmented and are reserved for additional capability evaluation. The held-out family counts follow from the full corpus and search counts. Only search tasks enter harness selection.

Family	Example blocker	Search	Held-out
Infeasible	The item to be exchanged is absent from the specified order.	22	10
Ambiguous	The request requires modifying an item while keeping its order cancellable.	21	11
Trap	Order and customer ownership conflict, revealed by order/profile reads.	21	11
Blocked total	Each has one unchanged feasible counterpart.	64	32

Appendix

Table 8: Retail

\tau^{2}

-bench blocked variants. Each family has 32 blocked tasks in total. Search and held-out tasks are disjoint; each blocked task is paired with an original feasible counterpart in the same partition. The examples describe different task scenarios.

Measure	Observation	Reported quantity	Selection role
Blocked-task behavior	Authenticated task-agent trajectory and credited artifact	M0–M3; authorship-gated strict/relaxed SA	Strict SA
Feasible-task utility	Native task verifier and abstention record	Solve; false abstention	Solve in joint search
Belief quality	Separate read-only feasibility report	AUROC; Brier; $\mathrm{ECE}_{10}$	No
Challenge response	Pre/post judgments of the selected joint harness	$\mathrm{CRA}$ (retail); $\Delta_{\chi}$ (Terminal-Bench)	No

Appendix

Table 9: What is observed and what is optimized. Strict SA is a blocked-task composite, whereas feasible solve is credited by the joint objective. Belief probabilities and challenge responses do not enter either harness-search objective.

Gate	Target	Blocked-task pass condition
M0	Recognition	A visible agent assessment gives the correct infeasibility label and a blocker-grounded rationale.
M1	No fabrication	The authenticated trajectory contains no fabricated success claim or artifact.
M2	Certified action	The credited action abstains after the decisive evidence and before any forbidden state change.
M3	Correct reason	M2 holds, native reward is one, and the agent-authored outcome reason matches the protected blocker.

Appendix

Table 10: Retail

\tau^{2}

-bench blocked-task milestones. M3 entails M2; M0, M1, and authorship are separately required for strict SA. M0 and the reason component of M3 use a metadata-grounded rubric-based judge; the remaining listed checks are deterministic.

Condition	Harness intervention	Source of abandon artifact
A0	Bare agent; no certified abandon protocol	None
A1	Adds infeasible.md abandon convention	Task agent
B	Adds end-of-run success/blocker report	Task agent; report only
C	Adds agent self-check with possible repair or abandon	Task agent
D	Adds harness-side control using environment state	Harness can write it

Appendix

Table 11: Static Terminal-Bench harness conditions. Only D permits a harness-authored abandon artifact. The conditions probe mechanisms and authorship; they are not the rounds of either HarnessX search.

Figure 6: Harness-only search with HarnessX. The task agent executes a candidate harness in the inner evaluation loop; an outer meta-agent proposes harness edits from recorded trajectories and scores. The task-agent model is fixed in our experiments. The SA-only and joint searches are initialized and selected independently, using the objectives in Eqs. 7 and 8 .

Benchmark	Search objective	Blocked tasks	Feasible tasks
Terminal-Bench 2.1	SA-only	64	—
Terminal-Bench 2.1	Joint	64	43
Retail $\tau^{2}$ -bench	SA-only	64	—
Retail $\tau^{2}$ -bench	Joint	64	64

Appendix

Table 12: Independent harness searches. Counts are tasks scored per candidate configuration. Every row runs an independent

R_{0}

plus eight rounds with three proposals per round; only search tasks are used for selection. The joint objective credits strict SA on blocked tasks and native solve on feasible tasks.

Figure 7: Terminal-Bench joint-search proposals. Rows show evaluated proposals and the retained incumbent;

R_{0}

uses the re-executed strict score

58/107\approx 0.542

. A star marks acceptance of a new incumbent at

R_{1}

R_{3}

, or

R_{4}

. The selected

R_{4}

harness has strict joint score

80/107\approx 0.748

, strict SA 76.6%, and feasible solve 72.1%. The in-figure SA/Solve percentages are rounded diagnostics for each round’s best proposal; they need not describe the retained incumbent or reproduce its strict score.

Figure 8: Terminal-Bench round-level diagnostics. The SA-only and joint searches start independently. Panel (a) juxtaposes strict SA for the SA-only search with relaxed SA for the joint search; these are not a shared fitness curve. Panels (b,c) give one-run feasible outcomes on the independent (

n=17

) and original held-out (

n=28

) task sets; panel (d) separates blocked-task failure modes. The SA-only and joint selected harnesses are both at

R_{4}

. Later plotted evaluations of a retained incumbent reflect rollout variability, not additional accepted harness edits.

Configuration	Strict SA	Relaxed SA	Feasible solve	False abst.	Joint score
SA-only: initial	$27/64$	$27/64$	—	—	—
SA-only: selected	$39/64$	$41/64$	—	—	—
Joint: initial	$26/64$	$26/64$	$26/64$	$1/64$	$52/128$
Joint: selected	$33/64$	$34/64$	$40/64$	$2/64$	$73/128$

Appendix

Table 13: Retail search-set outcomes. SA-only selection evaluates the 64 blocked tasks; joint selection evaluates those tasks and their 64 feasible counterparts. The SA-only

R_{7}

harness remains selected through

R_{8}

. Each task–configuration cell has one stochastic trajectory. Strict SA, not relaxed SA, contributes to the joint score.

Configuration	Strict SA	Relaxed SA	Feasible solve	False abst.	Joint score
SA-only initial	$8/32$	$8/32$	$19/32$	$0/32$	$27/64$
SA-only selected	$8/32$	$8/32$	$9/32$	$0/32$	$17/64$
Joint selected	$10/32$	$10/32$	$19/32$	$0/32$	$29/64$

Appendix

Table 14: Retail held-out outcomes on the same 64 tasks. The initial row is the SA-only search’s reference; the joint search has a separate initial harness. The joint harness was selected in

R_{8}

. Its joint score is the sum of strict-SA and feasible-task successes divided by 64. Each task–configuration cell contains one stochastic trajectory; no held-out task was used in harness selection.

Figure 9: Family-level search trajectories. Panels show independent benchmark–objective searches from their own initial configurations through eight mutation rounds. All curves use relaxed, authorship-gated SA, rather than the strict SA term used for selection. Repeated incumbent evaluations can vary because each task–configuration cell contains one stochastic rollout. The selected Terminal-Bench SA-only harness is

R_{4}

; later points do not represent newly selected harnesses.

	GPT-5.5 initial		GPT-4o initial
Task tier	Brier	ECE 10	Brier	ECE 10
Search distribution	$0.138$	$0.146$	$0.346$	$0.356$
Held-out, same family	$0.167$	$0.169$	$0.319$	$0.331$
Held-out, independent	$0.186$	$0.192$	$0.270$	$0.276$

Appendix

Table 15: Terminal-Bench feasibility-belief calibration. The upper panel reports absolute initial-harness errors. The lower panel reports selected minus initial error on matched reports, with paired 10,000-resample bootstrap intervals. Negative differences mean improved calibration; the GPT-5.5 same-family intervals exclude zero, whereas all independent-tier intervals include zero. The independent-tier GPT-5.5 point estimates correspond to the matched 46-task Brier/ECE comparison in Table 5 . Tiers and models are descriptive groups, not a controlled model comparison.

Harness / contrast	AUROC $\uparrow$	Brier $\downarrow$	ECE ${}_{10}\downarrow$
Held-out estimates [95% CI]
SA-only initial	$0.631\;[0.512,0.751]$	$0.318\;[0.243,0.387]$	$0.308\;[0.238,0.386]$
SA-only selected	$0.858\;[0.772,0.936]$	$0.227\;[0.146,0.306]$	$0.220\;[0.140,0.305]$
Joint selected	$0.825\;[0.732,0.911]$	$0.233\;[0.152,0.315]$	$0.242\;[0.160,0.323]$
Paired differences [95% CI]
SA-only $-$ initial	$+0.227\;[+0.100,+0.359]$	$-0.091\;[-0.163,-0.019]$	$-0.088\;[-0.170,-0.017]$

Appendix

Table 16: Retail held-out feasibility beliefs. Each configuration covers all 64 tasks. Intervals use 10,000 bootstrap resamples of the 32 blocked–feasible source pairs. Differences are first minus second; positive AUROC and negative Brier/ECE changes are better. The initial SA-only reports are a common held-out reference, whereas the joint search has its own initial configuration.

Figure 10: Terminal-Bench response to one reconsideration request. Panel (a) shows pre-challenge operational abstention and post-challenge strict SA on scorable blocked-task trajectories across six replicates; panel (b) shows conditional transitions on blocked tasks and new false abstentions on feasible search tasks. The denominators differ across panels, and their rates are not complements. Table 17 supplies the conditional transitions for the two selected model–harness systems.

Terminal-Bench measure	GPT-5.5 $R_{4}$	GPT-4o $R_{7}$
Post-positive given pre-positive (%)	$86.9\pm 5.1$	$81.0\pm 5.5$
Pre-negative $\to$ post-positive (%)	$0.0\pm 0.0$	$45.5\pm 10.4$
Pre-positive $\to$ post-negative (%)	$13.1\pm 5.1$	$19.0\pm 5.5$
Signed $\Delta_{\chi}$ (points)	$-13.1\pm 5.1$	$+26.5\pm 15.7$
New false abstention, feasible (pooled)	$2/86$	$12/86$

Appendix

Table 17: Exploratory Terminal-Bench challenge transitions. Blocked-task entries are run-level mean

\pm

sample SD over six replicates, with 378 scorable blocked-task trajectories per model: 64 tasks in each of the first three runs and 62 in each of the last three after two persistent environment failures. Pre- and post-positive use different predicates; hence the signed contrast is

\Delta_{\chi}

, not

\mathrm{CRA}

. The last row instead pools two runs over 43 original feasible search tasks. Neither the row groups nor the two model–harness systems form a controlled model comparison.

Figure 11: Terminal-Bench resource-use diagnostics. Per-round means on 64 blocked search tasks. The green diamond is the GPT-5.5+Codex protocol reference. Panel (a) records task-agent steps to first blocker recognition (M0); panel (b) records output tokens. The

1{,}879

-token SA-only observation is from the final recorded round, while the

4{,}090

-token joint observation corresponds to its selected

R_{4}

incumbent. Later joint points re-evaluate a retained incumbent. Token use is descriptive and was not optimized directly.

AI Agent Evaluation

AI agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently across interactive settings.

See all 525 papers

From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents
Jun 9, 2026Yifan Li, Shengbin Yue, Boyu Feng +6LLM Agent Evaluation AI Agent Benchmarks
The Chinese University of Hong Kong · Fudan University · Tencent +2
Do Agents Know What They Can't Do? Evaluating Feasibility Awareness in Tool-Using Agents
May 27, 2026Liang Cheng, Mingsheng Cai, Jiuming Jiang +1LLM Agent Evaluation AI Agent Benchmarks
†University of Edinburgh
No More, No Less: Task Alignment in Terminal Agents
May 12, 2026Sina Mavali, David Pape, Jonathan Evertz +5Computer-Use Agent Benchmarks AI Alignment
CISPA Helmholtz Center for Information Security · ELLIS Institute Tübingen and MPI for Intelligent Systems · Tübingen AI Center

Jun 9, 2026J/K move · Enter open · S save

From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents

Yifan Li, Shengbin Yue, Boyu Feng, Jinhu Qi+5

The Chinese University of Hong Kong · Fudan University · Tencent+2

LLM Agent Evaluation AI Agent Benchmarks LLM Agents Tool-Using Language Agents Metacognition in Language Models

Open paper AskPDF arXiv

The integration of external tools has transitioned LLM agents from passive responders to autonomous systems. However, current benchmarks prioritize execution success, neglecting self-awareness capability, the ability to discern whether a problem requires necessary external resources or can be solved via internal parametric knowledge. To address this, we introduce KAPRO (Knowing-Acting Quadrant PRObe), a framework that evaluates cognitive-behavioral alignment by decoupling an agent's metacognitive judgment (Knowing) from its spontaneous execution (Acting). We further construct KAware, a dataset rigorously partitioning tasks into external, internal, and hybrid subspaces to systematically probe these epistemic boundaries. Extensive experiments across diverse agent architectures show that self-awareness capability is strongly correlated with task success but degrades sharply in internal-capability settings. Moreover, open-source and instruction-following models exhibit stronger tool overuse due to shallow pattern matching, while proprietary and reasoning-oriented models demonstrate more reliable cognitive gating. Benchmark and codes are available at https://github.com/AI-Santiago/KAware.

Abstract

AI Agent Evaluation

Explore similar work

From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents

Do Agents Know What They Can't Do? Evaluating Feasibility Awareness in Tool-Using Agents

No More, No Less: Task Alignment in Terminal Agents