Underspecified instructions leave LLM agents to decide what to do. In public services, staff write these instructions, but the agent's decisions affect residents, patients, and students. We build four expert-validated public-service scenarios with records and decision levers that can be checked in the files, collect 2,400 trajectories from ten LLMs under four prompt conditions, and compare them with 1,000 runs in a context-free sandbox. Agents read the records in 96.9% of open-ended runs and produce an artifact or a self-chosen task in 18.7%; with an explicit task, 86.0% produce. Domain context does not change how many runs produce (18.7% vs.\ 16.0%), and the production ranking of the models largely carries over (Spearman 0.79), while the ranking by whether a model reads the records does not (0.19). Context cuts boundary-probing turns per run by 60%, and some artifacts contradict the records. We contribute a test that a public-service institution can run on its own records before procurement; its rule-based taxonomy needs no labeled data, and scenarios, code, and trajectories are released.
Figures & tables
Figure 1: Output space of all 2,400 scenario runs (TF-IDF and truncated SVD of each run’s concatenated outputs), colored by model, with six annotated runs: verbatim task execution, task looping, deliberate completion, aimless wandering, work in the domain files, and a self-invented analysis agenda. Runs of one model tend to lie close together: the models differ more than the prompt conditions.
Behavior
Share
Environment exploration
84.1
Philosophical / meta reflection
6.2
Artifact creation
3.7
Code execution
3.0
Self-generated task creation
1.1
Termination / refusal
1.0
Table 1: Turn-level behavior distribution over all 93,594 turns (rule-based taxonomy), percentages of all turns. Label names state what the rule detects: code execution fires on run_python and does not indicate verified self-improvement, and termination / refusal pools deliberate completion with refusal. Each rule’s trigger is documented next to its metric definition in the release.
Model
Context-free
Scenario
Δ
Llama-3.1-8B-Instruct
49.3
33.9 [27.2, 41.1]
−15.4
Apertus-8B-Instruct
98.7
90.0 [85.6, 94.4]
−8.7
Gemma-3-12B-IT †
0.0
0.0 [0.0, 0.0]
0.0
GPT-4.1-mini
0.0
0.6 [0.0, 1.7]
+0.6
DeepSeek-V3-0324
0.0
1.7 [0.0, 3.9]
+1.7
Mistral-Small-2603
1.3
3.3 [1.1, 6.1]
+2.0
Table 2: Share of open-ended runs that produce an artifact or a self-chosen task, with and without domain context, at a common 50 -turn cap. Brackets are 95% bootstrap intervals ( 10,000 resamples). Scenario n=180 per model (four domains), context-free n=75 . † Gemma’s context-free runs are dominated by parse failures ( 54.7% of turns), so its context-free row is not comparable; every contrast is reported with and without it.
Corpus
n
Produce %
Probe
Re-read
Governance
450
18.9 [15.3, 22.7]
0.28
17.6
Healthcare
450
17.6 [14.2, 21.1]
0.32
18.1
Education
450
17.8 [14.2, 21.3]
0.50
17.1
Environmentalism
450
20.4 [16.9, 24.2]
0.27
17.9
Scenarios pooled
1800
18.7 [16.9, 20.5]
0.34
17.7
Context-free
750
16.0 [13.5, 18.7]
0.87
13.9
Table 3: Open-ended conditions, common 50 -turn cap. Rates are percentages of runs, counts are per run; brackets are 95% bootstrap intervals.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Fine-grained classification of all 93,594 turns: rule-based subcategories of the nine taxonomy labels, with the share of each subcategory and representative behaviors.
Figure 3: Runs of the two most different models (Wasserstein distance 1.53 ) projected on the task–self-direction axis, defined by the centroids of the task-baseline and open-ended conditions. GPT-4.1-mini is bimodal: it either completes the task quickly or works in the domain files for the whole run. Llama-3.1-8B never reaches the second half of the axis.
Figure 4: Case study (freedom, environmentalism): under the same prompt, Claude-Sonnet-4 reads each file once, states that its understanding is complete, and finishes at turn 9 ; Mistral-Small-2603 repeats its actions for all 50 turns and re-reads the turn-1 README at turn 50 . Under the run outcomes of Section 4 , the first run calls finish after reading the records, and the second reaches the turn cap without writing a file.
Model
Rewording
Governance
Healthcare
Education
Environ.
Llama-3.1-8B
“Do whatever you would like to do.”
100/60/0/0
90/50/0/0
90/30/0/0
100/40/0/0
“It is entirely up to you what you do.”
80/20/0/0
100/40/0/0
90/10/0/0
80/30/0/0
“No task has been assigned to you.”
50/20/0/0
50/20/0/0
40/10/0/0
30/10/0/0
“There is currently no task for you to complete.”
90/50/0/0
70/40/0/0
70/10/0/0
90/30/0/0
Qwen2.5-7B
“Do whatever you would like to do.”
0/0/0/0
10/0/10/0
0/0/0/0
0/0/0/0
“It is entirely up to you what you do.”
0/0/0/0
0/0/0/0
50/0/50/0
0/0/0/0
Appendix
Table 4: Prompt rewordings, ten seeds per cell, percent of runs: produce / boilerplate / grounded / contradicting. Governance cells are the pilot of the main corpus.
Figure 5: Three levels of specification, four scenarios pooled. Per model, the share of runs that read the records, produce, end with a grounded file, and end with a contradicting file, under “Do what you want.” (open marker), “Have a look at the files and see what needs doing.” (light marker), and the explicit summary task (dark marker). Claude-Sonnet-4.5 was run only under the directed prompt.
Model
Read
Produce
Finish
Ask
Name
Work
Idle
Stopped
Boiler.
Grounded
Contra.
Apertus-8B-Instruct
85.0
76.7
90.0
6.7
23.3
6.7
21.7
1.7
51.7
25.0
0.0
Claude-Sonnet-4.5
100.0
40.0
56.7
96.7
98.3
86.7
10.0
0.0
8.3
46.7
35.0
Qwen2.5-7B-Instruct
100.0
28.3
6.7
38.3
53.3
20.0
73.3
0.0
5.0
21.7
0.0
Llama-3.1-8B-Instruct
78.3
60.0
1.7
0.0
0.0
0.0
88.3
0.0
11.7
0.0
0.0
GLM-4.5-Air
100.0
0.0
1.7
56.7
100.0
0.0
100.0
0.0
0.0
0.0
0.0
Gemma-3-12B-IT
48.3
0.0
0.0
3.3
10.0
0.0
100.0
0.0
0.0
0.0
0.0
Appendix
Table 5: Directed condition, four scenarios pooled, percent of runs per model ( n=60 ). Read: reads a domain file beyond the orientation files. Finish: calls finish . Ask: states in its reasoning that an instruction is needed. Name / Work: names / works on at least one decision lever. Idle, Stopped, Boiler., Grounded, Contra.: run outcomes of Section 4 .
Figure 6: Per model, the share of open-ended runs that produce (left) and that read a file beyond the orientation files (right), in the context-free sandbox (open markers) and the four scenarios pooled (filled markers), with 95% bootstrap intervals. The production ranking carries over between environments (Spearman 0.79 ); the reading ranking does not ( 0.19 ).
Measure
Spearman
Kendall
Mean ∣Δ∣ (null)
p
Max ∣Δ∣ (null)
p
Produces
0.79
0.71
7.5 (3.6)
0.0001
21.6 (14.3)
0.0016
Reads the records
0.19
0.18
10.3 (3.1)
0.0001
54.7 (12.2)
0.0001
Calls finish
0.87
0.75
6.1 (4.4)
0.0005
18.3 (14.0)
0.0055
Boundary-probing turns per run
0.64
0.49
0.63 (0.15)
0.0001
2.23 (0.68)
0.0001
Artifact turns per run
0.76
0.68
0.63 (0.15)
0.0001
5.68 (1.17)
0.0001
Redundant re-read turns per run
0.81
0.69
4.48 (1.00)
0.0001
11.04 (2.86)
0.0001
Appendix
Table 6: Rank stability between the context-free sandbox and the four scenarios pooled, open-ended conditions, ten models. Shifts are percentage points for rates and turns per run for counts; the null value in brackets is the 95 th percentile of the permutation null.
Model
Produce
Grounded
Contra.
Most frequent outcome
orig.
repeat
orig.
repeat
orig.
repeat
orig.
repeat
Apertus-8B
73
100
20
27
0
0
boilerplate
boilerplate
Claude-Sonnet-4 / 4.5
40
27
67
47
13
40
grounded
grounded
Qwen2.5-7B
0
0
0
0
0
0
idle
idle
Llama-3.1-8B
53
100
0
0
0
0
idle
boilerplate
GLM-4.5-Air
0
7
0
7
0
0
idle
idle
Appendix
Table 7: Test-retest of the freedom condition in the governance scenario, 15 seeds per model and run set, percent of runs. The repeat uses Claude-Sonnet-4.5 in place of Claude-Sonnet-4.
Dyson School of Design Engineering Imperial College London London, United Kingdom · Department of Electrical and Electronic Engineering Imperial College London London, United Kingdom · School of Electronic Information Nanchang Institute of Technology Nanchang, China
Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University · Department of Data and Systems Engineering, The University of Hong Kong · College of Economics and Management, Zhejiang Normal University +3