System 1 agents: a method for local, reliable and fast agentic AI
Organizations: Artificial Intelligence and Translational Imaging (ATI) Lab, Department of Radiology, School of Medicine, University of Crete, Heraklion, Crete, Greece. · Division of Radiology, Department of Clinical Science Intervention and Technology (CLINTEC), Karolinska Institute, Huddinge, Sweden. · Department of Medical Imaging, University Hospital of Heraklion, Crete, Greece.
Abstract
Agentic artificial intelligence could automate analyses of sensitive clinical data, but current agents depend on large language models (LLMs). Cloud LLMs expose patient data. Local LLMs hallucinate, and an invented value in a tool call propagates through every later step. Here we present a method to build System 1 agents: local agents that call tools without generating text. A deterministic compiler turns each tool's schema and the request into multiple-choice questions whose options come from the request and the schema, then a one-pass decision model answers them. On 1,952 requests to 157 biomedical tools, the best System 1 agent chose the right tool, or correctly declined, for 97.0% of requests, significantly ahead of all 18 local LLM-agent settings (best 95.2%). The System 1 agent filled 97.0% of 6,056 stated argument values. No System 1 call contained invented content, against 5.5-19.7% of LLM calls. The System 1 Agents framework is freely available as open-source software at https://github.com/eltzanis/SystemOneAgents.
Figures & tables
| category | measure | System 1 (six agents) | LLM (18 agents) | pairs | pplx-decider-27b | OpenJev | Qwen3.5-27B, reasoning off |
|---|---|---|---|---|---|---|---|
| Accuracy | Fully correct requests (% of 1,952) | 81.4–93.9 | 61.4–91.3 | 73 / 11 / 24 | 93.9 | 92.9 | 91.3 |
| Accuracy | Tool calling (% of 1,952) | 91.7–97.0 | 66.3–95.2 | 76 / 5 / 27 | 97.0 | 96.1 | 95.1 |
| Accuracy | Argument filling (% of 6,056 values) | 89.8–97.0 | 56.6–97.2 | 46 / 19 / 43 | 97.0 | 96.1 | 97.2 |
| Reliability | Calls with invented content (% of calls) a | 0.0 | 5.5–19.7 | — | 0.0 | 0.0 | 7.4 |
| Reliability | Calls with argument names absent from the schema (n) | 0 | 0–53 | — | 0 | 0 | 0 |
| Reliability | Underspecified requests handled correctly (% of 174) | 86.2–93.7 | 0.0–5.7 | — | 93.7 | 92.0 | 0.0 |
| model | source | base model | file | protocol | context |
|---|---|---|---|---|---|
| decider-4b | Mapika, Mapika/decider-4b-GGUF | Qwen3.5-4B | Q8_0, 4.5 GB | decider | 16,384 |
| pplx-decider-flash v1.2 | Perplexity, perplexity-ai/pplx-decider-flash-gguf-v1.2 | Qwen3.5-4B | Q8_0, 5.2 GB | pplx-decider | 5,632 |
| decider-35b-a3b | Mapika, Mapika/decider-35b-a3b (GGUF by mradermacher) | Qwen3.5-35B-A3B (3B active) | Q4_K_M, 21.2 GB | decider | 16,384 |
| OpenJev | openjev/openjev (GGUF by ggml-org) | 27B, Qwen3.5 architecture | Q4_K_M, 19.4 GB | native SystemOne | 8,192 |
| Clef | Cloudflare (GGUF by ggml-org) | Qwen3.5-27B with a joint schema head | Q8_0, 28.7 GB | native SystemOne | 8,192 |
| pplx-decider-27b v1.2 | Perplexity, perplexity-ai/pplx-decider-v1.2 | Qwen3.5-27B | Q8_0, 28.6 GB, converted with llama.cpp | pplx-decider | 5,632 |
| model | parameters | file | role |
|---|---|---|---|
| NVIDIA Nemotron 3.5 Bolt E4B | 8B (4B effective) | Unsloth UD-Q8_K_XL | chosen |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | 74B, 9B active | NVIDIA 24 GB release | chosen |
| Qwen3.8-27B | 27B | ggml-org Q8_0 | chosen |
| Meta Muse Glimmer 30B | 30B | Unsloth Q8_0 | chosen |
| Google Gemma 4 E4B (instruction-tuned) | 8B (4B effective) | ggml-org Q8_0 | chosen |
| Google Gemma 4 26B-A4B (instruction-tuned) | 26B, 4B active | ggml-org Q8_0 | chosen |
| agent | kind | tool calling, of 1,952 | right tool, of 1,645 | declined, of 307 | argument values, of 6,056 | fully correct, of 1,952 |
|---|---|---|---|---|---|---|
| decider-4b | System 1 | 1,804 (92.4; 91.2–93.5) | 1,537 | 267 | 5,438 (89.8) | 1,589 (81.4; 79.6–83.1) |
| pplx-decider-flash | System 1 | 1,857 (95.1; 94.1–96.0) | 1,588 | 269 | 5,727 (94.6) | 1,752 (89.8; 88.3–91.0) |
| decider-35b-a3b | System 1 | 1,790 (91.7; 90.4–92.8) | 1,519 | 271 | 5,478 (90.5) | 1,692 (86.7; 85.1–88.1) |
| OpenJev | System 1 | 1,876 (96.1; 95.2–96.9) | 1,596 | 280 | 5,817 (96.1) | 1,813 (92.9; 91.7–93.9) |
| Clef | System 1 | 1,857 (95.1; 94.1–96.0) | 1,567 | 290 | 5,688 (93.9) | 1,778 (91.1; 89.7–92.3) |
| pplx-decider-27b | System 1 | 1,894 (97.0; 96.2–97.7) | 1,614 | 280 | 5,876 (97.0) | 1,832 (93.9; 92.7–94.8) |
| System 1 model | base model as LLM agent | tool calling | argument filling | fully correct |
|---|---|---|---|---|
| decider-4b | Qwen3.5-4B, reasoning on | +0.3 (0.72) | −2.8 (<0.001) | −1.9 (0.08) |
| decider-4b | Qwen3.5-4B, reasoning off | +2.4 (0.003) | −0.6 (0.47) | +1.3 (0.27) |
| pplx-decider-flash | Qwen3.5-4B, reasoning on | +3.0 (<0.001) | +1.9 (0.0064) | +6.4 (<0.001) |
| pplx-decider-flash | Qwen3.5-4B, reasoning off | +5.1 (<0.001) | +4.2 (<0.001) | +9.6 (<0.001) |
| Clef | Qwen3.5-27B, reasoning on | +2.3 (0.0017) | −1.8 (0.011) | +2.2 (0.019) |
| Clef | Qwen3.5-27B, reasoning off | 0.0 (1.0) | −3.3 (<0.001) | −0.2 (0.86) |
| subcategory | requests | values | tool calling | argument filling |
|---|---|---|---|---|
| source: generated requests | 1,829 | 5,467 | 94.3 / 86.5 / 89.8 | 93.1 / 86.1 / 90.8 |
| source: mAIstro notebooks | 50 | 214 | 100.0 / 96.4 / 97.1 | 98.2 / 92.5 / 92.6 |
| source: ReclAIm notebooks | 73 | 375 | 98.9 / 93.3 / 98.8 | 98.8 / 91.7 / 96.9 |
| tools: AutoML (PyCaret) | 112 | 374 | 99.9 / 86.8 / 98.0 | 98.5 / 84.6 / 94.3 |
| tools: chemistry (ChEMBL) | 32 | 61 | 93.8 / 95.1 / 97.2 | 89.6 / 93.6 / 91.6 |
| tools: clinical trials | 41 | 114 | 89.8 / 98.6 / 96.5 | 82.9 / 95.4 / 88.2 |
| agent | s per request (mean) | GPU energy per request (J) | generated tokens per request | GPU energy per fully correct request (Wh) | peak GPU memory (MiB) |
|---|---|---|---|---|---|
| decider-4b | 1.17 | 210 | 13 | 0.072 | 7,368 |
| pplx-decider-flash | 4.59 | 441 | 0 | 0.137 | 12,294 |
| decider-35b-a3b | 2.24 | 413 | 15 | 0.133 | 23,490 |
| OpenJev | 2.56 | 758 | 0 | 0.228 | 21,852 |
| Clef | 3.12 | 927 | 0 | 0.284 | 31,107 |
| pplx-decider-27b | 10.14 | 1,367 | 0 | 0.407 | 35,057 |
| agent | calls | nonexistent argument (calls) | calls with an invented value, adjusted (%) | underspecified requests: required value filled with no source / with a value written elsewhere | schema-invalid calls |
|---|---|---|---|---|---|
| decider-4b | 1,807 | 0 | 0 | 0 / 5 | 7 |
| pplx-decider-flash | 1,832 | 0 | 0 | 0 / 6 | 5 |
| decider-35b-a3b | 1,796 | 0 | 0 | 0 / 2 | 3 |
| OpenJev | 1,814 | 0 | 0 | 0 / 3 | 1 |
| Clef | 1,811 | 0 | 0 | 0 / 3 | 6 |
| pplx-decider-27b | 1,828 | 0 | 0 | 0 / 3 | 0 |
| analysis | fully correct | tool calling | argument filling |
|---|---|---|---|
| main, all pairs | 73 / 11 / 24 | 76 / 5 / 27 | 46 / 19 / 43 |
| main, reasoning on | 37 / 6 / 11 | 41 / 2 / 11 | 23 / 8 / 23 |
| main, reasoning off | 36 / 5 / 13 | 35 / 3 / 16 | 23 / 11 / 20 |
| each LLM in its better mode (54 pairs) | 33 / 8 / 13 | 33 / 4 / 17 | 19 / 12 / 23 |
| scoring variant: search text compared as written | 73 / 11 / 24 | 76 / 5 / 27 | 46 / 19 / 43 |
| scoring variant: AutoML model names through PyCaret’s name table | 73 / 11 / 24 | 76 / 5 / 27 | 45 / 19 / 44 |
LLM Agent Reliability
Large language model (LLM) agent reliability concerns dependable, safe, and verifiable behavior across tools, multi-step tasks, and changing environments.
See all 899 papers