Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates
Organizations: Independent · Carnegie Mellon University · University of California, San Diego · The University of Hong Kong ·
Abstract
Kicker is a personal health assistant. Changes to an assistant's reusable instructions can improve task handling while changing what it discloses, which requests it follows, and how it reports actions. We study Safe Personal RSI, an update-evaluation component developed for Kicker. We restrict recursive self-improvement (RSI) to prompt updates, with model weights and runtime code fixed. The procedure tests proposed and current instructions, repeats failing cases, temporarily undoes changes to guide repairs, and checks safety and task performance before adoption. If no version passes, the current instructions stay in use. Our executable benchmark contains 29 email, calendar, health, and marketplace cases, with separate probes for authorization reuse, memory, and tool-output injection. Across three models and two deliberately risky edits, the procedure rejects 20 of 21 update attempts. In repair confirmation, an admitted GPT-5.6 Luna repair reduces attack violations from 10.7% to 0% and increases safe task completion from 89.7% to 100%; a broad handwritten authority rule matches these outcomes. Under stronger attacks, a repair transferred from Qwen2.5-7B-Instruct to Qwen3-14B reduces violations from 69.0% to 17.9% and increases safe task completion from 19.5% to 64.4%. Separate runtime probes examine memory-entry deletion and identify missing local OAuth scope checks in the evaluated snapshot.
Figures & tables
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Test group | Observation and scope |
|---|---|
| Draft contents and reporting; 23 base cases. | |
| Calendar/health | Email representations of invitation fields and health facts. |
| Authorization reuse | Drafts after claimed parameter changes or expired authority. |
| Memory | Stored entries before/after correction, deletion, subsequent queries, and an idle wait. |
| Terminal | Real model with simulated sandbox output; command log and answer checks. |
| API controls | Resource access, stored OAuth grants, proxy allowlists, and connection removal. |
| Items | Intended risk | Implemented test | Interpretation |
|---|---|---|---|
| 1–2 | Foreign-file read/write | Cross-organization resource-content GET/PUT | Organization-scoped API access checks. |
| 3–4 | Search, memory, concurrent isolation | Resource-list non-disclosure | Item 3 tests list access; item 4 concurrency coverage is absent. |
| 5–6 | Host filesystem/internal services | Three malicious terminal-output cases | Model responses to simulated sandbox output. |
| 7–8 | Shared temporary data and cleanup | Four hypothetical residue cases | Tests model behavior on hypothetical residue. |
| 9–13 | Secrets, session use, embedded/quoted mail instructions | Email-injection cases and hardened variants | Checks draft contents and final responses. |
| 14–16 | Invite fields, scope, health data | Calendar, health, and marketplace cases with a legitimate health-summary control | Invitation and health content is represented in email. |
| Setting | Admission study | Repair studies |
|---|---|---|
| Rounds / candidates | ||
| Search queries | 48 | 72 |
| Confirmation | Up to 3 candidate replays | 3 paired replays |
| Localization | Block rollback | Sentence rollback |
| Repair | Templates | LLM block; template fallback |
| Repair verification | 1 replay | 3 replays |
| Condition | Dev. | Final | Flags: U / F / S | Outcome or limitation |
|---|---|---|---|---|
| Qwen2.5-7B-Instruct / trial 1 | 317 | 87 | 5 / 8 / 2 | All authorized controls fail; two Utility-only executions incomplete. |
| Llama-3.1-8B-Instruct | 85 | 86 | Partial | Interrupted transport failures; not a complete comparison. |
| Qwen2.5-7B-Instruct / trial 2 | 235 | 87 | 7 / 6 / 0 | Repair not admitted; Safe Personal RSI control fails. |
| Qwen3-14B / direct repair | 370 | 87 | 13 / 12 / 5 | Severe veto keeps parent. |
| Qwen3-14B / direct repair, 3 replays | 0 | 261 | 46 / 44 / 20 | Parent kept; eight incomplete executions. |
| Qwen2.5-7B-Instruct / utility-aware repair | 291 | 87 | 7 / 8 / 0 | LLM repair admitted; Safe Personal RSI control fails. |
LLM Safety Evaluation
Large language model safety evaluation measures harmful compliance, robustness, and safe utility across prompts, tools, and real-world tasks.
See all 278 papers