Where It Breaks: Diagnosing Perception, Memory, and Decision Failures of a Vision-Language Agent in Pac-Man
Abstract
Training an agent to play Pac-Man with only up, down, left and right actions raises a basic question: when performance is poor, what should be improved? Difficulty can arise from reading the screen or from choosing actions over a long sequence. We examine these problems in a Qwen3.5-9B-based agent through separate perception checks and primitive-policy reinforcement-learning studies. In the policy study, which uses textual boards to remove image transcription from the decision interface, completion on eight held-out mazes increased from 0% before this optimization to 80.2% at a selected checkpoint, with three extra lives. The perception checks show that accurate static screenshot reading can coexist with errors during interaction. A separate policy comparison also shows that further optimization can accompany worse action choices. Together, these findings motivate concrete checks for post-training: evaluate perception in use, distinguish training success from full-game validation, and examine decisions and later performance rather than relying on one selected score. The study provides empirical lessons for improving future agent training and evaluation.
Figures & tables
| Stage | At 249 | At 425 | Change |
|---|---|---|---|
| T1 | 95 | 96 | +1 |
| T2 | 96 | 96 | 0 |
| T3 | 80 | 76 | -4 |
| T4 | 91 | 89 | -2 |
| T5 | 83 | 91 | +8 |
| T6 | 79 | 85 | +6 |
| Ordinary pellets remaining | Restart states | Attempts |
|---|---|---|
| 1 | 4 | 48 |
| 10–15 | 4 | 48 |
| 58–60 | 3 | 36 |
| Query panel | Valid | Exact |
|---|---|---|
| Raw greedy, 24 images | 24/24 | 24/24 |
| Raw sampled, three per image | 72/72 | 69/72 |
| Initial-image greedy repeats | 10/10 | 10/10 |
| Periodic full-map queries | 20/20 | 18/20 |
| Actor queries | 138/138 | 131/138 |
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
| Input | Agent | Clears | Win rate | Mean score |
|---|---|---|---|---|
| Image | Primitive | 0/50 | 0% | 490.2 |
| Image | Route-option harness | 10/50 | 20% | 5104.5 |
| ASCII | Primitive | 0/50 | 0% | 242.6 |
| ASCII | Route-option harness | 14/50 | 28% | 5571.0 |
| Model / interface | Clears | Score | Game steps |
|---|---|---|---|
| GPT-6.1 Sol / Codex CLI | 1/1 | 11,920 | 386 |
| Setting | Vision stage | Language stage |
|---|---|---|
| Trainable part | Encoder and merger | Language LoRA only |
| Updates / batch | 6,000 / 16 | 1,000 / 16 |
| Peak learning rate | ||
| LoRA configuration | Not used | Rank 16, alpha 32 |
| LoRA dropout / bias | Not used | 0.05 / none |
| Optimizer | AdamW | AdamW |
| Extra lives | Updates | Clears | Deaths | Agent | Reads |
|---|---|---|---|---|---|
| 0 | 3 | 18/64 | 43 | 6,698 | 16,139 |
| 0 | 20 | 9/64 | 49 | 6,309 | 15,564 |
| 3 | 3 | 55/64 | 101 | 10,941 | 25,628 |
| 3 | 20 | 50/64 | 109 | 10,796 | 25,993 |
| Model | Arm | 0 extra lives | 3 extra lives |
|---|---|---|---|
| Qwen | B0S0 | 2/64 (44) | 17/64 (114) |
| Qwen | B0S1 | 11/64 (35) | 37/64 (49) |
| Qwen | B1S0 | 3/64 (50) | 27/64 (113) |
| Qwen | B1S1 | 9/64 (34) | 37/64 (58) |
| D5A | B0S0 | 7/64 (46) | 40/64 (92) |
| D5A | B0S1 | 18/64 (35) | 59/64 (55) |
| Condition | Clears | Deaths | Refusals | Limit |
|---|---|---|---|---|
| Direct base | 0 | 12 | 0 | 0 |
| Base + harness | 4 | 0 | 7 | 1 |
| RL update 19 + harness | 3 | 1 | 8 | 0 |
| RL update 48 + harness | 3 | 0 | 9 | 0 |
| RL update 49 + harness | 2 | 0 | 8 | 2 |
| Condition | Win rate |
|---|---|
| Base model, primitive moves (U/D/L/R) | 0% |
| Base model selects harness options | 22.92% |
| RL model selects harness options | 55.17% |
| Extra lives | Observed change | Conservative 95% bound |
|---|---|---|
| 0 | ||
| 3 |
| Checkpoint | Clears | Game over | Time limit | Decisions |
|---|---|---|---|---|
| 270 | 0/96 | 92 | 4 | 25,066 |
| 310 | 0/96 | 84 | 12 | 28,994 |
| Update | T7 clears | Update | T7 clears |
|---|---|---|---|
| 330 (restored) | 54/96 | 380 | 29/96 |
| 335 | 78/96 | 385 | 67/96 |
| 340 | 76/96 | 390 | 68/96 |
| 345 | 57/96 | 395 | 73/96 |
| 350 | 59/96 | 400 | 69/96 |
| 355 | 39/96 | 405 | 51/96 |
| Extra lives | Seed | Steps | Deaths | Pellets left | Agent calls |
|---|---|---|---|---|---|
| 0 | 986000 | 83 | 1 | 160 | 73 |
| 0 | 986001 | 30 | 1 | 192 | 30 |
| 3 | 986000 | 180 | 4 | 130 | 157 |
| 3 | 986001 | 138 | 4 | 144 | 123 |
| Method | Eligible | Correct/ known | No heading | Un- resolved |
|---|---|---|---|---|
| Current image | 40 | 1/8 | 32 | 8 |
| Two images | 37 | 2/21 | 16 | 11 |
| Position difference | 40 | 23/27 | 13 | 8 |
| States | Baseline | Direction | Pixel | Both |
|---|---|---|---|---|
| All recorded (431) | 292 | 92 | 292 | 93 |
| Strict matched (321) | 190 | 1 | 190 | 2 |
| Recorded statistic | CP4–13 | CP14–15 |
|---|---|---|
| Clears / planned attempts | 57/2,304 | 1/2,964 |
| Technical faults (separate) | 155 | 168 |
| Time limits / valid games | 76/2,149 | 352/2,796 |
| Mean valid-game steps | 248.5 | 301.5 |
| Mean ordinary pellets left | 43.8 | 46.4 |
| Field group | Consumer and protocol permission | Source confirmation and remaining gap |
|---|---|---|
| RGB frame | Visual reader; current engine-reference policy also receives the screenshot. ASCII policy uses text instead. | Current engine messages and ASCII text-only adapter inspected; screenshot-public decision serialization not recovered. |
| Raw info /snapshot | Environment/runner data include absolute positions, ghost/timer fields and saved state. Availability is not permission to pass them to every agent. | Wrapper confirms these fields; raw source events are not already public receipts. |
| Public receipt | Later typed-receipt screenshot-public harness may consume frame/elapsed ticks, realized relative movement, localized pickups, ghost-eaten, death/respawn and termination events. | Paper contract and logs support these categories; exact projector keys/types, portal/respawn projection and model-facing subset remain unlocated. |
| Maintained state | Public terrain, pellet sets and actor estimates support candidates and permitted summaries; unknown values stay unknown. | Contract and bounded audits exist; the complete frozen public updater and final serializer remain unlocated. |
| Engine-derived summaries | Current engine planner reads level/snapshot information. Its model gets selected position, ghost state/id, timer, navigation-history and candidate summaries, not all memory. | Current prompts confirm selected fields. The latest planner also uses pixel/motion fields and excludes stored ghost paths/targets; this does not establish every historical planner’s behavior. |
| Private audit/restore data | Hidden maps, engine coordinates, ghost internals and RNG may support restore or diagnosis. Screenshot-public forbids their use to repair policy/harness inputs. | Panel-specific information contracts govern access; current recipe is not a complete mapping to frozen factorial/public/head panels. |
| Stage | Evidence in this study | Next intervention to validate |
|---|---|---|
| Perception | Transcription errors and separate static/closed-loop query audits | Align prompt and format; assess visual adaptation under its declared protocol |
| State | A traced actor-to-terrain write; bounded revised-memory checks | Enforce typed writes and explicit unknowns; compare full episodes |
| Decision | ASCII decision failures; no gain on the paired action-head panel | Retain matched risk-check comparisons; test corrective supervision separately |
| Independently loaded condition | Clears / attempts |
|---|---|
| Base, primitive moves | 0/12 |
| Base + harness | 4/12 |
| RL update 19 + harness | 3/12 |
| RL update 48 + harness | 3/12 |
| RL update 49 + harness | 2/12 |
| Execution | Route gate | 0 extra | 3 extra |
|---|---|---|---|
| Target replan | historical | 7/64 | 40/64 |
| Target replan | improved | 18/64 | 59/64 |
| Bound route | historical | 9/64 | 39/64 |
| Bound route | improved | 27/64 | 62/64 |
| Extra lives | Head3 clears | Head20 clears | Deaths (3/20) |
|---|---|---|---|
| 0 | 18/64 | 9/64 | 43/49 |
| 3 | 55/64 | 50/64 | 101/109 |
| Recorded states | Original | Heading | Pixel | Both |
|---|---|---|---|---|
| All 431 | 292 | 92 | 292 | 93 |
| Strict 321 | 190 | 1 | 190 | 2 |
Game-Playing Agents
Game-playing agents use search, learning, and reasoning to choose actions in games with strategic, uncertain, or changing environments.
See all 215 papers