Clean Tickets, Wrong Orders: Catching Silent Errors with a Language Model in Your Browser Tab
Organizations: ITMO University, Saint Petersburg, Russia
Abstract
A small language model that turns a typed order into structure can fail in two ways. It can print something broken, which any system rejects. Or it can print a clean, well formed order that nobody placed. Our companion work calls the second kind a silent error, and it is the dangerous one. We present a web demo in which a fine-tuned 0.8B parameter model runs entirely inside the visitor's browser tab, through llama.cpp compiled to WebAssembly. No order text leaves the device. The page parses orders as the visitor types, scores every ticket with the PIZZA benchmark's own matcher ported to TypeScript, refuses anything off the menu with "?" so the model can face the open web, and lets visitors hunt for silent errors or try to cause one. Run in the browser on all 1,357 PIZZA test orders, the deployed model writes the same output as the server evaluation for 99.0% of them and reaches the same exact match (0.882); we also report its latency on 3 consumer laptops. Demo: https://wrong.cuyor.com
Figures & tables
get me two pepsis a coke and five large fantas'', with a coke'' emphasised. An arrow leads to a dark circle labelled 0.8B LLM, then to a receipt reading 5 x large fanta, 2 x pepsi''. Three rows judge it: Parsed (check mark), Plausible (check mark), and Wrong (cross, highlighted), noting there is no coke. Below, a second input, two large horses'', goes through the same model and comes out as a question mark, refused so the system asks again. Outside the tab, a blocked dashed line leads to a grey server cloud, above the text ``0 bytes of the order leave the tab''.browser tab, on the visitor's device'': the page sends the order to a Web Worker running llama.cpp in WebAssembly and gets the ticket back; the worker reads the model from an OPFS cache keyed by its SHA-256; the page passes tickets to a scorer (decoder and matcher); a service worker supplies the headers that allow threads; a bench page uses the same engine. Outside the box, Hugging Face sends the model file once, and a dashed line to any server'' is blocked, marked ``0 bytes of order text''.get me a small pizza with red pep'' and a ticket shows 1 x small pizza, red peppers''. Panel 2, spot it: the order one large cheese and a small pepperoni pizza'' and a ticket 1 x large pizza'' with chips cheese, small, pepperoni, above buttons send it back'' and serve it'' and ten progress dots. Panel 3, break it: an input box, a ticket drawn as grey bars, and buttons right'' and wrong''. Panel 4, the gate: two large horses'', hello'' and asdf'' bounce off a bar labelled model with question marks, while get me a small pizza with red peppers'' passes through to a ticket.| Setting | Exact | Silent | Refused | No parse |
|---|---|---|---|---|
| Cited, 5 seeds ( Ahmed et al., 2026 ) | 0.875 | 0.045 | 0.034 | 0.046 |
| Deployed, browser | 0.882 | 0.036 | 0.020 | 0.062 |
| Browser output identical to the server’s, order by order: 99.0% | ||||
| Device, browser | Median | p90 | 0.1 s | 1 s | Load |
|---|---|---|---|---|---|
| M3 Air, 16 GB, Chrome 157 | 383 ms | 517 ms | 0% | 100% | 42 s |
| M1 MacBook, 8 GB, Chrome 156 | 600 ms | 830 ms | 0% | 98% | 40 s |
| Dell i7-1165G7, 16 GB, Edge 156 | 2.99 s | 4.56 s | 0% | 0% | 45 s |
Language Model Error Detection
Language model error detection identifies unreliable answers, reasoning steps, and actions so systems can flag, verify, or repair them.
See all 113 papers