Where a Model Sends Its Own Repeated Token
Organizations: Independent Researcher, Chile
Abstract
Black-box model identification works by scoring a model's response to natural-language prompts. One line of work feeds models a degenerate input -- their own token, repeated -- to find a failure mode rather than an identity. We take that input and ask where the model goes when it does not. For each token t, read argmax p(. | t, t) in one forward pass; the result is a map on the whole vocabulary, with two halves. The first -- which tokens are fixed points -- is partially anticipated, and we report it as a failed estimand: the natural distance on it is 83% cardinality, separates a corpus manipulation by two bits in 3471 against a precision floor of zero, and attributes families at 0.5833. The second half, where the map sends tokens that are not fixed points, is unrecorded; the one paper holding those tokens logged them as a zero. Pairing on the source token removes the cardinality confound by construction (r from 0.9128 to -0.0932) and attributes families at 0.8333 -- twelve models scored against a pool of nineteen -- with chance 0.1389, across seven tokenizer groups and several corpora. Two nulls clear it: frequency-matched destinations agree at 0.1429, independent marginals at 0.0798. Family predicts agreement better than tokenizer (0.2031 against 0.1205), and recurrent architectures cluster at balanced accuracy 1.0 against a 0.7895 majority rate, or 0.90 once each model's dominant destination is excluded -- the figure we stand behind. We measure the robustness envelope: 8-bit weight rounding moves the map less than deduplicating the training corpus does (0.9004 against 0.6353, on one support), 4-bit destroys it (0.0098; 0.1812 at deployment granularity, so not a coarseness artefact), and the precision floor varies by model from 0.201 to 0.9778. All estimands and kill conditions were registered before the data, and the failed one is reported as fully as the surviving one.
Figures & tables
| comparison | Hamming | robust at |
|---|---|---|
| pythia-410m vs -deduped (a corpus manipulation) | ||
| vs gpt-neo-125m | ||
| vs rwkv-4-430m | ||
| vs mamba-370m | ||
| the same weights at two numeric precisions |
| comparison | agreement | disagreement vs the floor |
|---|---|---|
| the same weights at two numeric precisions | — | |
| pythia-410m vs -deduped | ||
| vs gpt-neo-125m | ||
| vs mamba-370m | ||
| vs rwkv-4-430m |
| question | statistic | value | baseline |
|---|---|---|---|
| family attribution | rank-1, leave-one-out | chance | |
| architecture class | balanced accuracy | majority | |
| family vs. tokenizer | lift in mean agreement | tokenizer |
Language Model Fingerprinting
Language model fingerprinting identifies and verifies models or their lineage from distinctive outputs, internal representations, or embedded signals.
See all 77 papers