Independent Verification Paths Are Not Independent: A Case Study of Common-Mode Failure in a Satellite Catalogue Pipeline
Organizations: The Tesseract Academy London, United Kingdom
Abstract
A common safeguard for a data pipeline is redundant computation: derive each published number by two routes built on different technology and refuse to exit when they disagree. We report one such gate failing, in a cross-catalogue integrity study of two open registers of Earth-orbiting objects. A gate comparing a set-based Python path with SPARQL queries over the emitted RDF graph printed ALL CROSS-CHECKS AGREE on seven counts. Three were wrong, one overstated more than fourfold (932 against 220). Both paths imported the same constants, which encoded a misreading of the source's status vocabulary, so the error was common-mode and the gate could not see it. We give the mechanism, an object-level ledger reconciling every figure, and three checks that go back to the source's documentation, measured on the defective code and on its correction. We then checked that correction against each object's phase history, held in a source file the pipeline never read. The correction was also wrong: 42 of its 261 disagreements are artefacts, and none of our three checks flagged them. Finally, in a controlled replication with three pinned models and tools disabled, 72 of 75 paths generated on request as independent checks computed the defective count, 29 of 30 even when the prompt carried the source's own definitions of the codes. The evidence is one pipeline and one defect family. Within it, redundancy verified implementation, and the errors that reached publication were errors of meaning.
Figures & tables
| Defect class | Python | SPARQL | Agree | First correction | Full history |
| PhantomEntry | 22 | 22 | yes | 22 | 22 |
| PhantomEntryOnOrbit | 1 | 1 | yes | 1 | 1 |
| UndisclosedTrackingLoss | 1,104 | 1,104 | yes | 1,094 | 1,094 |
| DispositionDisagreement | 932 | 932 | yes | 261 | 220 |
| CoverageGap | 900 | 900 | yes | 622 | 622 |
| UnnumberedObject | 605 | 605 | yes | 605 | 605 |
| Gate | Defective code | First correction, against the second error |
|---|---|---|
| Total-function check on the status column | Fires : 9 unclassified codes over 1,099 rows ( ATT , C , DK , E , EVA DP , GRP , N , REL , TFR ) | Silent : 0 unclassified codes. E and C are classified, wrongly |
| Residue decomposition | Shows it : 20 groups, top 5 hold 729 of 932; 500 pair docking or attachment with impact or landing | Shows it, unread : 44 of 261 in groups pairing E or C with a CelesTrak object in orbit |
| Source inventory reconciliation | Fires : satcat100k listed, never fetched | Silent : scoped to the four main catalogues; the event catalogue is on the same index |
| Request | opus-5-5 | sonnet-5 | haiku-4-5 |
|---|---|---|---|
| Neutral | 5/5 | 5/5 | 5/5 |
| “Independent” path | 5/5 | 5/5 | 5/5 |
| Sharing forbidden | 4/5 | 4/5 | 5/5 |
| “Independent” + definitions | 5/5 | 5/5 | 5/5 |
| Sharing forbidden + definitions | 4/5 | 5/5 | 5/5 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Published count | 932 |
| GCAT gone, CelesTrak gives no decay date | 163 |
| CelesTrak decayed, GCAT not gone | 769 |
| docking or attachment ( DK , ATT ) | 500 |
| other transition codes ( TFR , GRP , N , …) | 140 |
| explosion or collision ( E , C ) | 57 |
| in orbit but tracking lost ( OX ) | 31 |