Prints in the Magnetic Dust: Robust Similarity Search in Legacy Media Images Using Checksum Count Vectors

Maciej Grzeszczuk; Kinga Skorupska; Grzegorz M. Wójcik

cs.CVMar 30, 2026

Prints in the Magnetic Dust: Robust Similarity Search in Legacy Media Images Using Checksum Count Vectors

Authors: Maciej Grzeszczuk, Kinga Skorupska, Grzegorz M. Wójcik

Organizations: Polish-Japanese Academy of Information Technology, Warsaw, Poland · The Foundation for the History of Home Computers, Warsaw, Poland · University of Maria Curie-Skodowska, Lublin, Poland

Abstract

Digitizing magnetic media containing computer data is only the first step towards the preservation of early home computing era artifacts. The audio tape images must be decoded, verified, repaired if necessary, tested, and documented. If parts of this process could be effectively automated, volunteers could focus on contributing contextual and historical knowledge rather than struggling with technical tools. We therefore propose a feature representation based on Checksum Count Vectors and evaluate its applicability to detecting duplicates and variants of recordings within a large data store. The approach was tested on a collection of decoded tape images (n=4902), achieving 58% accuracy in detecting variants and 97% accuracy in identifying alternative copies, for damaged recordings with up to 75% of records missing. These results represent an important step towards fully automated pipelines for restoration, de-duplication, and semantic integration of historical digital artifacts through sequence matching, automatic repair and knowledge discovery.

Figures & tables

Figure 1: Distribution of record checksum values for all the CAS files in the repository, by source. Visible spikes represent checksum values of 167 and 169.

Figure 2: File: Wumpus Adventure - Expanded Version v8.7b (1981-03-19) (Sebree’s Computing)(US)[CLOAD+RUN][BASIC].hex and its 9 damaged derivatives created according to the test procedure. Missing records in pink.

Figure 3: Correlation between title similarity and content similarity for file pairs. Each point represents a pair of files with title similarity

\geq 0.8

(cosine similarity on title strings). Content similarity (CS) is calculated using 256-dimensional vectors. Colors indicate content similarity ranges: dark green (CS

\geq 0.95

), light green (

0.80\leq

<0.95

), orange (

0.50\leq

<0.80

), and red (CS

<0.50

). The dashed line represents perfect correlation (

y=x

). Pearson correlation coefficient:

r=0.280

, Spearman rank correlation:

\rho=0.177

Figure 4: Top-1 Accuracy degradation across vector representations. Top-1 accuracy measures the percentage of cases where the correct file (self) has the highest similarity score among all candidates. Green bars indicate >90% accuracy, orange 70-90%, and red <70%. The accuracy decreases both with increased data loss and reduced vector dimensionality, with 16-dimensional space showing critical failure at >50% data loss.

Figure 5: Confusion matrices for 25% data loss cases with match threshold at 0.7.

Panel of Figure 6 — Figure 6: Top: separation (d’) between signal (self-matches) and noise (other-matches) distributions. Bottom: mean similarity between degraded and original versions of the same file, with standard deviation bands.

Library and Information Science

Library and information science studies how information is organized, preserved, discovered, and used across libraries, archives, and digital systems.

See all 19 papers

Identity-Duplication Auditing in National-Scale Neuroimaging Repositories
Oct 7, 2026Jiheng Li, Michael E. Kim, Trent M. Schwartz +19Neuroimaging Brain MRI
HABS-HD · Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA · AI2D Center for AI and Data Science for Integrated Diagnostics, University of Pennsylvania, Philadelphia, PA, USA +2
A machine-readable catalogue of the Tsiolkovsky papers (fond 555, Archive of the Russian Academy of Sciences), and a way to measure how well its handwriting can be read
Aug 4, 2026Vladimir BeskorovainyiLibrary and Information Science Online Handwriting Recognition
Besk Tech | Moscow Institute of Physics and Technology (MIPT)
Bag of Bags: Adaptive Visual Vocabularies for Genizah Join Image Retrieval
Apr 9, 2026Sharva Gogawale, Gal Grudka, Daria Vasyutinsky-Shapira +3Visual Document Retrieval Fine-Grained Image Retrieval
School of Computer Science and AI, Tel Aviv University, Ramat Aviv, Israel

Oct 7, 2026J/K move · Enter open · S save

Identity-Duplication Auditing in National-Scale Neuroimaging Repositories

Jiheng Li, Michael E. Kim, Trent M. Schwartz, Yuhan Cui+18

HABS-HD · Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA · AI2D Center for AI and Data Science for Integrated Diagnostics, University of Pennsylvania, Philadelphia, PA, USA+2

Neuroimaging Brain MRI

Open paper AskPDF arXiv

National-scale magnetic resonance imaging (MRI) repositories increasingly integrate data from different studies and institutions. However, subject identifiers assigned to study participants are typically specific to individual datasets, so the same person may appear under different identifiers when these datasets are aggregated, a problem we define as identity duplication. Such duplication can create leakage between training and test data and inflate apparent performance in downstream biomedical studies. Existing methods do not provide an end-to-end, image-based workflow to address this problem. In this work, we present HAPPEN, a human-in-the-loop pipeline for auditing identity duplication in T1-weighted brain MRI repositories. It combines SHA-256 fingerprinting for exact-duplicate detection with embeddings learned through supervised contrastive learning to retrieve near-duplicate candidates: pairs of non-identical scans that may originate from the same person. Retrieved pairs are reviewed as candidates in a locally hosted interface rather than automatically classified as duplicates. We deployed the workflow in a national-scale aggregated repository containing 95,129 scans and assessed end-to-end recovery of 54 pairs of subject records identified as duplicates using genome-wide genotype data. Transferability was assessed by locally deploying the same workflow on 22,386 scans at an independent institution without model retraining or image transfer. Deployment in the study repository identified 1,316 exact-duplicate scan groups and 1,275 near-duplicate groups of subject records supported by human review. Of these groups, 56% and 82%, respectively, crossed dataset boundaries. All 54 genetic-reference pairs were recovered. The external team independently completed the full workflow using a locally selected operating threshold and review standard.

Abstract

Library and Information Science

Explore similar work

Identity-Duplication Auditing in National-Scale Neuroimaging Repositories

A machine-readable catalogue of the Tsiolkovsky papers (fond 555, Archive of the Russian Academy of Sciences), and a way to measure how well its handwriting can be read

Bag of Bags: Adaptive Visual Vocabularies for Genizah Join Image Retrieval