Retail product catalogs assembled from web image search are cheap to build and pervasively noisy: queries return packaging shots instead of products, and entries proliferate under names that cannot be distinguished visually. Consolidating such a catalog can look like an alternative to scaling the model. We show this is not a comparison between two ways of improving a recognizer, but between improving a recognizer and changing what counts as correct. We study a public archive of 165,769 cheese images across 1,824 query-named entries. A packaging filter and near-duplicate removal reduce it to a 998-class recognition set on one fixed, group-aware split. Canonicalizing entries to their base type (844 classes) raises top-1 accuracy by 4.0 points. Scoring the unchanged model against the merged label space, by summing its predicted probability within each canonical group, recovers +4.27 points with no retraining (+4.25 over six seeds). Retraining on the consolidated catalog then adds -0.10±0.53, +0.08±0.65 and -0.10±0.65 points at 28, 50 and 88 M parameters, indistinguishable from zero at every capacity (± is the across-seed standard deviation). That component is specific, not mechanical: random merges with the identical size profile yield only +0.68 points, and the three backbones agree to within 0.1 points. But it is overwhelmingly category coarsening, not alias correction: across the 37 merged groups, type coarsenings supply 94% of the gain and alias collapses only 7%. We also detect no interaction between consolidation and backbone size: tripling backbone parameters is worth +1.50±0.57 points on the raw catalog and +1.42±00.44 on the canonical one, a within-seed interaction of -0.08 [-0.79,+0.63]. On this archive, the reported consolidation gain primarily reflects the evaluation label space, and separating that out costs one evaluation
Figures & tables
Figure 1: The two defects. Top: the Stage-1 probe scores packaging shots near zero and product shots near one. The leftmost image shows a packaged snack rather than unpackaged cheese, returned by “cheddar cheese” . Bottom: four distinct photographs from folders the scrape treats as four classes and Stage 3 maps to cheddar . This is why no per-image relabeling fixes a defect of this kind.
Quantity
Value
Folders (cheese names)
1,824
Total readable images
165,769
Images kept after packaging filter
62,174 (37.5%)
Distinct photographs after dedup
33,583 (54.0% of kept)
Cheeses with ≥20 distinct images
998
Distinct images/class, median / mean
35 / 36.1
Table 1: Archive and label-set statistics. The recognition subset exceeds the distinct-photograph count because a duplicate group spanning several folders contributes one image to each (Section 4.2 ).
Figure 2: The pipeline. Near-duplicate grouping reduces the filtered images to 33,583 distinct photographs before any split is drawn. Stage 3 changes the label space, not the weights, so Stage 2’s frozen model can be read out over it without retraining. The two branches illustrate this comparison, decomposed in Figure 3 . Six-seed ConvNeXt-Small means (Table 3 ).
Figure 3: (a) Raw-label evaluation, two canonical rescoring rules, and canonical-label retraining, across three backbone sizes (six-seed means). The filled marker shows the accuracy obtained from the frozen model’s posterior over canonical labels, without retraining. It matches the retrained model at every size. (b) Leave-one-out sensitivity: the gain is concentrated in blue , which coarsens 43 entries into a broad type rather than collapsing aliases.
Method
top-1
top-5
Uniform random baseline
0.10%
0.50%
Zero-shot CLIP (text-name match)
5.4%
11.1%
CLIP linear probe (frozen)
20.4%
37.7%
ConvNeXt-Tiny fine-tuned, 25 ep
25.6%
44.1%
ConvNeXt-Small fine-tuned, 25 ep
26.3%
44.7%
ConvNeXt-Base fine-tuned, 25 ep
26.6%
45.4%
Table 2: Stage 2: cheese-type recognition ( 998 classes). Every row is scored on the same held-out test split ( N=5,317 ) of the deduplicated 80 -capped subset. The probe is fit on that split’s 25,325 training images. ConvNeXt-Small remains the reference model for Sections 6.3 to 6.4 , with six-seed results in Table 3 . ConvNeXt rows here use seed 42 . No row is bolded: no single capacity step separates from its neighbor (Table 4 ).
Figure 4: Validation top-1 per epoch, seed 42 . The three raw-label backbones each peak before epoch 25 and their final accuracies differ by under two points, so Table 2 compares them at a fixed epoch budget. The two 844 -class runs are plotted for reference. Inset: epochs 15 to 25 .
Model
raw
marg.
retr.
defin.
resid.
Tiny ( 28 M)
25.30
29.60
29.51
+4.30±.24
−0.10±.53
Small ( 50 M)
26.11
30.36
30.44
+4.25±.09
+0.08±.65
Base ( 88 M)
26.80
31.03
30.93
+4.23±.20
−0.10±.65
CLIP probe, frozen
20.44
24.21
23.34
+3.76
−0.87
Small, filter 0.7
25.66
30.75
30.70
+5.09
−0.05
Table 3: Decomposing the merge gain: six-seed mean top-1 for the three ConvNeXt backbones. Accuracies are percentages; components are percentage points with six-seed standard deviations. Each row’s own raw model is rescored under the same canonical map by the marginal rule. The last two rows are single runs, and the 0.7 -filter row rebuilds the split, so only its decomposition is comparable.
Comparison
Δ
b / c
McNemar p
Tiny → Small (capacity)
+0.8
322 / 363
0.13
Small → Base (capacity)
+0.2
317 / 328
0.69
Tiny → Base (capacity)
+1.0
302 / 354
0.046
Small → Base, merged labels
+1.0
279 / 333
0.03
Merged → merged + class-bal.
−4.1
429 / 213
1×10−17
Raw → rescored, marginal
+4.3
32 / 259
3×10−45
Table 4: Exact McNemar tests on paired per-image outcomes, shared test split ( N=5,317 ), with the discordant counts used in each test: b correct before and wrong after, c wrong before and correct after.