Vision foundation models produce semantically strong features on a coarse patch grid, and dense prediction needs them at pixel resolution. Existing learned upsamplers close that gap by regressing the high-resolution feature map from the low-resolution one, typically through cross-attention driven by the input image. We observe that most of what they regress needs no learning: plain bilinear interpolation already reproduces the smooth regions of the target, and recent benchmarks even report bilinear and bicubic on par with several learned upsamplers. The rest is mostly edges and fine details, which suggests that these learned upsamplers spend much of their capacity re-computing the smooth content that classical interpolation provides. We therefore introduce AnchorUp, which stops predicting the high-resolution feature map and predicts only its correction from the interpolation. It keeps a parameter-free classical upsampler as an anchor, obtains a second prediction from a small guidance network, and blends the two through a single learnable scalar gate. Because the anchor supplies the smooth content, the guidance network only has to correct what remains. An analysis of the learned correction confirms that it concentrates on edges and fine details, and so the network can be small: AnchorUp uses 167,425 parameters, a quarter of the budget of the strongest baseline, NAF, and runs at 64% of its latency. Trained once on DINOv3 with only the first of NAF's two training stages, and evaluated on that backbone plus six held-out ones, it is already on par with NAF trained through both stages on Pascal VOC and Cityscapes. Our method surpasses it on DAVIS-2017 video object segmentation; the gain holds on every backbone and is larger in contour accuracy F. Code is available at https://github.com/KotaroHorikawa/AnchorUp.
Figures & tables
Figure 1: Accuracy against latency on Cityscapes averaged over seven backbones. Circle size shows the number of parameters (Tables 2 and 4 ). Top: AnchorUp achieves a better accuracy-speed trade-off than baselines: NAF. Bottom: Because our learned part supplies only a correction to a classical interpolation rather than the whole feature map, it focuses solely on missing details (outlines, whiskers, and fur) while remaining flat over the background. Feature maps are projected onto the first three principal components of each image; in the correction, zero appears as mid-gray.
Figure 2: AnchorUp. (a) The low-resolution feature map is upsampled in two ways: by a classical upsampler with no parameters, which gives the anchor, and by a NAF-style upsampler, a guidance network followed by neighborhood cross-attention. In the attention, queries come from the encoded image, keys from its average pooling, and values from the feature map itself. Only the guidance network and the gate α=σ(γ) have learnable parameters; the gate controls how far the output moves from the anchor toward the NAF-style prediction. (b) The output is a point on the segment between the two predictions: the network picks the direction, the gate picks the distance. The dashed line is the remaining error to the target.
Images only
Images + text
Distilled
Method
Anchor
d
#Params
Stage 2
DINOv3
Franca
CLIP
SigLIP2
PEcore
PEspatial
RADIO
Mean
Δ
(a) Pascal VOC
NAF [ 2 ]
–
256
662,528
✓
87.81
84.30
80.25
77.83
72.61
78.04
86.61
81.06
–
NAF
–
128
167,424
✓
87.83
84.21
79.98
77.40
72.16
78.16
86.51
80.89
− 0.17
NAF
–
128
167,424
87.34
84.24
80.53
78.85
73.62
77.60
86.55
81.24
+ 0.18
AnchorUp
bilinear
128
167,425
87.71
84.27
80.23
77.80
72.63
78.18
86.61
81.06
0.00
Table 1: Cross-backbone linear probing. All upsamplers are trained once on DINOv3-B/16 and applied unchanged to the other six backbones. Anchor is the interpolation used in Eq. 1 (“–” for NAF, which has none), d is the guidance dimension (Section 3.3 ), #Params the number of learnable parameters, and a check under Stage 2 marks training through the second stage of NAF’s recipe (Section 3.5 ). Rows are ordered so that each differs from the row above in a single factor; the third NAF row therefore differs from AnchorUp only by Eq. 1 . The best result per column is in bold, the second best is underlined; Δ is the difference in mean from the two-stage NAF row.
Images only
Images + text
Distilled
Method
DINOv3
Franca
CLIP
SigLIP2
PEcore
PEspatial
RADIO
Mean
Δ
(a) Pascal VOC
Bilinear
86.97
82.12
76.28
71.94
66.66
76.62
84.47
77.87
–
FeatUp [ 8 ]
86.60
81.68
77.29
72.17
66.28
74.28
82.95
77.32
− 0.54
JAFAR [ 4 ]
87.08
83.65
79.80
77.51
74.23
74.97
85.97
80.46
+ 2.59
AnyUp [ 29 ]
86.56
83.26
79.76
78.56
74.09
76.77
85.63
80.66
+ 2.80
Table 2: Comparison with other feature upsamplers. Every method is trained here under its own published configuration. NAF and AnchorUp are trained once on DINOv3-B/16 and applied unchanged; FeatUp and JAFAR require one model per backbone (Section 4 ). The NAF and AnchorUp rows are the models of Table 1 . Best per column in bold, second best underlined; Δ is the difference in mean from bilinear.
Images only
Images + text
Distilled
Method
Anchor
d
Stage 2
DINOv3
Franca
CLIP
SigLIP2
PEspatial
RADIO
Mean
Δ
J -Mean ↑
NAF [ 2 ]
–
256
✓
66.00
68.70
62.80
62.20
67.00
66.40
65.52
–
NAF
–
128
✓
65.60
68.40
62.90
61.90
66.80
65.90
65.25
− 0.27
NAF
–
128
62.79
66.59
61.77
62.31
65.88
65.02
64.06
− 1.46
AnchorUp
bilinear
128
66.30
69.50
63.50
62.40
67.70
67.20
66.10
+ 0.58
Table 3: Semi-supervised video object segmentation on DAVIS-2017. Label propagation uses the upsampled features directly, without any training on video. Rows follow the order of Table 1 , each differing from the row above it in one factor, so the third NAF row differs from AnchorUp only by Eq. 1 . The third NAF row is trained using Stage 1 only, following the AnchorUp recipe. The gain is largest on F -Mean, the boundary-accuracy term (Section 5 shows the correction concentrates on boundaries). Best per column in bold, second best underlined; Δ is the difference in mean from the two-stage NAF row.
Method
Params ↓
Latency (ms) ↓
Peak mem. (MiB) ↓
Bilinear
0
0.30
297.4
Bicubic
0
0.84
297.4
FeatUp [ 8 ]
173,540
27.95
2674.0
JAFAR [ 4 ]
628,480
45.82
5410.9
AnyUp [ 29 ]
845,056
49.27
3253.8
NAF [ 2 ]
662,528
25.31
1685.5
Table 4: Cost of one upsampling pass for the upsampler alone, at 448×448 output from the 28×28 features of 384 channels that a 448×448 input produces, FP32 and batch 1 on an RTX PRO 6000 Blackwell. Latencies are means over 500 forwards and the standard deviation is below 0.07 ms for every row. AnchorUp is 1.6× faster than NAF, with a quarter of its parameters.
Method
VOC
Cityscapes
DAVIS
Control
81.36±0.10
59.44±0.70
65.32±1.05
AnchorUp
81.19±0.18
60.57±0.08
68.87±0.15
Difference
−0.16±0.09
+1.14±0.62
+3.54±0.92
Table 5: Comparison with the control over three training seeds. The control is the third NAF row of Tables 1 and 3 ( d=128 , stage 1 only), and AnchorUp uses the bilinear anchor. Each entry is the mean and standard deviation over seeds 0–2 of the mean across backbones (mIoU on VOC and Cityscapes, J&F on DAVIS); the difference is taken between models of the same seed. Seed 0 is the model from Tables 1 and 3 , with the linear probes retrained.
Figure 3: The correction is a boundary signal. Columns: input, low-resolution features, anchor Fanchor , learned estimate Flearn , correction Cθ=α(Flearn−Fanchor) , and AnchorUp output F^HR . Feature maps are projected onto the first three principal components of each image and shown as RGB; in the correction column, zero appears as mid-gray. The correction is flat inside regions and concentrates on contours and on the fine repeated structure the anchor cannot resolve (rows 2 and 3). Added to the anchor (third column), this is the detail the output (last column) recovers.
Figure 4: The correction is high-pass. Radially averaged Fourier power of each term of Eq. 1 : the anchor Fanchor , the learned estimate Flearn , the correction Cθ=α(Flearn−Fanchor) , and the output F^HR . Two readings matter. At the lowest frequencies the correction carries about two orders of magnitude less power than the anchor ( 106 against 2×108 ): the smooth content of the output comes from the anchor, not from the network. Between roughly 0.06 and 0.3 cycles per pixel the correction carries more power than the anchor: the learned part spends its output where the interpolation cannot reach. The measurement protocol is in the supplementary material.
Anchor
learned α
α⋆
difference
Bilinear
0.620
0.551
+0.069
Bicubic
0.771
0.748
+0.023
Table S1: The learned gate against the gate the loss would choose. α⋆ is Eq. 3 of the main paper solved on 200 Cityscapes validation images with DINOv3 features, no labels used, each row using its own trained model. The gate is trained by the same loss as the network, with no schedule and no separate term.
Figure S1 : The error is flat around the learned gate α , and its minimum moves with the anchor. The loss of Section 3.4 of the main paper as a function of α , averaged over 200 Cityscapes validation images, with the guidance network of the bilinear-anchor model frozen and the anchor swapped. Dots mark minima; the vertical line marks the learned gate. The norm sums over all feature elements of an image, setting the scale of the axis.
Backbone
Checkpoint
Patch
DINOv3
vit_base_patch16_dinov3.lvd1689m
16
CLIP
16
vit_base_patch16_clip_384.
laion2b_ft_in12k_in1k
SigLIP2
vit_base_patch16_siglip_512.v2_webli
16
Franca
franca_vitb14
14
Table S2 : Frozen backbones. All are base-size vision transformers; Franca uses a patch size of 14 and the rest 16.
Setting
Value
Data
ImageNet, 512×512 HR targets
Degradation
fixed, ×2
Guidance resolution
min(224,4×HR feature) per axis
Loss
ℓ2 feature reconstruction, no auxiliary terms
Batch size
4
Optimizer
AdamW, β=(0.9,0.999)
Table S3: Training setup in full. The position encoding is worth stating explicitly because a later default in the same codebase disables the RoPE rescale: these results use rescale 2.
Images only
Images + text
Distilled
Method
DINOv3
Franca
CLIP
SigLIP2
PEcore
PEspatial
RADIO
Mean
(a) Pascal VOC
Control
86.93±0.35
84.02±0.19
80.70±0.15
79.50±0.57
74.50±0.76
77.42±0.16
86.45±0.09
81.36±0.10
AnchorUp
87.58±0.13
84.32±0.02
80.47±0.20
78.38±0.56
73.26±0.64
77.68±0.02
86.67±0.02
81.19±0.18
Difference
+0.65±0.22
+0.30±0.18
−0.23±0.06
−1.12±0.01
−1.24±0.12
+0.26±0.17
+0.23±0.10
−0.16±0.09
(b) Cityscapes
Table S4 : Per-backbone values behind Table 5 of the main paper. Mean and standard deviation over training seeds 0–2. The control is the third NAF row of Tables 1 and 3 of the main paper ( d=128 , stage 1 only), and AnchorUp uses the bilinear anchor; the difference is taken between models of the same seed. DAVIS excludes PEcore, as in Table 3 of the main paper.
Backbone
Image–text
Δ VOC
Δ Cityscapes
PEcore
✓
+1.98
+0.42
SigLIP 2
✓
+1.73
+0.30
CLIP
✓
+0.55
−0.29
RADIO
+0.03
−1.12
PEspatial
−0.03
−0.79
Franca
−0.07
−0.51
Table S5: Bicubic anchor minus bilinear anchor, by backbone. A check under Image–text marks the backbones pretrained with image–text supervision. Rows are ordered by the VOC gain. The three image–text backbones take the top three places on both benchmarks and the four others take the bottom four, with no overlap. One caveat for the PEcore–PEspatial reading of Section 4.1 of the main paper: the two checkpoints differ in nominal resolution as well, 224 and 512.
Anchor
d
α
DINOv3
CLIP
SigLIP 2
PEcore
Mean
#Params (M)
Bilinear
32
0.579
65.66
56.01
57.19
54.06
58.23
0.011
64
0.607
66.09
56.53
57.81
54.69
58.78
0.043
128
0.620
66.45
57.07
58.28
55.25
59.26
0.167
256
0.630
66.49
57.09
58.53
55.32
59.36
0.663
Bicubic
32
0.772
65.38
56.03
57.40
54.30
58.28
0.011
64
0.771
65.74
56.42
57.87
54.78
58.70
0.043
Table S6: Guidance dimension and anchor. AnchorUp trained with the recipe of Section 3.5 of the main paper at four guidance dimensions for both anchors. Cityscapes mIoU is evaluated by linear probing; the mean is calculated over the four backbones shown, not the seven in Table 1 of the main paper. The d=128 rows correspond to the models in that table, whose costs are reported in Table 4 of the main paper. α is the trained gate.
Training steps
Benchmark
25,000
100,000
Difference
VOC mIoU
81.06
81.08
+0.02
Cityscapes mIoU
60.77
60.94
+0.17
DAVIS J&F
65.90
66.29
+0.39
Table S7: Training length. AnchorUp with a bilinear anchor, mean over the seven backbones of Table 1 of the main paper. The DAVIS row therefore includes PEcore and is not comparable to Table 3 of the main paper, which averages over six. The main paper reports the 25,000-step checkpoint throughout.
Figure S2 : Where the extra training goes. Radially averaged power of the AnchorUp prediction with a bilinear anchor, at six checkpoints of the same run. The band below 0.05 cycles per feature pixel is settled at the first checkpoint and moves by less than 0.5% thereafter; all of the growth is in the shaded band above 0.25 , which gains 29% between 25,000 and 100,000 steps and 56% between 25,000 and 300,000.
Figure S3 : The correction split at the crossings of Figure 4 of the main paper. Rows are the three scenes of its Figure 3. The middle band, where the correction carries more power than the anchor, holds the fine contour lines; the band below holds the broad response around the same boundaries; the band above holds almost nothing.
Figure S4 : Decomposition, all ten images. Columns as in Figure 3 of the main paper, which shows the first three rows.
Figure S5 : Decomposition on held-out images, rows 1–10 of 30. ImageNet validation images, unseen during training; columns and projection as in Figure S4 .
Figure S6 : Decomposition on held-out images, rows 11–20 of 30. Columns and projection as in Figure S4 .
Figure S7 : Decomposition on held-out images, rows 21–30 of 30. Columns and projection as in Figure S4 .