BINO: Encoder Centric Self Supervised Stereo With Native Pair Input
Organizations: Imperial College London
Abstract
Stereo needs features that preserve fine cross view correspondence rather than only semantic similarity. Recent self supervised vision models transfer well, but they are not built for this goal, and geometry focused methods often rely on a binocular decoder or another explicit linkage module during pretraining. BINO asks whether strong binocular structure can instead be learned inside a compact encoder. It does this by fusing the rectified pair at the input stage, forming stereo micro cell tokens, and using a row aware patch phase positional encoding. Training uses one view masked token only distillation together with occlusion and view specific appearance mismatch. In a strict low resource setting with pretraining only on KITTI object, BINO gives the best frozen descriptor results under a no linkage probe among all compared baselines on proxy dense stereo, hard negative retrieval, and KITTI Stereo2012 disparity. With the same lightweight stereo head for every encoder, it stays near CroCov2 while using a much smaller encoder. Supplementary transfer experiments on KITTI Stereo~2015 show the same qualitative trend. These results suggest that much of the cross view reasoning often assigned to a separate linkage module can be learned inside a compact and reusable encoder.
Figures & tables
| Variant | train_s | params (M) | Ntok | PCK@0 (%) | PCK@1 (%) | PCK@2 (%) | EPE |
|---|---|---|---|---|---|---|---|
| Interleave, stride 1, width 4 | 162.3 | 2.233 | 128 | 75.96 | 87.29 | 91.76 | 0.557 |
| Concat, width 4 | 109.9 | 2.233 | 128 | 71.19 | 84.64 | 89.92 | 0.670 |
| Interleave, width 2 | 289.6 | 2.229 | 256 | 74.96 | 86.59 | 91.03 | 0.591 |
| Interleave, width 8 | 95.3 | 2.242 | 64 | 73.28 | 86.01 | 90.89 | 0.612 |
| Interleave, width 1 | 350.3 | 2.226 | 512 | 60.24 | 77.28 | 85.08 | 0.959 |
| Interleave, stride 2, width 4 | 160.0 | 2.233 | 128 | 74.00 | 86.48 | 91.39 | 0.587 |
| Variant | params (M) | train_s | PCK@0 (%) | PCK@1 (%) | PCK@2 (%) | EPE |
|---|---|---|---|---|---|---|
| 1D positional embedding | 2.233 | 161.0 | 76.07 | 87.37 | 91.80 | 0.553 |
| Factorized 2D row+col | 2.149 | 159.6 | 75.24 | 87.03 | 91.40 | 0.578 |
| Full 2D grid | 2.233 | 159.4 | 75.61 | 87.17 | 91.76 | 0.559 |
| De-interleaved center 2D | 2.149 | 159.4 | 72.97 | 84.51 | 89.51 | 0.674 |
| Patch-phase 2D (ours) | 2.139 | 159.8 | 75.90 | 88.76 | 92.95 | 0.507 |
| Variant | train_s | params (M) | PCK@0 (%) | PCK@1 (%) | PCK@2 (%) | EPE |
|---|---|---|---|---|---|---|
| Token only, uniform, EMA, scheduled mask | 188.5 | 2.139 | 79.46 | 90.85 | 93.68 | 0.453 |
| Token only, weighted, EMA, scheduled mask | 187.9 | 2.139 | 79.11 | 90.18 | 93.15 | 0.479 |
| Token+CLS, uniform, EMA, scheduled mask | 188.7 | 2.139 | 76.72 | 88.87 | 92.48 | 0.528 |
| Token+CLS, weighted, EMA, scheduled mask | 191.6 | 2.139 | 75.51 | 87.83 | 91.84 | 0.565 |
| Token+CLS, weighted, EMA, fixed mask 0.50 | 173.0 | 2.139 | 74.64 | 87.17 | 91.50 | 0.588 |
| Token+CLS, weighted, EMA, mask both views | 192.6 | 2.139 | 71.07 | 88.33 | 92.04 | 0.600 |
| Variant | train_s | params (M) | HARD_S1 PCK@1 (%) | HARD_S1 EPE | HARD_S2 PCK@1 (%) | HARD_S2 EPE | EASY_S1 PCK@1 (%) | EASY_S1 EPE |
|---|---|---|---|---|---|---|---|---|
| Easy only | 176.9 | 1.670 | 89.63 | 0.518 | 86.01 | 0.613 | 95.56 | 0.293 |
| Occlusion only | 175.1 | 1.670 | 90.35 | 0.476 | 87.83 | 0.530 | 95.75 | 0.268 |
| Photometric+noise only, shared | 180.9 | 1.670 | 90.43 | 0.486 | 87.41 | 0.554 | 95.75 | 0.286 |
| Occlusion+photometric, shared | 179.3 | 1.670 | 90.65 | 0.462 | 88.29 | 0.510 | 95.83 | 0.261 |
| Full hard, shared perturbation | 181.9 | 1.670 | 90.79 | 0.456 | 88.27 | 0.509 | 95.85 | 0.261 |
| Full hard, independent perturbation | 182.0 | 1.670 | 91.17 | 0.448 | 88.30 | 0.513 | 96.43 | 0.247 |
| Method | PCK@1tok (%) | EPE (px) |
|---|---|---|
| OURS | 84.07 | 11.82 |
| CroCo-v1 (sincos) | 52.36 | 18.68 |
| CroCo-v2 (RoPE) | 79.09 | 16.53 |
| iBOT-style | 60.15 | 29.49 |
| DINOv2-style | 60.15 | 29.43 |
| DINOv3-style | 71.55 | 20.11 |
| Method | Top-1 (%) | Top-5 (%) | Hard@1 (%) | Hard@5 (%) | Margin |
|---|---|---|---|---|---|
| OURS | 65.2 | 82.4 | 76.1 | 88.4 | +0.0007 |
| CroCo-v1 (sincos) | 45.1 | 69.1 | 59.1 | 79.5 | -0.0015 |
| CroCo-v2 (RoPE) | 56.3 | 78.6 | 69.2 | 85.7 | -0.0011 |
| iBOT-style | 12.4 | 32.9 | 24.1 | 51.1 | -0.0009 |
| DINOv2-style | 14.5 | 32.2 | 25.1 | 49.2 | -0.0040 |
| DINOv3-style | 37.9 | 62.1 | 51.7 | 74.0 | -0.0035 |
| Method | GT WTA EPE | GT SGMLOC EPE | GT SGMLOC Bad@1tok (%) | GT+LR SGMLOC EPE | GT+LR SGMLOC Bad@1tok (%) | LRkeep (%) |
|---|---|---|---|---|---|---|
| OURS | 9.74 | 4.62 | 2.19 | 4.57 | 1.91 | 92.5 |
| CroCo-v1 (sincos) | 14.76 | 10.25 | 21.36 | 10.22 | 21.29 | 96.8 |
| CroCo-v2 (RoPE) | 14.32 | 5.93 | 8.37 | 5.70 | 7.85 | 87.0 |
| iBOT-style | 28.78 | 7.63 | 11.93 | 7.55 | 11.57 | 71.7 |
| DINOv2-style | 34.07 | 8.44 | 15.68 | 8.39 | 15.44 | 67.1 |
| DINOv3-style | 19.53 | 8.01 | 14.22 | 7.98 | 14.03 | 80.9 |
| Method | Encoder params (M) | GT WTA EPE | GT WTA Bad@1tok (%) | GT SGMLOC EPE | GT SGMLOC Bad@1tok (%) | GT+LR SGMLOC EPE | GT+LR SGMLOC Bad@1tok (%) | LRkeep (%) |
|---|---|---|---|---|---|---|---|---|
| OURS | 1.339 | 4.423 | 0.78 | 4.258 | 0.70 | 4.215 | 0.56 | 99.4 |
| CroCo-v1 (sincos) | 3.337 | 5.095 | 2.99 | 4.759 | 2.65 | 4.422 | 1.64 | 77.6 |
| CroCo-v2 (RoPE) | 3.337 | 4.300 | 0.46 | 4.285 | 0.38 | 4.291 | 0.38 | 99.7 |
| iBOT-style | 2.020 | 5.606 | 4.04 | 5.742 | 3.64 | 5.718 | 3.62 | 97.7 |
| DINOv2-style | 2.022 | 5.593 | 4.02 | 6.232 | 3.54 | 6.210 | 3.49 | 99.3 |
| DINOv3-style | 1.928 | 5.684 | 4.63 | 5.806 | 3.65 | 5.803 | 3.65 | 99.4 |
| Method | Large-disparity LOCAL Bad@1tok (%) |
|---|---|
| Ours (native fused) | 15.53 |
| Ours (export control) | 21.77 |
| CroCo-v1 (sincos) | 16.29 |
| CroCo-v2 (RoPE) | 17.66 |
| iBOT-style | 20.93 |
| DINOv2-style | 21.23 |
| Method | Clean EPE | Clean Bad@1tok (%) | Occ40 EPE | Occ40 Bad@1tok (%) | OccAUC EPE | OccAUC Bad@1tok (%) | 40 EPE | 40 Bad@1tok (%) |
|---|---|---|---|---|---|---|---|---|
| Ours | 5.403 | 4.37 | 6.886 | 9.74 | 6.284 | 7.50 | 1.483 | 5.37 |
| CroCo-v1 (sincos) | 5.360 | 4.70 | 7.081 | 9.88 | 6.366 | 7.89 | 1.721 | 5.18 |
| CroCo-v2 (RoPE) | 5.028 | 3.53 | 8.741 | 19.74 | 7.380 | 13.71 | 3.713 | 16.22 |
| iBOT-style | 5.831 | 5.06 | 10.725 | 25.71 | 9.032 | 18.39 | 4.894 | 20.65 |
| DINOv2-style | 5.855 | 5.21 | 8.625 | 17.97 | 7.619 | 13.46 | 2.770 | 12.76 |
| DINOv3-style | 5.562 | 4.58 | 8.928 | 18.69 | 8.218 | 16.25 | 3.365 | 14.11 |
| Head | Params (M) | Top-1 (%) | Top-3 (%) | MRR | LOCAL EPE | LOCAL Bad@1tok (%) | BND EPE | BND Bad@1tok (%) | LARGE EPE | LARGE Bad@1tok (%) |
|---|---|---|---|---|---|---|---|---|---|---|
| Linear | 0.032 | 69.1 | 97.6 | 0.830 | 6.381 | 6.23 | 6.676 | 8.46 | 9.668 | 22.30 |
| Tiny64 1 1 | 0.106 | 78.6 | 97.9 | 0.881 | 5.745 | 5.67 | 6.218 | 8.14 | 8.953 | 21.39 |
| Small96 1 2 | 0.309 | 77.3 | 97.8 | 0.873 | 5.689 | 5.52 | 6.094 | 7.90 | 8.670 | 21.08 |
| Medium128 2 2 | 0.687 | 78.2 | 98.9 | 0.880 | 5.591 | 5.04 | 5.972 | 7.22 | 8.236 | 19.41 |
| Large192 3 3 | 2.186 | 80.0 | 99.0 | 0.891 | 5.604 | 5.44 | 5.990 | 7.75 | 8.598 | 20.93 |
| XLarge256 4 4 | 5.045 | 80.4 | 98.9 | 0.893 | 5.428 | 5.03 | 5.813 | 7.22 | 8.472 | 19.71 |
| Variant | Total params (M) | GT WTA EPE | GT WTA Bad@1tok (%) | GT SGMLOC EPE | GT SGMLOC Bad@1tok (%) | GT+LR SGMLOC EPE | GT+LR SGMLOC Bad@1tok (%) | Top-1 (%) | Hard@1 (%) | Margin |
|---|---|---|---|---|---|---|---|---|---|---|
| Base model | 1.771 | 5.184 | 14.66 | 2.202 | 1.73 | 2.192 | 1.64 | 64.3 | 76.3 | 0.0026 |
| + register tokens | 1.772 | 5.134 | 14.44 | 2.266 | 1.88 | 2.256 | 1.78 | 58.9 | 72.8 | 0.0015 |
| + SwiGLU FFN | 1.779 | 5.003 | 13.83 | 2.144 | 1.43 | 2.131 | 1.33 | 64.2 | 76.9 | 0.0028 |
| + dual token/global heads | 1.797 | 4.450 | 12.38 | 2.308 | 2.01 | 2.300 | 1.96 | 70.8 | 82.8 | 0.0034 |
| Synthetic stereo | KITTI Stereo 2012 val | |||||||
|---|---|---|---|---|---|---|---|---|
| Layer | RowConc | GT@1 | MRR | Entropy | RowConc | GT@1 | MRR | Entropy |
| layer0_raw | 0.1036 | 0.0990 | 0.2200 | 3.4568 | 0.0956 | 0.0718 | 0.3875 | 3.5953 |
| layer1_raw | 0.1408 | 0.1563 | 0.2642 | 2.9820 | 0.1402 | 0.1416 | 0.4406 | 3.1353 |
| layer2_raw | 0.1612 | 0.1753 | 0.2760 | 2.8855 | 0.1677 | 0.1691 | 0.4317 | 2.9826 |
| layer3_raw | 0.1602 | 0.1782 | 0.2823 | 2.9319 | 0.1701 | 0.1726 | 0.4361 | 2.9892 |
| layer4_raw | 0.1647 | 0.1806 | 0.2852 | 2.9476 | 0.1784 | 0.1778 | 0.4285 | 2.9695 |
| Synthetic stereo | KITTI Stereo 2012 val | |||||||
|---|---|---|---|---|---|---|---|---|
| Condition | RowConc | Tgt@1 | MRR | Entropy | RowConc | Tgt@1 | MRR | Entropy |
| Original (raw) | 0.1726 | 0.1782 | 0.2835 | 2.9708 | 0.1787 | 0.1781 | 0.4284 | 2.9672 |
| Original (DePos) | 0.1722 | 0.1783 | 0.2835 | 2.9695 | 0.1782 | 0.1784 | 0.4283 | 2.9658 |
| Replace-right | 0.1530 | 0.1603 | 0.2754 | 3.0595 | 0.1550 | 0.1612 | 0.4281 | 3.0712 |
| Row-shuffle-right | 0.1723 | 0.1163 | 0.2339 | 3.2364 | 0.1762 | 0.1047 | 0.2607 | 3.2632 |
| Duplicate-left (zero) | 0.1676 | 0.2603 | 0.6638 | 2.8850 | 0.1745 | 0.2650 | 0.7340 | 2.9336 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Top-1 (%) | Top-5 (%) | GT WTA EPE | GT SGM EPE |
|---|---|---|---|---|
| OURS | 70.0 | 100.0 | 12.728 | 6.728 |
| CroCo-v1 (sincos) | 35.0 | 70.0 | 9.076 | 10.849 |
| CroCo-v2 (RoPE) | 68.3 | 98.3 | 15.400 | 7.605 |
| iBOT-style | 18.3 | 41.7 | 28.327 | 8.151 |
| DINOv2-style | 16.7 | 55.0 | 23.254 | 9.985 |
| DINOv3-style | 55.0 | 90.0 | 15.059 | 8.437 |
| Method | Encoder params (M) | GT WTA EPE | GT WTA Bad@1tok (%) | GT SGMLOC EPE | GT SGMLOC Bad@1tok (%) | GT+LR SGMLOC EPE | GT+LR SGMLOC Bad@1tok (%) | LRkeep (%) |
|---|---|---|---|---|---|---|---|---|
| OURS | 1.339 | 4.742 | 0.95 | 3.888 | 1.29 | 3.831 | 1.06 | 99.7 |
| CroCo-v1 (sincos) | 3.337 | 5.022 | 1.97 | 4.313 | 2.05 | 4.209 | 1.63 | 98.4 |
| CroCo-v2 (RoPE) | 3.337 | 4.637 | 0.71 | 4.224 | 0.92 | 4.190 | 0.77 | 99.7 |
| iBOT-style | 2.020 | 5.316 | 2.58 | 4.599 | 2.58 | 4.598 | 2.57 | 98.9 |
| DINOv2-style | 2.022 | 5.719 | 4.21 | 5.377 | 4.13 | 5.377 | 4.13 | 100.0 |
| DINOv3-style | 1.928 | 5.294 | 2.47 | 6.401 | 3.63 | 6.401 | 3.63 | 100.0 |
Visual Representation Learning
Visual representation learning develops image and video features that capture useful structure for recognition, retrieval, spatial reasoning, and action.
See all 224 papers