ReCoCo: From Patches to Compositions for Visual Representation Learning
Organizations: Information Processing and Telecommunications Center (IPTC), Universidad Politécnica de Madrid, ES · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK · Sony Interactive Entertainment, DE · Surrey Institute for People-Centred Artificial Intelligence (PAI), University of Surrey, UK
Abstract
Self-supervised learning has produced powerful visual representations, yet learning strong dense features remains challenging. While consistency between augmented views provides strong image-level supervision, extending this principle to local visual content is less straightforward. Existing approaches often match individual patches across views or identify coherent regions, introducing uncertainties in correspondence estimation and assumptions about how visual content should be grouped. We introduce ReCoCo (Regional Composition Consistency), a principled approach that extends cross-view consistency to the regional level without requiring such correspondences or semantic grouping. ReCoCo represents the content shared between augmented views through compositions of their patch-level predictions over learned visual prototypes. By aligning these compositions, the model learns from regions containing multiple visual concepts without requiring them to be separated beforehand. ReCoCo integrates naturally with teacher-student self-distillation and introduces no additional learned parameters. Extensive experiments demonstrate substantial improvements in frozen-feature segmentation, clustering, and regional retrieval while maintaining competitive image-level recognition, alleviating the trade-off between dense and global representation quality.
Figures & tables
| Objects (COCO) | Regions (VOC) | |||
| Method | mAP | P@50 | mAP | R@1 |
| iBOT [ 6 ] | 20.9 | 32.1 | 49.3 | 82.3 |
| ReCoCo | 38.4 | 48.4 | 63.0 | 86.0 |
Visual Representation Learning
Visual representation learning develops image and video features that capture useful structure for recognition, retrieval, spatial reasoning, and action.
See all 249 papers