Organizations: Centre de Visió per Computador Universitat Autònoma de Barcelona Bellaterra, Catalonia · Computer Science Department Universitat Autònoma de Barcelona, Bellaterra, Catalonia
While time evolves linearly, the geometry of neural embedding spaces is inherently multi-dimensional, often chaotic, and difficult to interpret. In principle, one could constrain an embedding space to a single temporal dimension; however, such a reduction would sacrifice performance on downstream tasks, as one-dimensional embeddings cannot retain sufficient expressive capacity. This paper asks whether it is possible to learn representations that preserve temporal structure while remaining effective for image and object retrieval, and answers this question by building the mathematical foundations of such a system. We propose Temporally Decomposable Image Representations (TDIR), a representation learning algorithm that decomposes historical photographs into separate date and content components through orthogonal subspaces. We define and prove the conditions under which such a decomposition is achievable, characterize the error incurred when those conditions are only partially met, and show that orthogonality between temporal and categorical subspaces emerges naturally from the joint optimization, without requiring it to be imposed explicitly. Beyond its geometric properties, TDIR enables a class of transitive operations on embedding spaces: the temporal information of one image can be extracted and injected into the representation of another, with no label supervision required. All theoretical properties are grounded and validated in the real-world problem of Composed Image Retrieval on historical photographs, where a query simultaneously specifies object content and a target time period, either through labels or through example images. This in-the-wild setting serves as a concrete backing for the propositions we derive, offering an intuitive and interpretable way to navigate photographic archives while maintaining competitive performance in both date estimation and object retrieval.
Figures & tables
Figure 1: Visual abstract of the three proposed representation objectives, showing composed image retrieval for query-by-label (a), query-by-example (b) and query-by-examples (c), in the latter, the idea is to inject the temporal information of an image into the object representation of the other query.
Figure 2: Training pipeline. (a) Proxy loss forms year-agnostic category clusters. (b) Subtracting μc centers each cluster into a shared temporal subspace. (c) Year proxies are optimized on normalized residuals via cosine distance. (d) The swapping trick translates residuals across years, enforcing temporal transitivity.
Figure 3: Inference modes. (a) Label-based retrieval via proxy addition. (b) Image-guided retrieval with a target year label. (c) Zero-label retrieval: the year residual of xy is extracted and transplanted onto f(xc) .
Table 1: Example images and categories from the dataset.
Label+Label
Image+Label
Image+Image
Backbone
Obj. P@ K
Date P@ K
Obj. P@ K
Date P@ K
Obj. P@ K
Date P@ K
CLIP (arithmetic, see Appendix D.1 )
.152
.115
.387
.076
.389
.102
CLIP (prompting, see Appendix D.2 )
.415
.105
.372
.079
.520
.081
VGG + TDIR
.397
.712
.388
.579
.669
.234
ResNet + TDIR
.489
.803
.444
.665
.721
.306
ConvNeXt + TDIR
.443
.824
.416
.664
.725
.393
Table 2: Comparison across backbone architectures and the CLIP baseline (K=10).
Label+Label
Image+Label
Image+Image
Swap
Aux
Norm
Obj. P@ K
Date P@ K
Obj. P@ K
Date P@ K
Obj. P@ K
Date P@ K
✗
✓
✓
.362
.712
.342
.568
.683
.358
✓
✗
✓
.355
.822
.343
.676
.691
.354
✓
✓
✗
.358
.816
.349
.675
.708
.362
✓
✓
✓
.443
.824
.416
.664
.725
.393
Table 3: Ablation study (K=10, ConvNext ) for our method including the usage of the swapping trick, auxiliary proxies and normalization of the temporal component.
Figure 4: Calibration error bars when using query-by-examples year injection.
Table 4: Qualitative results with category error and temporal error .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Section
Content
A
Derivations and Demonstrations
B
Photographic Material Calculation
C
Object-Centric Analysis
D
CLIP Baselines
E
Frequently Asked Questions (FAQs)
Appendix
Table 9
Figure 5: Resulting graphical model from Proposition 2
Figure 6: Average Year Error decomposed by reference object (top) and target object (bottom) in the Image+Image inference mode. Reference objects with low error are reliable vessels for date injection, indicating clean category representations with low temporal entanglement. Target objects with high error are temporally ambiguous, making them difficult to anchor to a specific period regardless of the source image used.
Figure 7: Per-object precision radar plots for object retrieval (top row) and date estimation (bottom row) across Label+Label, Image+Label, and Image+Image inference modes (left to right). Object retrieval errors concentrate on visually sub-specific categories such as glasses, vehicle registration plates, and footwear, which share features across class boundaries. For date estimation, Label+Label achieves consistently strong performance, while image-based modes reveal a pronounced and category-dependent performance gap for temporally ambiguous objects.
Figure 8: Left: date estimation precision per source year across inference modes. Both Image+Label and Image+Image exhibit degraded performance for earlier decades, with Image+Label remaining more stable across the temporal range. Right: scalar summary of object and date precision per inference mode. In Image+Image retrieval, the relationship between object and date precision inverts relative to label-based modes, indicating that temporal injection is a stricter operation than category preservation under the learned decomposition.
Figure 9: Per-object drop in date retrieval precision.
Figure 10: Per-object drop in object retrieval precision.
Figure 11: Category-year cosine similarity across all 14 year proxies for the two most orthogonal ( Window , Bus ) and the most entangled ( Footwear ) of the 50 categories.
Museum and archival datasets do not mirror historical artistic production, but materialize the contingent histories of collecting, preservation, cataloging, and digitization. This has direct consequences for interpreting pretrained image representations: they may appear to encode historical time while actually encoding the institutional conditions under which objects become visible as data. We describe this phenomenon as temporal entanglement and investigate it by formulating artwork dating as an uncertainty-aware regression task over frozen image embeddings. We evaluate several pretrained vision models on a temporally controlled Wikidata corpus of artworks. Our results show that these models contain usable temporal information, with Vision-Language Models (VLMs) outperforming purely visual self-supervised baselines. However, a qualitative analysis indicates that this temporal knowledge is shaped by various biases.
Composed Image Retrieval (CIR) is a flexible image retrieval paradigm that enables users to accurately locate the target image through a multimodal query composed of a reference image and modification text. Although this task has demonstrated promising applications in personalized search and recommendation systems, it encounters a severe challenge in practical scenarios known as the Noise Triplet Correspondence (NTC) problem. This issue primarily arises from the high cost and subjectivity involved in annotating triplet data. To address this problem, we identify two central challenges: the precise estimation of composed semantic discrepancy and the insufficient progressive adaptation to modification discrepancy. To tackle these challenges, we propose a cHrono-synergiA roBust progressIve learning framework for composed image reTrieval (HABIT), which consists of two core modules. First, the Mutual Knowledge Estimation Module quantifies sample cleanliness by calculating the Transition Rate of mutual information between the composed feature and the target image, thereby effectively identifying clean samples that align with the intended modification semantics. Second, the Dual-consistency Progressive Learning Module introduces a collaborative mechanism between the historical and current models, simulating human habit formation to retain good habits and calibrate bad habits, ultimately enabling robust learning under the presence of NTC. Extensive experiments conducted on two standard CIR datasets demonstrate that HABIT significantly outperforms most methods under various noise ratios, exhibiting superior robustness and retrieval performance. Codes are available at https://github.com/Lee-zixu/HABIT
Composed image retrieval (CIR) aims to retrieve a desired target image from a query consisting of a reference image and a modification text. This task exhibits an unusual representational asymmetry: the modification text specifies a transition from the reference state, whereas retrieval candidates depict completed target states. This creates a representation mismatch for zero-shot methods that query pretrained vision-language spaces directly with transformation-oriented language. We study this mismatch and reformulate zero-shot composed image retrieval as target-state reconstruction followed by retrieval. We instantiate this formulation with ASAP-CIR, a training-free framework that reconstructs a static target representation using a frozen multimodal large language model (MLLM). The representation combines multiple holistic descriptions with a variable set of importance-weighted atomic semantics, thereby preserving both overall target identity and fine-grained visual constraints. Retrieval then integrates holistic state alignment, atomic constraint grounding, and calibrated target-state evidence aggregation. A controlled text-only diagnostic shows that target-side static query formulations achieve more reliable retrieval than dynamic composed query formulations, particularly when source-state semantics must be suppressed or transformed. Experiments on FashionIQ, CIRR, and CIRCO further characterize the effectiveness and limitations of this representation principle, with the clearest gains on the multi-target CIRCO benchmark. These results show that how composed intent is represented before retrieval is a consequential design choice, distinct from the choice of retrieval backbone itself.
Yihe Zhao, Songhe Feng
School of Computer Science and Technology, Beijing Jiaotong University, Beijing, 100044, China.