ORGAN: Object-Centric Representation Learning using Cycle Consistent Generative Adversarial Networks

Joël Küchler; Ellen van Maren; Vaiva Vasiliauskaitė; Katarina Vulić; Reza Abbasi-Asl; Stephan J. Ihle

cs.CVMar 2, 2026

ORGAN: Object-Centric Representation Learning using Cycle Consistent Generative Adversarial Networks

Authors: Joël Küchler, Ellen van Maren, Vaiva Vasiliauskaitė, Katarina Vulić, Reza Abbasi-Asl, Stephan J. Ihle

Organizations: Laboratory of Biosensors and Bioelectronics, Institute for Biomedical Engineering, University and ETH Zurich, Gloriastrasse 37/39, Zurich, 8092, Switzerland · Department of Neurology, Insel Gruppe, Bern, Switzerland · Department of Neurology, Department of Bioengineering and Therapeutic Sciences, University of California, San Francisco, USA · Department of Neurobiology, University of Chicago, 951 E 58th St, Chicago, 60637, IL, USA ·

Abstract

Although data generation is often straightforward, extracting information from data is more difficult. Object-centric representation learning can extract information from images in an unsupervised manner. It does so by segmenting an image into its subcomponents: the objects. Each object is then represented in a low-dimensional latent space that can be used for downstream processing. Object-centric representation learning is dominated by autoencoder architectures (AEs). Here, we present ORGAN, a novel approach for object-centric representation learning, which is based on cycle-consistent Generative Adversarial Networks instead. We show that it performs similarly to other state-of-the-art approaches on synthetic datasets, while at the same time being the only approach tested here capable of handling more challenging real-world datasets with many objects and low visual contrast. Complementing these results, ORGAN creates expressive latent space representations that allow for object manipulation. Finally, we show that ORGAN scales well both with respect to the number of objects and the size of the images, giving it a unique edge over current state-of-the-art approaches.

Figures & tables

Figure 1: Model architecture and training losses. ORGAN is based on the Cycle-Consistent Generative Adversarial Network (CycleGAN) architecture Zhu et al. (2017) , adapted to use an image (left) and a list (right) domain. Two GANs are used to transform between the two domains. By cycling through both generators, the cycle-consistency losses $\mathcal{L}_{Cyc_{L}}$ and $\mathcal{L}_{Cyc_{I}}$ can be enforced.

Figure 2: Network architectures used in this work. (A) List generator architecture, which transforms images into lists. First, the image is divided into patches and a score function is used, which predicts for each patch how likely it is to contain an object. After non-maximum suppression, a differentiable top-k operator detects k patches that contain objects. The feature network then extracts the object’s features. (B) Image generator architecture, which transforms lists into images. First, the list is transformed into an image based on its location and features. For that, each feature vector $\eta_{i}$ is projected ( $\zeta$ ) onto a unit hypersphere. Afterwards, the vector is scaled by the object existence probability $\alpha$ . Then, a similar approach as presented in BlobGAN is used Epstein et al. (2022) to convert the projections into an image. After adding a noise channel to allow for richer backgrounds, a U-Net transforms the style of the image.

Figure 3: Examples of object detection performance for four different datasets. We show performance for ORGAN (ours), SPACE Lin et al. (2020) , SLATE Singh et al. (2021) , LSD Jiang et al. (2023) and SPOT Kakogeorgiou et al. (2024) . We present both the reconstruction and the object detection in the form of fixed bounding boxes for ORGAN and SPACE. The color code of the bounding box represents the extracted feature vector. For SLATE, LSD, and SPOT we present the recovered slot masks (color-coded). SPOT does not allow for image reconstruction.

	Tetrominoes		Multi-MNIST		Sprites		Cells
Method	F1	Train/Infer	F1	Train/Infer	F1	Train/Infer	F1	Train/Infer
ORGAN	89.2 $\pm$ 20.5	44.0 / 2.5	86.1 $\pm$ 3.9	39.0 / 3.5	88.1 $\pm$ 2.3	36.0 / 4.0	77.9 $\pm$ 8.6	31.5 / 3.4
SPACE	76.0 $\pm$ 15.7	13.5 / 17.8	88.7 $\pm$ 4.7	16.5 / 15.0	88.5 $\pm$ 4.3	4.08 / 29.6	0.0 $\pm$ 0.0	13.0 / 15.0
SLATE	30.3 $\pm$ 24.8	6.75 / 3.4	5.8 $\pm$ 7.6	0.75 / 3.4	12.4 $\pm$ 10.6	0.75 / 3.0	46.6 $\pm$ 9.3	0.75 / 3.5
LSD	20.0 $\pm$ 39.8	7.5 / 8.0	5.7 $\pm$ 28.0	16.0 / 4.0	6.4 $\pm$ 17.7	11.5 / 7.0	53.9 $\pm$ 11.9	10.5 / 7.0
SPOT	13.8 $\pm$ 44.6	10.0 / 37.5	5.2 $\pm$ 43.0	10.0 / 36.3	0.5 $\pm$ 21.2	10.0 / 32.8	24.8 $\pm$ 0.0	10.0 / 32.8

Table 1: F1-scores (in %) and training/inference times (hours/ms) for all models on 128

\times

128 images.

Figure 4: Latent space quality of ORGAN. (A) The latent space as encoded by $\eta$ is presented for the Sprites dataset. Two of the three dimensions are sampled at 12 equally spaced values across the feature space. The third dimension is kept fixed at its center. Each combination of feature values is given to the image generator. The corresponding objects are generated and visualized in separate sub-images. The feature space encodes the color, shape, and size of the object. (B) An input image of the Tetrominoes dataset is cycled, while the list elements are modified. Top: All objects were moved closer to the center. Bottom: The object properties were swapped between the three objects counterclockwise. (C) To compare how well object clustering works on the feature space, the Davies–Bouldin Index Davies and Bouldin (1979) was calculated for ORGAN and SPACE, which detected distinct objects for Sprites and Tetrominoes . Our method achieves superior separability.

Figure 5: Analysis of the generalization capabilities of the object detection accuracy. (A) Recall and precision are plotted for both ORGAN and SPACE for Sprites . The architecture of SPACE can only be applied to a fixed input size (here $256\times 256$ ). ORGAN was trained on $128\times 128$ and applied to $256\times 256$ . This training approach gives SPACE an advantage because it is trained directly at the evaluation resolution. Yet, ORGAN outperforms in precision, while SPACE has a higher recall. (B) ORGAN can also generalize to much larger images. Trained on patches of size $128\times 128$ , prediction results on a $768\times 768$ image of the Cells dataset are presented. The color code of the bounding box illustrates differences in extracted features $\eta$ .

Unsupervised Visual Concept Discovery

See all 13 papers

Slot-RAE: Streamlining Object-Centric Learning via Direct Representation Auto-Encoders
Jul 13, 2026Alexandre Chapin, Emmanuel Dellandrea, Liming ChenLatent Diffusion Models Image Generation
LIRIS, imagine
Are Object-Centric Representations Better At Compositional Generalization?
Feb 18, 2026Ferdinand Kapl, Amir Mohammad Karimi Mamaghan, Maximilian Seitzer +4Compositional Generalization Object-Centric Representation Learning
Technical University of Munich · Helmholtz AI, Munich · KTH Royal Institute of Technology +2
Mask-supervised Object-centric Representation Learning with LeJEPA
Jul 2, 2026Jakob Geusen, Ender KonukogluContrastive Learning Representation Learning
Biomedical Image Computing Group, ETH Zurich

Jul 13, 2026J/K move · Enter open · S save

Slot-RAE: Streamlining Object-Centric Learning via Direct Representation Auto-Encoders

Alexandre Chapin, Emmanuel Dellandrea, Liming Chen

LIRIS, imagine

Latent Diffusion Models Image Generation Object-Centric Representation Learning

Open paper AskPDF arXiv

Deploying object-centric models for real-world scene understanding typically requires complex pipelines to achieve both robust scene decomposition and high-fidelity generation. Recent diffusion-based approaches have improved visual quality, but they almost universally rely on heavy, pretrained generative priors (e.g., Stable Diffusion) and external VAE latent spaces. In this paper, we propose Slot-RAE, a much simpler, fully integrated framework that operates directly within the continuous semantic feature space of visual foundation models (e.g., DINOv3). Slot-RAE employs a feature-space diffusion process using a Diffusion Transformer (DiT) decoder and a Representation Alignment (REPA) head. Unlike existing diffusion-based objectcentric methods that rely heavily on subsidized text-toimage priors, the generative core of Slot-RAE (Slot Attention and the DiT) is trained from scratch within the frozen VFM feature space. This eliminates the need for VAE bottlenecks and task-agnostic generative pre-training. Experiments on the COCO dataset demonstrate that despite its architectural simplicity, Slot-RAE achieves state-of-the-art results. It delivers comparable unsupervised object discovery, higher-fidelity image reconstruction, and robust zero-shot compositionality, all while being significantly faster and more computationally efficient than existing object-centric latent diffusion models.

Abstract

Unsupervised Visual Concept Discovery

Explore similar work

Slot-RAE: Streamlining Object-Centric Learning via Direct Representation Auto-Encoders

Are Object-Centric Representations Better At Compositional Generalization?

Mask-supervised Object-centric Representation Learning with LeJEPA