Square Superpixel Generation and Representation Learning via Granular Ball Computing

Shuyin Xia; Meng Yang; Dawei Dai; Fan Chen; Shilin Zhao; Junwei Han; Xinbo Gao; Guoyin Wang

cs.CVMar 31, 2026

Square Superpixel Generation and Representation Learning via Granular Ball Computing

Authors: Shuyin Xia, Meng Yang, Dawei Dai, Fan Chen, Shilin Zhao, Junwei Han, Xinbo Gao, Guoyin Wang,

Organizations: Shuyin Xia, Meng Yang, Dawei Dai, Fan Chen, Shilin Zhao and Junwei Han are with Chongqing University of Posts and Telecommunications, Chongqing, China. · Xinbo Gao and Wen Lu are with Xidian University, Xi’an, China. · Guoyin Wang is with Chongqing Normal University, Chongqing, China.

Abstract

Superpixels provide a compact region-based representation that preserves object boundaries and local structures, and have therefore been widely used in a variety of vision tasks to reduce computational cost. However, most existing superpixel algorithms produce irregularly shaped regions, which are not well aligned with regular operators such as convolutions. Consequently, superpixels are often treated as an offline preprocessing step, limiting parallel implementation and hindering end-to-end optimization within deep learning pipelines. Motivated by the adaptive representation and coverage property of granular-ball computing, we develop a square superpixel generation approach. Specifically, we approximate superpixels using multi-scale square blocks to avoid the computational and implementation difficulties induced by irregular shapes, enabling efficient parallel processing and learnable feature extraction. For each block, a purity score is computed based on pixel-intensity similarity, and high-quality blocks are selected accordingly. The resulting square superpixels can be readily integrated as graph nodes in graph neural networks (GNNs) or as tokens in Vision Transformers (ViTs), facilitating multi-scale information aggregation and structured visual representation. Experimental results on downstream tasks demonstrate consistent performance improvements, validating the effectiveness of the proposed method.

Figures & tables

Fig. 1: In contrast with SLIC, the first line depicts the input image, the second showcases the superpixel blocks generated by our method, and the last line displays those generated by the SLIC method.

Fig. 2: Overview of our superpixel generation approach: (a) Module for multi-scale superpixels; (b) Integration of the module into ViG for image classification; (c) Integration of the module into ViT for image-text retrieval; (d) Integration of the module into ViT for object detection.

Method	Time Complexity
Ncuts [ 11 ]	Superlinear in $N$ (global spectral/graph optimization)
ERS [ 12 ]	Typically superlinear or $O(N\log N)$ (solver-dependent)
SLIC	$O(NI)$
SEEDS [ 13 ]	$O(NI)$ (near-linear with iterative updates)
Watershed [ 14 ]	$O(N)$
TurboPixels [ 15 ]	$O(NI)$

TABLE I: Asymptotic time complexity comparison of representative superpixel methods.

N{=}HW

denotes the number of pixels,

I

is the number of iterations, and

L

is the number of coarse-to-fine stages.

Fig. 3: Our Method vs. SLIC on MNIST and CIFAR-10. Top: Our Method; Bottom: SLIC Method.

Model	Date	MNIST	CIFAR10
Model	Date	Test ACC	Test ACC
Vanilla GCN	ICLR 2017	90.70	55.71
GraphSage	NIPS 2017	97.31	65.76
MoNet	CVPR 2017	90.80	54.65
GatedGCN	Arxiv	97.34	67.31
GAT	ICLR 2018	95.53	64.22

TABLE II: Performance Comparison on MNIST and CIFAR10. The best results are boldfaced .

Dataset	Model	Image= $>$ Text			Text= $>$ Image
Dataset	Model	R@1	R@5	R@10	R@1	R@5	R@10
CelebA	FLIP	21.99	43.9	57.63	21.84	45.64	56.34
	FLIP*	22.64	43.96	57.77	22.42	46.12	56.48
	Ours	23.08	45.58	58.42	22.66	47.98	58.54
MM-CelebA	FLIP	7.63	19.15	26.06	8.85	21.5	28.86
	FLIP*	8.41	21.75	29.18	9.2	23.3	31.58

TABLE III: Performance Comparison on CelebA and MM-CelebA Zero-Shot Tasks. The best results are boldfaced .

Dataset	Model	Image= $>$ Text		Text= $>$ Image
Dataset	Model	R@5	R@10	R@5	R@10
CelebA	ALIGN	6.99	11.09	5.08	8.65
	BLIP	24.56	34.09	24.35	34.15
	CLIP	21.42	30.3	26.52	37.22
	FLIP	50.04	62.04	50.96	62.98
	Ours	50.12	62.36	50.78	62.2

TABLE IV: Performance Comparison on CelebA and MM-CelebA. All pretrained model backbones are frozen, with only the linear layer being fine-tuned. The best results are boldfaced .

Fig. 4: Visualization of multi-granularity token selection on COCO image. Left: Original image with objects at various scales (cows and pedestrians). Right: Selected tokens represented by square blocks of varying sizes. Our method adaptively assigns larger blocks to homogeneous background regions and smaller blocks to object-dense areas, effectively reducing tokens while preserving critical information for detection.

Model	Backbone	#Epochs	#Params (M)	GFLOPs	$AP^{val}$	$AP^{val}_{50}$	$AP^{val}_{75}$	$AP^{val}_{S}$	$AP^{val}_{M}$	$AP^{val}_{L}$
Real-time Object Detectors
YOLOv6-L	–	300	59	150	52.8	70.3	57.7	34.4	58.1	70.1
YOLOv7-L	–	300	36	104	51.2	69.7	55.5	35.2	55.9	66.7
YOLOv7-X	–	300	71	189	52.9	71.1	57.4	36.9	57.7	68.6
YOLOv8-L	–	–	43	165	52.9	69.8	57.5	35.3	58.3	69.8
YOLOv8-X	–	–	68	257	53.9	71.0	58.7	35.7	59.3	70.7

TABLE V: Performance comparison on COCO object detection. The best results are boldfaced .

Model	Tokens	Removed	AP	AP 50	AP 75	AP S	AP M	AP L
RT-DETR	400	0	53.1	71.3	57.7	34.8	58.0	70.0
Ours	350	50	52.5	70.6	56.5	34.2	56.6	69.8
Ours	300	100	52.3	70.4	56.6	35.4	56.7	69.9
Ours	200	200	52.3	70.3	56.6	33.4	56.3	69.8

TABLE VI: Performance Comparison on COCO Object Detection with Different Token Pruning Ratios. All models use RT-DETR-R50 backbone trained on COCO 2017. The best results are boldfaced .

Visual Representation Learning

Visual representation learning develops image and video features that capture useful structure for recognition, retrieval, spatial reasoning, and action.

See all 213 papers

H-SPAM: Hierarchical Superpixel Anything Model
Apr 13, 2026Julien Walther, Rémi Giraud, Michaël ClémentImage Segmentation
Univ. Bordeaux, CNRS, Bordeaux INP, IMS, UMR 5218, France · Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, France
Is an Image Also Worth 16x16=256 Superpixels? A Framework for Attentional Image Classification
May 26, 2026Pedro Henrique da Costa Avelar, Anderson R. Tavares, Luís C. LambGraph Attention Networks Vision Transformer
ufrgs · uom
Computer Vision with a Superpixelation Camera
Mar 27, 2026Sasidharan Mahalingam, Rachel Brown, Atul IngleImage Segmentation Computational Imaging
Portland State University · Willamette University

Apr 13, 2026J/K move · Enter open · S save

H-SPAM: Hierarchical Superpixel Anything Model

Julien Walther, Rémi Giraud, Michaël Clément

Univ. Bordeaux, CNRS, Bordeaux INP, IMS, UMR 5218, France · Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, France

Image Segmentation

Open paper AskPDF arXiv

Superpixels offer a compact image representation by grouping pixels into coherent regions. Recent methods have reached a plateau in terms of segmentation accuracy by generating noisy superpixel shapes. Moreover, most existing approaches produce a single fixed-scale partition that limits their use in vision pipelines that would benefit multi-scale representations. In this work, we introduce H-SPAM (Hierarchical Superpixel Anything Model), a unified framework for generating accurate, regular, and perfectly nested hierarchical superpixels. Starting from a fine partition, guided by deep features and external object priors, H-SPAM constructs the hierarchy through a two-phase region merging process that first preserves object consistency and then allows controlled inter-object grouping. The hierarchy can also be modulated using visual attention maps or user input to preserve important regions longer in the hierarchy. Experiments on standard benchmarks show that H-SPAM strongly outperforms existing hierarchical methods in both accuracy and regularity, while performing on par with most recent state-of-the-art non-hierarchical methods. Code and pretrained models are available: https://github.com/waldo-j/hspam.

Abstract

Visual Representation Learning

Explore similar work

H-SPAM: Hierarchical Superpixel Anything Model

Is an Image Also Worth 16x16=256 Superpixels? A Framework for Attentional Image Classification

Computer Vision with a Superpixelation Camera