Regressing 3D rotations of objects from 2D images is a crucial yet challenging task, with broad applications in autonomous driving, virtual reality, and robotic control. Existing rotation regression models often rely on large amounts of labeled data for training or require additional information beyond 2D images, such as point clouds or CAD models. Therefore, exploring semi-supervised rotation regression using only a limited number of labeled 2D images is highly valuable. While recent work FisherMatch introduces semi-supervised learning to rotation regression, it suffers from rigid entropy-based pseudo-label filtering that fails to effectively distinguish between reliable and unreliable unlabeled samples. To address this limitation, we propose a hardness-aware curriculum learning framework that dynamically selects pseudo-labeled samples based on their difficulty, progressing from easy to complex examples. We introduce both multi-stage and adaptive curriculum strategies to replace fixed-threshold filtering with more flexible, hardness-aware mechanisms. Additionally, we present a novel structured data augmentation strategy specifically tailored for rotation estimation, which assembles composite images from augmented patches to introduce feature diversity while preserving critical geometric integrity. Comprehensive experiments on PASCAL3D+ and ObjectNet3D demonstrate that our method outperforms existing supervised and semi-supervised baselines, particularly in low-data regimes, validating the effectiveness of our curriculum learning framework and structured augmentation approach.
Figures & tables
Figure 1: The overall framework (HACMatch). (Middle) Semi-supervised Pipeline: The framework operates in two stages: supervised pre-training with labeled images, followed by semi-supervised learning where strongly augmented unlabeled images are fed to the student network while weakly augmented versions go to the teacher network. Curriculum pseudo-labeling selects reliable pseudo-labels {R^u} based on prediction entropy. (Left) Structured Data Augmentation (PoseMosaic): Given an original image xu , we apply n different augmentation transformations {Ai} to generate augmented views {x~u(i)} . These views are spatially assembled into an a×b grid where a⋅b=n , creating a composite image that preserves global structure while enhancing local diversity. (Right) Curriculum Learning Strategies: We propose two approaches: Multi-stage Curriculum Learning divides training into discrete stages with linearly increasing selection proportions using entropy H(R^u) ; Adaptive Curriculum Learning continuously adjusts the entropy threshold throughout training, offering smoother transitions.
Figure 2: The illustration of limitation of using a fixed SSL filtering threshold τ . τ=−0.0 means entropy-based filtering does not work. (a) The evolution of the Mask Ratio (the proportion of unlabeled samples in each training iteration whose entropy below τ ) for different fixed entropy thresholds ( τ ). For any given τ , the ratio stays within a narrow band. (b) Final model performance (Mean Med and ACC@30 ∘ ) across different τ values. Performance is insensitive to the choice of the fixed threshold.
Comparison Aspect
Multi-stage CL
Adaptive CL
Hyper-parameters
3 parameters
2 parameters
( αstart , αend , nstage )
( τstart , τend )
Core Mechanism
Adjusts the selection proportion ( αi )
Directly adjusts the entropy threshold ( τt )
of samples per batch.
for filtering.
Adjustment Schedule
Discrete, stepped
Continuous, linear
Proportion αi is fixed within each stage.
Threshold τt is updated every iteration.
Table 1: Comparison of Method Implementation Details for Multi-stage Curriculum Learning and Adaptive Curriculum Learning.
αi←αstart+nstage−1i−1(αend−αstart)
Algorithm 1 Stage-Specific Threshold Calculation for Multi-stage CL
Figure 3: Visualization of our proposed PoseMosaic method with n=4 . The original image (a) is reassembled from four independently augmented versions into grids of (b) 4 × 1, (c) 2 × 2, and (d) 1 × 4 patches.
Figure 4: Visualization of various image augmentation methods. Each technique introduces different visual perturbations, with varying impacts on the task of rotation estimation.
Figure 5: Performance comparison of 16 different augmentation strategies in a semi-supervised setting (PASCAL3D+, 5% labeled data). The dashed lines represent the baseline (‘raw’) performance. We selected methods with ACC@30 ∘ ¿ 76% (indicated by bold labels) for our final augmentation pool.
Method
Ratio of Labled Samples
5%
10%
20%
All
Sup.-Fisher
37.43 ± 0.57 /50.22 ± 0.62
32.97 ± 0.35 /55.87 ± 0.29
24.09 ± 0.42 /63.38 ± 0.65
12.63 ± 0.55 /83.76 ± 0.76
Sup.-Laplace
36.29 ± 0.55 /52.73 ± 0.68
28.03 ± 0.51 /60.13 ± 0.75
22.66 ± 0.64 /65.38 ± 0.57
11.21 ± 0.74 /84.22 ± 0.62
FisherMatch
16.48 ± 0.59 /74.73 ± 0.82
13.65 ± 0.56 /79.13 ± 0.64
11.42 ± 0.37 /84.07 ± 0.58
–/–
Ours(multi-stage)
14.29 ± 0.44 /78.90 ± 0.74
12.26 ± 0.49 /82.68 ± 0.56
10.73 ± 0.68 /85.19 ± 0.47
–/–
Ours(adaptive)
14.17 ± 0.36 / 79.24 ± 0.61
12.37 ± 0.43 / 83.25 ± 0.57
10.78 ± 0.52 / 85.69 ± 0.66
–/–
Table 2: The average 3D rotation regression performance ( Mean Med( ∘ ) ↓ /ACC@30 (%)∘↑ ) across all 12 categories on PASCAL3D+ Xiang et al. (2014) under different ratios of labeled training data. Each value is reported as the mean ± standard deviation, computed over three independent runs. The best results are marked as bold .
Method
Ratio of Labled Samples
5%
10%
20%
All
Sup.-Fisher
46.79 ± 0.67 /42.93 ± 0.63
44.85 ± 0.47 /44.49 ± 0.71
40.68 ± 0.63 /48.12 ± 0.38
27.09 ± 0.52 /59.34 ± 0.65
Sup.-Laplace
45.97 ± 0.45 /44.22 ± 0.51
44.38 ± 0.53 /46.16 ± 0.42
40.18 ± 0.62 /49.71 ± 0.39
25.76 ± 0.57 /61.67 ± 0.73
FisherMatch
41.19 ± 0.54 /48.02 ± 0.78
36.18 ± 0.57 /52.59 ± 0.62
33.17 ± 0.67 /55.43 ± 0.68
–/–
Ours(multi-stage)
38.14 ± 0.64 /51.79 ± 0.47
35.17 ± 0.52 /53.38 ± 0.77
32.01 ± 0.42 /55.82 ± 0.65
–/–
Ours(adaptive)
37.26 ± 0.51 / 52.45 ± 0.58
34.89 ± 0.53 / 53.93 ± 0.66
31.58 ± 0.69 / 55.97 ± 0.59
–/–
Table 3: The average 3D rotation regression performance ( Mean Med( ∘ ) ↓ /ACC@30 ∘(%)↑ ) across all 100 categories on ObjectNet3D Xiang et al. (2016) under different ratios of labeled training data. Each value is reported as the mean ± standard deviation, computed over three independent runs. The best results are marked as bold .
Aug
CL-M
CL-A
Mean Med ↓
ACC@30 ∘ ↑
16.54
74.68
✓
15.17
76.86
✓
15.02
76.84
✓
14.31
78.74
✓
✓
14.22
79.04
✓
✓
14.17
79.24
Table 4: The model performance with different Curriculum Pseudo Labeling strategies.
Hard_Aug
Mean Med ↓
ACC@30 ∘ ↑
–
16.54
74.68
CutOut
16.84
75.20
CutMix
16.21
75.68
CutOcc
16.02
75.47
YOCO
15.31
76.66
Ours(all)
15.11
77.35
Table 5: The model’s performance with different advanced strong augmentations.
Figure 6: Effect of the number of patch splits ( n ) on model performance, with αstart fixed at 65 and αend fixed at 95.
Figure 7: Ablation study on the hyperparameters of the multi-stage curriculum learning strategy. All experiments were conducted on PASCAL3D+ with 5% labeled data, and no strong augmentation is applied. (a) Impact of the starting sample proportion ( αstart ), with αend fixed at 100 and nstage=4 . (b) Impact of the ending sample proportion ( αend ), with αstart fixed at 65 and nstage=4 . (c) Impact of the number of curriculum stages ( nstage ), with αstart=65 , αend=95 , and no strong augmentation.
Figure 8: Illustration of incompatible augmentation methods in PoseMosaic.
Numbers of Patches
1
2
3
4
5
6
Training Time(min)
394
401
409
418
434
440
Table 6: Training time (in minutes) under different numbers of patches when applying PoseMosaic.
Figure 9: Effect of the initial threshold ( τstart ) on performance in our adaptive CL strategy.
Point cloud registration critically depends on local features that are both distinctive and robust to arbitrary 3D rotations. Existing learning-based methods typically approximate rotation invariance via fragile local reference frames or extensive data augmentation, providing only empirical invariance and often degrading under unseen rotational transformations. In this paper, we propose SHReg, a strictly rotation-equivariant point cloud registration framework grounded in the representation theory of SO(3). By representing local geometric features as irreducible representations of SO(3), SHReg guarantees exact equivariance under arbitrary rotations without relying on local reference frames. Built upon a spherical-harmonics-based equivariant backbone, SHReg jointly learns rotation-invariant descriptors for robust correspondence matching and rotation-equivariant features that preserve fine-grained orientation information. The preserved equivariant structure enables each correspondence to directly hypothesize a rigid transformation, reducing reliance on large-scale hypothesis sampling in conventional RANSAC-based pipelines and leading to improved robustness under challenging rotational variations. Extensive experiments on 3DMatch, 3DLoMatch, and KITTI demonstrate that SHReg consistently outperforms state-of-the-art methods in registration accuracy, particularly under large rotational perturbations.
Chongjian Wang, Junjie Gao
College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao, China · School of Artificial Intelligence, Shandong Women’s University, Jinan, China
We consider monocular 3D human pose estimation (HPE), where the goal is to predict 3D human skeletal joints from a single 2D image, typically via 2D keypoint detection followed by 2D-to-3D lifting. Despite their success, we find that current lifting models exhibit strong performance degradation under rotations. We systematically study rotation equivariance across lifting architectures with increasing degrees of equivariant inductive bias, and investigate whether equivariance should be enforced by architectural design or learned from data, given the inherent ambiguity of monocular 2D-to-3D lifting. Utilising common HPE benchmarks, we demonstrate that rotation equivariance can be effectively learned via rotation-based data augmentation applied jointly to input and output poses. Compared with non-augmented models, this reduces error on rotated poses by over 30% across the benchmarks, with reductions of up to 72%. Moreover, augmentation-based models achieve up to 15% lower error than methods that are fully equivariant by design, while providing up to 37× faster inference. We further show that these findings generalise to real-world human pose sequences involving full-body rotations and to a state-of-the-art HPE model.
Pavlo Melnyk, Cuong Le, Urs Waldmann +2
CVL, Linköping University · Independent Researcher
3D object detection is essential for autonomous driving and robotic perception, yet its reliance on large-scale manually annotated data limits scalability and adaptability. To reduce annotation dependency, unsupervised and sparsely-supervised paradigms have emerged. However, they face intertwined challenges: low-quality pseudo-labels, unstable feature mining, and a lack of a unified training framework. This paper proposes SPL, a unified training framework for both unsupervised and sparsely-supervised 3D object detection via Semantic Pseudo-labeling and prototype Learning. SPL first generates high-quality pseudo-labels by integrating image semantics, point cloud geometry, and temporal cues, producing both 3D bounding boxes for dense objects and 3D point labels for sparse ones. These pseudo-labels are not used directly but as probabilistic priors within a novel, multi-stage prototype learning strategy. This strategy stabilizes feature representation learning through memory-based initialization and momentum-based prototype updating, effectively mining features from both labeled and unlabeled data. Extensive experiments on KITTI and nuScenes datasets demonstrate that SPL significantly outperforms state-of-the-art methods in both settings. Our work provides a robust and generalizable solution for learning 3D object detectors with minimal or no manual annotations. Our code is available at https://github.com/TossherO/SPL.
Yushen He, Lei Zhao, Weidong Chen
School of Automation and Intelligent Sensing and Institute of Medical Robotics, Shanghai Jiao Tong University, and Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China.