Regressing 3D rotations of objects from 2D images is a crucial yet challenging task, with broad applications in autonomous driving, virtual reality, and robotic control. Existing rotation regression models often rely on large amounts of labeled data for training or require additional information beyond 2D images, such as point clouds or CAD models. Therefore, exploring semi-supervised rotation regression using only a limited number of labeled 2D images is highly valuable. While recent work FisherMatch introduces semi-supervised learning to rotation regression, it suffers from rigid entropy-based pseudo-label filtering that fails to effectively distinguish between reliable and unreliable unlabeled samples. To address this limitation, we propose a hardness-aware curriculum learning framework that dynamically selects pseudo-labeled samples based on their difficulty, progressing from easy to complex examples. We introduce both multi-stage and adaptive curriculum strategies to replace fixed-threshold filtering with more flexible, hardness-aware mechanisms. Additionally, we present a novel structured data augmentation strategy specifically tailored for rotation estimation, which assembles composite images from augmented patches to introduce feature diversity while preserving critical geometric integrity. Comprehensive experiments on PASCAL3D+ and ObjectNet3D demonstrate that our method outperforms existing supervised and semi-supervised baselines, particularly in low-data regimes, validating the effectiveness of our curriculum learning framework and structured augmentation approach.
Figures & tables
Figure 1: The overall framework (HACMatch). (Middle) Semi-supervised Pipeline: The framework operates in two stages: supervised pre-training with labeled images, followed by semi-supervised learning where strongly augmented unlabeled images are fed to the student network while weakly augmented versions go to the teacher network. Curriculum pseudo-labeling selects reliable pseudo-labels {R^u} based on prediction entropy. (Left) Structured Data Augmentation (PoseMosaic): Given an original image xu , we apply n different augmentation transformations {Ai} to generate augmented views {x~u(i)} . These views are spatially assembled into an a×b grid where a⋅b=n , creating a composite image that preserves global structure while enhancing local diversity. (Right) Curriculum Learning Strategies: We propose two approaches: Multi-stage Curriculum Learning divides training into discrete stages with linearly increasing selection proportions using entropy H(R^u) ; Adaptive Curriculum Learning continuously adjusts the entropy threshold throughout training, offering smoother transitions.
Figure 2: The illustration of limitation of using a fixed SSL filtering threshold τ . τ=−0.0 means entropy-based filtering does not work. (a) The evolution of the Mask Ratio (the proportion of unlabeled samples in each training iteration whose entropy below τ ) for different fixed entropy thresholds ( τ ). For any given τ , the ratio stays within a narrow band. (b) Final model performance (Mean Med and ACC@30 ∘ ) across different τ values. Performance is insensitive to the choice of the fixed threshold.
Comparison Aspect
Multi-stage CL
Adaptive CL
Hyper-parameters
3 parameters
2 parameters
( αstart , αend , nstage )
( τstart , τend )
Core Mechanism
Adjusts the selection proportion ( αi )
Directly adjusts the entropy threshold ( τt )
of samples per batch.
for filtering.
Adjustment Schedule
Discrete, stepped
Continuous, linear
Proportion αi is fixed within each stage.
Threshold τt is updated every iteration.
Table 1: Comparison of Method Implementation Details for Multi-stage Curriculum Learning and Adaptive Curriculum Learning.
αi←αstart+nstage−1i−1(αend−αstart)
Algorithm 1 Stage-Specific Threshold Calculation for Multi-stage CL
Figure 3: Visualization of our proposed PoseMosaic method with n=4 . The original image (a) is reassembled from four independently augmented versions into grids of (b) 4 × 1, (c) 2 × 2, and (d) 1 × 4 patches.
Figure 4: Visualization of various image augmentation methods. Each technique introduces different visual perturbations, with varying impacts on the task of rotation estimation.
Figure 5: Performance comparison of 16 different augmentation strategies in a semi-supervised setting (PASCAL3D+, 5% labeled data). The dashed lines represent the baseline (‘raw’) performance. We selected methods with ACC@30 ∘ ¿ 76% (indicated by bold labels) for our final augmentation pool.
Method
Ratio of Labled Samples
5%
10%
20%
All
Sup.-Fisher
37.43 ± 0.57 /50.22 ± 0.62
32.97 ± 0.35 /55.87 ± 0.29
24.09 ± 0.42 /63.38 ± 0.65
12.63 ± 0.55 /83.76 ± 0.76
Sup.-Laplace
36.29 ± 0.55 /52.73 ± 0.68
28.03 ± 0.51 /60.13 ± 0.75
22.66 ± 0.64 /65.38 ± 0.57
11.21 ± 0.74 /84.22 ± 0.62
FisherMatch
16.48 ± 0.59 /74.73 ± 0.82
13.65 ± 0.56 /79.13 ± 0.64
11.42 ± 0.37 /84.07 ± 0.58
–/–
Ours(multi-stage)
14.29 ± 0.44 /78.90 ± 0.74
12.26 ± 0.49 /82.68 ± 0.56
10.73 ± 0.68 /85.19 ± 0.47
–/–
Ours(adaptive)
14.17 ± 0.36 / 79.24 ± 0.61
12.37 ± 0.43 / 83.25 ± 0.57
10.78 ± 0.52 / 85.69 ± 0.66
–/–
Table 2: The average 3D rotation regression performance ( Mean Med( ∘ ) ↓ /ACC@30 (%)∘↑ ) across all 12 categories on PASCAL3D+ Xiang et al. (2014) under different ratios of labeled training data. Each value is reported as the mean ± standard deviation, computed over three independent runs. The best results are marked as bold .
Method
Ratio of Labled Samples
5%
10%
20%
All
Sup.-Fisher
46.79 ± 0.67 /42.93 ± 0.63
44.85 ± 0.47 /44.49 ± 0.71
40.68 ± 0.63 /48.12 ± 0.38
27.09 ± 0.52 /59.34 ± 0.65
Sup.-Laplace
45.97 ± 0.45 /44.22 ± 0.51
44.38 ± 0.53 /46.16 ± 0.42
40.18 ± 0.62 /49.71 ± 0.39
25.76 ± 0.57 /61.67 ± 0.73
FisherMatch
41.19 ± 0.54 /48.02 ± 0.78
36.18 ± 0.57 /52.59 ± 0.62
33.17 ± 0.67 /55.43 ± 0.68
–/–
Ours(multi-stage)
38.14 ± 0.64 /51.79 ± 0.47
35.17 ± 0.52 /53.38 ± 0.77
32.01 ± 0.42 /55.82 ± 0.65
–/–
Ours(adaptive)
37.26 ± 0.51 / 52.45 ± 0.58
34.89 ± 0.53 / 53.93 ± 0.66
31.58 ± 0.69 / 55.97 ± 0.59
–/–
Table 3: The average 3D rotation regression performance ( Mean Med( ∘ ) ↓ /ACC@30 ∘(%)↑ ) across all 100 categories on ObjectNet3D Xiang et al. (2016) under different ratios of labeled training data. Each value is reported as the mean ± standard deviation, computed over three independent runs. The best results are marked as bold .
Aug
CL-M
CL-A
Mean Med ↓
ACC@30 ∘ ↑
16.54
74.68
✓
15.17
76.86
✓
15.02
76.84
✓
14.31
78.74
✓
✓
14.22
79.04
✓
✓
14.17
79.24
Table 4: The model performance with different Curriculum Pseudo Labeling strategies.
Hard_Aug
Mean Med ↓
ACC@30 ∘ ↑
–
16.54
74.68
CutOut
16.84
75.20
CutMix
16.21
75.68
CutOcc
16.02
75.47
YOCO
15.31
76.66
Ours(all)
15.11
77.35
Table 5: The model’s performance with different advanced strong augmentations.
Figure 6: Effect of the number of patch splits ( n ) on model performance, with αstart fixed at 65 and αend fixed at 95.
Figure 7: Ablation study on the hyperparameters of the multi-stage curriculum learning strategy. All experiments were conducted on PASCAL3D+ with 5% labeled data, and no strong augmentation is applied. (a) Impact of the starting sample proportion ( αstart ), with αend fixed at 100 and nstage=4 . (b) Impact of the ending sample proportion ( αend ), with αstart fixed at 65 and nstage=4 . (c) Impact of the number of curriculum stages ( nstage ), with αstart=65 , αend=95 , and no strong augmentation.
Figure 8: Illustration of incompatible augmentation methods in PoseMosaic.
Numbers of Patches
1
2
3
4
5
6
Training Time(min)
394
401
409
418
434
440
Table 6: Training time (in minutes) under different numbers of patches when applying PoseMosaic.
Figure 9: Effect of the initial threshold ( τstart ) on performance in our adaptive CL strategy.
College of Mathematics and Systems Science, Shandong University of Science and Technology, Qingdao, China · School of Artificial Intelligence, Shandong Women’s University, Jinan, China
School of Automation and Intelligent Sensing and Institute of Medical Robotics, Shanghai Jiao Tong University, and Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China.