Continual learning enables models to acquire new knowledge over time while retaining previously learned capabilities. However, its application to text-to-3D generation remains unexplored. We present ReConText3D, the first framework for continual text-to-3D generation. We first demonstrate that existing text-to-3D models suffer from catastrophic forgetting under incremental training. ReConText3D enables generative models to incrementally learn new 3D categories from textual descriptions while preserving the ability to synthesize previously seen assets. Our method constructs a compact and diverse replay memory through text-embedding k-Center selection, allowing representative rehearsal of prior knowledge without modifying the underlying architecture. To systematically evaluate continual text-to-3D learning, we introduce Toys4K-CL, a benchmark derived from the Toys4K dataset that provides balanced and semantically diverse class-incremental splits. Extensive experiments on the Toys4K-CL benchmark show that ReConText3D consistently outperforms all baselines across different generative backbones, maintaining high-quality generation for both old and new classes. To the best of our knowledge, this work establishes the first continual learning framework and benchmark for text-to-3D generation, opening a new direction for incremental 3D generative modeling. Project page is available at: https://mauk95.github.io/ReConText3D/.
Figures & tables
Figure 2: Overview of the ReConText3D framework. Base captions are encoded with CLIP and selected via k-center sampling under a count-aware budget to form replay memory R , which is combined with novel data to incrementally train GθB∪N while preserving base-class synthesis.
Method
CLIP (↑)
FD Incep (↓)
FD Point (↓)
Base
Novel
All
F (%)
Base
Novel
All
F (%)
Base
Novel
All
F (%)
TRELLIS-XL (Flow-based)
Base Training
29.60
–
–
-
73.04
–
–
-
75.22
–
–
-
Fine-tuning
24.46
29.79
27.12
17.36
101.00
75.33
56.66
38.18
72.01
69.05
70.13
–4.27
L2-SP
24.50
29.63
27.06
17.23
102.88
75.24
57.65
40.85
68.47
69.05
68.33
–8.98
Ours
28.41
29.86
29.14
4.02
87.28
75.34
52.42
19.50
70.20
70.28
69.78
–6.68
Table 1: Quantitative comparision on the Toys4K-CL benchmark. We report CLIP similarity (↑) and Fréchet Distance scores computed on Inception and PointNet++ features (↓). Forgetting (%) measures base-class degradation after novel training. Results are reported on TRELLIS-XL and Shap-E backbones. The best results are in bold, and the second-best are underlined.
Figure 3: Qualitative comparision of continual text-to-3D baselines on base class assets: From left to right: Ground Truth, Base Training, Fine-tuning, L2-SP, and our method. Results on TRELLIS are shown on the top, while Shap-E is shown on the bottom.
Figure 4: Ablation on replay memory size. Performance of our replay strategy under varying replay memory budgets/sizes on TRELLIS-XL and Shap-E backbones.
Figure 5: Ablation on replay strategy. comparision with random replay strategy on TRELLIS-XL and Shap-E backbones.
Figure 6: Qualitative comparision of novel class assets across baselines.
Split
Classes
Train
Test
Total
Base
45
1352
225
1577
Novel
45
1243
225
1468
Table 2: Toys4K-CL Benchmark Statistics.
Figure 7: Class distribution statistics for the Toys4K-CL benchmark (train and test). Both splits exhibit similar long-tailed behaviour.
Figure 8: Number of base class assets (replay examplers) returned by our count-aware budget allocation
Class
TRELLIS-XL
Shap-E
Base Training
Fine-tuning
L2-SP
L2SP + Ours
Ours
Base Training
Fine-tuning
L2-SP
L2SP + Ours
Ours
airplane
29.37
22.83
21.52
30.85
30.77
30.83
25.14
23.09
30.54
29.76
bicycle
27.21
18.12
19.04
25.88
25.36
27.18
26.17
23.33
25.40
26.19
bottle
32.96
28.54
24.38
30.66
29.75
31.52
31.37
31.30
30.53
31.18
bowl
33.50
30.97
31.38
33.94
33.78
33.86
32.81
32.81
33.68
33.85
cake
28.12
21.60
21.68
27.06
27.35
29.28
28.63
28.70
29.57
29.49
Table 3: Class-wise Evaluation (CLIP (↑)) on Base-Class assets for TRELLIS-XL and Shap-E. Best scores are highlighted in bold , whereas our ReConText3D results are highlighted in purple. Best scores are shown excluding Base Training.
Class
TRELLIS-XL
Shap-E
Fine-tuning
L2-SP
L2SP + Ours
Ours
Fine-tuning
L2-SP
L2SP + Ours
Ours
apple
33.14
34.09
32.03
33.35
32.17
31.56
32.27
32.37
ball
29.32
31.29
31.24
31.41
28.29
28.15
27.54
28.44
banana
33.03
33.00
33.15
33.03
32.23
31.79
32.66
33.14
boat
29.36
27.53
29.00
29.99
23.82
22.92
21.67
21.46
bread
30.65
30.86
30.35
30.75
27.42
26.68
26.10
26.01
Table 4: Class-wise Evaluation on Novel-Class Assets (CLIP (↑)) for both TRELLIS-XL and Shap-E. Best scores are highlighted in bold , whereas our ReConText3D results are highlighted in purple.
Figure 9: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on base class assets using TRELLIS-XL .
Figure 10: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on novel class assets using TRELLIS-XL .
Figure 11: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on base class assets using Shap-E .
Figure 12: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on novel class assets using Shap-E .