Continual learning enables models to acquire new knowledge over time while retaining previously learned capabilities. However, its application to text-to-3D generation remains unexplored. We present ReConText3D, the first framework for continual text-to-3D generation. We first demonstrate that existing text-to-3D models suffer from catastrophic forgetting under incremental training. ReConText3D enables generative models to incrementally learn new 3D categories from textual descriptions while preserving the ability to synthesize previously seen assets. Our method constructs a compact and diverse replay memory through text-embedding k-Center selection, allowing representative rehearsal of prior knowledge without modifying the underlying architecture. To systematically evaluate continual text-to-3D learning, we introduce Toys4K-CL, a benchmark derived from the Toys4K dataset that provides balanced and semantically diverse class-incremental splits. Extensive experiments on the Toys4K-CL benchmark show that ReConText3D consistently outperforms all baselines across different generative backbones, maintaining high-quality generation for both old and new classes. To the best of our knowledge, this work establishes the first continual learning framework and benchmark for text-to-3D generation, opening a new direction for incremental 3D generative modeling. Project page is available at: https://mauk95.github.io/ReConText3D/.
Figures & tables
Figure 2: Overview of the ReConText3D framework. Base captions are encoded with CLIP and selected via k-center sampling under a count-aware budget to form replay memory R , which is combined with novel data to incrementally train GθB∪N while preserving base-class synthesis.
Method
CLIP (↑)
FD Incep (↓)
FD Point (↓)
Base
Novel
All
F (%)
Base
Novel
All
F (%)
Base
Novel
All
F (%)
TRELLIS-XL (Flow-based)
Base Training
29.60
–
–
-
73.04
–
–
-
75.22
–
–
-
Fine-tuning
24.46
29.79
27.12
17.36
101.00
75.33
56.66
38.18
72.01
69.05
70.13
–4.27
L2-SP
24.50
29.63
27.06
17.23
102.88
75.24
57.65
40.85
68.47
69.05
68.33
–8.98
Ours
28.41
29.86
29.14
4.02
87.28
75.34
52.42
19.50
70.20
70.28
69.78
–6.68
Table 1: Quantitative comparision on the Toys4K-CL benchmark. We report CLIP similarity (↑) and Fréchet Distance scores computed on Inception and PointNet++ features (↓). Forgetting (%) measures base-class degradation after novel training. Results are reported on TRELLIS-XL and Shap-E backbones. The best results are in bold, and the second-best are underlined.
Figure 3: Qualitative comparision of continual text-to-3D baselines on base class assets: From left to right: Ground Truth, Base Training, Fine-tuning, L2-SP, and our method. Results on TRELLIS are shown on the top, while Shap-E is shown on the bottom.
Figure 4: Ablation on replay memory size. Performance of our replay strategy under varying replay memory budgets/sizes on TRELLIS-XL and Shap-E backbones.
Figure 5: Ablation on replay strategy. comparision with random replay strategy on TRELLIS-XL and Shap-E backbones.
Figure 6: Qualitative comparision of novel class assets across baselines.
Split
Classes
Train
Test
Total
Base
45
1352
225
1577
Novel
45
1243
225
1468
Table 2: Toys4K-CL Benchmark Statistics.
Figure 7: Class distribution statistics for the Toys4K-CL benchmark (train and test). Both splits exhibit similar long-tailed behaviour.
Figure 8: Number of base class assets (replay examplers) returned by our count-aware budget allocation
Class
TRELLIS-XL
Shap-E
Base Training
Fine-tuning
L2-SP
L2SP + Ours
Ours
Base Training
Fine-tuning
L2-SP
L2SP + Ours
Ours
airplane
29.37
22.83
21.52
30.85
30.77
30.83
25.14
23.09
30.54
29.76
bicycle
27.21
18.12
19.04
25.88
25.36
27.18
26.17
23.33
25.40
26.19
bottle
32.96
28.54
24.38
30.66
29.75
31.52
31.37
31.30
30.53
31.18
bowl
33.50
30.97
31.38
33.94
33.78
33.86
32.81
32.81
33.68
33.85
cake
28.12
21.60
21.68
27.06
27.35
29.28
28.63
28.70
29.57
29.49
Table 3: Class-wise Evaluation (CLIP (↑)) on Base-Class assets for TRELLIS-XL and Shap-E. Best scores are highlighted in bold , whereas our ReConText3D results are highlighted in purple. Best scores are shown excluding Base Training.
Class
TRELLIS-XL
Shap-E
Fine-tuning
L2-SP
L2SP + Ours
Ours
Fine-tuning
L2-SP
L2SP + Ours
Ours
apple
33.14
34.09
32.03
33.35
32.17
31.56
32.27
32.37
ball
29.32
31.29
31.24
31.41
28.29
28.15
27.54
28.44
banana
33.03
33.00
33.15
33.03
32.23
31.79
32.66
33.14
boat
29.36
27.53
29.00
29.99
23.82
22.92
21.67
21.46
bread
30.65
30.86
30.35
30.75
27.42
26.68
26.10
26.01
Table 4: Class-wise Evaluation on Novel-Class Assets (CLIP (↑)) for both TRELLIS-XL and Shap-E. Best scores are highlighted in bold , whereas our ReConText3D results are highlighted in purple.
Figure 9: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on base class assets using TRELLIS-XL .
Figure 10: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on novel class assets using TRELLIS-XL .
Figure 11: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on base class assets using Shap-E .
Figure 12: Qualitative comparison of continual text-to-3D baselines against ReConText3D (Ours) on novel class assets using Shap-E .
While text-to-3D generation has advanced rapidly, achieving high geometric fidelity at low inference cost remains challenging. Existing text-to-3D methods either decode discrete shape tokens autoregressively or iteratively refine global 3D representations with diffusion or flow models. However, autoregressive decoding is sequential and cannot revise errors, whereas diffusion and flow-matching models repeatedly process the full representation, making high-quality generation increasingly expensive. In this paper, we propose Block3D, a block-wise diffusion framework that partitions the discrete shape-token sequence into contiguous blocks, generates the blocks autoregressively, and jointly denoises all tokens within the current block. To alleviate error accumulation, we introduce confidence-guided intra-block correction, which revises low-confidence tokens before each block is finalized. On a held-out set from TRELLIS-500K, Block3D reduces mean end-to-end generation time from 25.71 seconds to 4.99 seconds, achieving a 5.15× speedup over the fine-tuned autoregressive baseline without sacrificing geometric fidelity.
Bowen Cui, Weijie Wang, Zeyu Zhang +7
ZIP Lab, Zhejiang University · Monash University · University of Adelaide
Understanding and generating 3D objects as compositions of meaningful parts is fundamental to human perception and reasoning. However, most text-to-3D methods overlook the semantic and functional structure of parts. While recent part-aware approaches introduce decomposition, they remain largely geometry-focused, lacking semantic grounding and failing to model how parts align with textual descriptions or their inter-part relations. We propose DreamPartGen, a framework for semantically grounded, part-aware text-to-3D generation. DreamPartGen introduces Duplex Part Latents (DPLs) that jointly model each part's geometry and appearance, and Relational Semantic Latents (RSLs) that capture inter-part dependencies derived from language. A synchronized co-denoising process enforces mutual geometric and semantic consistency, enabling coherent, interpretable, and text-aligned 3D synthesis. Across multiple benchmarks, DreamPartGen delivers state-of-the-art performance in geometric fidelity and text-shape alignment.
Current text-to-3D generation methods excel in natural scenes but struggle with industrial applications due to two critical limitations: domain adaptation challenges where conventional LoRA fusion causes knowledge interference across categories, and geometric reasoning deficiencies where pairwise consistency constraints fail to capture higher-order structural dependencies essential for precision manufacturing. We propose a novel framework named ForgeDreamer addressing both challenges through two key innovations. First, we introduce a Multi-Expert LoRA Ensemble mechanism that consolidates multiple category-specific LoRA models into a unified representation, achieving superior cross-category generalization while eliminating knowledge interference. Second, building on enhanced semantic understanding, we develop a Cross-View Hypergraph Geometric Enhancement approach that captures structural dependencies spanning multiple viewpoints simultaneously. These components work synergistically improved semantic understanding, enables more effective geometric reasoning, while hypergraph modeling ensures manufacturing-level consistency. Extensive experiments on a custom industrial dataset demonstrate superior semantic generalization and enhanced geometric fidelity compared to state-of-the-art approaches. Code is available at https://github.com/Junhaocai27/ForgeDreamer
Junhao Cai, Deyu Zeng, Junhao Pang +3
Shenzhen University, Shenzhen, Guangdong 518060, China · Guangzhou Maritime University, Guangzhou, Guangdong 510725, China