cs.LGMar 28, 2026
SaveGIFT: Bootstrapping Image-to-CAD Program Synthesis via Geometric Feedback
Organizations: AI Innovation Team, Red Hat · DeCoDE Lab, MIT · Core AI, IBM · MIT-IBM Watson AI Lab
Abstract
Generating executable CAD programs from images requires alignment between visual geometry and symbolic program representations, a capability that current methods fail to learn reliably as design complexity increases. Existing fine-tuning approaches rely on either limited supervised datasets or expensive post-training pipelines, resulting in brittle systems that restrict progress in generative CAD design. We argue that the primary bottleneck lies not in model or algorithmic capacity, but in the scarcity of diverse training examples that align visual geometry with program syntax. This limitation is especially acute because the collection of diverse and verified engineering datasets is both expensive and difficult to scale, constraining the development of robust generative CAD models. We introduce Geometric Inference Feedback Tuning (GIFT), a data augmentation framework that leverages geometric feedback to turn test-time compute into a bootstrapped set of high-quality training samples. GIFT combines two mechanisms: Soft-Rejection Sampling (GIFT-REJECT), which retains diverse high-fidelity programs beyond exact ground-truth matches, and Failure-Driven Augmentation (GIFT-FAIL), which converts near-miss predictions into synthetic training examples that improve robustness on challenging geometries. By amortizing inference-time search into the model parameters, GIFT captures the benefits of test-time scaling while reducing inference compute by 80%. It improves mean IoU by 12% over a strong supervised baseline and remains competitive with more complex multimodal systems, without requiring additional human annotation or specialized architectures.
Figures & tables
Figure 1: Efficiency vs. Performance. We compare Pass@k (test set IoU) across compute budgets. GIFT (green) matches the peak performance of the CAD-Coder-SFT baseline (orange) while using 80% less compute (requiring far fewer samples). GIFT outperforms both SFT and GIFT-REJECT at every compute level, demonstrating that self-training with geometric feedback significantly enhances image-conditional CAD generation. Results reflect mean IoU on the GenCAD test subset, including failure cases. An extended scaling analysis is provided in Appendix C (Figure 12 ).
Figure 2: Robustness Analysis. ( ) GIFT achieves superior accuracy (Mean/Median IoU). ( ) While all models degrade with increased task complexity (token length), GIFT maintains higher resilience than SFT. ( ) GIFT solves 53% more problems compared to the baseline, highlighting the benefit of diverse training data.
Figure 3: The CAD-Coder pipeline processes multimodal inputs (text prompts and images) to generate executable CAD code. This code is converted into STEP files via geometric alignment and validated against the ground truth using IoU metrics.
Figure 4: Comparison of Vision-Language Model generations: (a) standard text description vs. (b) executable CAD code.
Figure 5: Standard SFT Baseline. The model is trained on static image-code pairs using a next-token prediction objective.
Figure 6: Soft Rejection Sampling (SRS). We sample programs, verify them with a geometric kernel, and retain diverse valid programs ( ) to expand the training distribution beyond the single ground truth.
Figure 7: Failure-Driven Augmentation (FDA). We identify hard negatives (programs with ), render them into synthetic images ( ), and train the model to map these ’noisy’ inputs back to the correct ground truth code ( ).
| Exp | Condition | Budget | Input | Output | Mean |
|---|---|---|---|---|---|
| (Image) | (Code) | ||||
| IoU Top 10% | w/ IoU=1 | (8, 16, 32, 64, 128) | 4k | 169,116 | 0.734 |
| w/o IoU=1 | (8, 16, 32, 64, 128) | 4k | 134,345 | 0.665 | |
| IoU Best (Top1) | w/ IoU=1 | (8, 16, 32, 64, 128) | 4k | 3,911 | 0.839 |
| w/o IoU=1 | (8, 16, 32, 64, 128) | 4k | 2,619 | 0.760 |
Table 1: Performance comparison between Top-10% and Top-1 inference-time selections on the GenCAD training subset. We report the mean IoU obtained by retaining either the top 10% of generated samples or only the single best sample per input, with and without exact ground-truth matches.
Figure 8: IoU distributions for samples obtained via ITS, selecting the top 10% and top-1 programs per image. Panels show: (a) all samples; (b) all samples excluding ground truth (GT) matches; (c) best-performing samples; and (d) best samples excluding GT. The persistence of low-quality generations, even in the best-performing subset, highlights the need to amortize offline ITS feedback to improve base model reliability.
Figure 9: Analysis of inference-time IoU statistics. (a) Pass@N scaling shows that best-case performance improves with compute budget, while mean and median gains saturate more quickly. (b) The cumulative IoU distribution motivates the filtering thresholds used for SRS and FDA by separating low-quality failures from high-fidelity solutions
Figure 10: GIFT strategies impact. We compare the SFT baseline against GIFT-REJECT (targeting output diversity) and the full GIFT framework (combining diversity and robustness). The full GIFT method yields the largest performance gain, confirming the critical value of learning from hard negatives.
| Method | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| SFT | 0.698 | 0.725 | 0.743 | 0.763 | 0.780 | 0.788 | 0.792 | 0.801 | 0.804 | 0.807 |
| GIFT-REJECT | 0.732 | 0.745 | 0.762 | 0.773 | 0.785 | 0.796 | 0.802 | 0.807 | 0.810 | 0.812 |
| GIFT-FAIL | 0.761 | 0.780 | 0.789 | 0.791 | 0.799 | 0.802 | 0.802 | 0.803 | 0.803 | 0.806 |
| GIFT | 0.779 | 0.802 | 0.809 | 0.811 | 0.812 | 0.816 | 0.816 | 0.817 | 0.818 | 0.819 |
| +11.60 % | +10.53 % | +8.92 % | +6.31 % | +4.04 % | +3.52 % | +3.02 % | +2.01 % | +1.80 % | +1.56 % |
Table 2: Test-set IoU across increasing inference-time compute budgets for CAD-Coder-SFT, GIFT-REJECT, GIFT-FAIL, and the full GIFT model. The final row reports the relative improvement of GIFT over the SFT baseline at each budget.
Figure 11: Evaluation results on standard captioning metrics for SFT and GIFT variants.
| Method | pass@1 | pass@5 | pass@10 | Gap | |
|---|---|---|---|---|---|
| SFT | 0.698 | 0.776 | 0.807 | 15.5 % | - |
| GIFT-REJECT | 0.732 | 0.788 | 0.812 | 10.9 % | -29.6 % |
| GIFT-FAIL | 0.761 | 0.792 | 0.806 | 5.9 % | -61.9 % |
| GIFT | 0.777 | 0.812 | 0.820 | 5.2 % | -66.4 % |
Table 3: Mean IoU on the GenCAD test set and amortization gap between pass@1 and pass@k under different dataset-augmentation strategies. Smaller gaps indicate that model quality is better amortized into single-shot generation.
| Modality | IoU | |
| frontier zero-shot | ||
| GPT4o | IMG-s | 0.378 |
| Gemini-2.5-PRO | IMG-s | 0.451 |
| Gemini-3.0-PRO | IMG-s | 0.540 |
| open-source zero-shot | ||
| Qwen-2.5-72B ( Bai et al., 2025 ) | IMG-s | 0.345 |
Table 4: Comparison with recent modality-to-CAD systems trained on DeepCAD variant datasets under single-sample inference. Inputs include point clouds (PC), text (TXT), multi-view images (IMG), and single-view images (IMG-s).
| Method | IoU |
|---|---|
| SFT (LONG) | 0.698 |
| SFT w/ AUG | 0.710 |
| GIFT-REJECT | 0.742 |
| GIFT-REJECT w/ AUG | 0.745 |
| GIFT-FAIL | 0.761 |
| GIFT | 0.782 |
Table 5: Ablation of GIFT variants on the GenCAD test set. AUG denotes standard augmentation.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: Extended inference-time scaling comparison including GIFT-FAIL. Across all compute budgets, GIFT achieves the highest IoU, while GIFT-FAIL and GIFT-REJECT each improve over the SFT baseline. The full GIFT model matches the high-compute performance of SFT with substantially less inference-time sampling.
| Base Model | Sampler |
|---|---|
| QwenVL-2.5-7B | CAD-Coder-7B |
| QwenVL-2.5-3B | CAD-Coder-7B |
| CAD-Coder-7B | CAD-Coder-7B |
Table 6: Base-model ablation for GIFT. We compare the sampler and training backbone combinations used to generate augmented data and analyze how base-model choice affects downstream performance.
Figure 13: Base-model ablation across training phases. We compare GIFT variants bootstrapped with different base and sampler models at 5k, 10k, and 15k training steps. The benefits of GIFT persist across training progress.
Figure 14: Sensitivity to temperature sampling for GIFT and SFT. GIFT is more robust in particular for large temperatures.
| Valid Sample Rate (%) | Error Analysis (%) | ||||
|---|---|---|---|---|---|
| Method | Split | pass@1 | Failed Gen | Failed OCC | Not Solids |
| GIFT | in-distribution | 99.06 | 0.20 | 0.38 | 0.35 |
| out-of-distribution | 96.75 | 0.60 | 2.45 | 0.20 | |
| SFT | in-distribution | 98.75 | 0.34 | 0.60 | 0.31 |
| out-of-distribution | 97.60 | 1.18 | 1.18 | 0.05 | |
Table 7: Valid sample rate and error analysis for GIFT and CAD-Coder-SFT on in-distribution and out-of-distribution test sets. We report pass@1 validity along with failure categories including generation failures, OpenCASCADE failures, and not-solid outputs.
Figure 15: Preliminary out-of-distribution evaluation using a set of 400 real-world images, based on the data split provided by ( Doris et al., 2025 ) . The results show that GIFT effectively leverages inference-time compute to handle challenging, unseen inputs.
| Modality | IoU (mean) | IoU (median) | |
|---|---|---|---|
| cadrille-RL (w/o f) | PC | 0.912 | 0.974 |
| cadrille-RL (w/o f) | PC,IMG | 0.920 | 0.978 |
| cadrille-RL (w/ f) | PC | 0.820 | 0.966 |
| cadrille-RL (w/ f) | PC, IMG | 0.827 | 0.972 |
| CAD-Coder-SFT (w/ f) | IMG-s | 0.698 | 0.803 |
| CAD-Coder-GIFT (w/ f) | IMG-s | 0.782 | 0.948 |
Table 8: Comparison with recent multimodal and specialized CAD-generation systems. PC denotes point clouds, IMG multi-view images, and IMG-s single-view images; w/ f includes failed generations, while w/o f excludes them. Reporting both mean and median IoU highlights the sensitivity of average performance to failed executions.
Figure 16: Performance metrics under different data-augmentation strategies. We evaluate Top-K ( ) generations filtered by geometric validity ( against the ground truth) and compare accuracy, solved-problem rate, and performance as a function of task complexity. GIFT improves both geometric fidelity and robustness, with the largest gains appearing on more complex problems.
Figure 17: Complexity statistics for ground-truth and generated programs with high-fidelity. We compare the distributions of operation counts and token lengths across SFT, GIFT-REJECT, and GIFT. The results show that GIFT improves alignment with valid target structures.
| Model | Mode | Input Res | IoU (mean) | IoU (median) |
|---|---|---|---|---|
| Gemini-3-PRO | Zero-Shot | Low | 0.200 | 0.141 |
| Gemini-3-PRO | Zero-Shot | High | 0.540 | 0.560 |
| CAD-Coder | GIFT | - | 0.782 | 0.948 |
Table 9: Mean and median IoU on a subset of the GenCAD test set for zero-shot Gemini 3.0 Pro and the specialized CAD-Coder-GIFT model. Results are shown for different input resolutions to assess the sensitivity of frontier multimodal models to visual detail.
Figure 18: Distribution of Top-1 samples with .
| Budget | Inputs | Samples | ( , ) |
| 10,000 | 10,000 | ||
| 10,000 | 20,000 | ||
| 40,000 | 120,000 |
Table 10: Dataset budget mix and sampling hyperparameter configurations used during data generation. The Samples column reports the selected subset size retained for each compute budget after top-10% filtering.
Figure 19: IoU distribution of sampled programs before and after balancing. Balancing reduces over-representation of the densest score ranges and produces a more even distribution of training examples across quality levels, improving the usefulness of both SRS and FDA data.
Figure 20: Rendered examples of generated CAD programs with intermediate IoU quality ( and ). These visualizations show how geometrically imperfect but executable programs can be projected back into the image domain and reused as synthetic inputs for Failure-Driven Augmentation.
Figure 21: Examples of low-IoU but executable generations used for Failure-Driven Augmentation. The rendered failures provide structured hard negatives that train the model to recover the original ground-truth program from imperfect visual geometry.
Figure 22: Workflow comparison between standard SFT and GIFT. In the SFT pipeline, each input image is paired with a single ground-truth program. In GIFT, multiple sampled programs are geometrically verified, and both high-quality alternatives and structured failures are converted into additional supervised training signals.
Figure 23: Example of high IoU CAD generations used for SRS augmentation.
Figure 24: Example of high IoU CAD generations used for SRS augmentation.
Figure 25: Example of high IoU CAD generations used for SRS augmentation.
| Hyperparameter | Value |
|---|---|
| Model Architecture | |
| Base Model | Qwen-VL-2.5-7B-Instruct/Qwen-VL-2.5-3B-Instruct |
| Parameter Count | 7B/3B |
| Datasets | |
| Source Dataset | GenCAD-Code / DeepCAD |
| Training Size (Original SFT) | 163k |
Table 11: Experimental setup and implementation details for training, inference-time scaling, and GIFT-specific augmentation. We summarize model backbones, dataset sizes, sampling budgets, filtering thresholds, optimization settings, and hardware.
More in this topic
Synthetic Data Augmentation
Synthetic data augmentation creates or modifies training examples to improve learning when real data are scarce, imbalanced, costly, or sensitive.
See all 453 papers