Grinding and sanding are fundamental processes in industrial robotic surface finishing. However, physical trials are expensive and consume workpieces, making reproducible experiments difficult. We present RoboFin3D, a sim-to-real platform built on Isaac Sim and the Newton physics engine, that provides physics-based grinding and sanding simulation for cheap and repeatable robotic surface finishing experiments. RoboFin3D utilizes a signed distance field (SDF) to model the changing geometry of the workpiece, enabling contact computation, live updates and rendering without an intermediate mesh. It additionally uses a separate surface field to model progressive surface appearance change during sanding. We also introduce WeldGen, a weld sampling module, to generate weld beads on 8,918 real-world workpiece meshes for providing diverse simulation assets. The simulation parameters are calibrated on real experimental results and our evaluation demonstrates our simulation's fidelity against the real world. We also demonstrate that simulation-generated data can be used to improve the performance of perception models. Simulation-only fine-tuning of SAM2 improves IoU for segmentation of unsanded regions from 77.15% to 84.47%, while combined synthetic and real training reaches 97.41%.
Figures & tables
Fig. 1: RoboFin3D is able to model grinding material removal and sanding appearance changes. (a) and (b) are simulated and real grinding processes. (c) and (d) are simulated and real sanding processes.
Fig. 2: RoboFin3D workflow. WeldGen generates welds on workpieces. They are then used as both part of the training data for the perception model and assets in the simulator. Our perception model is fine-tuned using synthetic and real images. Real experimental results are used to calibrate the simulation, and the simulator can be used to evaluate a grinding or sanding control policy before real deployment.
Fig. 3: Simulation architecture. (a) Surface geometry represented by the SDF ϕ and surface appearance represented by the field D reside in a shared CUDA/Vulkan buffer. Newton supplies hydroelastic contacts for updates using Warp, while custom shaders in Isaac Sim’s RTX renderer query the fields directly. (b) SDF representation of the workpiece geometry. (c) Surface field representing the workpiece surface appearance.
Fig. 4: Measured and simulated height profiles of a workpiece after two grinding passes. Left: optical scans before processing and after each pass, with the sections used for comparison marked at X=20 , 70 , and 120 mm. Right: measured (Real) and simulated (Sim) height profiles at those sections. The simulated height profiles resemble the real experimental results.
Pass
Real (mm 3 )
Sim (mm 3 )
Error (%)
1 (calibration)
139.1
111.0
20.2
2 (evaluation)
294.8
216.1
26.7
TABLE I: Cumulative grinding removal. Absolute relative errors use measured volumes as the reference.
Fine-tuning data
IoU
Dice
Precision
Recall
None (pretrained)
77.15
86.97
78.94
97.34
Sim only
84.47
91.42
91.18
91.73
Sim + Real
97.41
98.68
98.59
98.79
TABLE II: Segmentation of unsanded regions on real test images. All values are percentages; higher is better.
Fig. 5: Qualitative results for segmentation of unsanded regions on a real workpiece. (a) Input RGB image; (b) pretrained SAM2; (c) Sim-only; (d) Sim + real. Teal overlays indicate predicted unsanded regions.