Different shape representations facilitate different computations. Surface representations, in particular meshes, are often used for modeling, whereas volume representations are useful for spatial queries such as intersection or containment. Optimizing a surface representation based on a volumetric properties by gradient descent requires the derivatives of the volume relative to its bounding surface. We derive this gradient for winding numbers and show that it can be efficiently computed for volumetric values sampled on a regular grid (voxel representation) and surface parameters based on vertex sets (triangle meshes). This enables an efficient solution for a variety of optimization problems. We demonstrate the practical use of this approach at the examples of deforming meshes to resolve intersections, being manufacturable by cutting with a bandsaw from three directions, and creating shapes that are close to tiling 3D space.
Figures & tables
Figure 1. We present an efficient technique for differentiating voxel values with respect to the surface that generates the voxel grid. This enables the solution of various problems that can easily be solved on regular voxel grids, such as intersection detection, for surface representations like triangle meshes. The technique can be readily integrated into gradient-based optimization frameworks like PyTorch ( Paszke et al., 2019 ) . Moving the highlighted vertex (left) in the indicated direction changes the voxelization of all adjacent triangles (right). We discuss various applications, like a) the resolving of self-intersections, b) optimized cuts for manufacturing with bandsaws, and c) shape deformation to form repeatable tilings that fill the R3 .
Figure 2. We can differentiate voxelizations, i.e., point-sampled winding numbers, for arbitrary surfaces. This example shows piecewise linear surfaces and circles in 2D. Our derivatives match the finite difference reference. This example uses Monte Carlo integration and a box filter of diameter 3.
Figure 3. The winding number field w of a surface depicted in the left-most box is the sum of the four patch-wise contributions wi . Each wi defines for every point in space how often the patch pi (blue) is intersected in direction ±d . As the ray intersects the patch at most once, wi is piecewise constant and either −1,1 or 0 depending on the normal relative to the ground plane.
Figure 4. Given a polygon, the integrated winding numbers in each voxel are updated whenever a simplex intersects a voxel above the voxel. The volume of the intersection of the signed shadow solid volume (green for positive and red for negative contribution) and a voxel (small boxes) can be computed as described in Sec. 3.3 and added to the value of each voxel, resulting in an update of all voxel values ‘below’ the simplex segment. Therefore, the integrated winding number of all voxels in the highlighted grid column can be computed as the sum of ‘shadow’ contributions of the five blue segments.
Figure 5. For high grid resolutions (up to 1024 voxels in each dimension), the runtime stays below a second even for meshes over 100,000 faces for voxelization as well as the gradient computation. All timings are averaged over 30 repetitions and were computed on an Intel i9-12900K CPU. The number of faces is shown in parentheses.
Figure 6. Luo et al. (2024) propose differentiable voxelization by automatically differentiating through a (non-smoothed) winding number ( DOPH ). Since the winding number is piecewise constant, the derivatives of their voxelization are zero if computed numerically with finite differences (FD), and their automatic derivatives are only spuriously non-zero very close to the surface. In contrast, our technique ( DVX ) is based on the smoothed winding number, which leads to expected numerical derivatives that match our results exactly.
Figure 7. Differentiable voxelization allows resolving self-intersections by an ARAP deformation of the input shape.
Figure 8. Left: Wooden figure that was created by cutting along handmade cut silhouettes ( Gauthier, 2019 ) (used with permission). Right: The cut out surface is defined as the intersection of extruded silhouettes.
Figure 9. The energy Ecut measures the volume difference between the deformed mesh Vˉ and the deformed cut out shape S . Larger vertex distances of the deformed mesh Vˉ to V are shown in red (left). W is the discretized shape of Vˉ where yellow voxels have high volume and green voxels have low volume. Voxels with volume lower than 10−3 are hidden. W is projected to the silhouette shapes Sx,Sy and Sz that are used to generate the cut shape S . For visualization of the volume difference ( S−W ), we highlight all voxels in red that are part of the shape Sijk>0.5 but not in Wijk>0.5 .
Figure 10. When optimized (bottom row), much more material that does not belong to the actual shape can be removed by cutting, when compared to the cut-out shape for meshes without optimization (top row). All voxels of Sijk>0.5 that do not belong to Wijk>0.5 are highlighted in red. The color of the optimized triangle meshes shows the difference to the initial meshes. Gray voxels are part of the voxelization of the triangle meshes and red voxels are additional voxels that are not removed by cutting. The relative improvements denote how much unnecessary material of the initial meshes can be removed with the optimization ( 1−Ecut(Vˉ)/Ecut(V) ). Minimal deformations, such as straightening the fingers of the hand or adjusting the tail and ears of Tweety and Bunny, significantly improve the shapes’ cutability.
Figure 11. We test the ‘cutability’ of optimized cut shapes by cutting various optimized shapes using a bandsaw (left top and bottom) and a polystyrene foam cutter (bottom right). The references for the cut shapes are at the top right.
Figure 12. Differentiable voxelization can be used to optimize tilings in R3 to periodically fill the space. Left: Tiling procedure for a single shape instance. Right: Each row belongs to one optimized packing. The packed meshes (first column) can be repeated along all three axis to fill the space. The packed meshes are composed of one or more deformed meshes that are all similar (within a class of animals). When voxelized and repeated in all dimensions, the packing fills R3 (third column). The last column shows the interior of the space filling voxels by hiding some instances (for the space filling for multiple animal types with some hidden instances, all Tweety birds are shaded in reds and all Spots are shaded in blues).
Meshes remain the most practical representation for geometry reasoning and integration into graphics pipelines, yet existing reconstruction methods struggle to produce high-quality meshes. Most state-of-the-art approaches initially learn an intermediate representation (NeRF/3DGS) and treat mesh extraction as a post-processing step, which often leads to oversmoothed surfaces or poor quality meshes with excessive triangle counts.Existing mesh-native optimization methods alleviate some of these issues but suffer from fixed-resolution discretizations and unstable optimization behavior. In this paper, we introduce an adaptive mesh-based optimization framework and a practical mesh rendering technique to address these challenges. Our representation combines an optimizable Delaunay-triangulated tetrahedral grid with a multi-resolution hash grid. The former is refined through point pruning and insertion, while the latter provides latent features for SDF/appearance value predictions. We use volumetric rendering to bootstrap a coarse geometry while leveraging mesh-based rendering for recovering fine-grained details. Additionally, we propose a differentiable, rasterization-based depth-offset rendering formulation, reducing geometric artifacts and improving reconstruction quality. Our method significantly outperforms existing mesh optimization approaches across a variety of object-centric benchmarks while being competitive with state-of-the-art NeRF/3DGS methods.
Johannes Weidenfeller, Shaofei Wang, Philipp Fürnstahl +1
ETH Zurich, Zurich, Switzerland · Beijing Institute for General Artificial Intelligence, Beijing, China · University of Zurich, Zurich, Switzerland +1
Implicit surface representations have regained popularity because of their use in machine learning. A common component in optimization is regularization, penalizing the deviation of the surface from its original shape. The popular as-rigid-aspossible (ARAP) energy strikes a good compromise between realistic deformation behavior and efficient computation, at least for piecewise linear meshes. We develop an approach for computing the ARAP energy of a deformation function based on point sampling of the surface. The implicit representation is exploited to provide differentials in each sample. The evaluation is efficient and exact in each sample (up to numerical precision). We demonstrate the general applicability of the method to neural shape processing in several applications and contrast its properties with alternatives from the literature.
This paper studies tetrahedral meshes as the body representation for differentiable simulation and computational design. Fixed-connectivity meshes degrade under large morphs, while remeshing from scratch discards node correspondence. We present JamTet, a physics-based sphere-packing framework for volumetric meshing and morphing. We contribute (i) a GPU-parallel mesher combining octree-hierarchical packing with constrained Delaunay tetrahedralization, producing more uniform element volumes than TetGen and fTetWild; (ii) Lagrangian mesh morphing that preserves interior-node identities by re-equilibrating the same spheres within changing shapes and rebuilding the boundary and connectivity, remaining inversion-free where fixed-connectivity and TetSphere meshes invert; and (iii) a differentiable GPU simulator in JAX, with mass-spring edges and a volumetric Neo-Hookean term, integrated with mesh morphing in a design pipeline. In soft-robot morphology design experiments, interior-node gradients improve swimming fitness by 0.73-1.07 over a matched surface-only variant, while voxelized versions of the same designs yield 32-63% lower fitness. These results establish sphere packing as a practical volumetric mesh representation for gradient-based shape optimization. Code and media: under review.