RoboCloth: A Large-Scale Real Cloth Material Dataset for Neural Reflectance Reconstruction

Zhen Li; Haoran Qin; Francesco Di Sario; Daniel Rebain; Andrea Tagliasacchi

cs.CVOct 15, 2026

RoboCloth: A Large-Scale Real Cloth Material Dataset for Neural Reflectance Reconstruction

Authors: Zhen Li, Haoran Qin, Francesco Di Sario, Daniel Rebain, Andrea Tagliasacchi

Organizations: Simon Fraser University · Westlake University · University of Torino and Simon Fraser University · University of Torino ·

Abstract

We present RoboCloth, a dataset of 500 real cloth materials captured under flexible, calibrated camera and lighting configurations, where both are robotically controlled during acquisition. Each material is imaged in approximately 580 high-dynamic range frames spanning a wide view-light hemisphere; camera poses are refined via structure-from-motion and aligned to the robot frame, yielding per-material sparse surface geometry and structured per-point reflectance observations ready for neural material training. We demonstrate the value of this data through a two-stage neural reconstruction pipeline: Stage 1 trains a shared MLP decoder on our corpus to learn cloth material priors; Stage 2 then freezes the decoder and fits a dense 2048x2048 latent texture to each target material. Evaluating on held-out materials from our own dataset and on the independent UBO2014 benchmark, we show that a decoder trained on our captures outperforms decoders trained on existing measured reflectance datasets and analytic PBR fitting, transferring to unseen acquisition systems without modification. The dataset, calibration files, observation tensors, and train/test splits are released under CC BY 4.0 on the Hugging Face Hub.

Figures & tables

Dataset	Publication Year	Dataset Scale	Cloth Specific	Real World	Spatially Varying	Multiple Views	Flexible Coverage
MERL [ 19 ]	2003	100	✗	✓	✗	✓	✗
UTIA [ 5 ]	2014	150	✗	✓	✗	✓	✗
UBO2014 [ 41 ]	2014	84	✗	✓	✓	✓	✗
UBOFAB19 [ 21 ]	2019	378	✓	✓	✓	✓	✗
Real-SVBRDF [ 1 ]	2020	80	✗	✓	✓	✗	✗
PhotoMat [ 48 ]	2023	12,000	✗	✓	✓	✗	✗

Table 1: Comparison wrt. related works. We report whether each dataset is cloth-specific and provides real measurements, spatially varying appearance, multiple views, and flexible view-light coverage. With multiple views, we denote capture with more than two distinct viewing directions. RoboCloth is the only dataset to provide all five, and at scale.

Figure 2: Stage 1 — learning a material prior. A single BRDF decoder, shared across all training materials, is optimized jointly (flame) with per-point tables of latent codes, normals, and tangents; see Section 4.2 . N: normal, T: tangent, D: encoded local directions; each color denotes one material, with a few points drawn per material in every minibatch.

Figure 3: Stage 2 — dense material reconstruction. For a single target material, a dense latent-geometry texture is optimized (flame) through the parallax-aware lookup $P$ , the shared transform $\mathcal{T}$ , and the frozen decoder (snowflake); see Section 4.3 . Notation as in Figure 2 .

	Decoder training source			Baseline
Material	RoboCloth	Bonn	MERL	PBR	vs. best
Fabric02	34.57	28.56	27.88	30.96	$+3.61$
Fabric04	34.00	28.50	26.66	30.09	$+3.91$
Fabric09	39.50	36.08	33.96	36.96	$+2.54$
Fabric11	34.33	29.45	27.46	31.24	$+3.09$
Felt01	38.76	33.60	32.81	35.05	$+3.71$

Table 2: Cross-dataset transfer to UBO2014. Decoder columns differ only in Stage-1 training source; PSNR in dB (

\uparrow

). The last column reports the margin of RoboCloth over the best competing column (positive means RoboCloth leads).

	Decoder training source			Baseline
Material	RoboCloth	Bonn	MERL	PBR	vs. best
In-domain: RoboCloth test set (held-out)
226	29.24	25.63	24.24	25.35	$+3.61$
314	26.60	24.42	22.93	23.98	$+2.18$
370	29.71	29.21	27.75	29.29	$+0.42$
145	28.44	24.45	23.39	24.28	$+3.99$

Table 3: Per-material reconstruction. Top: in-domain on our held-out test set. Bottom: cross-dataset on held-out Bonn materials. The last column reports the margin of RoboCloth over the best competing column, in dB; positive means RoboCloth leads.

Panel of Figure 4 — Figure 4: Grazing-angle supervision. Polar BRDF-lobe plots of the learned contribution $f_{r}\cos\theta_{i}$ vs. incident elevation $\theta_{i}$ ( $0^{\circ}$ =normal at top, $\pm 90^{\circ}$ =grazing at the sides); each colored curve is a fixed outgoing direction (blue $\theta_{o}\!=\!15^{\circ}$ , orange $30^{\circ}$ , green $45^{\circ}$ , red $60^{\circ}$ ), and the dashed line marks the specular direction. Each row is a different surface point; columns are the four grazing-supervision variants. Only our loss (first column, Eq. 3 ) makes $f_{r}\cos\theta_{i}$ fall off smoothly to zero near grazing; ZeroExact, NearZeroBRDF and the no-grazing baseline all leave the near-grazing region uncontrolled, inflating the lobes toward $\pm 90^{\circ}$ .

Figure 5: Transfer PSNR vs. training-set size. Mean UBO2014 transfer PSNR (over all

12

transfer materials) against the number of training cloth materials (

80/240/400

, i.e. the

100/300/500

-material subsets). Accuracy rises consistently with more data and begins to saturate at our full corpus. The vertical axis is zoomed to make the trend visible.

Material	RoboCloth-100	RoboCloth-300	RoboCloth-500
	(80)	(240)	(400)
fabric02	34.38	34.60	34.57
fabric04	33.33	33.72	34.00
fabric09	38.93	39.40	39.50
fabric11	33.76	34.04	34.33
felt01	38.38	38.70	38.76

Table 4: Dataset-size ablation (UBO2014 transfer). Per-material PSNR (dB,

\uparrow

) for decoders trained on the

100/300/500

-material subsets (

80/240/400

training materials after the

0.8/0.2

split). PSNR rises consistently with more training data.

Figure 6: The coordinate-transform operator $\mathcal{T}$ . A per-point (Stage 1) or per-texel (Stage 2) normal $\mathbf{n}$ and tangent $\mathbf{t}$ are orthogonalized by Gram–Schmidt into a TBN frame $(\tilde{\mathbf{n}},\tilde{\mathbf{t}},\tilde{\mathbf{b}})$ . The world-space incident and outgoing directions $\boldsymbol{\omega}_{i},\boldsymbol{\omega}_{o}$ are rotated into this local frame; the local directions together with $\tilde{\mathbf{n}}$ are then encoded with degree- $3$ real spherical harmonics and concatenated with the latent code before being passed to the BRDF decoder.

Figure 7: Parallax-aware texture lookup $P$ (Stage 2). An initial sample at $\mathbf{u}$ provides the geometry latent $\mathbf{g}(\mathbf{u})$ . Together with the incident and outgoing directions $\boldsymbol{\omega}_{i},\boldsymbol{\omega}_{o}$ , a small geometry decoder maps it to a texture-space displacement $\Delta\mathbf{u}$ . The final point attributes are bilinearly sampled from the latent-geometry grid $\mathbf{Z}$ at $\mathbf{u}^{\prime}=\mathbf{u}+\Delta\mathbf{u}$ , reproducing yarn-level parallax without an explicit height field.

Source	Data type	# Materials
RoboCloth	Real robotic captures (HDR)	$\sim$ 400
Bonn	Multi-LED and linear-light-source	$311$
MERL	Measured isotropic BRDFs	$100$

Table 5: Training sources for the neural decoder (Stage 1). Each RoboCloth material contributes

{\sim}580

HDR frames. Every Bonn material is measured in three modalities: PAN (panchromatic,

24

unfiltered LEDs,

388

images), POLY (polychromatic, RGB-filtered LEDs,

100

images), and LLS (linear light source,

14

angles,

280

images). MERL supplies tabulated BRDFs rather than images and is sampled directly.

Table 12

Neural Rendering

Neural rendering uses learned 3D scene and light representations to synthesize, edit, and relight images from new viewpoints.

See all 120 papers

Extracting Neural Materials from Multi-view Images
Jun 25, 2026Kim Youwang, Jon Hasselgren, Peter Kocsis +3Inverse Rendering Differentiable Rendering
NVIDIA · POSTECH · 1NVIDIA 2POSTECH +3
3DReflecNet: A Large-Scale Dataset for 3D Reconstruction of Reflective, Transparent, and Low-Texture Objects
May 11, 2026Zhicheng Liang, Haoyi Yu, Boyan Li +63D Reconstruction Benchmarks 3D Reconstruction
The Chinese University of Hong Kong, Shenzhen · Capital Normal University · University of Southern California
CLOTH-HUGS: Cloth Aware Human Gaussian Splatting
Apr 17, 2026Sadia Mubashshira, Nazanin Amini, Kevin Desai3D Gaussian Splatting 3D Reconstruction

Jun 25, 2026J/K move · Enter open · S save

Extracting Neural Materials from Multi-view Images

Kim Youwang, Jon Hasselgren, Peter Kocsis, Andrea Weidlich+2

NVIDIA · POSTECH · 1NVIDIA 2POSTECH+3

Inverse Rendering Differentiable Rendering Neural Rendering

Open paper AskPDF arXiv

Neural materials can represent complex specular reflections and scattering effects in a compact, universal basis. However, acquiring and authoring such materials remains challenging. We present NeuMatEx, a differentiable inverse rendering method for extracting spatially varying neural materials from images. The nonlinear structure of neural material latent spaces makes optimization with naive inverse rendering infeasible. To address this, we train a Large Material Reconstruction Model (LMRM) that directly predicts initialbase color, neural material latents, and aleatoric uncertainty guides from images. This material prior provides a good initialization and better constrains our subsequent optimization using inverse path tracing. The predicted uncertainty further helps by anchoring high-confidence regions more tightly to the LMRM prediction, preventing lighting and complex specular effects from being baked into materials. Experiments on synthetic and real assets show that NeuMatEx extracts complex materials with better visual quality and material decomposition than PBR-based methods.

Abstract

Neural Rendering

Explore similar work

Extracting Neural Materials from Multi-view Images

3DReflecNet: A Large-Scale Dataset for 3D Reconstruction of Reflective, Transparent, and Low-Texture Objects

CLOTH-HUGS: Cloth Aware Human Gaussian Splatting