Event-based Photometric Stereo via Rotating Illumination and Per-Pixel Learning

Hyunwoo Kim; Won-Hoe Kim; Sanghoon Lee; Jianfei Cai; Giljoo Nam; Jae-Sang Hyun

cs.CVMar 11, 2026

Event-based Photometric Stereo via Rotating Illumination and Per-Pixel Learning

Authors: Hyunwoo Kim, Won-Hoe Kim, Sanghoon Lee, Jianfei Cai, Giljoo Nam, Jae-Sang Hyun

Organizations: Department of Mechanical Engineering, Yonsei University, Seoul, 03722, South Korea · Department of Electrical and Electronic Engineering, Yonsei University, Seoul, 03722, South Korea · Department of Data Science and AI, Monash University, Melbourne, VIC 3800, Australia · Meta Reality Labs, United States

Abstract

Photometric stereo is a technique for estimating surface normals using images captured under varying illumination. However, conventional frame-based photometric stereo methods are limited in real-world applications due to their reliance on controlled lighting, and susceptibility to ambient illumination. To address these limitations, we propose an event-based photometric stereo system that leverages an event camera, which is effective in scenarios with continuously varying scene radiance and high dynamic range conditions. Our setup employs a single light source moving along a predefined circular trajectory, eliminating the need for multiple synchronized light sources and enabling a more compact and scalable design. We further introduce a lightweight per-pixel multi-layer neural network that directly predicts surface normals from event signals generated by intensity changes as the light source rotates, without system calibration. Experimental results on benchmark datasets and real-world data collected with our data acquisition system demonstrate the effectiveness of our method, achieving a 7.12% reduction in mean angular error compared to existing event-based photometric stereo methods. In addition, our method demonstrates robustness in regions with sparse event activity, strong ambient illumination, and scenes affected by specularities.

Figures & tables

Figure 1: Overview of the proposed method. (a) Our system setup. A light source rotates around the optical axis of a fixed event camera. (b) Event signals observed over one rotation cycle of the light source. The signals are converted to a polarity-based event representation that encodes temporal illumination variations. (c) The polarity-based representation is aggregated at each pixel to form a polarity sum vector, which is used as the input to a surface normal estimation network. The network learns the nonlinear mapping from temporal event patterns to surface normals on a per-pixel basis and predicts dense surface normal maps without system calibration.

Figure 2: Schematic diagram of our system. The system consists of a fixed event camera and a light source. The light source rotates around the optical axis of the event camera at a constant angular speed.

Panel of Figure 3 — Figure 3: Illustration of the process of generating event representations. (a) Intensity variations are induced by a rotating light source. (b) Events triggered by these variations are temporally sliced to compute the event representation.

Layer	Description
1	Dense-(4096), Tanh, Dropout
2	Dense-(4096), Tanh, Dropout
3	Dense-(2048), Tanh, Dropout
4	Dense-(2048), Tanh, Dropout
5	Dense-(2048), Tanh, Dropout
6	Dense-(3)

Table 1: The network structure. The numbers in the parentheses represent the number of the output dimensions. Dropout rates for training are set to 0.2 for all Dense layers. The input dimension of the first Dense layer is set to 96.

Figure 4: Comparison between simulated and real event data over one cycle of the light source rotation. (a) Event signals during one cycle. Blue points represent ON events, and red points represent OFF events. (b) Estimated surface normal maps and corresponding angular error maps for each event signal.

Figure 5: Our experiment setting. The rotating platform consists of a DC motor, a timing belt, and a hollow disc. The disc is connected to the motor through the timing belt, and rotates together. The light source is attached to the edge of the disc.

Figure 6: Dataset creation with Blobby obejcts. (a) 3D-printed objects in the real world. (b) Corresponding ground truth 3D models. (c) The image captured in the real world. Averaged during one cycle of the light source. (d) Rendered scene with the same pose as captured image. (e) Corresponding ground truth surface normal map.

Figure 7: Light source trajectories used in each dataset. The bold circular markers indicate the initial point of each trajectory, and the arrows indicate the direction of the light source motion along each trajectory.

Method	DiLiGenT-EV semi-real dataset									CW real dataset			Average
Method	Ball	Buddha	Cat	Cow	Goblet	Harvest	Pot1	Pot2	Reading	Ball	Bunny	Tiger	Average
EventPS-OP	4.12	16.65	8.53	25.31	15.52	35.11	9.64	13.32	22.49	11.66	19.71	20.84	16.91
EventPS-FCN	7.37	15.84	8.45	17.41	12.79	26.09	10.84	13.87	20.75	8.82	17.58	18.71	14.88
EventPS-CNN	4.53	12.06	6.44	16.55	9.91	24.97	7.69	10.33	18.18	7.94	15.94	18.12	12.72
Ours	4.52	12.45	6.69	13.58	9.83	23.63	7.95	9.89	17.21	7.52	16.10	17.94	12.24

Table 2: Quantitative comparison (MAE in degrees) on the DiLiGenT-EV semi-real dataset and CW real dataset. Lower is better. The bolded values are the best performing values.

Figure 8: Comparison between analytical approach and learning-based approach. The number between predicted normal map and error map represents MAE.

Figure 9: Evaluation results on DiLiGenT-EV semi-real dataset. The number between predicted normal map and error map represents MAE.

Figure 10: Evaluation results on CW real dataset. The number between predicted normal map and error map represents MAE.

Method	Ball	blob01	blob02	blob03	blob04	blob06	blob07	blob08	blob09	blob10	Average
EventPS-OP	5.75	13.37	14.07	17.29	14.28	14.84	12.45	17.43	10.30	13.41	13.32
EventPS-FCN	8.30	9.57	10.11	11.19	10.41	9.76	10.46	11.44	8.33	9.30	9.89
EventPS-CNN	8.43	11.60	11.41	12.94	10.96	11.27	10.85	13.95	8.60	10.71	11.07
Ours	7.01	9.48	10.14	11.36	9.94	10.45	9.89	12.77	7.32	9.29	9.77

Table 3: Quantitative comparison (MAE in degrees) on the CCW real dataset. Lower is better. The bolded values are the best performing values.

Figure 11: Evaluation results on CCW real dataset. The number between predicted normal map and error map represents MAE.

Figure 12: Quantitative evaluation showing the influence of event count on angular error for each method.

Figure 13: Qualitative comparison of error maps across methods, where the red boxes indicate regions with sparse event activity. Our method achieves the lowest angular error in these areas, demonstrating its robustness under low event density conditions.

Figure 14: Surface normal estimation of a complex shape object under a high exposure scenario. (a) Captured image of the object, (b) Captured image under high exposure scenario. (c) Predicted normal map using event signal.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 15: All evaluation results on DiLiGenT-EV semi-real dataset. The number between predicted normal map and error map represents MAE.

Figure 16: All evaluation results on CCW real dataset. The number between predicted normal map and error map represents MAE.

Surface Normal Estimation

Surface Normal Estimation recovers the orientation of visible 3D surfaces from images and other sensor data for shape understanding and reconstruction.

See all 26 papers

PIE-PS: Photometric Stereo from Physical Irradiance Event Streams
Oct 6, 2026Xiangze Meng, Guangyu Li, Jing Li +4Event-Based Vision Surface Normal Estimation
Jilin University, China · Guangdong Institute of Intelligence Science and Technology, China · Beijing Institute of Technology, China +1
Near-light Photometric Stereo with Symmetric Lights
Mar 17, 2026Lilika Makabe, Heng Guo, Hiroaki Santo +2Surface Normal Estimation Depth Estimation
Graduate School of Information Science and Technology, Osaka University, Japan.
E2EGS: Event-to-Edge Gaussian Splatting for Pose-Free 3D Reconstruction
Mar 16, 2026Yunsoo Kim, Changki Sung, Dasol Hong +13D Reconstruction Novel View Synthesis
Urban Robotics Lab, School of Electrical Engineering, KAIST

Oct 6, 2026J/K move · Enter open · S save

PIE-PS: Photometric Stereo from Physical Irradiance Event Streams

Xiangze Meng, Guangyu Li, Jing Li, Di Mei+3

Jilin University, China · Guangdong Institute of Intelligence Science and Technology, China · Beijing Institute of Technology, China+1

Event-Based Vision Surface Normal Estimation

Open paper AskPDF arXiv

Event cameras record asynchronous log-image-irradiance changes with microsecond latency and high dynamic range. These properties are useful for photometric stereo under moving illumination, but raw events are sparse and depend on an unknown contrast threshold. We start from the event trigger model and derive a physical relation between adjacent events, light motion, and surface normals. This relation gives a direct physics-only solver, but the solver needs the threshold, enough events at each pixel, and independent per-pixel optimization. To address these limits, we introduce PIE-PS, a learning-based framework for dense surface normal reconstruction from raw event streams and known lighting. We form Physical Irradiance Events (PIEs) by pairing two adjacent events at the same pixel with their corresponding light directions. Each PIE provides a Physical Irradiance Event Feature (PIEF), defined as the signed event rate. PIEF does not require the unknown contrast threshold. To share spatial and temporal context across nearby PIEs, we introduce PIE-GNN, which treats each PIE as a graph node and encodes it with its light-pair geometry. Since the reliability of PIE observations can vary with local appearance, illumination geometry, and sensor noise, Reliability-Grading Attention (RGA) predicts reliability weights to down-weight unreliable PIEs. Pixel aggregation then produces dense normals. Experiments on synthetic and real data show that PIE-PS outperforms prior event-based photometric stereo methods and the direct solver baseline.

Abstract

Surface Normal Estimation

Explore similar work

PIE-PS: Photometric Stereo from Physical Irradiance Event Streams

Near-light Photometric Stereo with Symmetric Lights

E2EGS: Event-to-Edge Gaussian Splatting for Pose-Free 3D Reconstruction