Surface Normal Estimation From Optimized and Distributed Light Sources Using DNN-Based Photometric Stereo

Photometric stereo method can estimate high density normal. Theoretically, it is possible to reconstruct normals from three different shading images corresponding to different light source directions. However, in practical environments with materials that include specular reflections, global illumination, and measurement noise, a large number of shading images are required for robust normal estimation. To achieve this, multiple light sources must be positioned and calibrated for measurements. In this study, we propose a simplified measurement method using a near light source setup with a diffuser that can generate arbitrary light sources. Additionally, we efficiently estimate normals from a small number of measurements using deep learning by learning the light source distribution.

The measurement setup consists of a camera, projector, and diffuser. The diffuser, illuminated by projecting patterns from the projector, emits transmitted light to illuminate the target object. The use of the diffuser allows for the placement of multiple light sources close to the object. By expanding the illuminated area, it becomes possible to create area light sources and achieve fewer occlusions during measurements.

For pattern learning, we use a framework consisting of differentiable rendering and convolutional neural networks. In the rendering module, a three-dimensional shape is input to generate shading images under different light sources represented by weights. In the subsequent photometric stereo network, shading images are input to output normals. Training is conducted end-to-end using synthetic data. Furthermore, by inputting parameters for light source placement and orientation, the same model can be used without retraining for different setups.

The results of simulation experiments confirm that accurate normal estimation is possible for various materials, surpassing previous research in terms of accuracy. Particularly, correct normals are estimated in regions with self-occlusion and specular reflections, demonstrating the effectiveness of the patterned area light sources.

In real-world environments, it has been confirmed that normal estimation is feasible with qualitatively sufficient accuracy. Normal estimation accuracy in spherical shapes is higher than that of previous methods.

Publications

T. Iwaguchi and H. KawasakiSurface Normal Estimation From Optimized and Distributed Light Sources Using DNN-Based Photometric Stereo, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023.
T. Iwaguchi and H. Kawasaki, Efficient light transport acquisition by coded illumination and robust photometric stereo by dual photography using deep neural network, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCVW), 2021.

Back

Computer Vision and Graphics Laboratory