University of Leicester
final.pdf (7.76 MB)

GR-PSN: Learning to Estimate Surface Normal and Reconstruct Photometric Stereo Images

Download (7.76 MB)
journal contribution
posted on 2023-11-13, 14:12 authored by Y Ju, B Shi, Y Chen, Huiyu Zhou, J Dong, K-M Lam

In this paper, we propose a novel method, namely GR-PSN, which learns surface normals from photometric stereo images and generates the photometric images under distant illumination from different lighting directions and surface materials. The framework is composed of two subnetworks, named GeometryNet and ReconstructNet, which are cascaded to perform shape reconstruction and image rendering in an end-to-end manner. ReconstructNet introduces additional supervision for surface-normal recovery, forming a closed-loop structure with GeometryNet. We also encode lighting and surface reflectance in ReconstructNet, to achieve arbitrary rendering. In training, we set up a parallel framework to simultaneously learn two arbitrary materials for an object, providing an additional transform loss. Therefore, our method is trained based on the supervision by three different loss functions, namely the surface-normal loss, reconstruction loss, and transform loss. We alternately input the predicted surface-normal map and the ground-truth into ReconstructNet, to achieve stable training for ReconstructNet. Experiments show that our method can accurately recover the surface normals of an object with an arbitrary number of inputs, and can re-render images of the object with arbitrary surface materials. Extensive experimental results show that our proposed method outperforms those methods based on a single surface recovery network and shows realistic rendering results on 100 different materials. Our code can be found in .


Author affiliation

School of Computing and Mathematical Sciences, University of Leicester


  • AM (Accepted Manuscript)

Published in

IEEE Transactions on Visualization and Computer Graphics


Institute of Electrical and Electronics Engineers



Copyright date


Available date




Usage metrics

    University of Leicester Publications