University of Leicester
Browse
final.pdf (7.76 MB)

GR-PSN: Learning to Estimate Surface Normal and Reconstruct Photometric Stereo Images

Download (7.76 MB)
journal contribution
posted on 2023-11-13, 14:12 authored by Y Ju, B Shi, Y Chen, Huiyu Zhou, J Dong, K-M Lam

In this paper, we propose a novel method, namely GR-PSN, which learns surface normals from photometric stereo images and generates the photometric images under distant illumination from different lighting directions and surface materials. The framework is composed of two subnetworks, named GeometryNet and ReconstructNet, which are cascaded to perform shape reconstruction and image rendering in an end-to-end manner. ReconstructNet introduces additional supervision for surface-normal recovery, forming a closed-loop structure with GeometryNet. We also encode lighting and surface reflectance in ReconstructNet, to achieve arbitrary rendering. In training, we set up a parallel framework to simultaneously learn two arbitrary materials for an object, providing an additional transform loss. Therefore, our method is trained based on the supervision by three different loss functions, namely the surface-normal loss, reconstruction loss, and transform loss. We alternately input the predicted surface-normal map and the ground-truth into ReconstructNet, to achieve stable training for ReconstructNet. Experiments show that our method can accurately recover the surface normals of an object with an arbitrary number of inputs, and can re-render images of the object with arbitrary surface materials. Extensive experimental results show that our proposed method outperforms those methods based on a single surface recovery network and shows realistic rendering results on 100 different materials. Our code can be found in https://github.com/Kelvin-Ju/GR-PSN .

History

Author affiliation

School of Computing and Mathematical Sciences, University of Leicester

Version

  • AM (Accepted Manuscript)

Published in

IEEE Transactions on Visualization and Computer Graphics

Publisher

Institute of Electrical and Electronics Engineers

issn

1941-0506

Copyright date

2023

Available date

2023-11-13

Language

en

Usage metrics

    University of Leicester Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC