Text

Outdoor Inverse Rendering From a Single Image Using Multiview Self-Supervision

In this paper we show how to perform scene-level inverse rendering to recover shape, reflectance and lighting from a single, uncontrolled image using a fully convolutional neural network. The network takes an RGB image as input, regresses albedo, shadow and normal maps from which we infer least squares optimal spherical harmonic lighting coefficients. Our network is trained using large uncontrolled multiview and timelapse image collections without ground truth. By incorporating a differentiable renderer, our network can learn from self-supervision. Since the problem is ill-posed we introduce additional supervision. Our key insight is to perform offline multiview stereo (MVS) on images containing rich illumination variation. From the MVS pose and depth maps, we can cross project between overlapping views such that Siamese training can be used to ensure consistent estimation of photometric invariants. MVS depth also provides direct coarse supervision for normal map estimation. We believe this is the first attempt to use MVS supervision for learning inverse rendering. In addition, we learn a statistical natural illumination prior. We evaluate performance on inverse rendering, normal map estimation and intrinsic image decomposition benchmarks.

Ketersediaan

Informasi Detail

Judul Seri

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE; Vol.44 No.7 July 2022

No. Panggil

-

Penerbit

: .,

Deskripsi Fisik

p. 3659-3675

Bahasa

English

ISBN/ISSN

-

Klasifikasi

NONE

Tipe Isi

-

Tipe Media

-

Tipe Pembawa

-

Edisi

-

Subjek

INVERSE RENDERING
ILLUMINATION ESTIMATION
INTRINSIC IMAGE DECOMPOSITION
SHAPE-FROM-SHADING

Info Detail Spesifik

DOI: 10.1109/TPAMI.2021.3058105

Pernyataan Tanggungjawab

Ye Yu, William A. P. Smith

Versi lain/terkait

Lampiran Berkas

Komentar