Abstract

The estimation of 3D face shape from a single image must be robust to variations in lighting, head pose, expression, facial hair, makeup, and occlusions. Robustness requires a large training set of in-the-wild images, which by construction, lack ground truth 3D shape. To train a network without any 2D-to-3D supervision, we present RingNet, which learns to compute 3D face shape from a single image. Our key observation is that an individual’s face shape is constant across images, regardless of expression, pose, lighting, etc. RingNet leverages multiple images of a person and automatically detected 2D face features. It uses a novel loss that encourages the face shape to be similar when the identity is the same and different for different people. We achieve invariance to expression by representing the face using the FLAME model. Once trained, our method takes a single image and outputs the parameters of FLAME, which can be readily animated. Additionally we create a new database of faces “not quite in-the-wild” (NoW) with 3D head scans and high-resolution images of the subjects in a wide variety of conditions. We evaluate publicly available methods and find that RingNet is more accurate than methods that use 3D supervision. The dataset, model, and results are available for research purposes.

Video

Challenge NoW

The goal of this benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods under variations in viewing angle, lighting, and common occlusions. Please click on the Challenge page to check the state-of-the-art comparisons. If you want to participate on the challenge follow the instructions on the Challenge and Downloads page from where you can download the data.

Update: NoW dataset is divided into Validation and Test set. Ground Truth Scans are available for the Validation Set.

More Information

• The pdf preprint can be downloaded from here.
• Please register and accept the License agreement in this website in order to get access to the NoW Dataset
• RingNet on github
• Pre-trained models are available in the Downloads page
• RingNet project page at MPI-IS
• The arxiv version is also available now
• For questions, please contact ringnet@tue.mpg.de

Referencing RingNet and the NoW Dataset

@inproceedings{RingNet:CVPR:2019,
title = {Learning to Regress {3D} Face Shape and Expression from an Image without {3D} Supervision},
author = {Sanyal, Soubhik and Bolkart, Timo and Feng, Haiwen and Black, Michael},
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
month = jun,
pages = {7763--7772},
year = {2019},
month_numeric = {6} 
}