Guy Gafni - NerFACE

Talking Papers Podcast

PAPER TITLE:
Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
AUTHORS:
Guy Gafni Justus Thies Michael Zollhöfer Matthias Nießner
Project page: https://gafniguy.github.io/4D-Facial-Avatars/
CODE:
💻https://github.com/gafniguy/4D-Facial-Avatars
ABSTRACT:
We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face. Digitally modeling and reconstructing a talking human is a key building-block for a variety of applications. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoint or head-poses is required. In contrast to state-of-the-art approaches that model the geometry and material properties explicitly, or are purely image-based, we introduce an implicit representation of the head based on scene representation networks. To handle the dynamics of the face, we combine our scene representation network with a low-dimensional morphable model which provides explicit control over pose and expressions. We use volumetric rendering to generate images from this hybrid representation and demonstrate that such a dynamic neural scene representation can be learned from monocular input data only, without the need of a specialized capture setup. In our experiments, we show that this learned volumetric representation allows for photo-realistic image generation that surpasses the quality of state-of-the-art video-based reenactment methods.
RELATED PAPERS:
📚Representing Scenes as Neural Radiance Fields for View Synthesis
📚Deep Video Portraits
📚Nerfies: Deformable Neural Radiance Fields
📚AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
CONTACT:
-----------------
If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: [e-mail]
TIME STAMPS
-----------------------
00:00
00:07 Intro
00:27 Authors
01:16 Abstract / TLDR
02:54 Motivation
12:24 Related Work
13:20 Approach
17:10 Results
27:05 Conclusions and future work
32:12 Outro
#talkingpapers #CVPR2021 #NeRF
#machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence #FacialAvatars
Recorded on April, 2nd 2021.
SUBSCRIBE AND FOLLOW:
🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com
📧Subscribe to our mailing list: http://eepurl.com/hRznqb
🐦Follow us on Twitter: https://twitter.com/talking_papers
🎥YouTube Channel: https://bit.ly/3eQOgwP
🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

📧Subscribe to our mailing list: http://eepurl.com/hRznqb

🐦Follow us on Twitter: https://twitter.com/talking_papers

🎥YouTube Channel: https://bit.ly/3eQOgwP