References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Human Gaussian Splatting: Real-time Rendering of Animatable Avatars
(Submitted on 28 Nov 2023 (v1), last revised 28 Mar 2024 (this version, v2))
Abstract: This work addresses the problem of real-time rendering of photorealistic human body avatars learned from multi-view videos. While the classical approaches to model and render virtual humans generally use a textured mesh, recent research has developed neural body representations that achieve impressive visual quality. However, these models are difficult to render in real-time and their quality degrades when the character is animated with body poses different than the training observations. We propose an animatable human model based on 3D Gaussian Splatting, that has recently emerged as a very efficient alternative to neural radiance fields. The body is represented by a set of gaussian primitives in a canonical space which is deformed with a coarse to fine approach that combines forward skinning and local non-rigid refinement. We describe how to learn our Human Gaussian Splatting (HuGS) model in an end-to-end fashion from multi-view observations, and evaluate it against the state-of-the-art approaches for novel pose synthesis of clothed body. Our method achieves 1.5 dB PSNR improvement over the state-of-the-art on THuman4 dataset while being able to render in real-time (80 fps for 512x512 resolution).
Submission history
From: Arthur Moreau [view email][v1] Tue, 28 Nov 2023 12:05:41 GMT (1065kb,D)
[v2] Thu, 28 Mar 2024 17:07:28 GMT (1649kb,D)
Link back to: arXiv, form interface, contact.