References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: GaussianCube: Structuring Gaussian Splatting using Optimal Transport for 3D Generative Modeling
(Submitted on 28 Mar 2024 (v1), last revised 5 Apr 2024 (this version, v2))
Abstract: 3D Gaussian Splatting (GS) have achieved considerable improvement over Neural Radiance Fields in terms of 3D fitting fidelity and rendering speed. However, this unstructured representation with scattered Gaussians poses a significant challenge for generative modeling. To address the problem, we introduce GaussianCube, a structured GS representation that is both powerful and efficient for generative modeling. We achieve this by first proposing a modified densification-constrained GS fitting algorithm which can yield high-quality fitting results using a fixed number of free Gaussians, and then re-arranging the Gaussians into a predefined voxel grid via Optimal Transport. The structured grid representation allows us to use standard 3D U-Net as our backbone in diffusion generative modeling without elaborate designs. Extensive experiments conducted on ShapeNet and OmniObject3D show that our model achieves state-of-the-art generation results both qualitatively and quantitatively, underscoring the potential of GaussianCube as a powerful and versatile 3D representation.
Submission history
From: Bowen Zhang [view email][v1] Thu, 28 Mar 2024 17:59:50 GMT (8739kb,D)
[v2] Fri, 5 Apr 2024 09:35:37 GMT (8739kb,D)
Link back to: arXiv, form interface, contact.