Single-Image 3D Human Digitization with Shape-guided Diffusion

dc.contributor.authorAlbahar, Badouren
dc.contributor.authorSaito, Shunsukeen
dc.contributor.authorTseng, Hung-Yuen
dc.contributor.authorKim, Changilen
dc.contributor.authorKopf, Johannesen
dc.contributor.authorHuang, Jia-Binen
dc.date.accessioned2024-03-01T13:18:41Zen
dc.date.available2024-03-01T13:18:41Zen
dc.date.issued2023-12-10en
dc.date.updated2024-01-01T08:55:39Zen
dc.description.abstractWe present an approach to generate a 360-degree view of a person with a consistent, high-resolution appearance from a single input image. NeRF and its variants typically require videos or images from different viewpoints. Most existing approaches taking monocular input either rely on ground-truth 3D scans for supervision or lack 3D consistency. While recent 3D generative models show promise of 3D consistent human digitization, these approaches do not generalize well to diverse clothing appearances, and the results lack photorealism. Unlike existing work, we utilize high-capacity 2D diffusion models pretrained for general image synthesis tasks as an appearance prior of clothed humans. To achieve better 3D consistency while retaining the input identity, we progressively synthesize multiple views of the human in the input image by inpainting missing regions with shape-guided diffusion conditioned on silhouette and surface normal. We then fuse these synthesized multi-view images via inverse rendering to obtain a fully textured high-resolution 3D mesh of the given person. Experiments show that our approach outperforms prior methods and achieves photorealistic 360-degree synthesis of a wide range of clothed humans with complex textures from a single image.en
dc.description.versionPublished versionen
dc.format.mimetypeapplication/pdfen
dc.identifier.doihttps://doi.org/10.1145/3610548.3618153en
dc.identifier.urihttps://hdl.handle.net/10919/118232en
dc.language.isoenen
dc.publisherACMen
dc.rightsIn Copyrighten
dc.rights.holderThe author(s)en
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.titleSingle-Image 3D Human Digitization with Shape-guided Diffusionen
dc.typeArticle - Refereeden
dc.type.dcmitypeTexten

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
3610548.3618153.pdf
Size:
73.47 MB
Format:
Adobe Portable Document Format
Description:
Published version
License bundle
Now showing 1 - 1 of 1
Name:
license.txt
Size:
1.5 KB
Format:
Item-specific license agreed upon to submission
Description: