RODIN Diffusion
scroll to discover
Drag to view generated 3D avatar vs. mesh
RODIN Diffusion
This 3D avatar diffusion model is an AI system that automatically produces highly detailed 3D digital avatars. The generated avatars can be freely viewed in 360 degrees with unprecedented quality. The model significantly accelerates traditionally sophisticated 3D modeling process and opens new opportunities for 3D artists.
RODIN Model Architecture
This 3D avatar diffusion model is trained to generate 3D digital avatars represented as neural radiance fields. We build on the state-of-the-art generative technique (diffusion models) for 3D modeling. We use tri-plane representation to factorize the neural radiance field of avatars, which can be explicitly modeled by diffusion models and rendered to images via volumetric rendering. The proposed 3D-aware convolution brings the much-needed computational efficiency while preserving the integrity of diffusion modeling in 3D. The whole generation is a hierarchical process with cascaded diffusion models for multi-scale modeling. Once the generative model is trained, one can control the avatar generation based on the latent code derived from either an input image, text prompt or random noise.
Visualization of the RODIN Diffusion Model.
Please see the paper Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion for more details.
Portrait-guided Avatar Creation
Create a high-fidelity personalized 3D avatar from one’s portrait
Original Portrait
Avatar Created by Our Model
Text-guided Avatar Creation
Build an elaborate 3D avatar from scratch couldn't be easier.
A bearded man with curly hair posing in a black leather jacket
A woman with afro hairstyle wearing red
A young man with curly hair wearing glasses
A woman with long red hair wearing white
Avatar Created by Our Model
Text-guided Avatar Editing
Our Rodin model supports text-guided manipulation for the customized 3D avatar. One can intuitively edit a wide variety of attributes for reconstructed 3D avatar using natural language.
Appearance Editing
- Blonde Hair
- Short Hair
- Braids
- Beards
- Glasses
- Pink Suit
- White Suit
- Smiling Face
Rich Avatar
Diversity
Our Rodin Model shows impressive generation diversity in terms of gender, age, ethnicity, expression, face accessories, etc.
Responsible AI considerations
This algorithm has been trained to produce 3D digital avatars based on provided natural language sentences or photographs. However, it is crucial to acknowledge that the potential for misuse exists, as is the case with other AI-driven content generation models. In order to mitigate the risk of malicious dissemination of disinformation, it is recommended to incorporate tags or watermarks when distributing the photos generated by this model.
Acknowledgement
Research Paper
Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion, arXiv preprint, 2022.
Research Authors
Tengfei Wang*, Bo Zhang*, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, Baining Guo
*Equal contribution
Project Contribution
Product | Yu Liu, Jieyu Xiao°, Scarlett Li
Design | Yang Ou, Geli Guo
Engineering & Prototyping | Yan Xia, Bo Zhang, Dong Chen, Ting Song, Tiantian Xue, Tengfei Wang°
°Intern
