Animate3D
3D model animation generation
- Multi View Video Diffusion Model (MV-VDM): Multi view rendering based on static 3D objects, trained on large-scale multi view video datasets.
- Spatiotemporal Attention Module: Enhance spatial and temporal consistency, integrate 3D and video diffusion models.
- 4D Score Distillation Sampling (4D-SDS): Combining reconstruction and sampling to refine appearance and motion.
- Large scale multi view video dataset (MV Video): containing 115K animations, covering 53K animated 3D objects, rendered into over 1.8M multi view videos.
- Animation reconstruction: Directly reconstruct motion from generated multi view videos.
- Animation refinement: Further optimize appearance and motion through 4D-SDS.
- Open release of data, code, and models: providing resources for further research and application.
Product Details
Animate3D is an innovative framework used to generate animations for any static 3D model. Its core concept includes two main parts: 1) proposing a new multi view video diffusion model (MV-VDM) based on multi view rendering of static 3D objects and training on the large-scale multi view video dataset (MV Video) we provide. 2) Based on MV-VDM, a framework combining reconstruction and 4D score distillation sampling (4D-SDS) is introduced to generate animations for 3D objects using multi view video diffusion priors. Animate3D enhances spatial and temporal consistency by designing a new spatiotemporal attention module, and maintains the identity of static 3D models through multi view rendering. In addition, Animate3D proposes an effective two-stage process to generate animations for 3D models: first, directly reconstruct motion from the generated multi view video, and then refine appearance and motion through the introduction of 4D-SDS.
