HoloDreamer
Generate holographic 3D panoramic world from text description
- Stylish and detailed rectangular panorama generation: Combining multiple diffusion models to generate stylized and detailed rectangular panoramas from complex text prompts.
- Enhance two-stage panoramic reconstruction: perform depth estimation and project RGBD data to obtain point clouds, and use base and auxiliary cameras for projection and rendering in different scenes.
- 3D Gaussian Scattering (3D-GS): Quickly reconstruct 3D scenes and enhance scene integrity.
- Multi view supervision: Using a 2D diffusion model to generate initial local images, and then gradually generating scenes to improve global consistency.
- Panoramic image rotation without cracks: Apply circular blending technology to avoid cracks when rotating the panoramic image.
- Two stage optimization: In the transfer optimization stage, the rendered images of the reconstructed scene are internally drawn, 3D-GS is optimized, and the final reconstructed scene is generated.
- High definition panoramic initialization: Generate high-definition panoramic images as the overall initialization of the 3D scene to improve the quality and consistency of reconstruction.
Product Details
HoloDreamer is a text driven 3D scene generation framework that can generate immersive and perspective consistent fully enclosed 3D scenes. It consists of two basic modules: stylized rectangular panorama generation and enhanced two-stage panorama reconstruction. The framework first generates high-definition panoramic images as the overall initialization of the complete 3D scene, and then uses 3D Gaussian scattering (3D-GS) technology to quickly reconstruct the 3D scene, thereby achieving consistent and completely enclosed 3D scene generation. The main advantages of HoloDreamer include high visual consistency, harmony, reconstruction quality, and rendering robustness.
