SV4D
A model for generating multi perspective videos
- Generate a 4D image matrix of 40 frames with a resolution of 576x576.
- Generate track videos using SV3D as reference views for SV4D.
- Input the video as a reference frame and perform 4D sampling.
- Generate longer new perspective videos by densely sampling (interpolating) the remaining frames.
- Suitable for generating art works and design processes.
- Applied to educational or creative tools.
- Research on generative models, including understanding the limitations of generative models.
Product Details
Stable Video 4D (SV4D) is a generative model based on Stable Video Diffusion (SVD) and Stable Video 3D (SV3D), which accepts videos from a single perspective and generates multiple new perspective videos of the object (4D image matrix). The model is trained to generate 40 frames (5 video frames x 8 camera perspectives) at 576x576 resolution, given 5 reference frames of the same size. Generate track videos by running SV3D, then use the track videos as reference views for SV4D, and input the videos as reference frames for 4D sampling. The model also generates longer new perspective videos by using the generated first frame as an anchor point and then densely sampling (interpolating) the remaining frames.
