Thursday, July 25, 2024

Install Stable Video 4D (SV4D) Locally for Video to Video Generation

 Stable Video 4D (SV4D) is a generative model based on Stable Video Diffusion (SVD) and Stable Video 3D (SV3D), which takes in a single-view video of an object and generates multiple novel-view videos (4D image matrix) of that object.


Code:

conda create -n sdv4 python=3.11 -y && conda activate sdv4

conda install -c conda-forge nccl conda install pytorch torchvision torchaudio -c pytorch

git clone https://github.com/Stability-AI/generative-models.git && cd generative-models

pip install accelerate huggingface_hub

pip install black==23.7.0
pip install chardet==5.1.0
pip install clip@git+https://github.com/openai/CLIP.git
pip install einops>=0.6.1
pip install fairscale>=0.4.13
pip install fire>=0.5.0
pip install fsspec>=2023.6.0
pip install invisible-watermark>=0.2.0
pip install kornia==0.6.9
pip install matplotlib>=3.7.2
pip install natsort>=8.4.0
pip install ninja>=1.11.1
pip install numpy>=1.24.4
pip install omegaconf>=2.3.0
pip install open-clip-torch>=2.20.0
pip install opencv-python==4.6.0.66
pip install pandas>=2.0.3
pip install pillow>=9.5.0
pip install pudb>=2022.1.3
pip install pytorch-lightning==2.0.1
pip install pyyaml>=6.0.1
pip install rembg
pip install scipy>=1.10.1
pip install streamlit>=0.73.1
pip install tensorboardx==2.6
pip install timm>=0.9.2
pip install tokenizers
pip install torchdata==0.6.1
pip install torchmetrics>=1.0.1
pip install tqdm>=4.65.0
pip install transformers==4.19.1
pip install triton==2.0.0
pip install urllib3<1.27,>=1.25.4
pip install wandb>=0.15.6
pip install webdataset>=0.2.33
pip install wheel>=0.41.0
pip install xformers>=0.0.20
pip install gradio
pip install streamlit-keyup==0.2.0
pip install imageio==2.19.3 pip install imageio-ffmpeg==0.4.7

huggingface-cli login   # get the key from huggingface.co
export HF_TOKEN=""

wget --header="Authorization: Bearer $HF_TOKEN" -O checkpoints/sv4d.safetensors https://huggingface.co/stabilityai/sv4d/resolve/main/sv4d.safetensors
wget --header="Authorization: Bearer $HF_TOKEN" -O checkpoints/sv3d_u.safetensors https://huggingface.co/stabilityai/sv3d/resolve/main/sv3d_u.safetensors
ls -lh checkpoints/*.safetensors

python3 scripts/sampling/simple_video_sample_4d.py --input_path assets/test_video1.mp4 --output_folder outputs/sv4d

No comments: