Thursday, June 13, 2024

How-To Train Stable Diffusion 3 Medium On Your Image Dataset Locally

This video is a step-by-step tutorial to fine-tune Stable Diffusion 3 Medium locally on your own custom image dataset. 

 



Code:

conda create -n sdft python=3.11 -y

pip install peft
pip install datasets
pip install huggingface_hub
pip install wandb
pip install bitsandbytes
pip install pillow
pip install git+https://github.com/huggingface/transformers
pip install accelerate
pip install sentencepiece

git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .

cd examples/dreambooth

pip install -r requirements_sd3.txt

huggingface-cli login            

accelerate config default



from huggingface_hub import snapshot_download

mkdir /home/Ubuntu/dog
local_dir = "/home/Ubuntu/dog"
snapshot_download(
    "diffusers/dog-example",
    local_dir=local_dir, repo_type="dataset",
    ignore_patterns=".gitattributes",
)

============

export MODEL_NAME="stabilityai/stable-diffusion-3-medium-diffusers"
export INSTANCE_DIR="/home/Ubuntu/dog"
export OUTPUT_DIR="trained-sd3-lora"

accelerate launch train_dreambooth_lora_sd3.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --instance_data_dir=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --mixed_precision="fp16" \
  --instance_prompt="a photo of sks dog" \
  --resolution=512 \
  --train_batch_size=1 \
  --gradient_accumulation_steps=4 \
  --learning_rate=1e-5 \
  --report_to="wandb" \
  --lr_scheduler="constant" \
  --lr_warmup_steps=0 \
  --max_train_steps=500 \
  --validation_prompt="A photo of sks dog in a bucket" \
  --validation_epochs=25 \
  --seed="0" \
  --push_to_hub


2 comments:

Muhammad Nouman Khan said...

I am running the same code, but i got a weard error: it says that CLIPTokenizer is not there in 'stabilityai/stable-diffusion-3-medium-diffusers' model on huggingface

ERROR:
[rank1]: Traceback (most recent call last):
[rank1]: File "/var/Nouman/ME/diffusers/examples/dreambooth/train_dreambooth_lora_sd3.py", line 1861, in
[rank1]: main(args)
[rank1]: File "/var/Nouman/ME/diffusers/examples/dreambooth/train_dreambooth_lora_sd3.py", line 1119, in main
[rank1]: tokenizer_one = CLIPTokenizer.from_pretrained(
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/anaconda3/envs/sdft_env/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2147, in from_pretrained
[rank1]: raise EnvironmentError(
[rank1]: OSError: Can't load tokenizer for 'stabilityai/stable-diffusion-3-medium-diffusers'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'stabilityai/stable-diffusion-3-medium-diffusers' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.
W0710 18:23:23.854000 140416574048064 torch/distributed/elastic/multiprocessing/api.py:851] Sending process 744735 closing signal SIGTERM
E0710 18:23:23.919000 140416574048064 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: 1) local_rank: 1 (pid: 744736) of binary: /home/anaconda3/envs/sdft_env/bin/python
Traceback (most recent call last):
File "/home/anaconda3/envs/sdft_env/bin/accelerate", line 8, in
sys.exit(main())
^^^^^^
File "/home/anaconda3/envs/sdft_env/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 48, in main
args.func(args)
File "/home/anaconda3/envs/sdft_env/lib/python3.11/site-packages/accelerate/commands/launch.py", line 1088, in launch_command
multi_gpu_launcher(args)
File "/home/anaconda3/envs/sdft_env/lib/python3.11/site-packages/accelerate/commands/launch.py", line 733, in multi_gpu_launcher
distrib_run.run(args)
File "/home/anaconda3/envs/sdft_env/lib/python3.11/site-packages/torch/distributed/run.py", line 870, in run
elastic_launch(
File "/home/anaconda3/envs/sdft_env/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
return launch_agent(self._config, self._entrypoint, list(args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/sdft_env/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 263, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
train_dreambooth_lora_sd3.py FAILED
------------------------------------------------------------
Failures:

ABS said...

I am getting the same above error. Please help