inference_with_deliberate.md

Inference with the Deliberate Model

Introduction

Deliberate is a highly specialized Image generation AI Model of type Safetensors / Checkpoint AI Model created by AI community user XpucT. An extensive fine-tuning process based on SD 1.5 makes it capable of generating images highly relevant to the specific use cases it was designed for, such as character, girl, 3d.

Take Deliberate_v2 as an example, this doc shows how to infer with the Deliberate model based on Mindone SD 1.5.

Step 1. Model conversion

Download Deliberate_v2.safetensor from the XpucT/Deliberate model card website. Tools for SD 1.x or SD 2.x model conversion from torch to MindSpore format are provided. Convert the Deliberate_v2.safetensor by running:

cd examples/stable_diffusion_v2
python tools/model_conversion/convert_weights.py \
  --source /path/to/Deliberate_v2.safetensor \
  --target /path/to/Deliberate_v2.ckpt \
  --model sdv1 \
  --source_version pt

Step 2. Inference

To perform image-to-image generation with the converted checkpoints, run the following commands. By default, 8 images (n_iter * n_samples) will be sampled for the prompt and wrote to output_path='output'.

# dpm++ 2M sampler (by default)
python text_to_image.py \
  -v 1.5 \
  --ckpt_path /path/to/Deliberate_v2.ckpt \
  --prompt {text prompt}
  --n_iter 2 # by default
  --n_samples 4 # by default

Run with more prompts by using --data_path. Save the prompts as prompts.txt with each line as a single one and run the command:

python text_to_image.py \
  -v 1.5 \
  --ckpt_path /path/to/Deliberate_v2.ckpt \
  --data_path /path/to/prompts.txt

Run with different schedulers:

# dpm++ 2M sampler (by default)
python text_to_image.py \
  -v 1.5 \
  --ckpt_path /path/to/Deliberate_v2.ckpt \
  --prompt {text prompt}

# dpm 2M
python text_to_image.py \
  -v 1.5 \
  --ckpt_path /path/to/Deliberate_v2.ckpt \
  --prompt {text prompt}
  --dpm_solver

# plms (equivalent to pndm)
python text_to_image.py \
  -v 1.5 \
  --ckpt_path /path/to/Deliberate_v2.ckpt \
  --prompt {text prompt}
  --plms