stablepy_demo.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/R3gm/stablepy/blob/main/stablepy_demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BQeX1ykNNMla"
      },
      "source": [
        "# Stablepy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TYqyA785NZF8"
      },
      "source": [
        "Install dependencies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "itGk9x06NL5O"
      },
      "outputs": [],
      "source": [
        "!pip install stablepy==0.6.0 -q"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7YIpscy8sjgs"
      },
      "source": [
        "To use the version with the latest changes, you can install directly from the repository.\n",
        "\n",
        "`pip install -q git+https://github.com/R3gm/stablepy.git`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KdNshU7kNbLj"
      },
      "source": [
        "Download our models and other stuffs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GtTdza7SNexT"
      },
      "outputs": [],
      "source": [
        "%cd /content/\n",
        "\n",
        "# Model\n",
        "!wget https://huggingface.co/frankjoshua/toonyou_beta6/resolve/main/toonyou_beta6.safetensors\n",
        "!wget https://huggingface.co/RunDiffusion/Juggernaut-XL-v9/resolve/main/Juggernaut-XL_v9_RunDiffusionPhoto_v2.safetensors\n",
        "\n",
        "# VAE\n",
        "!wget https://huggingface.co/fp16-guy/anything_kl-f8-anime2_vae-ft-mse-840000-ema-pruned_blessed_clearvae_fp16_cleaned/resolve/main/anything_fp16.safetensors\n",
        "\n",
        "# LoRAs\n",
        "!wget https://civitai.com/api/download/models/183149 --content-disposition\n",
        "!wget https://civitai.com/api/download/models/97655 --content-disposition\n",
        "\n",
        "# Embeddings\n",
        "!wget https://huggingface.co/embed/negative/resolve/main/bad-hands-5.pt\n",
        "!wget https://huggingface.co/embed/negative/resolve/main/bad-artist.pt\n",
        "\n",
        "# Upscaler\n",
        "!wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "amYJfvMwOKnL"
      },
      "source": [
        "# Inference with Stable diffusion 1.5"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PvTT0AiKRX0m"
      },
      "source": [
        "First, we pass the path of the model we will use.\n",
        "\n",
        "The default task is txt2img but it can be changed to other tasks like canny"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9hvZngGDOhh5"
      },
      "outputs": [],
      "source": [
        "from stablepy import Model_Diffusers\n",
        "import torch\n",
        "\n",
        "model_path = \"./toonyou_beta6.safetensors\"\n",
        "vae_path = \"./anything_fp16.safetensors\"\n",
        "\n",
        "model = Model_Diffusers(\n",
        "    base_model_id = model_path, # path to the model\n",
        "    task_name = \"canny\", # task\n",
        "    vae_model = vae_path, # path vae\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HFJUWOzmMElv"
      },
      "source": [
        "You can see the different tasks that can be used with sd1.5 with the following code:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i8WGUSPN05RD"
      },
      "outputs": [],
      "source": [
        "from stablepy import SD15_TASKS\n",
        "\n",
        "SD15_TASKS"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CQNbptvYSs4z"
      },
      "source": [
        "To switch tasks or models, we can call `model.load_pipe()` and specify the new task or model. This will load the necessary components."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jnO6p80qPRS9"
      },
      "outputs": [],
      "source": [
        "model.load_pipe(\n",
        "    base_model_id = model_path, # path to the model\n",
        "    task_name = \"txt2img\", # task\n",
        "    vae_model = None, # Use default VAE\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hNdnzYZXMZwO"
      },
      "source": [
        "Simple generation using a sampler and a specified prompt weight."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FCYd5Ya5z1Ic"
      },
      "outputs": [],
      "source": [
        "images, img_info = model(\n",
        "    prompt = \"cat, (masterpiece), (best quality)\",\n",
        "    sampler=\"DPM++ SDE\",\n",
        "    schedule_type=\"Karras\",\n",
        "    syntax_weights=\"Classic-original\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The output consists of the images generated in a list"
      ],
      "metadata": {
        "id": "A5ZXM34tjfYU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "images[0]"
      ],
      "metadata": {
        "id": "p2-OHc1Dje55"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The output also includes img_info, which contains details like the **seed**, the **path** where the image was saved, and the **generation metadata**."
      ],
      "metadata": {
        "id": "6k8hSFEIkWzo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "img_info"
      ],
      "metadata": {
        "id": "w707FqK4jMoh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "scqyCMnoNJ1Q"
      },
      "source": [
        "The different samplers that can be used can be checked in the following way:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZJz5pVDx1-UJ"
      },
      "outputs": [],
      "source": [
        "from stablepy import scheduler_names\n",
        "\n",
        "scheduler_names"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "And for the schedule types:"
      ],
      "metadata": {
        "id": "eKMErUczihf-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from stablepy import SCHEDULE_TYPE_OPTIONS\n",
        "\n",
        "SCHEDULE_TYPE_OPTIONS"
      ],
      "metadata": {
        "id": "akYwc8mWie_9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZStd44a_NRIU"
      },
      "source": [
        "Prompt weight is the syntax and method used to emphasize certain parts of the prompt. If you want to get results similar to other popular implementations, you can use \"Classic-original\"."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4oesBsst1u7i"
      },
      "outputs": [],
      "source": [
        "from stablepy import ALL_PROMPT_WEIGHT_OPTIONS\n",
        "\n",
        "ALL_PROMPT_WEIGHT_OPTIONS"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fqmU2o0fTUvZ"
      },
      "source": [
        "We will use a basic txt2img task in which we can specify different common parameters, such as Loras, embeddings, upscaler, etc."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zVrtRLuFPSiQ"
      },
      "outputs": [],
      "source": [
        "from IPython.display import display\n",
        "\n",
        "lora1_path = \"./EmptyEyes_Diffuser_v10.safetensors\"\n",
        "lora2_path = \"./FE_V2.safetensors\" # pixel art lora\n",
        "upscaler_path = \"./RealESRGAN_x4plus.pth\"\n",
        "\n",
        "images, image_list = model(\n",
        "    prompt = \"pixel art (masterpiece, best quality), 1girl, collarbone, wavy hair, looking at viewer, blurry foreground, upper body, necklace, contemporary, plain pants, ((intricate, print, pattern)), ponytail, freckles, red hair, dappled sunlight, smile, happy,\",\n",
        "    negative_prompt = \"(worst quality, low quality, letterboxed), bad_artist_token, bad_hand_token\",\n",
        "    img_width = 513,\n",
        "    img_height = 1024,\n",
        "    num_images = 1,\n",
        "    num_steps = 30,\n",
        "    guidance_scale = 8.0,\n",
        "    clip_skip = True, # Clip skip to the penultimate layer, in other implementations it is equivalent to use clipskip 2.\n",
        "    seed = -1, # random seed\n",
        "    sampler=\"DPM++ SDE\",\n",
        "    schedule_type=\"Karras\",\n",
        "    syntax_weights=\"Classic-original\",\n",
        "\n",
        "    lora_A = lora1_path,\n",
        "    lora_scale_A = 0.8,\n",
        "    lora_B = lora2_path,\n",
        "    lora_scale_B = 0.9,\n",
        "\n",
        "    textual_inversion=[(\"bad_artist_token\", \"./bad-artist.pt\"), (\"bad_hand_token\", \"./bad-hands-5.pt\")], # Is a list of tuples with [(\"<token_activation>\",\"<path_embeding>\"),...]\n",
        "\n",
        "    upscaler_model_path = upscaler_path, # Upscale the image and Hires-fix\n",
        "    upscaler_increases_size=1.5,\n",
        "    hires_steps = 25,\n",
        "    hires_denoising_strength = 0.35,\n",
        "    hires_prompt = \"\", # If this is left as is, the main prompt will be used instead.\n",
        "    hires_negative_prompt = \"\",\n",
        "    hires_sampler = \"Use same sampler\",\n",
        "\n",
        "    #By default, the generated images are saved in the current location within the 'images' folder.\n",
        "    image_storage_location = \"./images\",\n",
        "\n",
        "    #You can disable saving the images with this parameter.\n",
        "    save_generated_images = False,\n",
        ")\n",
        "\n",
        "for image in images:\n",
        "  display(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V0-kKkqzbLwa"
      },
      "source": [
        "The upscaler_model_path can be used with different models and can also be used any of the builtin upscalers.\n",
        "Example: `upscaler_model_path=\"Latent (bicubic)\",`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Pc6cjdVfeOWM"
      },
      "outputs": [],
      "source": [
        "from stablepy import ALL_BUILTIN_UPSCALERS\n",
        "\n",
        "ALL_BUILTIN_UPSCALERS"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zSYYzJ7FXO2d"
      },
      "source": [
        "## ControlNet"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4BsMsk0SXjtN"
      },
      "outputs": [],
      "source": [
        "model.load_pipe(\n",
        "    base_model_id = model_path,\n",
        "    task_name = \"canny\",\n",
        "    # Use default VAE\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PF1PCOyKXvSx"
      },
      "source": [
        "Our control image will be this one, to which the processor will apply Canny, and then use ControlNet to generate the final image."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ia5fu84QW1MM"
      },
      "outputs": [],
      "source": [
        "from PIL import Image\n",
        "import os\n",
        "\n",
        "def download_image(img_url):\n",
        "    filename = os.path.basename(img_url)\n",
        "    !wget -q {img_url} -O {filename}\n",
        "    img = Image.open(filename)\n",
        "    img.thumbnail((300, 300))\n",
        "    display(img)\n",
        "    return filename  # return the path\n",
        "\n",
        "control_image = download_image(\"https://huggingface.co/lllyasviel/sd-controlnet-canny/resolve/main/images/bird.png\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3AjvhCa3dsBn"
      },
      "source": [
        "Inference with canny"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NefvFTvCWL8v"
      },
      "outputs": [],
      "source": [
        "images, image_list = model(\n",
        "    prompt = \"(masterpiece, best quality), bird\",\n",
        "    negative_prompt = \"(worst quality, low quality, letterboxed)\",\n",
        "    image = control_image,\n",
        "    preprocessor_name = \"Canny\", # Needed to activate the Canny preprocessor\n",
        "    preprocess_resolution = 512, # It is the resize of the image that will be obtained from the preprocessor.\n",
        "    image_resolution = 768, # The max proportional final resolution based on the provided image.\n",
        "    controlnet_conditioning_scale = 1.0, # ControlNet Output Scaling in UNet\n",
        "    control_guidance_start = 0.0, # ControlNet Start Threshold (%)\n",
        "    control_guidance_end= 1.0, # ControlNet Stop Threshold (%)\n",
        "\n",
        "    upscaler_model_path = upscaler_path,\n",
        "    upscaler_increases_size=1.3,\n",
        "\n",
        "    # By default, 'hires-fix' is applied when we use an upscaler; to deactivate it, we can set 'hires steps' to 0\n",
        "    hires_steps = 0,\n",
        ")\n",
        "\n",
        "for image in images:\n",
        "  display(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_4ETyFwVm4wt"
      },
      "source": [
        "Valid `preprocessor_name` depending on the task:\n",
        "\n",
        "\n",
        "| Task name    | Preprocessor Name |\n",
        "|----------|-------------------|\n",
        "|canny|\"None\" \"Canny\"|\n",
        "|mlsd|\"None\" \"MLSD\"|\n",
        "| openpose | \"None\" \"Openpose\" \"Openpose core\" |\n",
        "|scribble|\"None\" \"HED\" \"PidiNet\" \"TEED\" |\n",
        "|softedge|\"None\" \"HED\" \"PidiNet\" \"HED safe\" \"PidiNet safe\" \"TEED\" |\n",
        "|segmentation|\"None\" \"UPerNet\" \"SegFormer\"|\n",
        "|depth|\"None\" \"DPT\" \"Midas\" \"ZoeDepth\" \"DepthAnything\"|\n",
        "|normalbae|\"None\" \"NormalBae\"|\n",
        "|lineart|\"None\" \"Lineart\" \"Lineart coarse\" \"None (anime)\" \"LineartAnime\" \"Lineart standard\" \"Anyline\"|\n",
        "|lineart_anime|\"None\" \"Lineart\" \"Lineart coarse\" \"None (anime)\" \"LineartAnime\" \"Lineart standard\" \"Anyline\"|\n",
        "|tile|\"None\" \"Blur\"|\n",
        "|recolor|\"None\" \"Recolor luminance\" \"Recolor intensity\"|\n",
        "|shuffle|\"None\" \"ContentShuffle\"|\n",
        "|repaint|\"None\"|\n",
        "|pattern|\"None\"|\n",
        "|ip2p|\"None\"|\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fcuPwmjudAc5"
      },
      "source": [
        "## Adetailer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o_0OE-PaPGOl"
      },
      "outputs": [],
      "source": [
        "model.load_pipe(\n",
        "    base_model_id = model_path,\n",
        "    task_name = \"txt2img\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RxBK7tcLdENa"
      },
      "source": [
        "There must be a match of parameters for good results to be obtained with adetailer, it is also useful to use `strength` in adetailer_inpaint_params with low values ​​below 0.4."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "F_b2jCSBcVBH"
      },
      "outputs": [],
      "source": [
        "# These are the parameters that adetailer A uses by default, but we can modify them if needed, the same applies to adetailer B.\n",
        "adetailer_params_A = {\n",
        "    \"face_detector_ad\" : True,\n",
        "    \"person_detector_ad\" : True,\n",
        "    \"hand_detector_ad\" : False,\n",
        "    \"prompt\": \"\", # The main prompt will be used if left empty\n",
        "    \"negative_prompt\" : \"\",\n",
        "    \"strength\" : 0.35, # need low values\n",
        "    \"mask_dilation\" : 4,\n",
        "    \"mask_blur\" : 4,\n",
        "    \"mask_padding\" : 32,\n",
        "    \"inpaint_only\" : True, # better\n",
        "    \"sampler\" : \"Use same sampler\",\n",
        "}\n",
        "\n",
        "images, image_list = model(\n",
        "    prompt = \"(masterpiece, best quality), 1girl, collarbone, wavy hair, looking at viewer, blurry foreground, upper body, necklace, contemporary, plain pants, ((intricate, print, pattern)), ponytail, freckles, red hair, dappled sunlight, smile, happy,\",\n",
        "    negative_prompt = \"(worst quality, low quality, letterboxed)\",\n",
        "    img_width = 512,\n",
        "    img_height = 1024,\n",
        "    num_images = 1,\n",
        "    num_steps = 30,\n",
        "    guidance_scale = 8.0,\n",
        "    clip_skip = True,\n",
        "    seed = 33,\n",
        "    sampler=\"DPM++ SDE\",\n",
        "    schedule_type=\"Karras\",\n",
        "\n",
        "    FreeU=True, # Improves diffusion model sample quality at no costs.\n",
        "    pag_scale=3.0,  # PAG enhances image quality and is typically set to 3.0, which can be effective in some cases.\n",
        "    adetailer_A=True,\n",
        "    adetailer_A_params=adetailer_params_A,\n",
        "\n",
        "    adetailer_B=True, # \"If we don't use adetailer_B_params, it will use default values.\n",
        "\n",
        "    # By default, the upscaler will be deactivated if we don't pass a model to it.\n",
        "    # It's also valid to use a url to the model, Lanczos or Nearest.\n",
        "    #upscaler_model_path = \"Lanczos\",\n",
        ")\n",
        "\n",
        "for image in images:\n",
        "  display(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aDnuReyiiB7i"
      },
      "source": [
        "## Inpaint"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9YlKpQ9liDbx"
      },
      "outputs": [],
      "source": [
        "model.load_pipe(\n",
        "    base_model_id = model_path,\n",
        "    task_name = \"inpaint\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GF2hjejxjPgp"
      },
      "source": [
        "We can specify the directory of our mask image, but we can also generate it, which is what we'll do in this example\n",
        "\n",
        "You need a mouse to draw on this canvas."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TTnxYmsGJYj9"
      },
      "outputs": [],
      "source": [
        "images, image_list = model(\n",
        "    image = control_image,\n",
        "    # image_mask = \"/my_mask.png\",\n",
        "    prompt = \"a blue bird\",\n",
        "    strength = 0.5,\n",
        "    negative_prompt = \"(worst quality, low quality, letterboxed)\",\n",
        "    image_resolution = 768,  # The equivalent resolution to be used for inference.\n",
        "    sampler=\"DPM++ SDE\",\n",
        "    schedule_type=\"Karras\",\n",
        ")\n",
        "\n",
        "for image in images:\n",
        "  display(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a3Bci8VHmG5N"
      },
      "source": [
        "If you're using a device without a mouse or Jupyter Notebook outside of Colab, the function to create a mask automatically won't work correctly. Therefore, you'll need to specify the path of your mask image manually."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U8mc0S5vmQ7s"
      },
      "source": [
        "# Styles\n",
        "These are additions to the prompt and negative prompt to utilize a specific style in generation. By default, there are only 9 of these, and we can know their names by using:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o8bnYMsXm9o2"
      },
      "outputs": [],
      "source": [
        "model.STYLE_NAMES"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gl5IE01_nSTl"
      },
      "source": [
        "But if we want to use other styles, we can load them through a JSON, like this one for example.\n",
        "Here are more JSON style files: [PromptStylers](https://github.com/wolfden/ComfyUi_PromptStylers), [sdxl_prompt_styler](https://github.com/ali1234/sdxl_prompt_styler/tree/main)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eM3aiE1RjoQN"
      },
      "outputs": [],
      "source": [
        "!wget https://raw.githubusercontent.com/ahgsql/StyleSelectorXL/main/sdxl_styles.json"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V-IfivFJnijz"
      },
      "outputs": [],
      "source": [
        "model.load_style_file(\"sdxl_styles.json\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FY5LCRDRoWBM"
      },
      "source": [
        "The file was loaded with 77 styles replacing the previous ones, now we can see the new names:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ldNxLj_ooXxX"
      },
      "outputs": [],
      "source": [
        "model.STYLE_NAMES"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FQkfWmvRow7Q"
      },
      "source": [
        "Now we can use the style in the inference."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NKYtqaxlo1_f"
      },
      "outputs": [],
      "source": [
        "# Image to Image task.\n",
        "model.load_pipe(\n",
        "    base_model_id = model_path,\n",
        "    task_name = \"img2img\",\n",
        ")\n",
        "\n",
        "# We can also use multiple styles in a list [\"Silhouette\", \"Kirigami\"]\n",
        "images, image_list = model(\n",
        "    style_prompt = \"Silhouette\", # The style will be added to the prompt and negative prompt\n",
        "    image = control_image,\n",
        "    prompt = \"a bird\",\n",
        "    negative_prompt = \"worst quality\",\n",
        "    strength = 0.48,\n",
        "    image_resolution = 512,\n",
        "    sampler=\"DPM++ SDE\",\n",
        "    schedule_type=\"Karras\",\n",
        ")\n",
        "\n",
        "for image in images:\n",
        "  display(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1v2n80ujO6My"
      },
      "source": [
        "#Verbosity Level\n",
        "To change the verbosity level, you can use the logger from StablePy\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "97XN4eNfsgWu"
      },
      "outputs": [],
      "source": [
        "import logging\n",
        "from stablepy import logger\n",
        "\n",
        "logging_level_mapping = {\n",
        "    'DEBUG': logging.DEBUG,\n",
        "    'INFO': logging.INFO,\n",
        "    'WARNING': logging.WARNING,\n",
        "    'ERROR': logging.ERROR,\n",
        "    'CRITICAL': logging.CRITICAL\n",
        "}\n",
        "\n",
        "Verbosity_Level = \"WARNING\" # Messages INFO and DEBUG will not be printed\n",
        "\n",
        "logger.setLevel(logging_level_mapping.get(Verbosity_Level, logging.INFO))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6YY8c3T2gcpq"
      },
      "source": [
        "# LCM and TCD\n",
        "\n",
        "Latent Consistency Models (LCM) can generate images in a few steps. When selecting the 'LCM Auto-Loader' or 'TCD Auto-Loader' sampler, the model automatically loads the LCM_LoRA for the task. Generally, guidance_scale is used at 1.0 or a maximum of 2.0, with steps between 4 and 8.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "O3TVADzdge3T"
      },
      "outputs": [],
      "source": [
        "# Generating an image with txt2img\n",
        "model.load_pipe(\n",
        "    base_model_id = model_path,\n",
        "    task_name = \"txt2img\",\n",
        ")\n",
        "images, image_list = model(\n",
        "    prompt = \"(masterpiece, best quality), 1girl, collarbone, wavy hair, looking at viewer, blurry foreground, upper body, necklace, contemporary, plain pants, ((intricate, print, pattern)), ponytail, freckles, red hair, dappled sunlight, smile, happy,\",\n",
        "    negative_prompt = \"(worst quality, low quality, letterboxed)\",\n",
        "    num_images = 1,\n",
        "    num_steps = 7,\n",
        "    guidance_scale = 1.0,\n",
        "    sampler=\"LCM Auto-Loader\",  # or 'TCD Auto-Loader'\n",
        "    syntax_weights=\"Classic\", # (word:weight) and (word) for prompts weights\n",
        "    disable_progress_bar = True,\n",
        "    save_generated_images = False,\n",
        "    display_images = True,\n",
        ")\n",
        "\n",
        "# Using the image generated in img2img\n",
        "# If we use the same model and VAE, we can switch tasks quickly\n",
        "model.load_pipe(\n",
        "    base_model_id = model_path,\n",
        "    task_name = \"img2img\",\n",
        ")\n",
        "images_i2i, image_list = model(\n",
        "    prompt = \"masterpiece, sunlight\",\n",
        "    image = images[0], # only one image\n",
        "    style_prompt = \"Disco\", # Apply a style\n",
        "    strength = 0.70,\n",
        "    num_steps = 5,\n",
        "    guidance_scale = 1.0,\n",
        "    sampler=\"LCM Auto-Loader\",\n",
        "    disable_progress_bar = True,\n",
        "    save_generated_images = False,\n",
        "    display_images = True,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vsC_l6lxhj7r"
      },
      "outputs": [],
      "source": [
        "logger.setLevel(logging.INFO) # return info"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GY0skyTXntas"
      },
      "source": [
        "# Inference with SDXL"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ue9h7FKGWc1j"
      },
      "source": [
        "The tasks that can be used with SDXL:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iXAiSLXx94-3"
      },
      "outputs": [],
      "source": [
        "from stablepy import SDXL_TASKS\n",
        "\n",
        "SDXL_TASKS"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zsqajNfYX44N"
      },
      "source": [
        "\n",
        "When switching between different tasks, you may encounter an out-of-memory (OOM) issue if there isn't enough GPU memory available, particularly with SDXL. To avoid this, you can set `retain_task_model_in_memory=False` in `model.load_pipe` or `Model_Diffusers` to save some VRAM. However, in many cases, this may not be sufficient, and you may need to restart the kernel or runtime in Colab to resolve the issue."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "H5L6K3dr9c3y"
      },
      "outputs": [],
      "source": [
        "model_name = \"./Juggernaut-XL_v9_RunDiffusionPhoto_v2.safetensors\"  # SDXL safetensors\n",
        "\n",
        "model.load_pipe(\n",
        "    base_model_id = model_name,\n",
        "    task_name = \"openpose\",\n",
        "    retain_task_model_in_cache=False,  # version 0.6.0 default is False\n",
        ")\n",
        "\n",
        "model.advanced_params(image_preprocessor_cuda_active=True)  # Default is False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SUAHnEhfZIkv"
      },
      "source": [
        "We will perform OpenPose with the following image using SDXL."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "G7OxI1mWqDwL"
      },
      "outputs": [],
      "source": [
        "control_image_2 = download_image(\"https://upload.wikimedia.org/wikipedia/commons/f/f8/Model_Posing_On_Typical_Studio_Set.jpg\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3eKyULPjCJuG"
      },
      "outputs": [],
      "source": [
        "images, image_list = model(\n",
        "    image = control_image_2,\n",
        "    prompt = \"a man with a pink jacket in the jungle\",\n",
        "    negative_prompt = \"worst quality\",\n",
        "\n",
        "    # Relative resolution\n",
        "    image_resolution = 1024,\n",
        "    preprocessor_name = \"Openpose\", # for get the image preprocessor\n",
        "    sampler=\"Euler a\",\n",
        ")\n",
        "\n",
        "for image in images:\n",
        "  display(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bjfi-n3ShMzb"
      },
      "source": [
        "# Diffusers format"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KjHk_2ZOjVrQ"
      },
      "source": [
        "\n",
        "You can also load models in the Diffusers format. This format divides the model into different parts, which allows you to load individual sections from various models more easily. For instance, models like SD 1.5 and SDXL can be loaded using the repository name as shown in this example: [RealVisXL_V2.0](https://huggingface.co/SG161222/RealVisXL_V2.0/tree/main). This repository contains folders corresponding to each section of the model such as unet, vae, text encoder, and more.\n",
        "\n",
        "Another characteristic of the diffusers format is that it can use either the safetensors or bin extension. Currently, you can only use diffuser models in the safetensors extension because they offer better performance and are safer than bin files. To verify if a diffuser model is in safetensors format, check the [unet folder](https://huggingface.co/SG161222/RealVisXL_V2.0/tree/main/unet) and see if it ends with the safetensors extension."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Dzpva586tcvT"
      },
      "outputs": [],
      "source": [
        "repo = \"SG161222/RealVisXL_V2.0\"\n",
        "\n",
        "model.load_pipe(\n",
        "    base_model_id = repo, # path to the model\n",
        "    task_name = \"canny\", # task\n",
        "    vae_model = repo, # backed vae\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mLzuQDBfny1x"
      },
      "source": [
        "The T2I-Adapter depth is similar to that of ControlNet and uses less VRAM"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PJ3nryKR4f8A"
      },
      "outputs": [],
      "source": [
        "# Example sdxl_depth-midas\n",
        "model.load_pipe(\n",
        "    base_model_id = repo, # sdxl repo\n",
        "    task_name = \"sdxl_depth-midas_t2i\",\n",
        "    retain_task_model_in_cache=False,\n",
        ")\n",
        "\n",
        "# We can also use multiple styles in a list [\"Silhouette\", \"Kirigami\"]\n",
        "images, image_list = model(\n",
        "    image = control_image,\n",
        "    prompt = \"a green bird\",\n",
        "    negative_prompt = \"worst quality\",\n",
        "\n",
        "    # If we want to use the preprocessor\n",
        "    t2i_adapter_preprocessor = True,\n",
        "    preprocess_resolution = 1024,\n",
        "\n",
        "    # Relative resolution\n",
        "    image_resolution = 1024,\n",
        "\n",
        "    sampler=\"DPM++ 2M SDE\", # We can also use euler at final with \"DPM++ 2M SDE Ef\"\n",
        "\n",
        "    t2i_adapter_conditioning_scale = 1.0,\n",
        "    t2i_adapter_conditioning_factor = 1.0,\n",
        "\n",
        "    display_images = True,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TTl8AaZ9RNzo"
      },
      "source": [
        "# ControlNet pattern"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FqdEZa6BRQSS"
      },
      "source": [
        "It is used to generate images with a QR code but can also be used to generate optical patterns."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j8Yws7siRza9"
      },
      "outputs": [],
      "source": [
        "spiral_image = download_image(\"https://upload.wikimedia.org/wikipedia/en/6/6c/Screwtop_spiral.jpg\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7qXf9YsvSQmm"
      },
      "outputs": [],
      "source": [
        "repo = \"SG161222/RealVisXL_V2.0\"\n",
        "\n",
        "model.load_pipe(\n",
        "    base_model_id = repo,\n",
        "    task_name = \"pattern\",\n",
        "    retain_task_model_in_cache=False,\n",
        ")\n",
        "\n",
        "images, image_list = model(\n",
        "    image = spiral_image,\n",
        "    prompt = \"a jungle landscape\",\n",
        "    negative_prompt = \"worst quality\",\n",
        "    sampler=\"DPM++ 2M SDE\",\n",
        "    schedule_type=\"Lambdas\",\n",
        "    image_resolution = 1024,\n",
        ")\n",
        "\n",
        "for image in images:\n",
        "  display(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3UuGqmo9gwaH"
      },
      "source": [
        "# IP Adapter"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MJoMRPFAbUMw"
      },
      "source": [
        "IP-Adapter enhances diffusion models by adding a dedicated image cross-attention layer for better image-specific feature learning and adaptability."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5vaBv_1NgyYp"
      },
      "outputs": [],
      "source": [
        "from stablepy import IP_ADAPTERS_SD, IP_ADAPTERS_SDXL\n",
        "\n",
        "IP_ADAPTERS_SDXL"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kROso95NcMCl"
      },
      "source": [
        "\n",
        "You can specify a list of different IP adapter models with their respective images, but combinations between normal IP adapters and FaceID adapters are not enabled. Additionally, base_vit_G models cannot be combined with other models because they use a different image encoder. This image encoder is necessary for most IP adapter models and will occupy additional space in VRAM.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aKtEMwB4nC0g"
      },
      "outputs": [],
      "source": [
        "img_ip = download_image(\"https://upload.wikimedia.org/wikipedia/commons/3/3f/TechCrunch_Disrupt_2019_%2848834434641%29_%28cropped%29.jpg\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lfnxb92TlncN"
      },
      "outputs": [],
      "source": [
        "model_name = \"./Juggernaut-XL_v9_RunDiffusionPhoto_v2.safetensors\"  # SDXL safetensors\n",
        "\n",
        "model.load_pipe(\n",
        "    base_model_id = model_name,\n",
        "    task_name = \"txt2img\",\n",
        "    retain_task_model_in_cache=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vTsErl_3iNU2"
      },
      "outputs": [],
      "source": [
        "images, image_list = model(\n",
        "    prompt = \"a man with a pink jacket in the jungle\",\n",
        "    negative_prompt = \"worst quality\",\n",
        "\n",
        "    img_width = 1024,\n",
        "    img_height = 1024,\n",
        "    num_images = 1,\n",
        "    num_steps = 30,\n",
        "    guidance_scale = 8.0,\n",
        "    sampler=\"Euler a\",\n",
        "\n",
        "    ip_adapter_image = [img_ip],\n",
        "    ip_adapter_mask = [],\n",
        "    ip_adapter_model = [\"plus_face\"],\n",
        "    ip_adapter_scale = [0.9],\n",
        "    ip_adapter_mode = [\"original\"],\n",
        "\n",
        "    display_images = True,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oWJGI6IdtxW1"
      },
      "source": [
        "#### Multi IP-Adapter\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XCVO3eEutxHS"
      },
      "outputs": [],
      "source": [
        "images, image_list = model(\n",
        "    prompt = \"a man with a pink jacket in the jungle\",\n",
        "    negative_prompt = \"worst quality\",\n",
        "\n",
        "    img_width = 1024,\n",
        "    img_height = 1024,\n",
        "    num_images = 1,\n",
        "    num_steps = 30,\n",
        "    guidance_scale = 8.0,\n",
        "    sampler=\"Euler a\",\n",
        "\n",
        "    ip_adapter_image = [img_ip, control_image],  # face image and the bird image\n",
        "    ip_adapter_mask = [],                        # You can specify masks to isolate the influence area of each (similar to the inpainting mask)\n",
        "    ip_adapter_model = [\"plus_face\", \"base\"],\n",
        "    ip_adapter_scale = [0.9, 1.0],               # Scale 0.0 is equivalent to disabled\n",
        "    ip_adapter_mode = [\"original\", \"style\"],     # The second IP adapter will take the style properties of the bird image\n",
        "\n",
        "    display_images = True,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dF8D0zej2xOc"
      },
      "source": [
        "- `ip_adapter_mode` specifies which layers are active in the IP adapter model, certain layers have specific influence on how features are extracted from the IP image, the valid options for this are \"original\", \"style\", \"layout\" and \"style+layout\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bF-15bKGT9W6"
      },
      "source": [
        "- You can also use multiple images with a single IP adapter as follows by placing them in a list:\n",
        "\n",
        "```\n",
        "    ip_adapter_image = [[img_ip_1, img_ip_2, img_ip_3, img_ip_4]],\n",
        "    ip_adapter_mask = [],\n",
        "    ip_adapter_model = [\"plus_face\"],\n",
        "    ip_adapter_scale = [0.9],\n",
        "    ip_adapter_mode = [\"original\"],\n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8XbyGTTlDTPc"
      },
      "source": [
        "# Displaying preview images during generation steps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "L1xb0G07DTPe"
      },
      "outputs": [],
      "source": [
        "model.load_pipe(\n",
        "    base_model_id = repo,\n",
        "    task_name = \"txt2img\",\n",
        "    retain_task_model_in_cache=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "10DTNoR3DTPe"
      },
      "source": [
        "By setting `image_previews=True`, an iterable generator object for image previews will be created."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8TgHk9SxDTPf"
      },
      "outputs": [],
      "source": [
        "stream = model(\n",
        "    prompt = \"a cat\",\n",
        "    negative_prompt = \"worst quality\",\n",
        "    sampler=\"DPM++ 2M SDE\",\n",
        "    schedule_type=\"Lambdas\",\n",
        "    img_width = 768,\n",
        "    img_height = 768,\n",
        "    image_previews=True,\n",
        ")\n",
        "\n",
        "# Iterate over the generator object to get each value\n",
        "for img, info_img in stream:\n",
        "    display(img[0])\n",
        "    if info_img[1]:\n",
        "        print(info_img)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YqETYZi3DTPf"
      },
      "source": [
        "### Config the stream parameters\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "21-BMmjyDTPg"
      },
      "outputs": [],
      "source": [
        "model.stream_config(\n",
        "    concurrency=2,\n",
        "    latent_resize_by=1,\n",
        "    vae_decoding=False,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TJjHktt4DTPh"
      },
      "source": [
        "- **concurrency**: `int`\n",
        "  - **Default**: 5\n",
        "  - **Description**: Controls how often the preview images are generated and displayed in relation to the steps. For example, a value of 2 displays an image every 2 steps.\n",
        "\n",
        "- **latent_resize_by**: `int`\n",
        "  - **Default**: 8\n",
        "  - **Description**: Controls the scaling size of the latent images. A value of 1 is useful for achieving high performance.\n",
        "\n",
        "- **vae_decoding**: `bool`\n",
        "  - **Default**: `False`\n",
        "  - **Description**: Use the VAE to decode the preview images. If set to `True`, it may negatively impact performance."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Nlyj1jqro3lR"
      },
      "source": [
        "# Utils"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OISDbhbljYNu"
      },
      "outputs": [],
      "source": [
        "# Load beta styles\n",
        "model.load_beta_styles()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "C_tzObXyxGic"
      },
      "outputs": [],
      "source": [
        "model.STYLE_NAMES"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3H3SWosLTgpF"
      },
      "outputs": [],
      "source": [
        "# For more details about the parameters\n",
        "help(model.__call__)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "help(Model_Diffusers.load_pipe)"
      ],
      "metadata": {
        "id": "OiYJ3TxI3IAw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Run the upscaling tool"
      ],
      "metadata": {
        "id": "tm4qRiNh51kI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Load the model"
      ],
      "metadata": {
        "id": "_CapLd-oAdiv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from PIL import Image\n",
        "from stablepy import BUILTIN_UPSCALERS, load_upscaler_model\n",
        "\n",
        "scaler_beta = load_upscaler_model(\n",
        "    model=\"./RealESRGAN_x4plus.pth\",\n",
        "    tile=192,\n",
        "    tile_overlap=8,\n",
        "    device=\"cuda\",\n",
        "    half=True\n",
        ")"
      ],
      "metadata": {
        "id": "AaVwojmt6CMD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "upscale"
      ],
      "metadata": {
        "id": "n440GHcLAiN8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "image_path = \"bird.png\"\n",
        "image_pil_base = Image.open(image_path)\n",
        "image_pil_base = image_pil_base.convert(\"RGB\")\n",
        "upscaler_size = 1.3\n",
        "\n",
        "image_upscaler = scaler_beta.upscale(image_pil_base, upscaler_size)\n",
        "\n",
        "image_upscaler"
      ],
      "metadata": {
        "id": "e4_uI6b97XNo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can also use any for the built-in upscalers."
      ],
      "metadata": {
        "id": "IKXcMzrX6tn3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "BUILTIN_UPSCALERS"
      ],
      "metadata": {
        "id": "YuXbE1RH6s7K"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "More info usage:"
      ],
      "metadata": {
        "id": "AmI3d70L9myP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "help(load_upscaler_model)"
      ],
      "metadata": {
        "id": "AgrvQt5u9pBK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Run the preprocessing tool"
      ],
      "metadata": {
        "id": "YzDoSWRg59kc"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Load the model and set it on CUDA"
      ],
      "metadata": {
        "id": "R1iMXysqGGBw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from stablepy import Preprocessor, ALL_PREPROCESSOR_TASKS\n",
        "\n",
        "preprocessor = Preprocessor()\n",
        "\n",
        "preprocessor.load(\"Openpose\", use_cuda=True)\n",
        "# preprocessor.to(\"cuda\")"
      ],
      "metadata": {
        "id": "k5P3JUkh6DHf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Inference"
      ],
      "metadata": {
        "id": "bzoC4clhGNnL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "result_image = preprocessor(\n",
        "    image=\"Model_Posing_On_Typical_Studio_Set.jpg\",\n",
        "    image_resolution=1024,  # The final proportional resolution based on the provided image\n",
        "    detect_resolution=512,  # The resolution at which the detector will perform the inference\n",
        ")\n",
        "\n",
        "result_image"
      ],
      "metadata": {
        "id": "kml7pIZ09-FD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Valid preprocessor names:"
      ],
      "metadata": {
        "id": "0Xs6g2ijGQcf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ALL_PREPROCESSOR_TASKS"
      ],
      "metadata": {
        "id": "2gRq-7uQAcD1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "help(Preprocessor)"
      ],
      "metadata": {
        "id": "_fALLNaa_DwO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JwtC4KLm3bte"
      },
      "source": [
        "# Common issues and potential solutions:\n",
        "\n",
        "- **Black image output**: This problem happens when a tensor reaches a certain value and \"explodes\". To prevent this, avoid using high weights with LoRAs or IP Adapter. FreeU can also worsen the issue. A common fix is to use more stable VAE. For SDXL, you don't need to specify a VAE because the most stable one `madebyollin/sdxl-vae-fp16-fix`, automatically loads for execution.\n",
        "\n",
        "- **Distorted or very strange images**: This usually occurs due to prompt weight. In this implementation, the emphasis level set using Compel or Classic is particularly sensitive. It's best to use low prompt weights. Similarly, for LoRAs, it's recommended to use low scales. Also, using Classic variants like Classic-original can help; It has a normalization method to avoid extreme peaks that can greatly distort the outcome.\n",
        "\n",
        "- **Pony Diffusion not producing good images**: Compatibility with the model can be tricky. However, you can try using sampler DPM++ 1s or DPM2 with Compel or Classic prompt weights to improve results.\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}