ComfyUI Setup

Requirements for ComfyUI

You can use the Krita plugin with your custom installation of ComfyUI. When connecting it will check for a number of custom node extensions and models which it requires to function. These are listed below.

Required custom nodes

Install custom nodes according to the instructions of the respective projects, or use ComfyUI Manager.

• ControlNet preprocessors
• IP-Adapter
• Inpaint nodes
• External tooling nodes

Required models

Download models to the paths indicated below. If you are using extra_model_paths.yml, those will also work. Shared models are always required, and you can choose SD1.5 or SDXL or install both.

This can be done manually or via script (see below).

Shared

• Clip-Vision to models/clip_vision/SD1.5
• NMKD Superscale SP_178000_G to models/upscale_models
• OmniSR_X2_DIV2K to models/upscale_models
• OmniSR_X3_DIV2K to models/upscale_models
• OmniSR_X4_DIV2K to models/upscale_models
• MAT_Places512_G_fp16 to models/inpaint

SD 1.5

• ControlNet inpaint to models/controlnet
• ControlNet tile to models/controlnet
• IP-Adapter (SD1.5) to models/ipadapter
• Hyper-SD-LoRA (SD1.5) to models/loras/

SD XL

• IP-Adapter (SDXL) to models/ipadapter
• Hyper-SD-LoRA (SDXL) to models/loras
• Fooocus Inpaint (Head) to models/inpaint
• Fooocus Inpaint (Patch) to models/inpaint

Checkpoints

The following checkpoints are used by the default styles:

• Realistic Vision
• DreamShaper
• Juggernaut XL
• ZavyChroma XL

At least one checkpoint is required, but it doesn't have to be one of the above.

Download Script

The models above mostly list strict requirements, there are a lot of additional models (like ControlNet) which activate optional features in the plugin. If you have them already - great. Otherwise you can use the download_models.py script to fetch all required and optional models.

After installing the plugin you can find the script in the plugin folder (called ai_diffusion, see Paths). Open a command prompt and run:

python -m pip install aiohttp tqdm
python download_models.py /path/to/your/comfyui

This will download all models supported by the plugin directly into the specified folder with the correct version, location, and filename. The download location does not have to be your ComfyUI installation, you can use an empty folder if you want to avoid clashes and copy models afterwards. There are also options to only download a subset, or list all relevant URLs without downloading.

python download_models.py --help

Note: The script downloads models only. It does not install or modify ComfyUI or custom nodes!

Troubleshooting

If you're getting errors about missing resources, or workload not being installed, it's probably because one of the models wasn't found. You can find the client.log file by clicking the "View Log files" link in the Connection settings. Check the log for warnings. Here you will also find which models were found in your installation, and the patterns the plugin looks for.

Model paths must contain one of the search patterns entirely to match. The model path is allowed to be longer though: you may place models in arbitrary subfolders and they will still be found. If there are multiple matches, any files placed inside a krita subfolder are prioritized.

Questions and Support

• ComfyUI, comfy.org for issues related to ComfyUI
• Discussions and Issues for things concerning the setup for Krita Diffusion Plugin
• Please don't use official Krita channels for questions related to the Plugin