ROCm support

[renich]

Would it be possible for you to implement ROCm support?

[xhinker]

Sorry for my late response, I don't use the ROCm and AMD card, so I don't think ROCm will be supported in the foreseeable future, but welcome merging your code if you want to contribute to this tool. Thank You

[renich]

No worries. I don't think my python chops are up to par with what is required. I can give it a shot, though. Yet, hopefully, someone else can hop in and help us.

[Teriks]

I do not think there is currently anything in the code preventing ROCm support, if you have an environment setup where diffusers is running on ROCm, this code should be able to run in that environment.

ROCm uses the same device names as the CUDA backend for compatibility, the difficult part of ROCm is getting a correct environment to run the code

I have had some success setting this up on runpod.io on an AMD card, here are some steps I took to create the environment: https://github.com/Teriks/dgenerate?tab=readme-ov-file#linux-with-rocm-amd-cards

mostly just that, minus installing my project

[renich]

I think this is the main reason I opened up this bug:

https://github.com/xhinker/sd_embed?tab=readme-ov-file#install

torchao has a PR that seems to be introducing ROCm support: pytorch/ao#1201

[Teriks]

I think this is the main reason I opened up this bug:

https://github.com/xhinker/sd_embed?tab=readme-ov-file#install

torchao has a PR that seems to be introducing ROCm support: pytorch/ao#1201

Ah yea, torchao is used for quantization in the examples. It seems a lot of quant libraries are still WIP on ROCm support.

The actual prompt weighting code does not involve torchao and runs fine I believe, fyi @xhinker

You just have skip out on using quantization features for ROCm, I could not get bitsandbytes working on ROCm with their multiplatform wheel for instance. The support for it is just spotty at the moment.