Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for ComfyUI #254

Merged
merged 9 commits into from
Jun 16, 2024
Merged

Conversation

MieMieMieeeee
Copy link
Contributor

@MieMieMieeeee MieMieMieeeee commented Jun 13, 2024

功能

  • 试着支持了一下comfyui
    • 只支持了基本的text2image和image2image功能
      • 可以自定义workflow
      • 用户没有定义时,提供了最最最基础的默认workflow的模版

杂谈

  • 感觉自己有的地方写的不太优雅..请多指教!
  • 因为图生图的时候comfyui需要先把图片上传到服务器,然后使用返回的图片文件名再进行提交,所以为了执行ctx.http把getPayload改成了async
  • 测试的时候用的是秋叶版整合的comfyui
    • 记录一下版本号ComfyUI: 2052[55f37b](2024-03-07)
  • workflow可以支持自定义json文件上传,但是需要提供api版本的workflow。然而ComfyUI需要开启开发者模式才能导出api版本的workflow。希望用户不会搞错...。姑且在配置项里提了一嘴需要api格式的workflow

src/config.ts Outdated Show resolved Hide resolved
src/config.ts Outdated Show resolved Hide resolved
src/config.ts Outdated Show resolved Hide resolved
src/index.ts Outdated
@@ -453,6 +526,31 @@ export function apply(ctx: Context, config: Config) {
const b64 = Buffer.from(imgRes.data).toString('base64')
return forceDataPrefix(b64, imgRes.headers.get('content-type'))
}
if (config.type === 'comfyui') {
//get filenames from history
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//get filenames from history
// get filenames from history

下面也改一下

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

类似的地方已修改

src/index.ts Outdated
await sleep(config.pollInterval)
}
//get images by filename
const imagesOutput: Buffer[] = [];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const imagesOutput: Buffer[] = [];
const imagesOutput: Buffer[] = []

附近还有几个分号,你处理一下

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

类似的地方已修改

src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated
const workflowImage2Image = config.workflowImage2Image ? resolve(ctx.baseDir, config.workflowImage2Image) : '../data/default-comfyui-i2i-wf.json'
const workflow = image ? workflowImage2Image : workflowText2Image
logger.debug('workflow:', workflow)
const prompt= require(workflow)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

你都改 async 了这里不要 require 比较好。

Suggested change
const prompt= require(workflow)
const prompt = JSON.parse(await readFile(workflow, 'utf8'))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commit 66bdd52
为readFile修改了一下路径的获取

src/index.ts Show resolved Hide resolved
@MieMieMieeeee
Copy link
Contributor Author

  • 已修正所有review指出的问题
  • 为sample增加了一个required 4cb3b4c

已使用当前的branch重新对comfyui测试,可以正常使用

src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated
}
}
// return first image
return forceDataPrefix(Buffer.from(imagesOutput[0]).toString('base64'))
Copy link
Contributor

@idranme idranme Jun 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MIME 类型是否固定为 image/png

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

确实返回的图片类型是不固定的,修改了一下

src/index.ts Outdated Show resolved Hide resolved
src/index.ts Outdated Show resolved Hide resolved
@shigma shigma merged commit b27a02c into koishijs:main Jun 16, 2024
1 of 2 checks passed
@MaikoTan MaikoTan added the comfyui Related to ComfyUI backend label Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comfyui Related to ComfyUI backend
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: 能支持ComfyUI吗?
4 participants