Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add onediff_comfy_nodes/benchmarks #956

Merged
merged 44 commits into from
Jun 25, 2024
Merged

Conversation

ccssu
Copy link
Contributor

@ccssu ccssu commented Jun 17, 2024

@strint
Copy link
Collaborator

strint commented Jun 25, 2024

这个 pr 可以解决之前 ci 无法捕获 comfyui test 报错的问题么

@strint
Copy link
Collaborator

strint commented Jun 25, 2024

本 pr 一个主要功能是增加了 comfyui 的 benchmark 对吧

@ccssu
Copy link
Contributor Author

ccssu commented Jun 25, 2024

本 pr 一个主要功能是增加了 comfyui 的 benchmark 对吧

  • 主要功能是对 工作流进行了更严格测试,比如动态shape, sd3添加多prompt,不停留在跑通 (测试的工作流和文档挂钩,减少社区用户反馈跑不通文档工作流的问题)
  • 通过 websocket-client 获取图片方法解决ci 无法捕获 comfyui test 报错

用于测试和处理图像生成的工作流程code:

with comfy_client_context(port=comfy_port) as client:
logger.info(f"Testing workflows: {workflow}")
for i, comfy_graph in enumerate(dispatch_generator(workflow)):
start_time = time.time()
images = client.get_images(comfy_graph.graph)
end_time = time.time()
if not images:
logger.error("No images generated")
raise ValueError("No images generated")
if len(images) != 1:
logger.error(
f"Expected 1 image, but got {len(images)} images. Batch Size == 1"
)
raise ValueError(
f"Expected 1 image, but got {len(images)} images. Batch Size == 1"
)
for images_output in images.values():
for image_data in images_output:
processor.process_image(image_data, i)
e2e_time = end_time - start_time
logger.info(f"Workflow {i} E2E: {e2e_time:.2f} seconds")
# if i>0: # TODO refine
# assert e2e_time < 10 # sec

@ccssu ccssu requested a review from doombeaker June 25, 2024 09:42
@ccssu ccssu enabled auto-merge (squash) June 25, 2024 18:17
@ccssu ccssu merged commit 705a3cb into main Jun 25, 2024
7 checks passed
@ccssu ccssu deleted the add_onediff_comfy_nodes_benchmarks branch June 25, 2024 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants