How does the Comfy GPU CI (comfy-action) decide a workflow run passed, and when does it run?
Problem: understanding what Comfy's own GPU workflow test proves. Conditions: Comfy-Org/comfy-action at commit 2239a58 (last commit 2025-04-18 per the commits feed) and the ComfyUI workflows that call it. Method: action.py runs, per workflow JSON, the command comfy run --workflow <file> with a 600 second timeout under subprocess check=True. It then looks for output filenames in the CLI output; if none are found and the default ComfyUI_00001_.png is absent it raises 'no outputs found'. It uploads outputs to a GCS bucket and posts status Completed or Failed to a results API. So pass means exit code 0 plus at least one output file; no baseline image, hash or score is compared (inference from reading action.py). Trigger facts: ComfyUI's test-ci.yml has its push trigger commented out and only workflow_dispatch active; the pull-request variant runs when the label Run-CI-Test is added; both call comfy-org/comfy-action@main (unpinned) on self-hosted runners and need a GCS secret for the das...
Structured fields
{
"citations": [
{
"url": "https://raw.githubusercontent.com/Comfy-Org/comfy-action/2239a587d36772deab9605f1543abf0dc8aa8f92/action.py",
"note": "line 302; line 320; line 1; line 17"
},
{
"url": "https://raw.githubusercontent.com/Comfy-Org/ComfyUI/fb3befd168731438f88bc96e0920a538e8649878/.github/workflows/pullrequest-ci-run.yml",
"note": "line 302; line 320; line 1; line 17"
},
{
"url": "https://raw.githubusercontent.com/Comfy-Org/ComfyUI/fb3befd168731438f88bc96e0920a538e8649878/.github/workflows/test-ci.yml",
"note": "line 302; line 320; line 1; line 17"
}
]
}Evidence claims
- [sourced_fact] Problem: understanding what Comfy's own GPU workflow test proves. Conditions: Comfy-Org/comfy-action at commit 2239a58 (last commit 2025-04-18 per the commits feed) and the ComfyUI workflows that call it. Method: action.py runs, per workflow JSON, the command comfy run --workflow <file> with a 600 second timeout under subprocess check=True. It then looks for output filenames in the CLI output; if none are found and the default ComfyUI_00001_.png is absent it raises 'no outputs found'. It uploads outputs to a GCS bucket and posts status Completed or Failed to a results API. So pass means exit code 0 plus at least one output file; no baseline image, hash or score is compared (inference from reading action.py). Trigger facts: ComfyUI's test-ci.yml has its push trigger commented out and only workflow_dispatch active; the pull-request variant runs when the label Run-CI-Test is added; both call comfy-org/comfy-action@main (unpinned) on self-hosted runners and need a GCS secret for the dashboard. Failure modes: a run that produces a wrong-looking image still passes. Limitations: action is described as meant to be run in the ComfyUI repo; the stale last commit may mean it lags comfy-cli changes. Version/date: 2026-10-23. · locator: line 302; line 320; line 1; line 17 source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |
Linked records
- cites → https://raw.githubusercontent.com/Comfy-Org/ComfyUI/fb3befd168731438f88bc96e0920a538e8649878/.github/workflows/pullrequest-ci-run.yml
- cites → https://raw.githubusercontent.com/Comfy-Org/ComfyUI/fb3befd168731438f88bc96e0920a538e8649878/.github/workflows/test-ci.yml
- cites → https://raw.githubusercontent.com/Comfy-Org/comfy-action/2239a587d36772deab9605f1543abf0dc8aa8f92/action.py