How does ComfyUI's own repository run image-quality regression tests (baseline directory, SSIM threshold, matching by prompt metadata), and what can I copy or must I fix for my own workflows?

Problem. Detecting unintended output changes after upgrading ComfyUI, nodes or models. Conditions. ComfyUI v0.42.5 tests/ directory; pytest, opencv-python, scikit-image (README names scikit-image==0.21.0). Method. Step 1: run pytest tests/inference with --output_dir to produce baseline images (the test starts main.py with --disable-assets on a port, queues graphs for sampler/scheduler/prompt combinations through the API, and asserts images exist and are not blank). Step 2: change code. Step 3: run inference again plus tests/compare/test_quality.py, which pairs each baseline PNG with a test PNG whose embedded 'prompt' metadata string is identical (matching file name tried first; empty prompts never match) and computes skimage structural_similarity with channel_axis=-1; the pass threshold is SSIM > 0.95; it writes PASS/FAIL rows, a difference image and a grid image. Failure modes. Metadata match requires identical prompt JSON, so any workflow edit orphans its baseline; only SSIM is implemented (a METRICS dict allows adding more); the threshold is a single global constant, not per-image. scikit-image docs warn that data_range is guessed from dtype (wrong by 2x for float images) and that matching Wang et al. needs gaussian_weights=True, sigma=1.5, use_sample_covariance=False plus data_range; the test calls it with defaults on uint8 RGB. Limitations. Images only: no video, no seed or hardware recording beyond what the prompt metadata contains, and results are not stable across GPUs (see C3-B574-05 and prior reproducibility items). Built for the project's own CI, not as a product API. Version and date. v0.42.5. Curator qualifications: All cited documentation excerpts located on the live pages by curator GET; no video sources. Limitations: Read from code; not run. Whether 0.95 is appropriate for other models is untested. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://raw.githubusercontent.com/comfyanonymous/ComfyUI/483dae4326356b81de05d27aca3f433dd323cc25/tests/compare/test_quality.py",
      "note": "METRICS_PASS_THRESHOLD; ssim_score; Quality regression test; structural_similarity Notes"
    },
    {
      "url": "https://raw.githubusercontent.com/comfyanonymous/ComfyUI/483dae4326356b81de05d27aca3f433dd323cc25/tests/README.md",
      "note": "METRICS_PASS_THRESHOLD; ssim_score; Quality regression test; structural_similarity Notes"
    },
    {
      "url": "https://scikit-image.org/docs/stable/api/skimage.metrics.html",
      "note": "METRICS_PASS_THRESHOLD; ssim_score; Quality regression test; structural_similarity Notes"
    }
  ]
}