With Celery on Redis, when can a long GPU render run twice, and what do acks_late, idempotency and the visibility timeout mean for a job queue in front of ComfyUI?

Problem. Orchestrating paid or slow generative jobs through a broker without double-running them (see C3-B574-18 for the Comfy v2 reject-on-duplicate key). Conditions. Celery stable docs (Tasks, Using Redis, Configuration, Optimizing), fetched 2026-10-23. Method. Default: the worker acknowledges the message just before executing, so a started task is never re-delivered (at-most-once for started work). Setting acks_late acknowledges after the task returns: if the worker dies the message is redelivered to another worker (at-least-once), so the task must be idempotent; the docs say the worker cannot detect idempotency. Even with acks_late, a child process that exits or is killed gets acknowledged unless task_reject_on_worker_lost is enabled, which can cause message loops. On Redis, a message not acknowledged within the visibility timeout (default one hour) is redelivered to another worker, and tasks whose runtime exceeds it execute again, 'and again in a loop' for ETA/countdown/retry tasks; raising the timeout to the longest ETA is not recommended for reliability. Recipe (inference): give each render a client-chosen job id, write results to an idempotent location (see storage findings), have the task check for an existing finished result first, set visibility_timeout above the hard time limit or run renders as short tasks that poll an external job (e.g. ComfyUI prompt_id), and set acks_late with worker_prefetch_multiplier=1 so long jobs do not wait behind prefetched ones. Failure modes. Redelivery doubling GPU spend; killed workers silently dropping jobs under default acking; message loops from reject_on_worker_lost. Limitations. Describes Celery with the Redis transport; other brokers (e.g. SQS) have their own visibility semantics not read. Version and date. Celery stable docs, undated. Curator qualifications: All cited documentation excerpts located on the live pages by curator GET; no video sources. Limitations: Docs-based; the recipe is inference and untested. Evidence status: statements located in the cited sources by a correlated desk review (Claude); not executed; not independently reviewed.

{
  "citations": [
    {
      "url": "https://docs.celeryq.dev/en/stable/userguide/tasks.html",
      "note": "Task acknowledgment; Task acknowledgment; Caveats: Visibility timeout; task_reject_on_worker_lost"
    },
    {
      "url": "https://docs.celeryq.dev/en/stable/getting-started/backends-and-brokers/redis.html",
      "note": "Task acknowledgment; Task acknowledgment; Caveats: Visibility timeout; task_reject_on_worker_lost"
    },
    {
      "url": "https://docs.celeryq.dev/en/stable/userguide/configuration.html",
      "note": "Task acknowledgment; Task acknowledgment; Caveats: Visibility timeout; task_reject_on_worker_lost"
    }
  ]
}