How do I run a ComfyUI diffusion model with fp8 weights, what does that setting change, and is NF4 (bitsandbytes) still a supported route?
Problem. Reduce VRAM for a large diffusion model without leaving core ComfyUI, and decide between fp8 and NF4. Conditions. ComfyUI commit 9fe7ef76 (comfyui_version.py shows 0.42.0). Method. Two core routes exist. (a) Per workflow: the Load Diffusion Model node (UNETLoader) has an advanced weight_dtype widget with default, fp8_e4m3fn, fp8_e4m3fn_fast and fp8_e5m2; the code maps fp8_e4m3fn_fast to the same dtype plus model_options['fp8_optimizations'] = True. (b) Per server: --fp8_e4m3fn-unet, --fp8_e5m2-unet and --fp8_e8m0fnu-unet store diffusion-model weights in that dtype, and text-encoder variants exist; --supports-fp8-compute makes ComfyUI behave as if the device supports fp8 compute. Because the workflow field is saved inside the workflow JSON while the CLI flag is not, a workflow moved to another machine can load differently if the server flags differ (inference). NF4. The comfyanonymous/ComfyUI_bitsandbytes_NF4 repository's whole README, at the pinned commit, is one sentence s...
Structured fields
{
"citations": [
{
"url": "https://raw.githubusercontent.com/comfyanonymous/ComfyUI/9fe7ef764d0452dcc9dcb925efd076d4e68eece1/nodes.py",
"note": "UNETLoader.INPUT_TYPES; UNETLoader.load_unet; fpunet_group; parser; README.md"
},
{
"url": "https://raw.githubusercontent.com/comfyanonymous/ComfyUI/9fe7ef764d0452dcc9dcb925efd076d4e68eece1/comfy/cli_args.py",
"note": "UNETLoader.INPUT_TYPES; UNETLoader.load_unet; fpunet_group; parser; README.md"
},
{
"url": "https://raw.githubusercontent.com/comfyanonymous/ComfyUI_bitsandbytes_NF4/6c523bf1ee37900757ee538e6683f1312b8f425c/README.md",
"note": "UNETLoader.INPUT_TYPES; UNETLoader.load_unet; fpunet_group; parser; README.md"
}
]
}Evidence claims
- [sourced_fact] Problem. Reduce VRAM for a large diffusion model without leaving core ComfyUI, and decide between fp8 and NF4. Conditions. ComfyUI commit 9fe7ef76 (comfyui_version.py shows 0.42.0). Method. Two core routes exist. (a) Per workflow: the Load Diffusion Model node (UNETLoader) has an advanced weight_dtype widget with default, fp8_e4m3fn, fp8_e4m3fn_fast and fp8_e5m2; the code maps fp8_e4m3fn_fast to the same dtype plus model_options['fp8_optimizations'] = True. (b) Per server: --fp8_e4m3fn-unet, --fp8_e5m2-unet and --fp8_e8m0fnu-unet store diffusion-model weights in that dtype, and text-encoder variants exist; --supports-fp8-compute makes ComfyUI behave as if the device supports fp8 compute. Because the workflow field is saved inside the workflow JSON while the CLI flag is not, a workflow moved to another machine can load differently if the server flags differ (inference). NF4. The comfyanonymous/ComfyUI_bitsandbytes_NF4 repository's whole README, at the pinned commit, is one sentence saying not to use it; treat NF4 in ComfyUI as a third-party route with no core support (inference from that statement and from the absence of any NF4 option in the core loader code read). Failure modes. fp8 storage changes numerical results, so seeds will not reproduce bit-for-bit across dtypes (inference); the sources give no quality numbers; the _fast option implies hardware-specific fp8 matmul (inference from the flag names). Limitations. Source code and help strings only; nothing was run. Version and date. ComfyUI 9fe7ef76; NF4 repo 6c523bf1. · locator: UNETLoader.INPUT_TYPES; UNETLoader.load_unet; fpunet_group; parser; README.md source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |
Linked records
- cites → https://raw.githubusercontent.com/comfyanonymous/ComfyUI/9fe7ef764d0452dcc9dcb925efd076d4e68eece1/comfy/cli_args.py
- cites → https://raw.githubusercontent.com/comfyanonymous/ComfyUI/9fe7ef764d0452dcc9dcb925efd076d4e68eece1/nodes.py
- cites → https://raw.githubusercontent.com/comfyanonymous/ComfyUI_bitsandbytes_NF4/6c523bf1ee37900757ee538e6683f1312b8f425c/README.md