Can a ComfyUI diffusion model be exported to ONNX and TensorRT for speed, what is portable between machines, and which features break?
Problem. Speeding up a fixed image-generation workflow with TensorRT while keeping the workflow reproducible elsewhere. Conditions. comfyanonymous/ComfyUI_TensorRT commit 5bcc3f1e (MIT-style permission text, copyright ComfyUI contributors); supports SD1.5, SD2.1, SD3, SDXL, SDXL Turbo, SVD, SVD-XT, AuraFlow and Flux. Method. The conversion node exports the loaded model with torch.onnx.export (opset_version=17, dynamic axes) to a temporary model.onnx, parses it with the TensorRT ONNX parser, sets the FP16 builder flag (BF16 for Flux), and writes a .engine file; a timing_cache.trt next to the code speeds later builds. Choose a static engine (one batch, height, width; less VRAM) or a dynamic engine with min/opt/max ranges (best speed at opt). Engine file names encode the shape (stat-b-..., dyn-b-min-max-opt-...). At run time a TensorRT Loader replaces the model; CLIP and VAE still come from the original checkpoint. Portability. The README says the engine is specific to your GPU, so the...
Structured fields
{
"citations": [
{
"url": "https://raw.githubusercontent.com/comfyanonymous/ComfyUI_TensorRT/5bcc3f1e5c2424bb20bcb586e340c25ebe4a954f/README.md",
"note": "Description; Common Issues/Limitations; Building A TensorRT Engine; TRT_MODEL_CONVERSION_BASE; TRT_MODEL_CONVERSION_BASE"
},
{
"url": "https://raw.githubusercontent.com/comfyanonymous/ComfyUI_TensorRT/5bcc3f1e5c2424bb20bcb586e340c25ebe4a954f/tensorrt_convert.py",
"note": "Description; Common Issues/Limitations; Building A TensorRT Engine; TRT_MODEL_CONVERSION_BASE; TRT_MODEL_CONVERSION_BASE"
}
]
}Evidence claims
- [sourced_fact] Problem. Speeding up a fixed image-generation workflow with TensorRT while keeping the workflow reproducible elsewhere. Conditions. comfyanonymous/ComfyUI_TensorRT commit 5bcc3f1e (MIT-style permission text, copyright ComfyUI contributors); supports SD1.5, SD2.1, SD3, SDXL, SDXL Turbo, SVD, SVD-XT, AuraFlow and Flux. Method. The conversion node exports the loaded model with torch.onnx.export (opset_version=17, dynamic axes) to a temporary model.onnx, parses it with the TensorRT ONNX parser, sets the FP16 builder flag (BF16 for Flux), and writes a .engine file; a timing_cache.trt next to the code speeds later builds. Choose a static engine (one batch, height, width; less VRAM) or a dynamic engine with min/opt/max ranges (best speed at opt). Engine file names encode the shape (stat-b-..., dyn-b-min-max-opt-...). At run time a TensorRT Loader replaces the model; CLIP and VAE still come from the original checkpoint. Portability. The README says the engine is specific to your GPU, so the ONNX file is an intermediate (written to a temp directory in the code read), not the artifact you ship; moving a workflow to a different GPU means rebuilding the engine (inference from 'specific to your GPU', plus the pinned tensorrt>=10.0.1). Failure modes. Engines are 'not yet compatible with ControlNets or LoRAs'; Flux needs more than 24 GB VRAM; build time 3-10 minutes for image models, 10-25 minutes for SVD, up to an hour for SVD-XT; new engines do not appear in the loader until the browser is refreshed. Limitations. README and one source file read; no speed-up figure is stated in the README. Version and date. Commit 5bcc3f1e. · locator: Description; Common Issues/Limitations; Building A TensorRT Engine; TRT_MODEL_CONVERSION_BASE; TRT_MODEL_CONVERSION_BASE source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |