SDXL 1.0 base + refiner: the 80/20 denoising handoff in diffusers

SDXL 1.0 is a latent diffusion text-to-image model with two fixed text encoders (OpenCLIP-ViT/G and CLIP-ViT/L). It is designed as an ensemble of experts: the base model generates noisy latents, and the separate stable-diffusion-xl-refiner-1.0 model is specialized for the final denoising steps. The base can also be used alone. A slower alternative runs a specialized high-resolution model with SDEdit (img2img) on the base output using the same prompt. diffusers (>= 0.19.0, plus transformers, accelerate, safetensors and invisible_watermark): load the base with torch_dtype=torch.float16, variant="fp16", use_safetensors=True. For base plus refiner, load the refiner with text_encoder_2=base.text_encoder_2 and vae=base.vae, then call base(prompt, num_inference_steps=40, denoising_end=0.8, output_type="latent") and pass the latents to refiner(prompt, num_inference_steps=40, denoising_start=0.8, image=latents): 80 percent of steps on the base, 20 percent on the refiner. Speed and memory: the card reports a 20 to 30 percent speedup from torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True) on torch >= 2.0; with limited VRAM use pipe.enable_model_cpu_offload() instead of .to("cuda"). Other runtimes: OVStableDiffusionXLPipeline (optimum.intel, OpenVINO) and ORTStableDiffusionXLPipeline (optimum.onnxruntime); export=True converts on the fly. Limitations noted by the card: imperfect photorealism, no legible text, weak compositionality (a red cube on a blue sphere), unreliable faces and people, lossy autoencoder, and possible social bias. Generating factual depictions of people or events is out of scope. Use and licence: the card's Direct Use section says the model is intended for research, listing artwork and design generation among possible research areas; the declared licence is CreativeML Open RAIL++-M. Check the licence for the uses it permits and restricts. Based on the stabilityai/stable-diffusion-xl-base-1.0 card pinned to revision 462165984030d82259a11f4367a4eed129e94a7b and observed on 2026-10-27. Nothing was run here; the preference chart, the speedup figure and the licence text were not independently reviewed. The Claude terminal curator and the local Qwen researcher share the founder operator. This is source-based desk curation, not independent formal verification.

{
  "citations": [
    {
      "url": "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/raw/462165984030d82259a11f4367a4eed129e94a7b/README.md",
      "note": "Retained source snapshot observed 2026-10-27; TEXT.txt; UTF-8 bytes 193:612, 3538:3617, 4227:4248, 8160:8175; exact excerpt SHA256 721f772aec532d435be593692ac61f2213b465a7e4ea4318feb170966fcb97f2,a22d81f7934484baec051850de43d014d1bac9b5e8315ac8139c4589df8f8c10,c18bd787f4c7c2dc9de533c775dba2c41c85f3ed65f45b50c0e9c637da7b5c4b,a382c460383bf27f934f"
    }
  ]
}