For FLUX, how do I set PuLID's 'timestep to start inserting ID' and true-vs-fake CFG to trade identity fidelity against editability?
Problem. Identity-conditioning methods can overwrite style, lighting and composition. PuLID aims to keep the original model's output as unchanged as possible apart from the face. Conditions. FLUX.1-dev with PuLID-FLUX v0.9.1 (better ID fidelity than v0.9.0, about 5 points on the authors' similarity metric); runs on a 16 GB GPU with fp8 per the README (needs PyTorch >= 2.4.1 for flux-dev-fp8). Method. (1) 'timestep to start inserting ID': 0 inserts from the first step; earlier insertion gives higher ID fidelity but lower editability; later insertion lowers fidelity, raises editability and disturbs the base model less. Authors suggest 4 for realistic images and 0-1 for stylised images; lower it if similarity is too low. (2) CFG: FLUX.1-dev is guidance-distilled, so the usual guidance_scale (e.g. 4) is 'fake CFG'. Authors recommend fake CFG for photoreal scenes (better aesthetics and facial naturalness) and switching to true CFG if ID fidelity is unsatisfactory, or in stylised scenes w...
Structured fields
{
"citations": [
{
"url": "https://raw.githubusercontent.com/ToTheBeginning/PuLID/main/docs/pulid_for_flux.md",
"note": "Useful Tips (1); Useful Tips (1); Useful Tips (2); limitation; License; Model Zoo; Abstract"
},
{
"url": "https://raw.githubusercontent.com/ToTheBeginning/PuLID/main/README.md",
"note": "Useful Tips (1); Useful Tips (1); Useful Tips (2); limitation; License; Model Zoo; Abstract"
},
{
"url": "https://arxiv.org/abs/2404.16022",
"note": "Useful Tips (1); Useful Tips (1); Useful Tips (2); limitation; License; Model Zoo; Abstract"
}
]
}Evidence claims
- [sourced_fact] Problem. Identity-conditioning methods can overwrite style, lighting and composition. PuLID aims to keep the original model's output as unchanged as possible apart from the face. Conditions. FLUX.1-dev with PuLID-FLUX v0.9.1 (better ID fidelity than v0.9.0, about 5 points on the authors' similarity metric); runs on a 16 GB GPU with fp8 per the README (needs PyTorch >= 2.4.1 for flux-dev-fp8). Method. (1) 'timestep to start inserting ID': 0 inserts from the first step; earlier insertion gives higher ID fidelity but lower editability; later insertion lowers fidelity, raises editability and disturbs the base model less. Authors suggest 4 for realistic images and 0-1 for stylised images; lower it if similarity is too low. (2) CFG: FLUX.1-dev is guidance-distilled, so the usual guidance_scale (e.g. 4) is 'fake CFG'. Authors recommend fake CFG for photoreal scenes (better aesthetics and facial naturalness) and switching to true CFG if ID fidelity is unsatisfactory, or in stylised scenes where fake CFG gave lower ID similarity and poorer style response. Failure modes. Beta limitation: ID fidelity may be low for some male inputs. Limitations. Settings come from the authors' comparisons shown as images; no numbers extracted. Rights. Repo code is Apache-2.0; usage with FLUX.1-dev must follow the FLUX.1-dev model licence; the face-encoder licence is not stated in the files I read. Version/date. PuLID NeurIPS 2024 (arXiv 2404.16022); README lists FLUX v0.9.1. · locator: Useful Tips (1); Useful Tips (1); Useful Tips (2); limitation; License; Model Zoo; Abstract source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |