How do I tune InstantID's two weights, what does the keypoint ControlNet lock, and what does its licence permit?
Problem. Storyboard or character stills need one face identity from a single photo while text still controls scene and style. Conditions. SDXL; InstantID checkpoints (ControlNetModel IdentityNet + ip-adapter.bin); insightface antelopev2 for the face embedding. Method. (1) Extract the largest face with FaceAnalysis(name='antelopev2', det_size=(640,640)); take face_info['embedding'] and draw_kps(face_image, face_info['kps']). (2) Call the pipeline with image_embeds=face_emb, image=face_kps, controlnet_conditioning_scale=0.8, ip_adapter_scale=0.8 (README example). (3) Tuning from the README: raise both scales for more similarity; if over-saturated, lower ip_adapter_scale first, then controlnet_conditioning_scale; lower ip_adapter_scale for stronger text control. (4) With LCM-LoRA use about 10 steps and guidance_scale in [0,1]. Failure modes. Only the largest face is used (no multi-person); inference (labelled): because IdentityNet is conditioned on five facial keypoints from the refere...
Structured fields
{
"citations": [
{
"url": "https://raw.githubusercontent.com/instantX-research/InstantID/main/README.md",
"note": "Usage Tips; Usage Tips; Usage Tips; Usage Tips; code example; Speed Up with LCM-LoRA; Disclaimer; Disclaimer; keypoint conditioning (code)"
}
]
}Evidence claims
- [sourced_fact] Problem. Storyboard or character stills need one face identity from a single photo while text still controls scene and style. Conditions. SDXL; InstantID checkpoints (ControlNetModel IdentityNet + ip-adapter.bin); insightface antelopev2 for the face embedding. Method. (1) Extract the largest face with FaceAnalysis(name='antelopev2', det_size=(640,640)); take face_info['embedding'] and draw_kps(face_image, face_info['kps']). (2) Call the pipeline with image_embeds=face_emb, image=face_kps, controlnet_conditioning_scale=0.8, ip_adapter_scale=0.8 (README example). (3) Tuning from the README: raise both scales for more similarity; if over-saturated, lower ip_adapter_scale first, then controlnet_conditioning_scale; lower ip_adapter_scale for stronger text control. (4) With LCM-LoRA use about 10 steps and guidance_scale in [0,1]. Failure modes. Only the largest face is used (no multi-person); inference (labelled): because IdentityNet is conditioned on five facial keypoints from the reference image, head pose and framing in the output are likely to follow the reference keypoints unless you supply different keypoints, which is the usual 'pose lock'. Rights. Code is Apache-2.0, but insightface models (manual or auto download) are non-commercial research only and the released InstantID checkpoints are research-only. Limitations. README gives no measurements for the scale interactions. Version/date. README on main, fetched 2026-10-23. · locator: Usage Tips; Usage Tips; Usage Tips; Usage Tips; code example; Speed Up with LCM-LoRA; Disclaimer; Disclaimer; keypoint conditioning (code) source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |