What do I need to run StoryDiffusion's consistent-character comic generation for the first time (prompts, GPU, defaults), what does it keep consistent, and what is and is not released or licensed?
StoryDiffusion's image part (Consistent Self-Attention) is training-free: it lets the images of one batch share sampled tokens so faces and attire converge. Per the README the user must give at least 3 prompts and 5-6 are recommended; the low-VRAM Gradio script was tested on a 24 GB A10 with 30 GB RAM. In that script the first 4 prompt rows are identity prompts, the sampling rate is 0.5 and output is 768x768; the reference-image mode is implemented with PhotoMaker; cartoon reference images are marked unsupported. The paper's own author-run test shows slightly higher CLIP-based text and character similarity than IP-Adapter and PhotoMaker, and its stated limitation is minor clothing details. The video model's code and weights are unchecked TODO items in the README. Code is Apache-2.0; the third-party ViStoryBench run scores it below OmniGen2 and UNO on cross-CIDS and states that the method cannot take several character references in one shot.
Structured fields
{
"citations": [
{
"url": "https://github.com/HVision-NKU/StoryDiffusion/tree/8de45e424887766fdd84dc917436ff8605f00149",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/html/2405.01434v1",
"note": "candidate evidence excerpts"
},
{
"url": "https://arxiv.org/html/2505.24862v5",
"note": "candidate evidence excerpts"
}
]
}Evidence claims
- [sourced_fact] StoryDiffusion's image part (Consistent Self-Attention) is training-free: it lets the images of one batch share sampled tokens so faces and attire converge. Per the README the user must give at least 3 prompts and 5-6 are recommended; the low-VRAM Gradio script was tested on a 24 GB A10 with 30 GB RAM. In that script the first 4 prompt rows are identity prompts, the sampling rate is 0.5 and output is 768x768; the reference-image mode is implemented with PhotoMaker; cartoon reference images are marked unsupported. The paper's own author-run test shows slightly higher CLIP-based text and character similarity than IP-Adapter and PhotoMaker, and its stated limitation is minor clothing details. The video model's code and weights are unchecked TODO items in the README. Code is Apache-2.0; the third-party ViStoryBench run scores it below OmniGen2 and UNO on cross-CIDS and states that the method cannot take several character references in one shot. · locator: candidate evidence excerpts source
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| documentation | unknown | permitted | permitted | unknown | unknown | Applies to this original summary only, not to the cited sources. |