bytedance/LatentSync
Source: https://github.com/bytedance/latentsync
Structured fields
{
"archived": false,
"checked_at": "2026-10-17T00:00:00.000Z",
"usefulness": "End-to-end lip sync using audio-conditioned latent diffusion (Whisper embeddings into a U-Net via cross-attention), versions 1.5 and 1.6 (512x512), with published VRAM figures for inference and training.",
"source_type": "repository",
"availability": "available",
"next_check_at": "2026-10-31T00:00:00.000Z",
"hardware_notes": "Inference minimum 8GB (1.5) / 18GB (1.6) per README.",
"known_limitations": [
"README page does not mention ComfyUI integration."
],
"upstream_updated_at": "2025-06-20T07:36:58.000Z"
}Evidence claims
- [upstream_claim] README: end-to-end lip-sync method on audio-conditioned latent diffusion with no intermediate motion representation; LatentSync 1.6 (2025/06/11) trained on 512x512 videos to reduce blurriness; 1.5 (2025/03/14) adds a temporal layer. source
- [upstream_claim] README VRAM: inference minimum 8GB for 1.5 and 18GB for 1.6; training 20-55GB depending on stage. source
- [upstream_claim] Research review verdict at export: reviewed (as of 2026-10-17). Recorded by the research pipeline, not a hub review outcome; evidence.review stays unreviewed until an independent reviewer records one.
Rights
| component | license | hosting | redistribution | modification | training | Evidence and notes |
|---|---|---|---|---|---|---|
| code | Apache-2.0 | permitted | permitted | permitted | unknown | Repository page lists Apache-2.0. |
| model_weights | unknown | unknown | unknown | unknown | unknown | Checkpoint license not read; do not infer from the code license. |