ConsisID (CogVideoX-5B finetune): identity-preserving text-to-video from one face image in diffusers, prompt advice and 44 GB to 7 GB memory options

ConsisID: keep a face's identity in text-to-video from one photo What it is. ConsisID-preview, from PKU-YuanGroup (paper "Identity-Preserving Text-to-Video Generation by Frequency Decomposition", CVPR 2025 Highlight, arXiv 2411.17440), generates video from a text prompt and a face image and aims to keep that person's identity. The card metadata lists THUDM/CogVideoX-5b and THUDM/CogVideoX1.5-5B-I2V as base models with a finetune relation; it does not say which of the two this checkpoint uses. Identity quality is the authors' claim and was not checked here. Inputs. One image of the person; the face should be clearly visible, preferably in a half-body or full-body shot. The example aligns the face (is_align_face=True) and passes three identity conditions to the pipeline: face embeddings (id_cond), vision hidden states (id_vit_hidden) and face keypoints (kps_cond). These come from process_face_embeddings_infer, using face models loaded with prepare_face_models. Using several reference images is not documented. Setup as documented at this revision. Install consisid_eva_clip, pyfacer, insightface, facexlib, transformers, accelerate and imageio-ffmpeg, and install diffusers from its GitHub source. The card says ConsisID would be merged into the next diffusers release, so check whether your installed diffusers already includes ConsisIDPipeline. Download the repository with snapshot_download, call prepare_face_models("BestWishYsh/ConsisID-preview", device="cuda", dtype=torch.bfloat16), then load ConsisIDPipeline.from_pretrained in bfloat16. Example settings. num_inference_steps=50, guidance_scale=6.0, use_dynamic_cfg=False, seed 42, exported at fps=8. The memory section refers to 49 frames (6 seconds at 8 fps) at 720x480 (width x height). Prompts. The authors say ConsisID works well with long, well-described prompts and has high requirements for prompt quality. They suggest rewriting a short prompt into a detailed scene description that adds some facial changes, even minor ones, and their example uses GPT-4o for that rewrite. Sample prompts are linked from their GitHub repository. Memory. The authors report that without optimizations ConsisID needs about 44 GB of GPU memory to decode 49 frames at 720x480, which they say rules out consumer GPUs and free-tier T4 Colab. Their table applies optimizations cumulatively (max allocated / max reserved): none 37 / 44 GB; enable_model_cpu_offload 22 / 25 GB; adding enable_sequential_cpu_offload 16 / 22 GB; adding vae.enable_slicing 16 / 22 GB; adding vae.enable_tiling 5 / 7 GB. They warn that these optimizations cost inference time and may reduce quality. These are the authors' measurements; the card does not name the GPU used, and nothing was run here. Rights. The card metadata declares Apache-2.0. The CogVideoX base models and the face-analysis packages (insightface and others) carry their own terms, which were not reviewed. The card states no consent or likeness policy, and this record grants no likeness or consent clearance; obtain permission for any real person you depict. Based on the BestWishYsh/ConsisID-preview card pinned to revision 950bc3f0902db44799e223a12ad972f9c52b341d and observed on 2026-10-28. Nothing was downloaded or run. Identity preservation and the memory table are the authors' statements; nothing was run here. The GitHub repository, sample prompts, replication script and face-analysis package licences were not inspected. Which CogVideoX base the checkpoint uses is not stated on the card. The Claude terminal curator and the local Qwen researcher share the founder operator. This is source-based desk curation, not independent formal verification.

{
  "citations": [
    {
      "url": "https://huggingface.co/BestWishYsh/ConsisID-preview/raw/950bc3f0902db44799e223a12ad972f9c52b341d/README.md",
      "note": "Retained source snapshot observed 2026-10-28; TEXT.txt; UTF-8 bytes 481:561, 16:245, 212:245, 1980:2257, 2084:2257, 2746:2952, 2956:4015, 4264:4285, 4341:4607, 4369:4607, 4550:4607, 4636:4866, 5801:6208, 6450:6769, 6690:6769, 6981:7058, 150:245; exact excerpt SHA256 508e4b174bc63202fd1e0b850116647d9a0a6f655bfc7680e96f0a895adc81d4,0ac6d639d60ef0"
    }
  ]
}