Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers

Instalación básica con Diffusers:

pip install -U diffusers transformers accelerate

Ejemplo simple con Diffusers:

import torch
from diffusers import DiffusionPipeline

# cambiar a "mps" para dispositivos Apple
pipe = DiffusionPipeline.from_pretrained(
    "Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers",
    dtype=torch.bfloat16,
    device_map="cuda"
)

prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]

Uso recomendado con SkyReelsV2DiffusionForcingPipeline:

# pip install ftfy
import torch
from diffusers import AutoModel, SkyReelsV2DiffusionForcingPipeline, UniPCMultistepScheduler
from diffusers.utils import export_to_video

model_id = "Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers"
vae = AutoModel.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)

pipeline = SkyReelsV2DiffusionForcingPipeline.from_pretrained(
    model_id,
    vae=vae,
    torch_dtype=torch.bfloat16
)

flow_shift = 8.0 # 8.0 para T2V, 5.0 para I2V
pipeline.scheduler = UniPCMultistepScheduler.from_config(
    pipeline.scheduler.config,
    flow_shift=flow_shift
)
pipeline = pipeline.to("cuda")

prompt = "A cat and a dog baking a cake together in a kitchen. The cat is carefully measuring flour, while the dog is stirring the batter with a wooden spoon. The kitchen is cozy, with sunlight streaming through the window."

output = pipeline(
    prompt=prompt,
    num_inference_steps=30,
    height=544,
    width=960,
    num_frames=97,
    base_num_frames=97,
    ar_step=5,
    causal_block_size=5,
    overlap_history=None,
    addnoise_condition=20,
).frames[0]

export_to_video(output, "T2V.mp4", fps=24, quality=8)

Instalación desde el repositorio:

git clone https://github.com/SkyworkAI/SkyReels-V2
cd SkyReels-V2
pip install -r requirements.txt

Ejemplo de inferencia Diffusion Forcing para video largo:

model_id=Skywork/SkyReels-V2-DF-14B-540P

python3 generate_video_df.py \
  --model_id ${model_id} \
  --resolution 540P \
  --ar_step 5 \
  --causal_block_size 5 \
  --base_num_frames 97 \
  --num_frames 737 \
  --overlap_history 17 \
  --prompt "A graceful white swan with a curved neck and delicate feathers swimming in a serene lake at dawn, its reflection perfectly mirrored in the still water as mist rises from the surface, with the swan occasionally dipping its head into the water to feed." \
  --addnoise_condition 20 \
  --offload