Skip to content

下载原始 Skill 文件

bash
curl https://ai.ospreyai.cn/docs/raw/skills/comfyui-fun-inpaint-video-generation.md -o comfyui-fun-inpaint-video-generation.md

对话式接入

本 Skill 文件可被 AI 助手(Claude Code、Cursor、ChatGPT 等)学习,通过自然语言对话完成首尾帧视频生成。

在 AI 对话中发送以下指令即可:

学习:https://ai.ospreyai.cn/docs/raw/skills/comfyui-fun-inpaint-video-generation.md,保存为本地的技能 skills

更多接入方式和使用示例详见 API 文档 — AI 助手对话式接入


ComfyUI Wan 2.2 Fun Inpaint 首尾帧视频生成

Overview

通过公网网关 https://ai.ospreyai.cn 使用 ComfyUI 将 首帧 + 尾帧两张图片 生成一段过渡视频。

使用 Wan 2.2 Fun Inpaint + LightX2V 4 步加速 工作流,核心节点 WanFunInpaintToVideo 接收首帧(start_image)和尾帧(end_image),自动补间生成从起始画面到结束画面的平滑过渡动画(640×640, 16fps, 81帧, ~5 秒)。

核心特性:

  • 首尾帧控制:精确从 start_image 过渡到 end_image
  • Fun Inpaint 模型:Wan 2.2 专为 inpaint 场景优化的扩散模型,比标准 I2V 更适合首尾帧补间
  • 双 UNET 采样:高噪声 + 低噪声模型分阶段采样(步骤 0→2 + 2→4),画质更稳定
  • LightX2V 4 步加速:LoRA 加速仅需 4 步采样,生成速度快
  • 采样偏移 shift=8:比标准 I2V(shift=5)更高的偏移值,适合首尾帧长距离过渡

与其他视频生成 Skill 的对比:

Skill核心节点输入输出适用场景
视频生成WanImageToVideo1 张起始图~5s 视频从单图生成动态视频
6 关键帧视频WanFirstLastFrameToVideo ×56 张关键帧~5s 视频多关键帧过渡
本 SkillWanFunInpaintToVideo首帧 + 尾帧~5s 视频两张图之间的精确过渡

所有 API 均需 Bearer Token 鉴权(Authorization: Bearer sk-xxx)。

Quick Start

bash
export GW="https://ai.ospreyai.cn"
export API_KEY="sk-your-api-key"

# 1. 上传首帧和尾帧图片
curl -s -H "Authorization: Bearer $API_KEY" -X POST "$GW/api/v1/upload" \
  -F "image=@start.png" -F "overwrite=true"
curl -s -H "Authorization: Bearer $API_KEY" -X POST "$GW/api/v1/upload" \
  -F "image=@end.png" -F "overwrite=true"

# 2. 提交 Fun Inpaint 视频工作流(完整 JSON 见下方 Route A)
curl -s -H "Authorization: Bearer $API_KEY" -X POST "$GW/api/v1/ai/video/generate" \
  -H "Content-Type: application/json" \
  -d '{"prompt":{...}}'

# 3. 查询任务状态
curl -s -H "Authorization: Bearer $API_KEY" "$GW/api/v1/ai/tasks/{prompt_id}"

# 4. 下载 MP4 视频
curl -s -H "Authorization: Bearer $API_KEY" \
  "$GW/api/v1/ai/image/view/?filename=output.mp4&subfolder=video&type=output" \
  -o output.mp4

Task Routing

场景动作
首次生成首尾帧视频→ Route A: Upload & Generate
需要查看任务是否完成→ Route B: Check Status
需要获取或下载 MP4→ Route C: Download
需要调优提示词、帧数或尺寸→ Route D: Tune Parameters
需要排查服务、节点、模型或文件问题→ Route E: Troubleshoot

Route A: Upload & Generate

服务信息

  • 网关地址: https://ai.ospreyai.cn
  • 上传接口: POST /api/v1/upload
  • 提交接口: POST /api/v1/ai/video/generate
  • 鉴权方式: Authorization: Bearer sk-xxx

基本流程

  1. 上传首帧(start)和尾帧(end)两张图片到 ComfyUI
  2. 提交 Fun Inpaint 视频工作流
  3. 工作流包含 17 个节点:
    • 2 × LoadImage — 加载首帧和尾帧图片
    • WanFunInpaintToVideo — 首尾帧转视频核心节点
    • CreateVideo — 创建视频,设置帧率
    • SaveVideo — 保存视频为 MP4
    • 其他 13 个节点 — CLIP/VAE/UNet 加载、LoRA 加速、双 KSamplerAdvanced 采样等

Step 1: 上传首帧和尾帧图片

bash
# 上传首帧图片
curl -s -H "Authorization: Bearer $API_KEY" -X POST "$GW/api/v1/upload" \
  -F "image=@/path/to/start.png" \
  -F "overwrite=true"

# 上传尾帧图片
curl -s -H "Authorization: Bearer $API_KEY" -X POST "$GW/api/v1/upload" \
  -F "image=@/path/to/end.png" \
  -F "overwrite=true"

响应(每次上传):

json
{"name": "start.png", "subfolder": "", "type": "input"}

记住两个返回的 name 值,工作流中 LoadImage 节点需要使用。

最大文件大小: 50MB。推荐使用 PNG 格式,两张图片尺寸应一致(建议 640×640)。

Step 2: 提交 Fun Inpaint 视频工作流

bash
curl -s -H "Authorization: Bearer $API_KEY" -X POST "$GW/api/v1/ai/video/generate" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": {
      "90": {"inputs": {"clip_name": "umt5_xxl_fp8_e4m3fn_scaled.safetensors", "type": "wan", "device": "default"}, "class_type": "CLIPLoader"},
      "91": {"inputs": {"text": "色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸变的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走", "clip": ["90", 0]}, "class_type": "CLIPTextEncode"},
      "92": {"inputs": {"vae_name": "wan_2.1_vae.safetensors"}, "class_type": "VAELoader"},
      "93": {"inputs": {"shift": 8, "model": ["116", 0]}, "class_type": "ModelSamplingSD3"},
      "94": {"inputs": {"shift": 8, "model": ["117", 0]}, "class_type": "ModelSamplingSD3"},
      "95": {"inputs": {"add_noise": "disable", "noise_seed": 0, "steps": 4, "cfg": 1, "sampler_name": "euler", "scheduler": "simple", "start_at_step": 2, "end_at_step": 4, "return_with_leftover_noise": "disable", "model": ["94", 0], "positive": ["111", 0], "negative": ["111", 1], "latent_image": ["96", 0]}, "class_type": "KSamplerAdvanced"},
      "96": {"inputs": {"add_noise": "enable", "noise_seed": 42, "steps": 4, "cfg": 1, "sampler_name": "euler", "scheduler": "simple", "start_at_step": 0, "end_at_step": 2, "return_with_leftover_noise": "enable", "model": ["93", 0], "positive": ["111", 0], "negative": ["111", 1], "latent_image": ["111", 2]}, "class_type": "KSamplerAdvanced"},
      "97": {"inputs": {"samples": ["95", 0], "vae": ["92", 0]}, "class_type": "VAEDecode"},
      "99": {"inputs": {"text": "你的正向提示词(英文动作描述)", "clip": ["90", 0]}, "class_type": "CLIPTextEncode"},
      "100": {"inputs": {"fps": 16, "images": ["97", 0]}, "class_type": "CreateVideo"},
      "101": {"inputs": {"unet_name": "wan2.2_i2v_high_noise_14B_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"},
      "102": {"inputs": {"unet_name": "wan2.2_i2v_low_noise_14B_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"},
      "110": {"inputs": {"image": "你的首帧图片.png"}, "class_type": "LoadImage"},
      "111": {"inputs": {"width": 640, "height": 640, "length": 81, "batch_size": 1, "positive": ["99", 0], "negative": ["91", 0], "vae": ["92", 0], "start_image": ["110", 0], "end_image": ["112", 0]}, "class_type": "WanFunInpaintToVideo"},
      "112": {"inputs": {"image": "你的尾帧图片.png"}, "class_type": "LoadImage"},
      "116": {"inputs": {"lora_name": "wan2.2_i2v_lightx2v_4steps_lora_v1_high_noise.safetensors", "strength_model": 1, "model": ["101", 0]}, "class_type": "LoraLoaderModelOnly"},
      "117": {"inputs": {"lora_name": "wan2.2_i2v_lightx2v_4steps_lora_v1_low_noise.safetensors", "strength_model": 1, "model": ["102", 0]}, "class_type": "LoraLoaderModelOnly"},
      "158": {"inputs": {"filename_prefix": "video/fun_inpaint", "format": "auto", "codec": "auto", "video": ["100", 0]}, "class_type": "SaveVideo"}
    }
  }'

响应:

json
{"prompt_id": "e1f2a3b4-...", "number": 150, "node_errors": {}}

node_errors 为空对象 {} 表示工作流校验通过。如有错误会在此列出。

使用前需修改:

节点字段修改为
99text你的正向提示词(英文动作描述)
110image首帧图片上传返回的 name
112image尾帧图片上传返回的 name
158filename_prefix输出文件名前缀,如 "video/my_fun_inpaint"

节点说明

节点 IDclass_type功能
90CLIPLoader加载 Wan 的 CLIP 模型(文本编码器)
91CLIPTextEncode负向提示词编码(中文质量排除词)
92VAELoader加载 VAE 模型
93ModelSamplingSD3高噪声模型采样偏移调整(shift=8)
94ModelSamplingSD3低噪声模型采样偏移调整(shift=8)
95KSamplerAdvanced后半段采样(去噪,步骤 2→4,低噪声模型)
96KSamplerAdvanced前半段采样(加噪,步骤 0→2,高噪声模型)
97VAEDecodeVAE 解码 latent → 像素帧
99CLIPTextEncode正向提示词编码(英文动作描述)
100CreateVideo创建视频,设置帧率
101UNETLoader加载高噪声 UNet 模型
102UNETLoader加载低噪声 UNet 模型
110LoadImage加载首帧图片
111WanFunInpaintToVideo首尾帧转视频核心节点
112LoadImage加载尾帧图片
116LoraLoaderModelOnly加载高噪声 LoRA(4 步加速)
117LoraLoaderModelOnly加载低噪声 LoRA(4 步加速)
158SaveVideo保存视频为 MP4

工作流连接图

LoadImage(110) ──→ WanFunInpaintToVideo(111) ──→ KSamplerHigh(96) ──→ KSamplerLow(95) ──→ VAEDecode(97) ──→ CreateVideo(100) ──→ SaveVideo(158)
LoadImage(112) ──→ │                          │                    │
                    │                          │                    │
CLIPLoader(90) ──→ CLIPTextEncode_Pos(99) ────┤                    │
               └→ CLIPTextEncode_Neg(91) ──────┤                    │
VAELoader(92) ──→ WanFunInpaintToVideo(111)    │                    │
VAELoader(92) ──→ VAEDecode(97)                                      │

UNETLoader(101) → LoRA_High(116) → ModelSamplingSD3(93, shift=8) ──→ KSamplerHigh(96)
UNETLoader(102) → LoRA_Low(117)  → ModelSamplingSD3(94, shift=8) ──→ KSamplerLow(95)

采样流程(两阶段):

WanFunInpaintToVideo(111) 输出 latent


KSamplerAdvanced(96)                    KSamplerAdvanced(95)
├─ add_noise: enable                    ├─ add_noise: disable
├─ model: 高噪声 (shift=8)              ├─ model: 低噪声 (shift=8)
├─ steps: 4                             ├─ steps: 4
├─ start_at_step: 0 → end_at_step: 2    ├─ start_at_step: 2 → end_at_step: 4
├─ return_with_leftover_noise: enable   ├─ return_with_leftover_noise: disable
└─ 输出: 部分去噪的 latent ──────────→  └─ 输出: 完全去噪的 latent ──→ VAEDecode

关键参数

节点字段说明示例值
110image首帧图片文件名(需先上传)"start.png"
112image尾帧图片文件名(需先上传)"end.png"
99text正向提示词(英文动作描述)"A cat waking up and stretching"
111width / height视频尺寸640 × 640
111length视频帧数(81帧≈5秒@16fps)81
100fps帧率16
96noise_seed随机种子(不同值生成不同视频)42
93/94shift采样偏移(Fun Inpaint 默认 8)8
158filename_prefix输出文件名前缀"video/fun_inpaint"

输出视频参数

  • 分辨率: 640 × 640
  • 帧数: 81 帧
  • 帧率: 16 fps
  • 时长: 约 5 秒
  • 格式: MP4
  • 输出路径: subfolder=video, type=output

Route B: Check Status

查询队列

bash
curl -s -H "Authorization: Bearer $API_KEY" "$GW/api/v1/ai/queue"

响应:

json
{
  "queue_running": [[150, "e1f2a3b4-...", {...}]],
  "queue_pending": []
}
  • queue_running 不为空 → 任务正在执行
  • queue_pending 不为空 → 任务排队等待

查询任务状态

bash
curl -s -H "Authorization: Bearer $API_KEY" "$GW/api/v1/ai/tasks/{prompt_id}"

进行中:

json
{
  "e1f2a3b4-...": {
    "status": {"status_str": "success", "completed": false},
    "outputs": {}
  }
}

已完成:

json
{
  "e1f2a3b4-...": {
    "status": {"status_str": "success", "completed": true},
    "outputs": {
      "158": {
        "images": [{"filename": "fun_inpaint_00001_.mp4", "subfolder": "video", "type": "output"}],
        "animated": [true]
      }
    }
  }
}

视频生成通常需要 30 秒至 2 分钟(取决于 GPU 和队列情况)。

Route C: Download

下载前先从任务状态中获取输出文件信息(filename, subfolder, type),然后通过查看接口下载。

bash
# 下载 MP4 视频(注意 subfolder=video)
curl -s -H "Authorization: Bearer $API_KEY" \
  "$GW/api/v1/ai/image/view/?filename=fun_inpaint_00001_.mp4&subfolder=video&type=output" \
  -o output.mp4

关键点:

  • 视频输出的 subfoldervideo(不是空字符串)
  • 视频输出的 typeoutput
  • 下载的是 MP4 格式,可直接播放

Python 调用示例

python
import requests
import time

GW = "https://ai.ospreyai.cn"
API_KEY = "sk-your-api-key"
headers = {"Authorization": f"Bearer {API_KEY}"}

# 1. 上传首帧和尾帧图片
def upload_image(path):
    with open(path, "rb") as f:
        resp = requests.post(f"{GW}/api/v1/upload", headers=headers,
                             files={"image": f}, data={"overwrite": "true"})
        name = resp.json()["name"]
        print(f"Uploaded: {name}")
        return name

start_name = upload_image("start.png")
end_name = upload_image("end.png")

# 2. 提交 Fun Inpaint 视频工作流
prompt = {
    "prompt": {
        "90": {"inputs": {"clip_name": "umt5_xxl_fp8_e4m3fn_scaled.safetensors", "type": "wan", "device": "default"}, "class_type": "CLIPLoader"},
        "91": {"inputs": {"text": "色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸变的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走", "clip": ["90", 0]}, "class_type": "CLIPTextEncode"},
        "92": {"inputs": {"vae_name": "wan_2.1_vae.safetensors"}, "class_type": "VAELoader"},
        "93": {"inputs": {"shift": 8, "model": ["116", 0]}, "class_type": "ModelSamplingSD3"},
        "94": {"inputs": {"shift": 8, "model": ["117", 0]}, "class_type": "ModelSamplingSD3"},
        "95": {"inputs": {"add_noise": "disable", "noise_seed": 0, "steps": 4, "cfg": 1, "sampler_name": "euler", "scheduler": "simple", "start_at_step": 2, "end_at_step": 4, "return_with_leftover_noise": "disable", "model": ["94", 0], "positive": ["111", 0], "negative": ["111", 1], "latent_image": ["96", 0]}, "class_type": "KSamplerAdvanced"},
        "96": {"inputs": {"add_noise": "enable", "noise_seed": 42, "steps": 4, "cfg": 1, "sampler_name": "euler", "scheduler": "simple", "start_at_step": 0, "end_at_step": 2, "return_with_leftover_noise": "enable", "model": ["93", 0], "positive": ["111", 0], "negative": ["111", 1], "latent_image": ["111", 2]}, "class_type": "KSamplerAdvanced"},
        "97": {"inputs": {"samples": ["95", 0], "vae": ["92", 0]}, "class_type": "VAEDecode"},
        "99": {"inputs": {"text": "A cat waking up, stretching, and looking around", "clip": ["90", 0]}, "class_type": "CLIPTextEncode"},
        "100": {"inputs": {"fps": 16, "images": ["97", 0]}, "class_type": "CreateVideo"},
        "101": {"inputs": {"unet_name": "wan2.2_i2v_high_noise_14B_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"},
        "102": {"inputs": {"unet_name": "wan2.2_i2v_low_noise_14B_fp8_scaled.safetensors", "weight_dtype": "default"}, "class_type": "UNETLoader"},
        "110": {"inputs": {"image": start_name}, "class_type": "LoadImage"},
        "111": {"inputs": {"width": 640, "height": 640, "length": 81, "batch_size": 1, "positive": ["99", 0], "negative": ["91", 0], "vae": ["92", 0], "start_image": ["110", 0], "end_image": ["112", 0]}, "class_type": "WanFunInpaintToVideo"},
        "112": {"inputs": {"image": end_name}, "class_type": "LoadImage"},
        "116": {"inputs": {"lora_name": "wan2.2_i2v_lightx2v_4steps_lora_v1_high_noise.safetensors", "strength_model": 1, "model": ["101", 0]}, "class_type": "LoraLoaderModelOnly"},
        "117": {"inputs": {"lora_name": "wan2.2_i2v_lightx2v_4steps_lora_v1_low_noise.safetensors", "strength_model": 1, "model": ["102", 0]}, "class_type": "LoraLoaderModelOnly"},
        "158": {"inputs": {"filename_prefix": "video/fun_inpaint", "format": "auto", "codec": "auto", "video": ["100", 0]}, "class_type": "SaveVideo"}
    }
}
resp = requests.post(f"{GW}/api/v1/ai/video/generate", headers=headers, json=prompt)
prompt_id = resp.json()["prompt_id"]
print(f"Task submitted: {prompt_id}")

# 3. 轮询任务状态
while True:
    resp = requests.get(f"{GW}/api/v1/ai/tasks/{prompt_id}", headers=headers)
    data = resp.json()
    task = data.get(prompt_id, {})
    status = task.get("status", {})
    if status.get("completed"):
        print("Task completed!")
        break
    print(f"Status: {status.get('status_str', 'unknown')}...")
    time.sleep(5)

# 4. 下载视频
outputs = task.get("outputs", {}).get("158", {})
video_info = outputs.get("images", [{}])[0]
resp = requests.get(f"{GW}/api/v1/ai/image/view/", headers=headers,
                    params={"filename": video_info["filename"],
                            "subfolder": video_info.get("subfolder", ""),
                            "type": video_info.get("type", "output")})
with open("fun_inpaint_output.mp4", "wb") as f:
    f.write(resp.content)
print(f"Downloaded: fun_inpaint_output.mp4 ({len(resp.content)} bytes)")

Route D: Tune Parameters

参数节点建议
首帧图片110推荐 PNG,清晰度越高越好
尾帧图片112推荐 PNG,尺寸应与首帧一致
正向提示词99 text英文动作描述效果更稳定,如 "A butterfly emerging from cocoon and spreading wings"
负向提示词91 text默认可保持,避免修改
视频尺寸111 width/height默认 640×640;支持 832×480 等,需为 16 的倍数
视频帧数111 length默认 81(≈5s@16fps);41≈2.5s, 161≈10s
帧率100 fps默认 16;视频时长 = length / fps
随机种子96 noise_seed不同值生成不同视频;相同种子 + 相同参数可复现
采样步数95/96 stepsLightX2V 加速模式下固定 4 步,不建议修改
采样偏移93/94 shift默认 8;Fun Inpaint 专用值,增大更锐利,减小更平滑

与标准 I2V 工作流的参数差异

参数标准 I2V (WanImageToVideo)Fun Inpaint (WanFunInpaintToVideo)
核心节点WanImageToVideoWanFunInpaintToVideo
输入图片仅 start_imagestart_image + end_image
采样偏移 shift58
模型文件相同(wan2.2_i2v_high/low_noise_14B)相同
LoRA 加速相同(lightx2v_4steps)相同
输出单张图驱动的自由运动首尾帧约束下的精确过渡

Fun Inpaint vs FirstLastFrameToVideo

特性Fun Inpaint (本 Skill)FLF2V (6 关键帧 Skill)
输入帧数2 帧(首+尾)6 帧(5 段首尾)
段数1 段5 段
每段帧数81 帧25 帧/段
输出帧率16 fps24 fps
输出分辨率640×640720×720
采样偏移shift=8shift=5
生成时间30s-2min1-5min
适用场景两张图之间的长过渡多关键帧串联

Route E: Troubleshoot

问题排查方法
上传图片失败检查文件大小(≤50MB)、格式(推荐 PNG)、Bearer Token 是否有效
工作流提交报 value_not_in_list模型文件名不正确,检查 UNet/CLIP/VAE/LoRA 名称是否与服务器一致
工作流提交报 return_type_mismatch节点间连接类型不匹配,检查节点输出→输入的链接是否正确
任务长时间未完成检查 /api/v1/ai/queue 是否有排队任务;GPU 可能忙碌
下载视频返回空检查 subfolder 是否为 video(不是空字符串)
首尾帧过渡不自然检查首尾帧图片风格/构图是否一致,差异过大会导致中间帧质量下降
视频动态不足正向提示词添加更多动作描述;增大 length 值增加帧数
401 鉴权失败检查 Bearer Token 是否有效:curl -H "Authorization: Bearer sk-xxx" $GW/api/v1/ai/queue
429 请求被限流AI 接口 10次/分/IP,稍后重试

内网直连 vs 公网网关

内网直连公网网关
地址192.168.1.236:8188ai.ospreyai.cn
鉴权Authorization: Bearer sk-xxx
上传路径/upload/image/api/v1/upload
提交路径/prompt/api/v1/ai/video/generate
任务查询/history/{id}/api/v1/ai/tasks/{id}
队列查询/queue/api/v1/ai/queue
下载路径/view?filename=.../api/v1/ai/image/view/?filename=...&subfolder=video&type=output
限流10次/分/IP

所需模型文件

类型文件名存放目录
Text Encoderumt5_xxl_fp8_e4m3fn_scaled.safetensorsmodels/text_encoders/
VAEwan_2.1_vae.safetensorsmodels/vae/
Diffusion (高噪声)wan2.2_i2v_high_noise_14B_fp8_scaled.safetensorsmodels/diffusion_models/
Diffusion (低噪声)wan2.2_i2v_low_noise_14B_fp8_scaled.safetensorsmodels/diffusion_models/
LoRA (高噪声加速)wan2.2_i2v_lightx2v_4steps_lora_v1_high_noise.safetensorsmodels/loras/
LoRA (低噪声加速)wan2.2_i2v_lightx2v_4steps_lora_v1_low_noise.safetensorsmodels/loras/

Verification Checklist

  • [ ] 首帧图片上传成功,返回 name
  • [ ] 尾帧图片上传成功,返回 name
  • [ ] 工作流提交成功,node_errors 为空
  • [ ] 任务在 /api/v1/ai/queue 中执行完成
  • [ ] 任务状态 completed: true,outputs 包含节点 158 的 MP4 文件信息
  • [ ] 成功下载 MP4 文件(注意 subfolder=video
  • [ ] 视频可正常播放,首尾帧过渡自然
  • [ ] 视频分辨率为 640×640,帧率 16fps

AI API Gateway Documentation