🚀 What's New in v2.15.0

• 🎭 MiniMax-Remover — new AI video object removal with 81-frame DiT inpainting, sliding-window batching for long videos, and automatic tiered fallback (MiniMax → FLUX Klein → LaMa → FFmpeg)
• 🎛️ 5 New No-LLM Modes — run ai_upscale, video_depth, flux_klein, minimax_remover, and animate_portrait directly from the dropdown without any LLM
• 🧠 Auto-VRAM Tile Sizing — AI upscaler auto-calculates optimal tile dimensions from available GPU memory; new tile_size parameter for manual VRAM control
• ⚡ VRAM Management Overhaul — pipelines transfer to CPU before cleanup, only loaded modules are processed, consolidated FFmpeg binary paths across all synthesizers
• 🎬 NLE Editor & Upscaler Improvements — video dimension padding, framerate compatibility fixes, 1-based frame indexing, dynamic mask fallback resolution

📄 See CHANGELOG.md for the complete version history.

NotebookLM Overview: Exploring the features and capabilities of ComfyUI-FFMPEGA. (Click to watch on YouTube)

✨ Features

🎬 Examples

See what FFMPEGA can do — each example shows the prompt or preset used, the input clip, and the result.

4×4 Video Grid

Prompt: Concatenate these clips in a 4x4 grid

Crossfade Transitions + Bouncing Logo

Prompt: Concatenate these clips with a crossfade transition between each, add a fade in at the start and fade out at the end and a bouncing image_path_a at 10% size and 30% opacity

Color Grade + Text Overlay + Compression

Prompt: Color grade with the cinematic teal orange LUT, normalize audio, add a text "water" in the bottom right corner, compress for web at 720p

Picture-in-Picture with Audio Mix

Prompt: Place video_b in the bottom-right corner at 25% size with a white border over video_a and mix the audio

Vintage Film Look + Subtitles

Prompt: Normalize audio to -14 LUFS, add a warm vintage film look with grain overlay, and burn in these subtitles

Datamosh Glitch Effects

Prompt: Apply datamosh glitch effect, add chromatic aberration with strong RGB split, pixelate slightly, add ghost trails

Cinematic Teal & Orange

Prompt: Add a cinematic teal and orange color grade, apply a subtle vignette, and fade in from black

Neon Glow Edge Detection

Prompt: Apply a neon glow edge detection effect, add chromatic aberration, and slow the video to 0.5x speed with smooth motion

Colorhold Noir

Prompt: Use colorhold to keep only the red, desaturate everything else, boost contrast to 1.5, add a strong vignette, apply noir style

Green Screen Removal

Prompt: Remove the green screen with chroma key, despill the green edges, sharpen slightly

📦 Installation

Requirements

• ComfyUI (latest)
• Python 3.10+
• FFMPEG installed and in PATH (install guide)
• Node.js 18+ (required for CLI tools: Gemini CLI, Claude CLI, Qwen CLI — download)
• Ollama (optional, for local LLM inference — download)

Option 1: ComfyUI Manager (Recommended)

1. Open ComfyUI Manager
2. Search for ComfyUI-FFMPEGA
3. Click Install

Option 2: Manual Install

cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/AEmotionStudio/ComfyUI-FFMPEGA.git
cd ComfyUI-FFMPEGA
pip install -r requirements.txt

cd C:\path\to\ComfyUI\custom_nodes
git clone https://github.com/AEmotionStudio/ComfyUI-FFMPEGA.git
cd ComfyUI-FFMPEGA
pip install -r requirements.txt

Note: Use whichever Python package manager your ComfyUI venv uses (pip, uv pip, etc.). The above commands assume pip is available in your ComfyUI virtual environment.

Restart ComfyUI after installation.

🚀 Quick Start

1. Add an FFMPEG Agent node to your workflow
2. Connect a video path or use the input field
3. Enter a natural language prompt
4. Select your LLM model
5. Run the workflow

💡 Tip: Right-click the FFMPEG Agent node to open the FFMPEGA Presets context menu — 200+ categorized effects you can apply with a single click, no prompt typing needed. Great for quick edits or discovering what's available.

Example Prompts

💬 Prompt Guide

When using an LLM, the AI agent interprets your natural language and maps it to skills with specific parameters. Here's how to get the best results. (For manual editing without an LLM, see the Effects Builder and Text Input sections.)

Specifying Exact Values

You can request specific parameter values and the agent will use them directly:

What Works Well ✅

• Explicit numbers: "brightness 0.2", "speed 1.5x", "CRF 20" — the agent maps these directly
• Named presets: "VHS look", "cinematic style", "noir" — triggers multi-step preset pipelines
• Chaining operations: "Trim first 5 seconds, resize to 720p, add vignette" — executes in order
• Descriptive goals: "Make it look warmer", "Remove the green screen" — the agent picks the right skills
• Technical terms: "denoise", "deband", "normalize audio" — maps to exact FFmpeg filters

What Might Not Work as Expected ⚠️

• Vague intensity words: "Make it very blurry" or "a little brighter" — the agent has to guess what number "very" or "a little" means. Tip: use a specific value instead: "blur with radius 15"
• Out-of-range values: Parameters are auto-clamped to their valid range. If you ask for "brightness 5.0" it caps at the max (1.0)
• Complex compositing: Multi-layer effects with precise timing may need to be broken into separate passes
• Format-dependent features: Some effects (like transparency) require specific output formats. H.264/MP4 doesn't support alpha channels

🧠 AI Models (Auto-Downloaded)

Some skills use AI models that auto-download on first use. You can disable automatic downloads with the allow_model_downloads toggle on the FFMPEG Agent node — runs requiring a missing model will fail with a clear message and a manual download link.

All models are mirrored to first-party AEmotionStudio HuggingFace repos for supply chain resilience. Downloads try the AEmotionStudio mirror first, then fall back to upstream sources.

🎛️ Nodes

FFMPEGA provides 11 nodes that work together:

🎯 Skill System

FFMPEGA includes a comprehensive skill system with 218 operations organized into categories. Use them in two ways: let the AI agent select skills from your prompt, or pick them yourself with the Effects Builder — no LLM needed.

📄 See SKILLS_REFERENCE.md for the complete skill reference with all parameters and example prompts.

🧪 See SKILL_TEST_PROMPTS.md for ready-to-use copy-and-paste test prompts for every skill.

🧠 Agentic Tools

Beyond the 200+ editing skills, the agent has built-in tools for analyzing media and making better decisions. In LLM mode, the agent calls these autonomously based on your prompt. Some (like analyze_video and search_skills) are also invoked directly by internal skills, the Effects Builder, and no-LLM modes.

# custom_skills/dreamy_blur.yaml
name: dreamy_blur
description: "Soft dreamy blur with glow"
category: visual
tags: [dream, blur, soft, glow]

parameters:
  radius:
    type: int
    default: 5
    min: 1
    max: 30

ffmpeg_template: "gblur=sigma={radius},eq=brightness=0.06"

# Linux / macOS / Windows (Git Bash or PowerShell)
cd custom_skills/
git clone https://github.com/someone/ffmpega-retro-pack retro-pack

🤖 LLM Configuration

Tested with: FFMPEGA has been primarily tested using Gemini CLI and Qwen3 8B (via Ollama). Results may vary with other models.

Author's pick: The CLI connectors (especially Gemini CLI) have been the most reliable option in my experience — they handle tool-calling, structured output, and long context exceptionally well. Highly recommended if you have access.

Choosing a Model

FFMPEGA works best with models that have strong JSON output and instruction-following abilities. The agent sends a structured prompt and expects a valid JSON pipeline back — models with tool-calling or function-calling capabilities tend to perform best.

Things to keep in mind:

• Some models work better than others — larger models and those trained for structured output (JSON/tool-calling) produce more reliable results
• Some models may need more retries — if the agent fails to parse the response, try running the same prompt again. Smaller models occasionally return malformed JSON on the first try
• Find what works best for you — experiment with different models to find the right balance of speed, quality, and reliability for your hardware

Ollama (Local — Free)

The default option. Runs locally, no API key needed.

Install Ollama: Download from ollama.com/download (available for Linux, macOS, and Windows).

# Start Ollama (all platforms)
ollama serve

# Pull a model
ollama pull qwen3:8b

Windows users: After installing Ollama, the ollama command is available in both PowerShell and Command Prompt. Ollama also runs as a system tray app.

Recommended local models (≤30B, consumer GPU friendly):

Tip: On the Ollama library, look for models with a tools tag — this indicates native tool/function-calling support, which produces the best results with FFMPEGA.

OpenAI

llm_model: gpt-5.2
api_key: your-openai-key

Gemini (Google API)

llm_model: gemini-3-flash
api_key: your-google-ai-key

Gemini CLI (Free with any Google Account)

Use the Gemini CLI to run Gemini models without an API key. Works with any Google account.

Install:

Tip: You can also use pnpm add -g or yarn global add if you prefer.

Authenticate (first time only):

gemini

This opens a browser to sign in with your Google account.

Use in FFMPEGA:

llm_model: gemini-cli

No API key is needed — authentication is handled by the CLI. Select gemini-cli from the model dropdown in the node.

Note: The Gemini CLI runs as a subprocess and is sandboxed to the custom node directory for security. On Windows, gemini.cmd is also detected automatically.

Plans & Usage Limits

Tip: Sign in with a Google account (free) for the best experience — 1,000 requests/day with access to Pro and Flash models. An unpaid API key limits you to 250/day on Flash only.

Available Models

The Gemini CLI auto-selects the best model, but the following are available:

Free tier may auto-switch to Flash models when Pro quota is exhausted.

Anthropic

llm_model: claude-sonnet-4-6
api_key: your-anthropic-key

Claude Code CLI (Free with Anthropic Account)

Use the Claude Code CLI as a local LLM backend. Uses its own authentication — no API key needed in FFMPEGA.

Install:

Tip: You can also use pnpm add -g or yarn global add if you prefer.

Authenticate (first time only):

claude

This opens a browser to sign in with your Anthropic account.

Use in FFMPEGA:

llm_model: claude-cli

Auto-detected on PATH. Select claude-cli from the model dropdown.

Cursor Agent CLI

Use Cursor's CLI in agent mode as an LLM backend.

Install (all platforms): Open Cursor IDE → Command Palette (Ctrl+Shift+P / Cmd+Shift+P) → "Install 'cursor' command"

Start the agent:

agent

Use in FFMPEGA:

llm_model: cursor-agent

Auto-detected on PATH. Select cursor-agent from the model dropdown.

Qwen Code CLI (Free — 2,000 requests/day)

Use Qwen Code as a free LLM backend. Powered by Qwen3-Coder with 2,000 free requests/day via OAuth — no credit card required.

Install:

Tip: You can also use pnpm add -g or yarn global add if you prefer.

Authenticate (first time only):

qwen

Select "Qwen OAuth (Free)" and follow the browser prompts to sign in.

Use in FFMPEGA:

llm_model: qwen-cli

Auto-detected on PATH. Select qwen-cli from the model dropdown.

👁️ CLI Agent Vision Support

When Frame Extraction is used, FFMPEGA saves extracted frames to a _vision_frames/ directory and passes the frame paths to the CLI agent. Agents with vision support can see and analyze the actual frame images to make better editing decisions.

Note: Agents without vision support still receive the frame file paths and can use video metadata (duration, resolution, FPS) from analyze_video to make editing decisions. When Qwen fixes their vision support upstream, it will work automatically since the frame paths are already passed correctly.

⚠️ Important: The _vision_frames/ directory must not be listed in .gitignore or .git/info/exclude — CLI agents respect these ignore patterns and will be unable to read the frames. FFMPEGA's cleanup_vision_frames() automatically deletes the directory after each pipeline run.

Custom Model

Select custom from the model dropdown and type any model name in the custom_model field. The provider is auto-detected from the name:

This lets you use any new model immediately without waiting for a code update.

🔒 API Key Security

Your API keys are automatically scrubbed and never stored in output files:

• Error messages — keys are redacted before being shown in the UI (e.g. ****abcd)
• Workflow metadata — ComfyUI embeds workflow data in output images/videos; FFMPEGA strips the api_key field from this metadata before saving
• HTTP errors — keys are removed from network error messages that might include auth headers
• Debug logs — LLMConfig redacts keys in all string representations

No configuration needed — this protection is always active when an API key is provided.

⚠️ Safety precaution: As with any software, always inspect your output files before sharing them publicly — in the unlikely event of a bug or edge case that bypasses the automatic scrubbing.

📊 Token Usage Tracking

Monitor your LLM token consumption with opt-in usage tracking. Enable via two toggles on the node:

Token data sources by connector:

When enabled, the analysis output includes a usage breakdown:

Token Usage:
  Prompt tokens:     4,200
  Completion tokens: 1,800
  Total tokens:      6,000
  LLM calls:         5
  Tool calls:        3
  Elapsed:           12.4s

The usage_log.jsonl file stores one JSON object per run for historical analysis. It is gitignored by default.

🐛 Troubleshooting

ffmpeg -version

ollama serve

ollama pull qwen2.5:8b

🤝 Contributing

Contributions are welcome! Whether it's bug reports, new skills, or improvements — your help is appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📝 License

This project is licensed under the GPL-3.0 License — see the LICENSE file for details.