README.md

June 12, 2026 · View on GitHub

PALLAIDIUM

A free generative AI movie studio integrated into the Blender Video Editor.

You only truly understand how your film should have been made once it is finished.

Pallaidium turns that insight into a workflow. Prototype the full production cycle with AI before committing to a single frame of real footage. Start from song lyrics or a scene description, convert it to a screenplay with shots, break it down into timed strips, generate the images, animate them, add voices and music, and edit based on the emotional weight of what you see and hear. Then reverse the process — extract captions from the result, reconstruct the screenplay, refine it, and go shoot your real film with the confidence of someone who has already made it.

All of this happens in one place: Blender's Video Sequence Editor. Free and open source.

Generation Matrix
Features
Requirements
How to install
Usage
Batch Processing
Tips
Useful add-ons
Credits

Generation Matrix

This matrix provides a quick overview of the core generative capabilities, mapping input types to possible outputs.

Input	Image	Video	Text	Audio	Music	Speech
Text	✅	✅	✅	✅	✅	✅
Image	✅	✅	✅	✅
Video	✅	✅	✅	✅

Features


Text to video	Text to image	Text to text
Text to speech	Text to audio	Text to music
Image to image	Image to video	Image to text
Video to video	Video to Image	Video to text
ControlNet	OpenPose	Canny
ADetailer	IP Adapter Face	IP Adapter Style
Multiple LoRAs	LoRA Weight	Style selector
Seed	Quality steps	Strip power
Frames (Duration)	Word power	Model card selector
Batch conversion	Batch refinement of images.	Prompt batching
Batch upscale & refinement of movies.	Render-to-path selector.	Render finished notification.
Render Queue (async batch jobs)	Job status tracking (PENDING / RUNNING / COMPLETED)	Start / Stop / Cancel controls
User-defined file path for generated files.	Seed and prompt added to strip name.	One-click install and uninstall dependencies.

Requirements

Windows. Limited support for Linux.
Blender 5.2 or later
A CUDA-supported Nvidia card with at least 6-16 GB VRAM.
CUDA: 12.8
20+ GB HDD. (Each model is 6+ GB).

For Linux, we'll have to rely on contributor support. Post your issues here: https://github.com/tin2tin/Pallaidium/issues/105

How to install

Video walkthrough: https://www.youtube.com/watch?v=jmSZlEV_ZLw

First, download and install git (must be on PATH): https://git-scm.com/downloads
Download Blender 5.2 or later: https://www.blender.org/download/ and unzip it into the Documents folder (avoid paths with spaces or special characters on Windows).
Download the add-on: https://github.com/tin2tin/Pallaidium/archive/refs/heads/main.zip
Install the add-on as usual: Preferences > Add-ons > Install > select file > enable the add-on.
In the Generative AI add-on preferences, hit the "Install Dependencies" button.
Restart the computer after installing dependencies, then open Blender normally.
Open the add-on UI in the Sequencer > Sidebar > Generative AI.
5-30 GB must be downloaded first the first time any model is executed.
When you have Pallaidium installed, reach out on Discord: https://discord.gg/HMYpnPzbTm or leave a note on how it is working for you. It means the world to me to know someone is using it!

If any Python modules are missing, use this add-on to install them manually:
https://github.com/tin2tin/blender_pip
If "WARNING: Failed to find MSVC", install "Tools for Visual Studio":
https://aka.ms/vs/17/release/vs_BuildTools.exe
If error: "Missing DLL", install Microsoft Visual C++ Redistributable:
https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170

Uninstall

Python dependencies are now installed into the Blender user data folder, separate from the Blender executable:

Windows: %APPDATA%\Blender Foundation\Blender\5.2\datafiles\Pallaidium\site-packages
Linux: ~/.config/blender/5.2/datafiles/Pallaidium/site-packages

To remove Pallaidium's Python libraries, delete the Pallaidium folder at the path above.

To remove the add-on itself, disable and uninstall it via Preferences > Add-ons, or delete the Blender 5.2 folder from your Documents folder (Windows portable install).

To remove only the downloaded AI model weights, delete the Hugging Face cache folder manually:

Linux: ~/.cache/huggingface/hub
Windows: %userprofile%\.cache\huggingface\hub

Usage

The addon panel is located in the Video Sequence Editor > Sidebar (N-Panel) > Generative AI tab.

Basic Workflow

Choose Output: In the "Output" section of the panel, select whether you want to generate an Image, Video, Audio, or Text.
Select Model: Based on your output choice, select a specific AI model from the dropdown list.
Set Input:
- For Prompts: Leave the Input dropdown on Prompts.
- For Strips: Select one or more strips in the VSE timeline and set the Input dropdown to Strips.
Configure: Enter your text prompts and adjust parameters like resolution, quality steps, and guidance scale.
Generate:
- If using Prompts, click the Generate button.
- If using Strips, click the Generate from Strips button.

The generated media will be saved to the directory specified in the addon preferences and automatically added to your VSE timeline on a new channel.

Change Log

2026-06-12: Fix: Ideogram 4 — switch default weights to ideogram-ai/ideogram-4-nf4-diffusers (~10.5 GB NF4) instead of the 17.9 GB SDNQ FP8 model. Adds HuggingFace token login before loading. Re-enables FP8 MatMul acceleration (SDNQ models only). Adds VAE tiling and slicing to reduce memory spikes during decoding. Adds optional torch.compile support for transformer blocks.

2026-06-12: Fix: Progress phase display — queue job runner now correctly shows "Downloading model" only for byte-unit tqdm bars and "Loading model" for all other tqdm activity. Applies to both the threaded and main-thread job execution paths.

2026-06-12: Add: Florence-2 → Mask Editor auto-routing — when Florence-2 mode is set to Box Json and "Send to Mask" is enabled, the generated JSON is automatically applied to the Mask Editor after text generation completes.

2026-06-12: Fix: Box Editor sidebar tab switch now deferred via bpy.app.timers so the Image Editor area is fully initialised before the N-panel category is set. Adds "Boxes" label with mask icon to the layer list box. Simplifies layer detail display (removes redundant labels).

2026-06-12: Change: Florence-2 "Ideogram 4" mode renamed to "Box Json" in the mode selector.

2026-06-12: Fix: UI panel layout — draw_post_seed_ui and "Open Box Editor" button now draw after Batch Count; image model card and HF token fields moved to their correct position below the movie section.

2026-06-09: Add: Claude agent integration via Blender MCP — pallaidium_mcp_tools.py provides a high-level Python API (generate_image(), generate_video(), generate_audio(), queue_generate(), list_image_models(), list_styles(), etc.) that Claude agents can call through the Blender MCP server (execute_blender_code). Load the helper once per session in Blender's Python Console, then ask Claude to generate, queue, or inspect content in natural language.

2026-06-09: Fix: Marlin Video Captions — switched to SDNQ int8 quantized weights (tintwotin/Marlin-2B-SDNQ-int8), display name updated to "Marlin: Video Captions (SDNQ)". Inference now runs on the main thread for stability. Added ctranslate2 to Windows and Linux requirements.

2026-06-09: Add: 20+ new prompt styles — Flux Klein / High-Fidelity Realism, LTX Video / Motion Still, Character Sheet, Storyboard, and a full set of Ads, Art (Vintage Pencil Sketch, Deconstructed Collage, Lithography variants, Folk Art, Sci-Fi Scale, 1980s Editorial, Storyboard Technical Study), Atmosphere (Volumetric Dispersion, Light Catching Particles), and Look (Clean Commercial, Dynamic Action, High-Performance Technical) templates.

2026-06-08: Change: Python dependencies location — packages are now installed into the Blender user data directory (%APPDATA%\Blender Foundation\Blender\5.2\datafiles\Pallaidium\site-packages on Windows; ~/.config/blender/5.2/datafiles/Pallaidium/site-packages on Linux) instead of the Blender executable folder. A computer restart is still required after installing dependencies. Running Blender as Administrator is no longer needed.

2026-06-07: Add: LTX-2.3 Multi-Input N-Anchor mode — place 3 or more Image strips inside a Meta strip to pin the generated video at multiple points in time. Each image's position within the Meta strip determines its frame anchor: the first image locks frame 0, the last image locks the final frame, and every image in between is mapped proportionally to the corresponding frame index. Works with the interactive generator and the Render Queue.

2026-06-07: Add: Display System Console preference — a new checkbox in add-on preferences (after "Use Local Files Only") controls whether the Windows system console is shown when generation starts. Unchecking it suppresses show_system_console and set_system_console_topmost across all four interactive operators (video, audio, image, text) and both queue execution paths.

2026-06-07: Fix: Local Files Only — local_files_only is now consistently applied to every from_pretrained and hf_hub_download call across all model plugins. When the setting is enabled and weights are not cached, a clear error is shown: "Weights missing. Uncheck 'Use Local Files Only' in Preferences to download."

2026-06-03: Add: FLUX.2 Klein 9B Schematic — transform images into depth, normal, pose, and segmentation maps via nomadoor schematic LoRAs.

2026-06-03: Add: OmniVoice — multilingual text-to-speech with broad language support.

2026-06-03: Add: Stem Splitter — separate audio tracks into stems (vocals, drums, bass, etc.) using AI source separation.

2026-05-31: Refactor: Complete plugin architecture — models are auto-discovered from models_plugins/. Drop a single .py file to register any new HuggingFace model. Unsupported models moved to unsupported_models_plugins/.

2026-05-31: Add: Blender 5.2 support.

2026-06-06: Add: Faster Whisper Transcription — multilingual speech-to-text with word-level timestamps directly into VSE subtitle strips. Select a SOUND strip, choose model size (tiny → large-v3) and language (or auto-detect), click Generate.

2026-06-06: Add: Redo buttons — a Redo button (↺) on each Render Queue job reloads its generation settings (prompt, model, steps, guidance, seed, dimensions, toggles) back into the Pallaidium panel. A matching Redo button in the AI Metadata panel does the same from any completed strip's stored metadata.

2026-06-05: Add: LTX-2.3 Multi-Input First Frame + Last Frame (FLF) mode — place two Image strips inside a Meta strip to anchor the first and last frames of the generated video. A single Image strip placed after all other children acts as a last-frame-only keyframe.

Older Changes

2026-06-03: Fix: Qwen Image Edit — reference images from pickers now correctly rendered through the VSE pipeline at queue time (handles video, meta, and transformed strips). Switched to SDNQ uint4 pre-quantized weights (Disty0/Qwen-Image-Edit-2511-SDNQ-uint4-svd-r32) for better quality. Removed Lightning LoRA, raised default steps to 50. Exposed resolution controls. Fixed black bars on reference images. Fixed playhead not moving when queuing. Fixed width/height passed to inference.

2026-05-31: Add: Render Queue — async batch generation with per-job status tracking (PENDING / RUNNING / COMPLETED / FAILED). Start / Stop / Cancel controls.

2026-05-31: Add: User-defined render-to path in add-on preferences.

2026-04-10: Add: Qwen3 TTS — voice-cloning text-to-speech with text reference transcript for precise timbre matching. Supports audiobook-length generation.

2026-04-05: Add: ACE Step — AI audio and music generation.

2026-04-01: Add: Hviske Subtitles — automatic subtitle generation via Whisper large-v3-turbo.

2026-03-20: Add: Marlin Video Captions.

2026-02-26: Add: LTX-2 Multi-Input — custom LoRA + detailer LoRA in last step.

2026-02-22: Add: LTX-2 Video as Distilled or Multi-Input (via Meta Strips).

2026-01-23: Add: BFL-FLUX.2 Klein 4B and 9B.

2025-11-28: Add: FLUX.2 & Z-Image & Optimized model-loading.

2025-10-27: Remove 30 dead models.

2025-10-1: Fix: Deps. Add: Qwen Multi-image Edit.

2025-7-05: Add: FLUX Kontext Relight.

2025-6-26: Add: FLUX.1 Dev Kontext. Update Diffusers by installing this: "git+https://github.com/huggingface/diffusers.git" with the Python Module Manager add-on (link below), and restart Blender.

2025-6-22: Add: Long string parsing for Chatterbox (for Audiobooks). Use Blender 5.0 Alpha.

2025-6-14: Add: Chroma txt2vid

2025-6-12: Add: ZuluVision/MoviiGen1.1_Prompt_Rewriter

2025-6-01: Add: Chatterbox with zero shot text to speech and speech to speech voice cloning.

2025_5_29: Add: Flux depth+canny+redux Fix: neg Flux input + py libs

2025-5-12: Add: F5-TTS Voice Cloning

2025-5-09: Update: LTX 0.9.7 w. img, txt & vid input

2025-5-08: FramePack (Img/+Last Img), Img+txt for MMAudio, Flux De-distilled

2025-4-11: Improved LTX 0.95, MetaData, StripPicker, Wan t2i

2025-2-25: Add: MMAudio for Video to Sync Audio

2025-2-21: Support for Skywork/SkyReels-V1-Hunyuan-T2V/I2V. Need a full update of dependencies! (Thx newgenai79 for int4 transformer)

2025-2-15: Add: LoRA support for HunyuanVideo + better preset

2025-2-12: Add multi-media prompting via: OmniGen

2025-2-10: Update: a-r-r-o-w/LTX-Video-0.9.1-diffusers ZhengPeng7/BiRefNet_HR MiaoshouAI/Florence-2-large-PromptGen-v2.0 New: ostris/Flex.1-alpha Alpha-VLLM/Lumina-Image-2.0 Efficient-Large-Model/Sana_1600M_1024px_diffusers Fix: Frame by frame (SD XL) Remove: Corcelio/mobius

2025-1-26: Add: MiniMax Cloud txt/img/subject to video (insert your MiniMax API key in MiniMax_API.txt) and fast FLUX LoRA

2025-1-15: FLUX: faster img2img and inpaint

2024-11-2: Add: Image Background Removal, Stable Diffusion 3.5 Medium, Fast Flux(t2i)

2024-9-19: Add: Image to Video for CogVideoX

2024-9-15: Add: LoRA import for Flux

2024-9-14: Add: Flux Inpaint & Img2img.

2024-9-4: Add: Florence 2 (Image Caption), AudioLDM2-Large, CogVideox-2b, flash_attn on Win.

2024-9-2: Add: Vid2vid for CogVideoX-5b and Parler TTS

2024-8-28: Make CogVideox-5b run on 6 GB VRAM & Flux on 2 GB VRAM

2024-8-27: Add: CogVideoX-5b Remove: Low-quality models

2024-8-5: Add: Flux Dev - NB. needs update of dependencies and 24 GB VRAM

2024-8-2: Add: Flux Schnell - NB. needs update of dependencies and 24 GB VRAM

2024-7-12: Add: Kwai/Kolors (txt2img & img2img)

2024-6-13: Add: SD3 - A "Read" token from HuggingFace must be entered, it's free (img2img). Fix: Installation of Dependencies

2024-6-6: Add: Stable Audio Open, Frame:-1 will inherit duration.

2024-6-1: IP Adapter(When using SDXL): Face (Image or folder), Style (image or folder) New image models: Mobius, OpenVision, Juggernaut X Hyper

2024-4-29: Add: PixArt Sigma 2k, PixArt 1024 and RealViz V4

2024-2-23: Add: Proteus Lightning and Dreamshaper XL Lightning

2024-2-21: Add: SDXL-Lightning 2 Step & Proteus v. 0.3

2024-1-02: Add: WhisperSpeech

2024-01-01: Fix installation and Bark bugs.

2024-01-31: Add OpenDalle, Speed option, SDXL, and LoRA support for Canny and OpenPose, including OpenPose rig images. Prune old models including SD.

2023-12-18: Add: Bark audio enhance, Segmind Vega.

2023-12-1: Add SD Turbo & MusicGen Medium, MPS device for MacOS.

2023-11-30: Add: SVD, SVD-XT, SDXL Turbo

Location

Install Dependencies, and set Sound Notification in the add-on preferences:

Video Sequence Editor > Sidebar > Generative AI:

Styles:

See SDXL handling most of the styles here: https://stable-diffusion-art.com/sdxl-styles/

Tips:

If your playback stutters, then select a Movie strip > Menu > Strip > Movie Strip > Set Render Size.
If you get the message that CUDA is out of memory, restart Blender to free up memory and make it stable again - and try with another model.
New to Blender? Watch this tutorial: https://youtu.be/4_MIaxzjh5Y?feature=shared

Batch Processing

Select multiple strips and hit Generate.

https://github.com/tin2tin/Pallaidium/assets/1322593/28098eb6-3a93-4bcb-bd6f-53b71faabd8d

Useful add-ons

GPT4BLENDER

Use GPT4ALL to generate image prompts or stories:

https://github.com/tin2tin/GPT4BLENDER

https://github.com/tin2tin/GPT4BLENDER/assets/1322593/ae0d54ae-f904-4eaa-a0c6-e8d0874f8249

The team behind Pallaidium does not endorse or take responsibility for third-party use.
The team behind Pallaidium requires verification or explicit permission for redistribution.
It is prohibited to use Pallaidium to generate content that is demeaning or harmful to people, their environment, culture, religion, etc.
It is prohibited to use Pallaidium for pornographic, violent, and bloody content generation.
It is prohibited to use Pallaidium for error and false information generation.
It is prohibited to use Pallaidium for commercial misuse or misrepresentation.

Restrictions on using the AI models:

Pallaidium does not include any genAI models(weights). If the user decides to use a model, it is downloaded from HuggingFace.
In general, the models can only be used for non-commercial purposes and are meant for research purposes.
Consult the individual models on HuggingFace to read up on their licenses and ex. if they can be used commercially.

Credits

The Diffusers lib makes the following weights accessible through the Pallaidium UI:

BiRefNet_HR

Captions, Subtitles & Prompt Enhancement:

PallAIdium