Modal Cloud GPU Setup
August 31, 2026 · View on GitHub
Modal is the recommended cloud GPU provider for the toolkit's AI tools. It offers $30/month free compute on the Starter plan, fast cold starts, and scale-to-zero billing.
Fastest path: Run
/setupin Claude Code — it handles Modal installation, deployment, and.envconfiguration interactively. This doc is the reference for what/setupdoes under the hood, and for manual setup.
Create a Modal Account
- Go to modal.com and sign up
- Choose the Starter plan — $30/month free compute, just requires a payment method
- Typical toolkit usage is $1-2/month, well within the free allowance
- All apps scale to zero — no charges when idle
Install & Authenticate
uv sync --extra modal # Installs the Modal CLI into the toolkit's .venv
uv run modal setup # Opens browser to authenticate, saves token to ~/.modal.toml
uv run modal app list # Verify it works
Windows prerequisites
Two Windows-specific issues will otherwise make the commands in this guide fail in ways that look like toolkit bugs.
Set PYTHONIOENCODING=utf-8 before deploying. Modal's progress output contains
characters the default cp1252 console codec can't encode, so modal deploy aborts
mid-build with 'charmap' codec can't encode characters in position .... The build
itself is fine — only the printing fails. Make it permanent:
[Environment]::SetEnvironmentVariable('PYTHONIOENCODING','utf-8','User')
Reopen your terminal afterwards so new processes inherit it.
Make sure python3 isn't the Microsoft Store stub. A default Windows install puts
C:\Users\<you>\AppData\Local\Microsoft\WindowsApps ahead of your real Python on
PATH. That directory holds zero-byte App Execution Aliases which fail with
Permission denied — and since this toolkit's docs and commands all invoke
python3 tools/..., everything breaks. Check what resolves:
(Get-Command python3).Source
If it points into WindowsApps, either turn off the python.exe / python3.exe
aliases in Settings → Apps → App execution aliases, or move your Python directory
ahead of WindowsApps in your user PATH.
FFmpeg is not bundled on Windows and several tools need it. If winget install Gyan.FFmpeg fails with a corrupted-source error, download
ffmpeg-release-essentials.zip from https://www.gyan.dev/ffmpeg/builds/, extract it,
and add its bin\ directory to PATH.
Deploy Tools
Each AI tool has its own Modal app. Deploy only what you need, or deploy all of them — idle apps cost nothing.
# Speech generation (most commonly used)
uv run modal deploy docker/modal-qwen3-tts/app.py
# Image generation & editing
uv run modal deploy docker/modal-flux2/app.py
uv run modal deploy docker/modal-image-edit/app.py
uv run modal deploy docker/modal-upscale/app.py
# Music generation
uv run modal deploy docker/modal-music-gen/app.py
# Video processing
uv run modal deploy docker/modal-sadtalker/app.py
uv run modal deploy docker/modal-propainter/app.py
# Talking head, diffusion-based. Weights live in a Volume, so populate it FIRST
# (one-off, ~15GB, ~5 min) or the app deploys with nothing to load.
uv run modal run docker/modal-soulx/app.py::populate_weights
uv run modal deploy docker/modal-soulx/app.py
# Video generation (see the LTX-2 prerequisites note below)
uv run modal deploy docker/modal-ltx2/app.py
Deploy them one at a time. Modal rate-limits AppCreate; launching several
modal deploy calls in parallel makes most of them fail with
App create rate limit exceeded. Image builds happen server-side, so serial
deploys cost you nothing extra in compute — only wall-clock.
GPU tier gates some tools. Most apps request an A10G, which works on the free
tier. Two request A100-class GPUs and fail at deploy time with
Please add a payment method to use A100-40GB GPU functions until a card is on the
account: image-edit (A100) and ltx2 (A100-80GB). The image still gets built and
cached before that error, so re-deploying after adding a payment method is fast.
ltx2 additionally needs a Modal secret named huggingface-token for its gated
weights — it fails on the missing secret before it ever reaches the GPU check.
Each deploy prints an endpoint URL like:
https://yourname--video-toolkit-qwen3-tts-ttsengine-generate.modal.run
Save each URL to your .env file:
# Add to .env (replace with your actual URLs from deploy output)
MODAL_QWEN3_TTS_ENDPOINT_URL=https://yourname--video-toolkit-qwen3-tts-...modal.run
MODAL_FLUX2_ENDPOINT_URL=https://yourname--video-toolkit-flux2-...modal.run
MODAL_IMAGE_EDIT_ENDPOINT_URL=https://yourname--video-toolkit-image-edit-...modal.run
MODAL_UPSCALE_ENDPOINT_URL=https://yourname--video-toolkit-upscale-...modal.run
MODAL_MUSIC_GEN_ENDPOINT_URL=https://yourname--video-toolkit-music-gen-...modal.run
MODAL_SADTALKER_ENDPOINT_URL=https://yourname--video-toolkit-sadtalker-...modal.run
MODAL_SOULX_ENDPOINT_URL=https://yourname--video-toolkit-soulx-...modal.run
MODAL_DEWATERMARK_ENDPOINT_URL=https://yourname--video-toolkit-dewatermark-...modal.run
MODAL_LTX2_ENDPOINT_URL=https://yourname--video-toolkit-ltx2-...modal.run
When an endpoint label exceeds Modal's length limit, Modal truncates it and appends a
hash — dewatermark typically becomes ...-dewatermark-de-3e6418.modal.run. Copy the
URL Modal actually prints; don't reconstruct it from the pattern. Note that the deploy
log wraps long URLs across lines, so check for a (label truncated) marker and rejoin
the pieces.
Tip:
/setupautomates this — it runs each deploy, parses the URL, and writes it to.envfor you.
Cloudflare R2 (Recommended)
R2 is free file storage that bridges your local machine and cloud GPUs. Without it, tools fall back to free upload services (slower, less reliable).
R2 free tier: 10GB storage, 10 million operations/month, zero egress fees.
See the R2 section in /setup, or configure manually:
- Sign up at dash.cloudflare.com
- Go to R2 Object Storage → Create bucket (name it
video-toolkit) - Create an API token: R2 → Manage R2 API Tokens → Object Read & Write
- Add to
.env:R2_ACCOUNT_ID=your_account_id R2_ACCESS_KEY_ID=your_access_key_id R2_SECRET_ACCESS_KEY=your_secret_access_key R2_BUCKET_NAME=video-toolkit
Use the Tools
All cloud GPU tools accept --cloud modal:
# AI voiceover
uv run tools/qwen3_tts.py --text "Hello world" --speaker Ryan --output hello.mp3 --cloud modal
# AI image generation
uv run tools/flux2.py --prompt "A sunset over mountains" --output sunset.png --cloud modal
# AI image editing
uv run tools/image_edit.py --input photo.jpg --style cyberpunk --cloud modal
# AI upscaling
uv run tools/upscale.py --input photo.jpg --output photo_4x.png --cloud modal
# AI music generation (acemusic cloud API is now default — no Modal needed)
uv run tools/music_gen.py --preset corporate-bg --duration 60 --output bg.mp3
# Or use Modal: uv run tools/music_gen.py --preset corporate-bg --duration 60 --output bg.mp3 --cloud modal
# Talking head from portrait + audio
uv run tools/sadtalker.py --image portrait.png --audio voiceover.mp3 --output talking.mp4 --cloud modal
uv run tools/soulx.py --image portrait.png --audio voiceover.mp3 --output talking.mp4
# Watermark removal
uv run tools/dewatermark.py --input video.mp4 --region 1080,660,195,40 --output clean.mp4 --cloud modal
Tools & Costs
| Tool | Backend | Use Case | Est. Cost |
|---|---|---|---|
qwen3_tts | Qwen3-TTS | AI speech generation | ~$0.005-0.02 |
flux2 | FLUX.2 Klein | AI image generation | ~$0.01-0.03 |
image_edit | Qwen-Image-Edit | AI image editing, style transfer | ~$0.02-0.05 |
upscale | RealESRGAN | AI image upscaling (2x/4x) | ~$0.005-0.02 |
music_gen | ACE-Step 1.5 | AI music generation | Free (acemusic) / ~$0.02-0.10 (Modal) |
sadtalker | SadTalker | Talking head video | ~$0.05-0.30 |
soulx | SoulX-FlashHead 1.3B Pro | Talking head video, aspect-preserving | ~$0.0024 per second of output |
dewatermark | ProPainter | AI video inpainting | ~$0.05-0.50 |
All apps use A10G GPUs (24GB VRAM) except image_edit which uses A100 for its 25GB model.
Weight storage
Most apps bake their model weights into the image at build time. soulx is the
exception: it keeps its ~15GB in a Modal Volume, which is why it needs the one-off
populate_weights run above before its first deploy.
The split is measured, not stylistic. Cold start and generation speed are the same either way; what differs is rebuild time after a dependency change — measured at 1.8-8.2s on a volume against 79-385s baked, because any invalidated layer re-downloads everything below it. Apps that still change often earn a volume; settled ones don't need one.
Volumes are optional and free at this scale. Modal charges $0.09/GiB/month for volume
storage with 1 TiB/month included free, so soulx's 14.7GB costs nothing — it is
~2.6% of the free allowance. There is no bill either way, which is precisely why the choice
comes down to rebuild speed versus having one self-contained artifact rather than to cost.
Every app can be built either way. soulx defaults to a volume; the rest bake by
default. See docs/soulx.md for the full
comparison.
Cold Starts
First request after idle triggers a cold start while Modal loads the model:
| Tool | Cold Start | Warm Request |
|---|---|---|
qwen3_tts | ~60-90s | ~5-15s |
flux2 | ~25-30s | ~1-3s |
image_edit | ~5-8min | ~15-20s |
upscale | ~25-30s | ~3-5s |
music_gen | ~60-90s | ~10-30s |
sadtalker | ~45-60s | ~30-60s |
soulx | ~15s + ~600s first-call torch.compile | 6.4-7.9x realtime |
dewatermark | ~60-70s | varies by video length |
After 60 seconds of no requests, containers scale back to zero. No charges while idle.
Monitoring & Billing
# Check what's running (Tasks column should be 0 when idle)
uv run modal app list
# Check today's spend
uv run modal billing report --for today --json
# View container logs
uv run modal app logs video-toolkit-upscale
# Verify your setup
uv run tools/verify_setup.py
Architecture
Each tool has its own Modal app (docker/modal-*/app.py), deployed independently:
- One app per tool — independent scaling, GPU assignment, and lifecycle
- Web endpoints — HTTP POST via
@modal.fastapi_endpoint, nomodalpip dependency needed on the client - R2 file transfer — large results upload to Cloudflare R2 (if configured), otherwise base64
- Scale to zero —
scaledown_window=60means containers shut down after 1 minute idle
The client-side abstraction lives in tools/cloud_gpu.py, which routes call_cloud_endpoint() to either _call_runpod() (submit + poll) or _call_modal() (synchronous POST).
RunPod (Alternative)
RunPod is also supported as a fallback provider. Use --cloud runpod on any tool.
| Aspect | Modal | RunPod |
|---|---|---|
| Free tier | $30/mo compute | None (pay-as-you-go) |
| Setup | modal deploy | --setup flag per tool |
| Cold start | Faster (cached layers) | Slower (Docker pull) |
| Invocation | Synchronous POST | Async submit + poll |
| Auth | Token optional for web endpoints | RUNPOD_API_KEY required |
See runpod-setup.md for RunPod-specific instructions.