README.md

July 18, 2026 · View on GitHub

MicroDrama AI

Agentic AI Micro-Drama Video Generator

🚨 The problem with AI video today:

❌ Manual, fragmented workflow — Separate tools needed for scripting, image gen, and video gen
❌ No narrative structure — Raw T2V models don't understand story arcs or scene continuity
❌ Character inconsistency — Characters look different in every shot
❌ Visual-only — Missing scripts, narrative depth, and cinematic storytelling

💡 MicroDrama AI Solution:

🎬 Screenwriter, Storyboard Artist, Frame Generator, and Video Producer — all in one automated pipeline.

Type an idea. MicroDrama AI autonomously handles everything: story development, character design, shot-by-shot storyboarding, frame generation, and video clip production — then stitches it all into a complete cinematic micro-drama. Powered entirely by a single MuAPI key.

🎨 Explore 50+ more open-source AI apps →

AI-Youtube-Shorts-Generator — Auto-clip and crop your AI dramas into viral YouTube Shorts
Clip-Anything — Clip any moment from your AI drama with text prompts
AI-Faceless-Video-Generator — Generate faceless AI videos for your drama channel
awesome-ai-video-models — compare AI video models by API, price & speed

🎥 Demo Video

https://github.com/user-attachments/assets/f8b5d09c-8c58-40c1-b171-e3f42b0d71a8

Watch MicroDrama AI transform ideas into fully cinematic micro-drama videos using autonomous AI agents, character-consistent image generation, storyboarding, and MuAPI-powered video creation.

💡 Key Features

🎭 Idea to Video

From Spark to Screen

Transform a raw idea into a complete micro-drama. The AI develops the story, writes scene scripts, designs the storyboard, and generates the full video — zero manual steps.

📝 Script to Video

Bring Your Script to Life

Already have a scene script? Skip the writing phase and let MicroDrama AI handle storyboarding, character portraits, frame generation, and video production.

🎨 Character Consistency

Coherent Visual Identity

Characters are extracted with detailed static and dynamic features. Reference portraits are generated and used to keep characters visually consistent across every shot.

📡 Real-time Progress

Live Pipeline Visibility

Watch every stage of the pipeline in real time via SSE streaming — from story development through to the final concatenated video, with a live animated stage timeline.

🏗️ Architecture

                   ┌──────────────────────────────────────────────────────────┐
                   │                MicroDrama AI Server                      │
  Idea / Script ─► │                                                          │
                   │  ┌────────────────────────────────────────────────────┐  │
                   │  │              Screenwriter Agent                     │  │
                   │  │  develop_story()  → write_script_based_on_story()  │  │
                   │  └──────────────────────────┬─────────────────────────┘  │
                   │                             │ story + scene scripts       │
                   │  ┌──────────────────────────▼─────────────────────────┐  │
                   │  │           Character Extractor Agent                 │  │
                   │  │      extract_characters(script) → List[char]       │  │
                   │  └──────────────────────────┬─────────────────────────┘  │
                   │                             │ characters[]               │
                   │  ┌──────────────────────────▼─────────────────────────┐  │
                   │  │           Storyboard Artist Agent                   │  │
                   │  │  design_storyboard() → List[ShotBriefDescription]  │  │
                   │  └──────────────────────────┬─────────────────────────┘  │
                   │                             │ shots[]                    │
                   │  ┌──────────────────────────▼─────────────────────────┐  │
                   │  │                  MuAPI Tools                        │  │
                   │  │  ① Portrait Gen  →  flux-dev-image  (T2I)          │  │
                   │  │  ② Frame Gen     →  flux-kontext-dev-i2i  (I2I)    │  │
                   │  │  ③ Video Gen     →  kling-v2.1-standard-i2v  (I2V) │  │
                   │  └──────────────────────────┬─────────────────────────┘  │
                   │                             │ video clips                │
                   │  ┌──────────────────────────▼─────────────────────────┐  │
                   │  │                moviepy Concat                       │  │
                   │  └──────────────────────────┬─────────────────────────┘  │
                   └────────────────────────────-┼──────────────────────────--┘
                                                 │
                                        final_video.mp4

All LLM calls (story, characters, storyboard) go through MuAPI's /claude-sonnet-4-6 endpoint. Only one API key is required for the entire pipeline.

🔄 Pipeline Stages

Idea to Video — Full Pipeline

#	Stage	Agent / Tool	Output
1	Story Development	Screenwriter (MuAPI LLM)	Story outline in prose
2	Character Extraction	CharacterExtractor (MuAPI LLM)	Characters with static + dynamic visual features
3	Scene Scripting	Screenwriter (MuAPI LLM)	2–4 individual scene scripts
4	Character Portraits	MuAPI `flux-dev-image` (T2I)	Reference portrait per character — runs in parallel
5	Storyboard Design	StoryboardArtist (MuAPI LLM)	3–5 shots per scene: visual + motion + audio desc
6	Frame Generation	MuAPI `flux-kontext-dev-i2i` (I2I)	First frame per shot using character reference images
7	Video Generation	MuAPI `kling-v2.1-standard-i2v` (I2V)	5-second video clip per shot — runs in parallel
8	Concatenation	moviepy	All clips joined into `final_video.mp4`

Script to Video — Short Pipeline

Skips stages 1–3. Starts directly from your provided script.

#	Stage	Agent / Tool	Output
1	Character Extraction	CharacterExtractor (MuAPI LLM)	Characters with visual descriptions
2	Character Portraits	MuAPI `flux-dev-image` (T2I)	Reference portrait per character
3	Storyboard Design	StoryboardArtist (MuAPI LLM)	Shots from your script
4	Frame Generation	MuAPI `flux-kontext-dev-i2i` (I2I)	First frame per shot
5	Video Generation	MuAPI `kling-v2.1-standard-i2v` (I2V)	5-second video per shot
6	Concatenation	moviepy	Final scene video

🚀 Quick Start

Prerequisites

Python 3.10+
Node.js 18+
A MuAPI API key

1. Clone the repo

git clone https://github.com/Anil-matcha/Open-AI-Micro-Drama-Generator.git
cd Open-AI-Micro-Drama-Generator

2. Set up the server

cd server
python -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env
# Open .env and set MUAPI_KEY=your_key_here

3. Start the server

uvicorn api:app --reload --port 8000

API available at http://localhost:8000. Health check: GET /api/health.

4. Set up and start the client

cd client
npm install
npm run dev

Open http://localhost:3000.

5. Generate your first micro-drama

Enter an idea — e.g. "A detective chases a shadow through a neon-lit cyberpunk city"
Optionally add requirements — e.g. "Keep it under 3 scenes, suitable for all ages"
Pick a visual style: Cinematic, Realistic, Anime, Fantasy, or Documentary
Click Generate Video and watch the pipeline run in real time
Download your finished micro-drama when it completes

🛠️ Project Structure

Open-AI-Micro-Drama-Generator/
│
├── server/                           # FastAPI backend
│   ├── api.py                        # App entry point, endpoints, SSE job runner
│   ├── requirements.txt
│   ├── .env.example
│   │
│   ├── agents/                       # LLM-powered AI agents (all via MuAPI)
│   │   ├── screenwriter.py           # develop_story() + write_script_based_on_story()
│   │   ├── character_extractor.py    # extract_characters() → List[CharacterInScene]
│   │   └── storyboard_artist.py      # design_storyboard() → List[ShotBriefDescription]
│   │
│   ├── interfaces/                   # Pydantic data models
│   │   ├── character.py              # CharacterInScene (idx, name, static/dynamic features)
│   │   └── shot.py                   # ShotBriefDescription, ShotDescription
│   │
│   ├── tools/                        # MuAPI integration layer
│   │   ├── muapi_llm.py              # MuAPILLM — submit + poll LLM completions
│   │   ├── muapi_image_generator.py  # T2I (flux-dev) + I2I (flux-kontext) image gen
│   │   ├── muapi_video_generator.py  # I2V (kling-v2.1) video gen with polling
│   │   └── muapi_uploader.py         # Upload local images to MuAPI for URL references
│   │
│   ├── pipelines/                    # Orchestration logic
│   │   ├── idea2video.py             # Full end-to-end pipeline
│   │   └── script2video.py           # Scene-level pipeline (portraits → storyboard → video)
│   │
│   └── utils/
│       └── video.py                  # moviepy concatenation helper
│
└── client/                           # Next.js 14 frontend
    ├── next.config.js                # Rewrites /api/* and /outputs/* → FastAPI server
    ├── package.json
    ├── tailwind.config.js
    │
    ├── app/
    │   ├── layout.js                 # Root layout + metadata
    │   ├── globals.css               # Global styles + gradient utilities
    │   ├── page.js                   # Landing page: hero + generation form
    │   └── generate/[jobId]/
    │       └── page.js               # SSE consumer: pipeline progress + result viewer
    │
    └── components/
        ├── IdeaForm.js               # Idea/script input, style picker, mode toggle
        ├── PipelineProgress.js       # Animated 7-stage timeline + live scrolling log
        └── VideoResult.js            # Video player with download + open controls

🌐 API Reference

`POST /api/generate` — Start a generation job

Request body:

{
  "idea": "A lone astronaut discovers alien ruins on Europa",
  "user_requirement": "Keep it under 3 scenes, suitable for all ages",
  "style": "Cinematic",
  "mode": "idea2video"
}

Field	Type	Default	Description
`idea`	string	required	The creative idea or concept
`user_requirement`	string	`""`	Optional constraints (audience, length, tone)
`style`	string	`"Cinematic"`	Visual style: Cinematic, Realistic, Anime, Fantasy, Documentary
`mode`	string	`"idea2video"`	`"idea2video"` or `"script2video"`
`script`	string	`""`	Scene script (only used when `mode = "script2video"`)

Response:

{ "job_id": "550e8400-e29b-41d4-a716-446655440000" }

`GET /api/status/{job_id}` — Stream progress (SSE)

GET /api/status/550e8400-...
Accept: text/event-stream

SSE event stream. Reconnect-safe — replays all past events on reconnect.

Progress event:

{"type": "progress", "stage": "screenwriter", "message": "Developing story...", "progress": 10}

Completion event:

{"type": "complete", "video_url": "/outputs/job_id/final_video.mp4", "progress": 100}

Error event:

{"type": "error", "message": "MuAPI image job failed: quota exceeded", "progress": -1}

Pipeline stages in order: screenwriter → characters → storyboard → portraits → frames → video → concat

`GET /api/result/{job_id}` — Get job result

{
  "job_id": "550e8400-...",
  "status": "completed",
  "video_url": "/outputs/550e8400-.../final_video.mp4",
  "error": null
}

Status values: running | completed | failed

`GET /api/health` — Health check

{ "status": "ok", "service": "microdrama-api" }

⚙️ Configuration

Environment variables

Variable	Required	Description
`MUAPI_KEY`	✅	MuAPI API key — used for LLM, image, and video generation

One key. That's it.

MuAPI endpoints used internally

Purpose	Endpoint	Model
LLM completions	`POST /claude-sonnet-4-6`	claude-sonnet-4-6
Character portraits (T2I)	`POST /flux-dev-image`	FLUX.1-dev
Shot frames (I2I + character ref)	`POST /flux-kontext-dev-i2i`	FLUX.1-Kontext-dev
Video clips (I2V)	`POST /kling-v2.1-standard-i2v`	Kling v2.1 Standard
File upload	`POST /upload_file`	—
Job polling	`GET /predictions/{id}/result`	—

All jobs follow the same async pattern: submit → get request_id → poll every 3 s → return on "completed".

🖥️ UI Overview

Landing page (`/`)

Dark cinematic design with animated radial glow background
Feature pills: Screenwriter Agent · Storyboard Artist · Frame Generator · Video Generator
Idea textarea, optional requirements field
Style picker: Cinematic · Realistic · Anime · Fantasy · Documentary
Mode toggle: Idea to Video / Script to Video

Generation page (`/generate/[jobId]`)

7-stage animated pipeline timeline with live status indicators
Progress bar with percentage
Scrolling live log showing every message emitted by the pipeline
Video player with autoplay once complete event is received
Download button saves microdrama-{jobId}.mp4
SSE auto-reconnects if the connection drops mid-generation

📄 License

MIT — see LICENSE for details.