Karaoke Music Video Generator

Generate karaoke-style music videos from an audio file and cover image with word-level lyric highlighting.

Features

• Word-level karaoke highlighting - Lyrics highlight word-by-word as they're sung (ASS format with \kf tags)
• LLM-powered phrase breaks - Uses GPT-4o-mini to determine natural line breaks in lyrics
• Ken Burns effect - Gentle 100-110% zoom oscillation on the cover image
• Waveform visualization - Audio waveform displayed at the bottom of the video
• VAAPI GPU acceleration - Hardware encoding with automatic software fallback
• Karaoke watermark - Optional overlay stamp for instrumental/karaoke versions

Requirements

• Python 3.10+
• ffmpeg with libass support
• OpenAI API key (for Whisper transcription and LLM phrase breaks)

Python Dependencies

pip install openai python-dotenv

Usage

Basic (with simple SRT subtitles)

python generate_karaoke_video.py audio.mp3 cover.jpg

Karaoke Mode (word-level highlighting)

python generate_karaoke_video.py audio.mp3 cover.jpg --karaoke

With Karaoke Watermark

python generate_karaoke_video.py audio.mp3 cover.jpg --karaoke --karaoke-stamp

No Subtitles

python generate_karaoke_video.py audio.mp3 cover.jpg --no-subtitles

All Options

positional arguments:
  audio                 Path to audio file (MP3, WAV, etc.)
  image                 Path to cover image (JPG, PNG)

options:
  -o, --output          Output video path (default: <audio_name>.mp4)
  -s, --srt             Existing SRT file (skip transcription)
  --ass                 Existing ASS file with karaoke timing (skip transcription)
  -r, --resolution      Output resolution (default: 1920x1080)
  --test SECONDS        Generate only first N seconds (for testing)
  --skip-transcription  Skip transcription (requires --srt or --ass)
  --karaoke             Enable karaoke mode with word-level highlighting
  --karaoke-stamp       Add KARAOKE watermark overlay to video
  --no-subtitles        Skip subtitle generation (video only)

Folder Structure

to-generate/          # Place input folders here (audio + image per folder)
generated/            # Output videos appear here
assets/               # Overlay images for karaoke stamp

Environment Variables

Create a .env file or export:

export OPENAI_API_KEY=sk-...

How It Works

1. Transcription - Audio is sent to OpenAI Whisper API with word-level timestamps
2. Phrase Breaking - GPT-4o-mini analyzes the lyrics and determines natural line breaks
3. ASS Generation - Creates an ASS subtitle file with \kf karaoke fill tags
4. Video Generation - ffmpeg combines:
   • Cover image with Ken Burns zoom effect
   • Audio waveform visualization
   • Burned-in karaoke subtitles
5. Overlay (optional) - Karaoke watermark stamp from assets/koverlay2.png

Assets

• assets/kareoke.png - Karaoke overlay option 1
• assets/koverlay2.png - Karaoke overlay option 2 (default)

Output

• 1920x1080 MP4 video
• AAC audio at 192kbps
• H.264 video (VAAPI or libx264)