Video Black Frame Detection

Detect black frames in video for scene transitions, ad break insertion, and content segmentation. Uses FFmpeg for fast detection and OpenCV for detailed analysis. Optional LLM integration for intelligent ad placement.

Use Cases

• Ad Break Insertion: Find optimal points to insert advertisements
• Scene Detection: Identify scene transitions and chapter boundaries
• Content Segmentation: Split videos at natural break points
• Broadcast Compliance: Detect black segments for broadcast standards

Features

• Multiple Detection Methods: FFmpeg (fast), OpenCV (detailed), Combined (accurate)
• LLM Integration: Optional GPT-4o/Claude scene analysis for intelligent ad placement
• Export Formats: JSON and SRT output
• REST API: FastAPI server for integration
• CLI Tool: Command-line interface for batch processing

Related Projects

For comprehensive ad break detection, combine with:

• video-silence-detection: Detect silence segments using FFmpeg and Whisper

Using both together identifies points where both black frames AND silence occur - the most reliable ad break markers.

---

Quick Start

Prerequisites

• Python 3.10+
• FFmpeg installed and in PATH
• (Optional) OpenAI/Anthropic API key for LLM analysis

Installation

git clone https://github.com/hasanhalacli/video-black-frame-detection.git
cd video-black-frame-detection

# Using uv (recommended)
uv sync

# Or pip
pip install -e .

# With LLM support
pip install -e ".[llm]"

Basic Usage

# Detect black frames
blackframe-detect video.mp4

# Find ad break points
blackframe-detect ad-breaks video.mp4 -n 4

# Export to SRT for video editing
blackframe-detect video.mp4 -f srt -o blacks.srt

---

Detection Parameters

Minimum Duration

Duration	Use Case	Example
0.1s	Single black frames	Hard cuts, scene changes
0.3s	Quick fades	Transitions
0.5s	Short black segments	Scene breaks
1.0s	Deliberate black	Chapter markers
2.0s+	Extended black	Ad break markers, credits

Examples

# Detect quick cuts (single frames)
blackframe-detect video.mp4 -m 0.1

# Scene transitions
blackframe-detect video.mp4 -m 0.5

# Ad break markers (longer black)
blackframe-detect video.mp4 -m 2.0

---

Detection Methods

1. FFmpeg (Default) - Fast

Uses FFmpeg's blackdetect filter. Best for:

• Quick processing
• Batch operations
• Most use cases

blackframe-detect video.mp4 --method ffmpeg

2. OpenCV - Detailed

Frame-by-frame analysis with OpenCV. Best for:

• Detailed luminance analysis
• Thumbnail extraction
• Custom detection logic

blackframe-detect video.mp4 --method opencv

3. Combined - Most Accurate

FFmpeg for speed, OpenCV for verification. Best for:

• Maximum accuracy
• When false positives are costly

blackframe-detect video.mp4 --method combined

---

Export Formats

JSON Export

blackframe-detect video.mp4 -f json -o output.json

{
  "file_path": "video.mp4",
  "duration": 3600.0,
  "total_segments": 12,
  "segments": [
    {
      "start": 450.2,
      "end": 451.5,
      "duration": 1.3,
      "midpoint": 450.85,
      "start_formatted": "00:07:30.200"
    }
  ],
  "ad_break_suggestions": [...],
  "scene_boundaries": [...]
}

SRT Export

blackframe-detect video.mp4 -f srt -o blacks.srt

1
00:07:30,200 --> 00:07:31,500
[BLACK] Duration: 1.30s

2
00:15:20,000 --> 00:15:23,000
>>> AD BREAK 1 <<<

---

LLM-Powered Scene Analysis

For intelligent ad placement, use vision-capable LLMs to analyze scenes:

from blackframe_detection import BlackFrameDetector
from blackframe_detection.core.opencv_detector import OpenCVBlackDetector
from blackframe_detection.core.llm_analyzer import LLMSceneAnalyzer
from pathlib import Path

# Detect black frames
detector = BlackFrameDetector()
result = detector.detect("video.mp4", min_duration=0.5)

# Extract thumbnails around black segments
opencv = OpenCVBlackDetector()
thumbnails = opencv.extract_thumbnails(
    Path("video.mp4"),
    result.segments[:10],
    Path("thumbnails/"),
)

# Analyze with LLM
analyzer = LLMSceneAnalyzer(provider="openai")  # or "anthropic"
analyses = analyzer.analyze_segments(
    result.segments[:10],
    Path("thumbnails/"),
)

# Get best ad breaks based on content analysis
best_breaks = analyzer.get_best_ad_breaks(analyses, count=4)

for analysis in best_breaks:
    print(f"Time: {analysis.timestamp:.2f}s")
    print(f"  Before: {analysis.before_description}")
    print(f"  After: {analysis.after_description}")
    print(f"  Ad Score: {analysis.ad_suitability_score:.2f}")
    print(f"  Reason: {analysis.reasoning}")

The LLM analyzes frames before/after black segments to determine:

• Whether it's a true scene change
• Content context (action, dialog, credits)
• Suitability for ad insertion

---

Python API

from blackframe_detection import BlackFrameDetector, JSONExporter, SRTExporter

# Initialize
detector = BlackFrameDetector()

# Detect black frames
result = detector.detect(
    "video.mp4",
    method="ffmpeg",
    min_duration=0.5,
)

print(f"Found {len(result.segments)} black segments")

# Get ad break points
breaks = result.get_ad_break_points(count=4)
for pos in breaks:
    print(f"Ad break at {pos:.2f}s")

# Get scene boundaries
for boundary in result.get_scene_boundaries():
    print(f"Scene at {boundary:.2f}s")

# Export
JSONExporter().export(result, "output.json")
SRTExporter(mode="all").export(result, "output.srt")

---

Combining with Silence Detection

For the most reliable ad break detection:

from blackframe_detection import BlackFrameDetector

# Also install: pip install video-silence-detection
from silence_detection import SilenceDetector

# Detect both
black_detector = BlackFrameDetector()
silence_detector = SilenceDetector()

black_result = black_detector.detect("video.mp4", min_duration=0.5)
silence_result = silence_detector.detect("video.mp4", min_duration=2.0)

# Find overlapping segments (both black AND silent)
optimal_breaks = []
for black in black_result.segments:
    for silence in silence_result.segments:
        # Check overlap
        if black.start < silence.end and black.end > silence.start:
            overlap_start = max(black.start, silence.start)
            overlap_end = min(black.end, silence.end)
            optimal_breaks.append((overlap_start + overlap_end) / 2)

print(f"Found {len(optimal_breaks)} optimal ad break points")
for i, pos in enumerate(optimal_breaks, 1):
    print(f"  {i}. {pos:.2f}s")

---

REST API

Start Server

blackframe-detect serve --port 8001

Endpoints

Method	Endpoint	Description
GET	/health	Health check
POST	/detect	Detect black frames
POST	/detect/ad-breaks	Find ad break points

Example

curl -X POST http://localhost:8001/detect \
  -F "file=@video.mp4" \
  -F "min_duration=0.5"

---

Project Structure

video-black-frame-detection/
├── src/blackframe_detection/
│   ├── core/
│   │   ├── detector.py       # Main unified detector
│   │   ├── ffmpeg.py         # FFmpeg-based detection
│   │   ├── opencv_detector.py # OpenCV analysis
│   │   └── llm_analyzer.py   # LLM scene analysis
│   ├── exporters/
│   │   ├── json_exporter.py
│   │   └── srt_exporter.py
│   ├── api/
│   │   └── app.py
│   └── cli.py
├── configs/
├── tests/
└── README.md

---

Requirements

• Python 3.10+
• FFmpeg (required)
• OpenCV (installed automatically)
• OpenAI/Anthropic API key (optional, for LLM features)

License

MIT License - see LICENSE

Author

Hasan Halacli - Website · GitHub