Reference Images Guide

November 5, 2025 · View on GitHub

Complete guide to working with reference images for Sora video generation.

Overview

Reference images allow you to start video generation from a specific visual starting point instead of generating from text alone. Sora will animate from your reference image, maintaining its style, composition, and subject matter.

Supported Formats

JPEG (.jpg, .jpeg)
PNG (.png)
WEBP (.webp)

All formats are supported for both input and output. Prepared images are saved as PNG for best quality.

File Management

Setting Up Your Reference Directory

Set IMAGE_PATH environment variable to your reference images directory
Place images in this directory
The server only has access to files in this directory (security sandbox)

export IMAGE_PATH=~/sanzaru/images
mkdir -p $IMAGE_PATH

Discovering Available Images

Use list_reference_images to search your reference directory:

# List all images
all_images = list_reference_images()

# Find specific images
dogs = list_reference_images(pattern="dog*", file_type="png")

# Get recent uploads
recent = list_reference_images(sort_by="modified", order="desc", limit=10)

# Find large files
large = list_reference_images(sort_by="size", order="desc", limit=5)

Dimension Requirements

Reference images must match your target video dimensions exactly:

Video Size	Required Image Size
`720x1280$	720 \times 1280 \text{pixels}
$1280x720$	1280 \times 720 \text{pixels}
$1024x1792$	1024 \times 1792 \text{pixels} (\text{pro} \text{only})
$1792x1024$	1792 \times 1024 \text{pixels} (\text{pro} \text{only})

\text{If} \text{dimensions} \text{don}'\text{t} \text{match}, \text{use} $prepare_reference_image` to resize automatically.

Automatic Image Resizing

Basic Workflow

List available images
Prepare image to target dimensions
Create video with prepared image

# 1. Find your image
images = list_reference_images(pattern="sunset*")

# 2. Resize to match video dimensions
result = prepare_reference_image(
    "sunset.jpg",
    "1280x720",
    resize_mode="crop"
)

# 3. Generate video
video = create_video(
    prompt="The sun rises slowly, casting golden light across the landscape",
    size="1280x720",
    input_reference_filename="sunset_1280x720.png"
)

Resize Modes Explained

`crop$ (\text{Default} - \text{Recommended})

\text{Best} \text{for}: \text{Most} \text{use} \text{cases} \text{where} \text{you} \text{want} \text{to} \text{avoid} \text{distortion}

\text{Scales} \text{image} \text{to} \text{cover} \text{the} \text{target} \text{dimensions}
\text{Centers} \text{the} \text{image} \text{and} \text{crops} \text{excess}
\text{Preserves} \text{aspect} \text{ratio} (\text{no} \text{distortion})
\text{May} \text{lose} \text{some} \text{edges}

\text{Example}: 1920 \times 1080 → 1280 \times 720

\text{Image} \text{is} \text{scaled} \text{to} 1280 \times 720 \text{coverage}
\text{Excess} \text{on} \text{edges} \text{is} \text{cropped} \text{off}
\text{Result}: \text{No} \text{distortion}, \text{but} \text{some} \text{content} \text{may} \text{be} \text{cut}

$pad$ (\text{Letterbox})

\text{Best} \text{for}: \text{When} \text{you} \text{need} \text{to} \text{preserve} \text{the} \text{entire} \text{image}

\text{Scales} \text{image} \text{to} \text{fit} \text{inside} \text{the} \text{target} \text{dimensions}
\text{Adds} \text{black} \text{bars} (\text{letterbox}/\text{pillarbox}) \text{to} \text{fill} \text{gaps}
\text{Preserves} \text{aspect} \text{ratio} (\text{no} \text{distortion})
\text{Preserves} \text{full} \text{image} \text{content}

\text{Example}: 1920 \times 1080 → 1280 \times 720

\text{Image} \text{is} \text{scaled} \text{to} \text{fit} \text{within} 1280 \times 720
\text{Black} \text{bars} \text{added} \text{on} \text{sides}
\text{Result}: \text{Entire} \text{image} \text{visible}, \text{but} \text{with} \text{black} \text{bars}

$rescale$ (\text{Stretch})

\text{Best} \text{for}: \text{Rare} \text{cases} \text{where} \text{distortion} \text{is} \text{acceptable}

\text{Stretches}/\text{squashes} \text{to} \text{exact} \text{dimensions}
\text{Uses} \text{full} \text{canvas} (\text{no} \text{cropping} \text{or} \text{padding})
\text{May} \text{distort} \text{the} \text{image}
\text{No} \text{content} \text{loss}, \text{but} \text{aspect} \text{ratio} \text{changes}

\text{Example}: 1920 \times 1080 → 1280 \times 720

\text{Image} \text{is} \text{stretched} \text{horizontally} \text{and}/\text{or} \text{vertically}
\text{Result}: \text{No} \text{bars} \text{or} \text{cropping}, \text{but} \text{may} \text{look} \text{distorted}

\text{Choosing} \text{the} \text{Right} \text{Mode}

$``python

Portrait photo → landscape video

Use crop to avoid black bars

prepare_reference_image("portrait.jpg", "1280x720", resize_mode="crop")

Logo or graphic that must stay visible

Use pad to preserve everything

prepare_reference_image("logo.png", "1280x720", resize_mode="pad")

Abstract art where distortion is ok

Use rescale for full canvas usage

prepare_reference_image("abstract.jpg", "1280x720", resize_mode="rescale")


## Prompting with Reference Images

**CRITICAL:** When using reference images, your prompt should describe **motion and action only**, not what's already in the image.

### ❌ Bad Prompts (Re-describing the image)
```python
# Don't do this:
create_video(
    prompt="A pilot in an orange suit sitting in a cockpit with glowing instruments...",
    input_reference_filename="pilot.png"
)

The image already shows: character, setting, framing, style, lighting.

✅ Good Prompts (Describing the action)

# Do this instead:
create_video(
    prompt="The pilot glances up, takes a breath, then returns focus to the instruments.",
    input_reference_filename="pilot.png"
)

What to include in your prompt:

What happens next
Motion and camera movement
Actions and transformations
Timing and pacing

What NOT to include:

Descriptions of what's already visible
Style, lighting, or composition (already in the image)
Character or setting descriptions

See docs/sora2-prompting-guide.md for comprehensive prompting guidance.

Complete Example Workflows

Workflow 1: Using Existing Images

# 1. Search for available images
sunset_images = list_reference_images(pattern="sunset*", file_type="jpg")

# 2. Check if image needs resizing
# (if you know the dimensions don't match 1280x720)
result = prepare_reference_image(
    "sunset_original.jpg",
    "1280x720",
    resize_mode="crop"
)

# 3. Generate video with simple motion prompt
video = create_video(
    prompt="The sun rises slowly over the horizon as clouds drift by",
    model="sora-2",
    seconds="8",
    size="1280x720",
    input_reference_filename="sunset_original_1280x720.png"
)

# 4. Poll for completion
status = get_video_status(video.id)
# Keep checking until status.status == "completed"

# 5. Download when ready
download_video(video.id, filename="sunrise_animation.mp4")

Workflow 2: Generate Reference Image First

# 1. Generate a reference image
resp = create_image(
    prompt="futuristic pilot in mech cockpit, cinematic lighting",
    model="gpt-5"
)

# 2. Wait for completion
status = get_image_status(resp.id)

# 3. Download the image
download_image(resp.id, filename="pilot.png")

# 4. Resize for Sora (if needed)
prepare_reference_image("pilot.png", "1280x720", resize_mode="crop")

# 5. Animate it
video = create_video(
    prompt="The pilot looks up and smiles",
    size="1280x720",
    input_reference_filename="pilot_1280x720.png"
)

# 1. Generate base image
resp1 = create_image(prompt="a cyberpunk character")
get_image_status(resp1.id)  # wait
download_image(resp1.id, filename="character_v1.png")

# 2. Refine the image
resp2 = create_image(
    prompt="add more neon details and a cityscape background",
    previous_response_id=resp1.id
)
get_image_status(resp2.id)  # wait
download_image(resp2.id, filename="character_v2.png")

# 3. Prepare for video
prepare_reference_image("character_v2.png", "1280x720", resize_mode="crop")

# 4. Animate
video = create_video(
    prompt="Character turns and walks into the neon-lit street",
    size="1280x720",
    input_reference_filename="character_v2_1280x720.png"
)

Security & Best Practices

File Security

All images must be in IMAGE_PATH directory
Path traversal is blocked (../ not allowed)
Symlinks are rejected
Only specify filenames, never full paths

Performance Tips

Prepare images before peak generation time
Use crop mode for best quality/speed balance
PNG output ensures no quality loss during resizing
Original files are always preserved

Naming Conventions

The tool automatically creates descriptive filenames:

original.jpg → original_1280x720.png (after prepare)
photo.png → photo_1920x1080.png (after prepare)

You can also specify custom names:

prepare_reference_image(
    "sunset.jpg",
    "1280x720",
    output_filename="my_custom_name.png"
)

Troubleshooting

"File not found" Error

Check that file is in IMAGE_PATH directory
Use list_reference_images() to verify filename
Filenames are case-sensitive

"Invalid dimensions" Error

Target size must be one of: 720x1280, 1280x720, 1024x1792, 1792x1024
Use exact string format (e.g., "1280x720")

Video Doesn't Match Reference

Check that you used the correctly sized image ({name}_{width}x{height}.png)
Verify the prompt focuses on action, not re-describing the image
Try a simpler prompt describing only motion

Image Looks Distorted

Try different resize mode (crop usually works best)
Check original image aspect ratio
Consider using pad to preserve full image without distortion