Replicate Integration

Replicate is a cloud platform for running ML models. SimpleTuner uses Replicate's Cog container system to run training jobs on cloud GPUs.

• Default model: simpletuner/advanced-trainer-h100
• Default GPU: H100
• Selectable profiles: h100, h100-x2, h100-x4, h100-x8, l40s, l40s-x2, l40s-x4, l40s-x8

Quick Start

1. Create a Replicate account and get an API token
2. Set the environment variable:

   export REPLICATE_API_TOKEN="r8_your_token_here"
   simpletuner server
   

3. Open the web UI → Cloud tab → click Validate to verify

Data Flow

Upload limit: Replicate's file upload API accepts archives up to 100 MiB. SimpleTuner blocks submissions when the packaged archive exceeds this limit.

Hardware & Costs {#costs}

The WebUI submit modal lets you choose a hardware profile per job. From the CLI:

simpletuner cloud jobs submit my-config --hardware-profile h100-x4

Typical Training Costs

Cost Protection

Set spending limits in Cloud tab → Settings:

• Enable "Cost Limit" with amount/period (daily/weekly/monthly)
• Choose action: Warn or Block

Results Delivery

Option 1: HuggingFace Hub (Recommended)

1. Set HF_TOKEN environment variable
2. Publishing tab → enable "Push to Hub"
3. Set hub_model_id (e.g., username/my-lora)

Option 2: Local Download via Webhook

1. Start a tunnel: ngrok http 8080 or cloudflared tunnel --url http://localhost:8080
2. Cloud tab → set Webhook URL to tunnel URL
3. Models download to ~/.simpletuner/cloud_outputs/

Option 3: External S3

Configure S3 publishing in the Publishing tab (AWS S3, MinIO, Backblaze B2, etc.).

Network Configuration {#network}

API Endpoints {#api-endpoints}

SimpleTuner connects to these Replicate endpoints:

Webhook Source IPs {#webhook-ips}

Replicate webhooks originate from Google Cloud's us-west1 region:

For the most current IP ranges:

• Check Replicate webhook documentation
• Or use Google's published IP ranges filtered for us-west1

curl -X PUT http://localhost:8080/api/cloud/providers/replicate \
  -H "Content-Type: application/json" \
  -d '{
    "webhook_allowed_ips": ["34.82.0.0/16", "35.185.0.0/16"]
  }'

Firewall Rules {#firewall}

Outbound (SimpleTuner → Replicate):

34.82.0.0/16
34.83.0.0/16
35.185.0.0/16 - 35.247.0.0/16  (all /16 blocks in this range)

Inbound (Replicate → Your Server):

Allow TCP from 34.82.0.0/16, 35.185.0.0/16 to your webhook port

Production Deployment

Webhook endpoint: POST /api/webhooks/replicate

Set your public URL (without path) in the Cloud tab. SimpleTuner appends the webhook path automatically.

upstream simpletuner {
    server 127.0.0.1:8080;
}

server {
    listen 443 ssl http2;
    server_name training.yourcompany.com;

    ssl_certificate     /etc/ssl/certs/training.crt;
    ssl_certificate_key /etc/ssl/private/training.key;

    location /api/webhooks/ {
        allow 34.82.0.0/16;
        allow 35.185.0.0/16;
        deny all;

        proxy_pass http://simpletuner;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    location / {
        allow 10.0.0.0/8;
        allow 172.16.0.0/12;
        allow 192.168.0.0/16;
        deny all;

        proxy_pass http://simpletuner;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

training.yourcompany.com {
    @webhook path /api/webhooks/*
    handle @webhook {
        reverse_proxy localhost:8080
    }

    @internal remote_ip 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
    handle @internal {
        reverse_proxy localhost:8080
    }

    respond "Forbidden" 403
}

services:
  simpletuner:
    image: simpletuner:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.simpletuner.rule=Host(`training.yourcompany.com`)"
      - "traefik.http.routers.simpletuner.tls=true"
      - "traefik.http.services.simpletuner.loadbalancer.server.port=8080"
      - "traefik.http.middlewares.replicate-ips.ipwhitelist.sourcerange=34.82.0.0/16,35.185.0.0/16"
      - "traefik.http.routers.webhook.rule=Host(`training.yourcompany.com`) && PathPrefix(`/api/webhooks`)"
      - "traefik.http.routers.webhook.middlewares=replicate-ips"
      - "traefik.http.routers.webhook.tls=true"

Webhook Events {#webhook-events}

Troubleshooting {#troubleshooting}

"REPLICATE_API_TOKEN not set"

• Export the variable: export REPLICATE_API_TOKEN="r8_..."
• Restart SimpleTuner after setting it

"Invalid token" or validation fails

• Token should start with r8_
• Generate a new token from Replicate dashboard
• Check for extra spaces or newlines

Job stuck in "queued"

• Replicate queues jobs when GPUs are busy
• Check Replicate status page

Training fails with OOM

• Reduce batch size
• Enable gradient checkpointing
• Use LoRA instead of full fine-tuning

Webhook not receiving events

• Verify tunnel is running and accessible
• Check webhook URL includes https://
• Test manually: curl -X POST https://your-url/api/webhooks/replicate -d '{}'

Connection issues through proxy

# Test proxy connectivity to Replicate
curl -x http://proxy:8080 https://api.replicate.com/v1/account

# Check environment
env | grep -i proxy

API Reference {#api-reference}

Links

• Replicate Documentation
• SimpleTuner on Replicate
• Replicate API Tokens
• Replicate Status Page