Local Development Setup

June 17, 2026 ยท View on GitHub

This repository contains the API, processor, shared Python code, frontend, Supabase migrations, and the deadtrees development CLI.

Prerequisites

  • Docker Desktop or Docker Engine with docker compose
  • Python 3.12+
  • Node.js 20+ and npm
  • Supabase CLI
  • make, curl, and unzip

First-Time Bootstrap

These are the steps that currently produce a working local setup from a fresh clone.

1. Clone with submodules

git clone https://github.com/Deadwood-ai/deadtrees.git
cd deadtrees
git submodule update --init --recursive

2. Create the Python environment and install the CLI

python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -e ./deadtrees-cli[test]

This installs the deadtrees command used for local stack management and test execution.

3. Install frontend dependencies

npm --prefix frontend ci

4. Create local env files from the checked-in examples

cp .env.example .env
cp frontend/.env.local.example frontend/.env.dev.local

The example values are set up for local Supabase CLI defaults and the local Docker test stack. Keep manual SSH/VPN credentials, MCP bearer tokens, and other agent access notes out of .env and frontend env files. See docs/agents/environment-and-access.md.

5. Start local Supabase

supabase start

Useful local endpoints after supabase start:

  • API gateway: http://127.0.0.1:54321
  • Postgres: postgresql://postgres:postgres@127.0.0.1:54322/postgres
  • Studio: http://127.0.0.1:54323
  • Mailpit: http://127.0.0.1:54324

If you need a clean local database:

supabase db reset

Optional: isolated Supabase per worktree

For parallel local worktrees, generate a separate Supabase workdir, project id, and port band without changing the tracked supabase/config.toml:

bash scripts/setup-worktree.sh --skip-assets
scripts/qa/env.sh up
scripts/qa/env.sh reset
scripts/qa/validate-isolated-env.sh

This writes ignored runtime state under .local/supabase/<worktree-slug>/, starts the light Supabase profile, app API/nginx/Mailpit services, Vite, and seeds deterministic QA data. The validation command checks that the generated URLs are local-only, the services are reachable, QA fixtures are present, and Auth recovery mail reaches the worktree's Mailpit instance.

To point shell commands, Docker Compose, Vite, and Playwright local E2E tests at that isolated stack:

set -a
source "$(scripts/dev/isolated-supabase.sh env)"
set +a

docker-compose.test.yaml, npm --prefix frontend run dev:local, and npm --prefix frontend run test:e2e:local use the generated ports when these variables are present. The default isolated Supabase profile excludes Studio, Realtime, Storage, Mailpit, Edge Runtime, analytics/logging, and PgMeta; it keeps Postgres, Kong, PostgREST, and Auth for normal local app flows.

Useful commands:

scripts/qa/env.sh status
scripts/qa/validate-isolated-env.sh
scripts/qa/env.sh down

# Lower-level Supabase-only controls:
scripts/dev/isolated-supabase.sh status
scripts/dev/isolated-supabase.sh stop

This layer only proves isolated dev instances. Browser Use, built-in Browser, and multi-agent QA orchestration are a separate layer on top of the running isolated frontend/API/Supabase stack.

6. Download required local assets and test fixtures

make download-assets

This now downloads the fixtures required for the local API test suite:

  • assets/test_data/test-data.tif
  • assets/test_data/test-data-small.tif
  • assets/test_data/yanspain_crop_124_polygons.gpkg
  • assets/test_data/raw_drone_images/test_no_rtk_3_images.zip
  • assets/test_data/raw_drone_images/test_minimal_5_images.zip
  • data/assets/dte_maps/*.tif test clips
  • model and GADM assets

For the extra processor-only support data, run:

make download-processor-assets

This downloads:

  • assets/biom/terres_ecosystems.gpkg
  • assets/pheno/modispheno_aggregated_normalized_filled.zarr
  • assets/test_data/worldview_uint16_crop.tif

7. Start the local development stack

deadtrees dev start

This builds and starts the local test/dev services, including:

  • api-test
  • processor-test
  • nginx
  • mailpit

To rebuild from scratch:

deadtrees dev start --force-rebuild

To stop the stack:

deadtrees dev stop

8. Start the frontend

npm --prefix frontend run dev:local

What Should Work After Bootstrap

Local services

  • Frontend: http://127.0.0.1:5173
  • API via nginx: http://127.0.0.1:8080/api/v1/
  • API docs: http://127.0.0.1:8080/api/v1/docs
  • Downloads docs: http://127.0.0.1:8080/api/v1/download/docs
  • COGs: http://127.0.0.1:8080/cogs/v1/
  • Thumbnails: http://127.0.0.1:8080/thumbnails/v1/
  • Supabase Studio: http://127.0.0.1:54323

Tests

Prefer the deadtrees CLI for tests and debugging.

source venv/bin/activate

# Full API suite
deadtrees dev test api

# Single API file
deadtrees dev test api api/tests/routers/test_download.py

# Processor suite
deadtrees dev test processor

# First-time processor test bootstrap on a new machine
make setup-local-test-ssh
make download-processor-assets
deadtrees dev test processor

# Frontend tests
npm --prefix frontend test

Current known-good local result:

  • deadtrees dev test api passes end-to-end after the bootstrap steps above
  • npm --prefix frontend test passes
  • deadtrees dev test processor passes after make setup-local-test-ssh and make download-processor-assets
  • the remaining processor skips are the intentionally skipped comprehensive GeoTIFF standardization tests

Debugging tests

deadtrees dev debug api --test-path=api/tests/routers/test_download.py
deadtrees dev debug processor --test-path=processor/tests/test_processor.py

Optional: Extra ODM Test Data

For the basic local API suite, make download-assets is enough.

If you want the larger locally generated ODM ZIP fixtures as well:

./scripts/create_odm_test_data.sh

This is optional and mostly useful for extra processor/ODM experiments.

Processor test prerequisites

The processor integration suite uses SSH against the local nginx test container to mimic the storage server. Generate the local test-only keypair once before running processor tests:

make setup-local-test-ssh
make download-processor-assets

This writes an ignored keypair to .local/ssh/processing-to-storage and .local/ssh/processing-to-storage.pub. If you prefer a different key location, set absolute paths in LOCAL_TEST_SSH_PRIVATE_KEY_PATH and LOCAL_TEST_SSH_PUBLIC_KEY_PATH before running docker compose -f docker-compose.test.yaml ....

Processor-server validation workflow

This local laptop/worktree is good for API, shared, frontend, docs, and non-GPU checks. Processor tests that need NVIDIA runtime, model checkpoints, or full combined-model execution should run on the processing server dev checkout:

ssh processing-server
cd /home/jj1049/dev/deadtrees
git fetch origin <branch>
git checkout -B <branch> origin/<branch>
source venv/bin/activate
deadtrees dev test processor processor/tests/test_processor.py
deadtrees dev test processor processor/tests/test_process_deadwood_treecover_combined_v2.py::test_model_loads

If /home/jj1049/dev/deadtrees is dirty or not the current monorepo checkout, move it aside first instead of deleting it:

cd /home/jj1049/dev
ts=$(date -u +%Y%m%dT%H%M%SZ)
mv deadtrees "deadtrees.backup-$ts"
git clone https://github.com/Deadwood-ai/deadtrees.git deadtrees
cd deadtrees
git fetch origin <branch>
git checkout -B <branch> origin/<branch>

Copy ignored local test assets or .env from the backup only when needed. For new status columns or tables, apply the migration to the dev/test Supabase DB before running processor tests and reload the PostgREST schema cache. Never use this workflow against /home/jj1049/prod/deadtrees unless the user explicitly asks for a production operation.

Codex app worktrees

If you use the Codex desktop app, configure a project Local Environment so new Dead Trees worktrees bootstrap themselves automatically.

  • Setup script: bash scripts/setup-worktree.sh
  • Suggested actions:
    • source venv/bin/activate && deadtrees dev test api
    • source venv/bin/activate && deadtrees dev test processor
    • npm --prefix frontend test

The setup script auto-detects the primary checkout from git worktree list, so a Codex-created worktree will reuse the main checkout for shared assets, data, .local/ssh, local .codex project config, and ignored local docs/ops playbooks without hardcoding a machine-specific path.

Tracked Codex instructions live in root AGENTS.md, frontend/AGENTS.md, and docs/agents/. Local-only access details stay in .codex/*, docs/ops/*, and ignored env files.

Project structure

/assets      - Downloaded data and models
  /gadm        - GADM geographic data
  /models      - ML models for deadwood segmentation
/test_data   - Test GeoTIFF files

/api         - FastAPI application
  /src       - Source code
  /tests     - API tests

/processor   - Data processing service
  /src       - Source code
  /tests     - Processor tests

/shared      - Shared code between API and processor

API - Deployment

Additional requirements

So far I found the following packages missing on the Hetzner ubuntu image:

apt install -y make unzip

Setup user

create a user for everyone to log in (using root)

useradd dendro
usermod -aG docker dendro

Init git and download repo

Next upload SSH keys for developers to home/dendro/.ssh/authorized_keys Add Env variables to the key, to set the git user for each developer

command="export $GIT_AUTHOR_NAME='yourname' && export $GIT_AUTHOR_EMAIL='your-email';exec $SHELL -l" key

Next change the /home/dendro/.bashrc to configure git, add to the end:

if  [[ -n "$GIT_AUTHOR_NAME" && -n "$GIT_AUTHOR_EMAIL" ]]; then
        git config --global user.name "$GIT_AUTHOR_NAME"
        git config --global user.email "$GIT_AUTHOR_EMAIL"
fi

Now, you can download the repo, including the private repo.

# Clone the repository
git clone git@github.com:Deadwood-ai/deadtrees.git
cd deadtrees

# Initialize and update submodules
git submodule update --init --recursive

Create a .env file with required environment variables:

On the Storage server, only the necessary env is set

SUPABASE_URL=your_supabase_url
SUPABASE_KEY=your_supabase_key

Download required assets:

# Create assets directory and download test data, models, and GADM data
make

Build the repo

Optionally, you can alias the call of the correct docker compose file. Add to .bashrc

alias serv='docker compose -f docker-compose.api.yaml'

Certificate issueing & renewal

The certbot service can be used to issue a certificate. For that, the ACME challange has to be served by a temporary nginx:

server {
    listen 80;
    listen [::]:80;
    server_name default_server;
    server_tokens off;

    # add ACME challange for certbot
    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }
}

Then run the certbot service with all the mounts as configured in the docker-compose.api.yaml:

serv certbot certonly

Once successfull, you can start a cronjob to renew the certificate:

crontab -e

And add the following cronjob to run every Sunday night at 1:30. Currently we are not notified if this fails and thus needs to be monitored for now:

30 1  * * 0 docker compose -f /apps/deadtrees/docker-compose.api.yaml run --rm certbot renew