GapFill Web App

This directory contains the Vite / React / TypeScript web application for GapFill. The application source lives under src/.

Unless a command explicitly changes into ml/, run the commands in this README from the web/ directory.

Quick Start

Requires Node.js 22.12.0 or later. From the repository root:

cd web
npm ci
npm run dev

Open http://localhost:5173/. npm ci runs the postinstall script, which copies the ONNX Runtime WASM binary into public/ort-wasm/. The app loads the committed model from public/models/unet32.onnx.

The model was trained with a binary float32 [1,2,32,32] tensor whose channel 0 is Line Art and whose channel 1 is the target gap. For compatibility with the Web product's established pre-addon behavior, the browser runtime supplies alpha(Line) OR alpha(effective Guides) in channel 0. For a Guide gap, pixels belonging to that target gap are removed from the effective Guide mask. This is a Web compatibility policy, not a claim that the model was trained on Guides. The output is scored against patch-local painted regions split by Line Art and effective Guides. Predictions retain explicit learned/fallback provenance; fallback carries no learned confidence.

The web application has been tested with Google Chrome on Windows.

Directory Layout

The web app is expected to have a structure similar to:

web/
├── README.md
├── index.html
├── package.json
├── vite.config.ts
├── tsconfig.json
├── tsconfig.app.json
├── tsconfig.node.json
├── public/
│   ├── _headers
│   ├── models/
│   │   └── unet32.onnx
│   ├── ort-wasm/
│   │   └── ort-wasm-simd-threaded.wasm
│   └── preset-images/
│       └── ...
└── src/
    ├── main.tsx
    ├── App.tsx
    ├── App.css
    ├── index.css
    ├── components/
    ├── config/
    │   └── presets.ts
    ├── hooks/
    ├── types/
    ├── utils/
    │   └── GapFill/
    └── tests/
        └── GapFill/

Important directories:

• src/components/: Canvas, GapFill, reference, layer, toolbar, and related UI components.
• src/hooks/: History, initialization, shortcuts, layer handling, file I/O, session, and preset logic.
• src/config/presets.ts: Study task and preset definitions.
• src/utils/: Canvas utilities, gap detection, model preload, and ONNX inference.
• src/tests/: Node test runner tests.
• public/models/: ONNX model files loaded by the browser.
• public/ort-wasm/: Minimal ONNX Runtime Web WASM files copied by npm install.
• public/preset-images/: Preset images used by src/config/presets.ts.

Tests

Run tests from web/:

npm test

Export the ML Model for the Web App

The web app loads this ONNX model by default:

web/public/models/unet32.onnx

The ML training pipeline saves its best PyTorch checkpoint here by default:

ml/saved_models/gapfill/checkpoints/best_model.pth

After training, run the export command from ml/:

cd ../ml
python -m src.export_onnx
cd ../web

This command uses the ML Python environment, so install ml/requirements.txt before exporting if you have not already trained the model in that environment.

Default conversion:

input:  ml/saved_models/gapfill/checkpoints/best_model.pth
output: web/public/models/unet32.onnx
docs:  web/public/models/model_info.json
shape:  [1, 2, 32, 32]

To export a different checkpoint or output path:

cd ../ml
python -m src.export_onnx \
  --model_path saved_models/gapfill/checkpoints/final_model.pth \
  --output_path ../web/public/models/unet32.onnx
cd ../web

The default crop_size is 32, matching the web inference code. If you change the ML crop size, the browser inference patch size must be updated accordingly. The exporter writes a self-contained ONNX file and a sibling model_info.json. The JSON file documents the Line-only training contract and the distinct Web compatibility runtime policy, but is not loaded by the web app. Runtime code uses the names exposed by the ONNX session and validates the output's float32 shape and value count, matching the pre-addon Web behavior. Cross-host parity tests apply stricter finite/range checks independently. Repository validation pins the artifact SHA-256.

Required Runtime Assets

Before publishing, make sure these files exist:

web/public/models/unet32.onnx
web/public/models/model_info.json (documentation only)
web/public/preset-images/ (distributed presets; Task C is excluded)
web/public/ort-wasm/

public/ort-wasm/ is generated by npm install from web/. The ONNX model is generated from the ML checkpoint with python -m src.export_onnx. Preset images are stored in web/public/preset-images/ with the paths referenced by src/config/presets.ts; npm run check:preset-assets verifies the distributed assets and intentionally skips Task C.

Image Availability

All preset image materials were used with permission from ©IIS-P / Ponnomichi Production Committee.

The repository and hosted demo grant no license to extract or use these image materials independently. Without separate permission from the copyright holder, they may not be copied, modified, reused, or redistributed.

Task C uses full, unmodified source images, so its assets cannot be redistributed for copyright reasons and are excluded from Git under public/preset-images/C/. The official hosted build may include them under separate permission. A source build automatically hides Task C unless all separately authorized local files exist at the paths defined in src/config/presets.ts. Set GAPFILL_INCLUDE_TASK_C=false while building to exclude them explicitly. The MIT license for the source code does not grant rights to these image materials.

The PNG files contain embedded copyright metadata identifying ©IIS-P / Ponnomichi Production Committee. Please do not remove this metadata when using, copying, or processing the images. CI verifies the metadata of all distributed preset images with npm run check:image-metadata. See ASSET_LICENSE.md for the complete notice.

Preset Configuration

Study presets are defined in src/config/presets.ts. The preset order follows src/config/userstudy_flow.png and the study flow described in GapFill_CHI.pdf Section 5.1.

Each preset is composed from small profiles:

• presetAssets(...): selects line.png, guide.png, ref.png, and optional coloring images under public/preset-images/.
• TOOL_PROFILES: switches available tools, such as the full baseline toolset, Ours painting tools, or GapFill-only tools.
• GAP_FILL_PROFILES: switches GapFill availability, default on/off state, lock state, threshold control, and highlight color control.
• SESSION_PROFILES: switches time limits and whether Done auto-saves the Coloring layer.

To add or adjust a study condition, prefer reusing these profiles in IMAGE_PRESETS instead of duplicating every boolean flag.

Paths and Deployment

vite.config.ts uses base: './', so built assets can be served from a subpath such as GitHub Pages or Cloudflare Pages.

The app resolves models, preset-images, and ort-wasm relative to Vite's base URL. TypeScript code uses import.meta.env.BASE_URL; index.html uses Vite's %BASE_URL% replacement.

ONNX Runtime Web can use threaded WASM only when the page is cross-origin isolated. For that reason, the Vite dev server, Vite preview server, and public/_headers set:

Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Opener-Policy: same-origin

public/_headers is recognized by hosts such as Cloudflare Pages and Netlify.

Build

Run the production build from web/:

npm run build

When every separately authorized Task C asset is present locally, this build includes Task C for the official hosted distribution. To reproduce the public source build without Task C, run:

GAPFILL_INCLUDE_TASK_C=false npm run build

Preview the resulting static site locally:

npm run preview -- --host 0.0.0.0