JPIP

OpenHTJ2K's JPIP pipeline implements ISO/IEC 15444-9 (JPIP, 3rd edition) for on-demand delivery of JPEG 2000 codestreams at the precinct level. Clients ask for a view-window (resolution frame size + region offset + region size); the server returns only the precincts that contribute to that region, in the JPP-stream wire format. Network transports: HTTP/1.1 and HTTP/3 over QUIC.

Three demos ship with the library:

Demo	Binary / Page	Purpose
Mouse-driven foveation	open_htj2k_jpip_demo (native GLFW), jpip_demo.html (browser)	Three concentric cones around the cursor (fovea / parafovea / periphery) at full / reduced / coarse resolutions
Gigapixel pan + zoom viewer	jpip_viewer.html (browser)	Viewport-region decode for images larger than GPU texture limits
Foveation vs full-image benchmark	open_htj2k_jpip_benchmark	Bandwidth reduction + decode speedup across an NxN gaze grid

All three share the same open_htj2k_jpip_server and the core source/core/jpip/ library.

Build prerequisites

The three native binaries have different build-flag requirements:

Binary	Build flag	Native deps
open_htj2k_jpip_server	none (always built)	None beyond the core library
open_htj2k_jpip_benchmark	none (always built)	None beyond the core library
open_htj2k_jpip_demo	-DOPENHTJ2K_RTP=ON (shares the GLFW renderer with open_htj2k_rtp_recv)	GLFW 3.x; OpenGL 3.3 on Linux/Windows or Metal on macOS — see building.md

For HTTP/3 over QUIC on either the server or the native demo, add -DOPENHTJ2K_QUIC=ON. Required libraries (MsQuic + nghttp3) and per-platform install commands are documented in building.md → JPIP HTTP/3 prerequisites. The HTTP/1.1 transport is always available.

Quick start

Build the server, benchmark, and native demo (the demo needs the extra -DOPENHTJ2K_RTP=ON flag because it shares its renderer with open_htj2k_rtp_recv):

cmake -B build -DCMAKE_BUILD_TYPE=Release -DOPENHTJ2K_RTP=ON
cmake --build build -j

If you only need the server and benchmark (e.g. for CI or headless deployment), drop -DOPENHTJ2K_RTP=ON and the cmake step has no window-system or GPU dependencies.

Serve a codestream and drive it from the native demo:

# Terminal 1 — stateless HTTP/1.1 server
./build/bin/open_htj2k_jpip_server input.j2c --port 8080

# Terminal 2 — mouse-driven foveation demo, fetches from the server
./build/bin/open_htj2k_jpip_demo --server localhost:8080

Or drive the in-process JPP round-trip directly (no server needed, still exercises every byte of the JPP-stream wire format):

./build/bin/open_htj2k_jpip_demo input.j2c

Browser demos are deployed at https://htj2k-demo.pages.dev/ and run in any current Chromium- or Firefox-based browser; point them at your own server with the ?server= URL parameter or the Server field in the top bar. Building the browser demos locally requires Emscripten — see building.md → Building for WebAssembly.

Server — open_htj2k_jpip_server

HTTP/1.1 (optionally HTTP/3) server. Loads one .j2c codestream, builds the precinct index + packet locator once, then serves view-window requests — statelessly by default (the client's model field carries its cache state), or within §B.2 sessions when the client establishes a channel with cnew=http (the server then keeps a per-channel cache model; see cnew/cid below). The HTTP/3 path is stateless only.

open_htj2k_jpip_server <input.j2c>
    [--port N=8080]
    [--h3 --cert server.cert --key server.key]   # HTTP/3 (requires -DOPENHTJ2K_QUIC=ON)
    [--chunked]                                  # opt in to Transfer-Encoding: chunked
    [--no-chunked]                               # accepted for back-compat (now the default)

Warning — --chunked is for the browser/WASM streaming demos only. Some interactive reference JPIP clients (native desktop viewers) cannot parse chunked responses and fail with "connection closed unexpectedly"; serve those clients with the default Content-Length mode. See Progressive (chunked) delivery.

Query grammar (§C.4):

GET /jpip?fsiz=<fx>,<fy>&roff=<ox>,<oy>&rsiz=<sx>,<sy>&type=jpp-stream
          [&len=<N>][&model=...][&cnew=http]

• fsiz — target resolution frame size; the server picks the smallest discard level whose decoded size is ≥ fsiz.
• roff, rsiz — region within the resolution frame (default: full frame).
• type=jpp-stream — JPP-stream response (the only wire format this server speaks).
• len=<N> — §C.6.1 Maximum Response Length, in bytes. The server fills the response up to N bytes (EOR does not count) and terminates with EOR reason=4 (ByteLimit) when it stops early, or reason=2 (WindowDone) when the entire view-window fit. Delivery is resumable at byte granularity: a data-bin larger than the remaining budget contributes a prefix now and continues from that byte offset on the next request (tracked per-session, or via the model field's :bytes qualifier in stateless mode), so byte-limited clients always make forward progress. len=0 is legal and yields an EOR-only response.
• model — §C.9 cache-model advertisement; see cache-model field.
• cnew=<transports> — §C.3.3 session/channel establishment. When the comma-separated list contains http (the only transport this server grants), the reply carries JPIP-cnew: cid=<token>,path=jpip,transport=http and the server creates a session: a per-channel cache model remembers every data-bin (and partial-bin byte offset) delivered on that cid, so follow-up requests — which session clients send without a model field — never receive the same bytes twice (§B.2). Once a window completes, repeating the request returns an empty body with EOR reason=2, as Table D.2 requires. If no offered transport is supported (e.g. cnew=http-tcp alone), no JPIP-cnew header is returned and the request is served statelessly per §C.3.3 — never a transport the client did not offer (Table D.1).
• cid=<token> — request within an existing channel. Unknown or expired cids get HTTP 404 so the client re-establishes rather than looping on stateless duplicates. Channels are evicted LRU beyond 64 concurrent sessions.
• cclose=* (or a cid list) — §C.3.4 channel close, honoured after the response completes.
• qid=<N> — §C.3.5 request ID; echoed back verbatim in a JPIP-qid response header (§D.2.4). Session responses also carry Cache-Control: no-cache.

HTTP POST is also accepted (§C.1): the query string moves into the request body when GET's URL-length limit would otherwise truncate large cache-model advertisements. The server caps the body at 16 MB.

Responses are complete JPP-streams. Per §A.3.6.1, metadata-bin 0 is emitted first — even empty, as a session-binding signal — followed by main-header (§A.3.6), one tile-header data-bin per tile whose index appears in the view-window result (§A.3.3), every precinct data-bin selected by the view-window resolver, and an EOR. Large precinct data-bins are chunked into 987-byte messages (is_last=0 on all but the final chunk). Precincts are delivered in ascending in-class-id order so clients that iterate their cache by bin-id see no gaps. CORS preflight (OPTIONS) is handled for cross-origin browser access.

Progressive (chunked) delivery

The server supports Transfer-Encoding: chunked progressive delivery: each completed JPP-stream message is flushed to the socket as soon as the server produces it, so chunked-aware clients can start decoding while later precincts are still being built. On a 24 MB full-canvas response, loopback time-to-first-byte drops from ~7.4 ms (Content-Length path) to ~0.44 ms (~17×); chunked and Content-Length emit byte-identical bodies, so the mode swap has no effect on correctness.

Default is Content-Length. The server ships with Transfer-Encoding: chunked off because some interactive reference JPIP clients do not implement chunked transfer parsing and report "connection closed unexpectedly" when the Content-Length header they expect is absent — a session that otherwise negotiates perfectly (JPIP-cnew granted, image data flowing) fails at the HTTP layer with no JPIP-level error to point at the cause. Use the default mode for native desktop viewers; reserve --chunked for the browser demos and JpipClient, which accept either format. The client-side chunked refactor that enabled the TTFB win stays useful wherever the server is started with --chunked. Opt in with:

open_htj2k_jpip_server <input.j2c> --chunked

The §C.6.1 len= cap works identically in chunked mode — data-bins are emitted through budget-aware byte windows, so each wire chunk is already guaranteed to fit under the cap when it is flushed, and a bin interrupted by the cap resumes from its recorded byte offset on the next request. HTTP/3 continues to use the buffered data-reader path; progressive DATA-frame delivery over H3 would require an nghttp3 data-reader refactor and is tracked as a follow-up.

Native demo — open_htj2k_jpip_demo

Mouse-driven foveation. Three concentric cones follow the cursor:

Cone	fsiz ratio	Default radius	What it delivers
Fovea	1.0 (full resolution)	canvas_w / 15	Sharp detail around the cursor
Parafovea	0.5 (half res; configurable)	canvas_w / 8	Mid-resolution context
Periphery	0.125 (1/8 res; configurable)	whole image	Low-res anchor

open_htj2k_jpip_demo [<input.j2c>]
    [--fovea-radius N]            # canvas px; default = canvas_w / 15
    [--parafovea-radius N]        # canvas px; default = canvas_w / 8
    [--parafovea-ratio F=0.5]     # fsiz ratio
    [--periphery-ratio F=0.125]   # fsiz ratio
    [--window-size WxH=1920x1080] # decouple window from canvas
    [--reduce N=0]                # discard DWT levels
    [--server host:port]          # HTTP/1.1 server mode
    [--server-h3 host:port]       # HTTP/3 server mode (requires -DOPENHTJ2K_QUIC=ON)
    [--use-filter]                # direct precinct filter (skip JPP round-trip)
    [--decode-on-move-only] [--no-vsync]

Three modes are always available:

• In-process JPP round-trip (default, needs <input.j2c>): per frame, emit JPP messages → parse them back → reassemble a sparse codestream → decode. Exercises the full wire format locally; useful for benchmarking without networking.
• HTTP/1.1 client (--server host:port): three concurrent JpipClient::fetch_streaming calls per frame (one per cone), pipelined on their own TCP connections. The client feeds each HTTP chunk into the StreamingJppParser as it arrives, so the DataBinSet fills up while later chunks are still in flight rather than after a drain-to-EOF buffer pass.
• HTTP/3 client (--server-h3 host:port): three requests multiplexed on one QUIC connection via H3Client::fetch_multi.

--use-filter bypasses the JPP round-trip and installs the precinct filter directly on the decoder — an A/B comparison path that still works against local files. The window size can exceed the canvas (decouples the GPU texture scaler from the decoded resolution) or be smaller for cheap previews; Metal's 16384-pixel texture cap is honoured so 21600 × 10800 NASA Blue Marble-sized canvases run cleanly on Apple silicon.

Benchmark — open_htj2k_jpip_benchmark

Measures bandwidth and decode time for foveated vs full-image delivery across a gaze grid. No GUI or server needed.

open_htj2k_jpip_benchmark <input.j2c>
    [--gaze-grid NxN=5]
    [--reduce N=0]
    [--csv output.csv]
``$

\text{Example} \text{output} \text{on} \text{the} 1920  \times  1920 $land_shallow_topo_1920_fov.j2c`
reference asset:

gaze_x gaze_y │ precincts bytes decode │ bw_% dec_% ────────────────┼────────────────────────────┼───────────── 0 0 │ 171 89305 15.5ms │ 25.6 67.8 ... AVERAGE │ 175 │ 21.0 69.9

Bandwidth reduction: 79.0% | Decode speedup: 1.4x

## Browser demos

### Foveation — `jpip_demo.html`

Mouse-over a canvas; three concurrent `fetch()` calls per gaze move
return fovea + parafovea + periphery JPP-streams, merged into a
WASM-side `DataBinSet`, reassembled into a sparse codestream, decoded,
and painted via WebGL2 (with bilinear filtering). Uses the
multi-threaded WASM variant (pthreads + SIMD) when the page is
cross-origin-isolated, single-threaded SIMD otherwise.

Each per-cone response is drained via `response.body.getReader()` and
fed into the WASM `jpip_feed_stream*` API as the browser yields bytes,
so when the server is launched with `--chunked` the progressive
delivery round-trips all the way into the decoder without buffering
the full body on the JS side first.  Against the default
Content-Length server the same code path still works — the browser
just yields larger chunks.

Per-frame cache semantics: the JS side calls `jpip_reset_cache` each
frame so the previous gaze's high-res precincts decay — without it,
the periphery would freeze under the last foveal hit. Headers
(main-header, tile-headers, metadata-bin 0) are preserved across the
reset, so `&model=Hm,Ht*,M0` is advertised on every request and the
server stops re-sending tens of KB of headers per frame.

URL parameters:

| Param | Meaning |
|---|---|
| `server=host[:port]` | JPIP server (matches the top-bar field) |
| `reduce=N` | discard N DWT levels |
| `para_ratio=F` | parafovea fsiz ratio |
| `peri_ratio=F` | periphery fsiz ratio |
| `variant={st,mt}` | force single- or multi-threaded WASM |

### Gigapixel viewer — `jpip_viewer.html`

Pan + zoom for images larger than the GPU texture limit (Metal:
16384 pixels on Apple silicon). Only the precincts that fall inside
the current viewport are fetched and decoded; the decoder's
viewport-region API (`jpip_end_frame_region`) runs the IDWT only on
the rows within the visible rectangle and a column-limited horizontal
1D IDWT skips samples outside the visible columns.

Mouse drag = pan. `Ctrl`-wheel = zoom. Trackpad: two-finger scroll =
pan, pinch = zoom. Keyboard: arrows = pan, `+/-` = zoom,
`Home` = reset. Reduce level auto-selects from zoom:
`reduce = ceil(log2(1/zoom)) − 1` clamped to `[0, 5]`.

URL parameters:

| Param | Meaning |
|---|---|
| `server=host[:port]` | JPIP server |
| `debounce=N` | trailing-debounce window in ms (default 60; `0` disables) |
| `debug` | per-frame timing console dump |
| `variant={st,mt}` | force single- or multi-threaded WASM |
| `maxSize=WxH` | cap the WebGL render target to `WxH` (default `1920x1080`). Bounds per-frame precinct fetch + decode work on 4K / ultrawide displays. Pass `maxSize=window` to disable the cap and render at full window resolution. |
| `fit={stretch,contain}` | `stretch` (default) scales the render target up to fill the window with GPU-side `GL_LINEAR`; `contain` shows the canvas at native pixel scale centered in the window — like the foveation demo — with the `maxSize` cap defining the centered rectangle. |
| `precinctCacheMB=N` | LRU precinct cache budget in megabytes (v0.18.0); default `64`, pass `0` to disable. Received precincts are tracked in `&model=` so the server skips redelivery on subsequent pans (70–95% byte reduction typical). |
| `prefetchMargin=N` | Adjacent-viewport halo prefetch (v0.18.1) — fetches precincts up to `N` canvas pixels outside the visible viewport so short follow-up pans hit in the LRU cache. Default `128`, pass `0` to disable. Cancelled on the next user interaction so continuous panning never pays bandwidth for it. |
| `prefetchDelayMs=N` | Idle delay in ms before the halo prefetch fires. Default `150`. |
| `midPaintMs=N` | Mid-decode paint (v0.18.2) — on slow fetches, render the partial DataBinSet after `N` ms so the user sees progress before the full response arrives. Default `200`; `0` disables. Fast fetches (LAN / cached) never trigger it. |

Pan events are debounced + coalesced: during an in-flight fetch, new
events flip a "pending" slot rather than queue a second request, so
the final viewport the user aimed at is always what lands on screen.
Fetch responses are streamed into the WASM `jpip_feed_stream*` API
per chunk.  Against a `--chunked` server this matches the wire-level
chunked delivery; against the default Content-Length server the JS
side still avoids the full-buffer `arrayBuffer()` round-trip, it just
sees fewer/larger chunks from the browser.

## Core architecture

source/core/jpip/ precinct_index.{hpp,cpp} — (t, c, r, p_rc) → JPIP sequence number s, identifier I view_window.{hpp,cpp} — §C.4 view-window → precinct set vbas.{hpp,cpp} — VBAS codec (§A.2.1) jpp_message.{hpp,cpp} — JPP message headers (§A.2, Tables A.1, A.2) data_bin_emitter.{hpp,cpp} — header + precinct data-bin emission packet_locator.{hpp,cpp} — per-precinct byte ranges in the codestream codestream_walker.{hpp,cpp} — one-time codestream layout pass jpp_parser.{hpp,cpp} — wire stream → DataBinSet (one-shot parse_jpp_stream + resumable StreamingJppParser for chunked clients; they share a single try_decode_one_message helper) codestream_assembler.{hpp,cpp} — DataBinSet → sparse codestream cache_model.{hpp,cpp} — §C.9 client cache model jpip_request.{hpp,cpp} — query parser jpip_response.{hpp,cpp} — HTTP framer; chunked + Content-Length writers + parsers jpip_client.{hpp,cpp} — HTTP/1.1 client with incremental chunked-transfer state machine (fetch_streaming + progress callback) tcp_socket.{hpp,cpp} — cross-platform TCP wrapper h3_server.{hpp,cpp} — MsQuic + nghttp3 HTTP/3 server h3_client.{hpp,cpp} — MsQuic + nghttp3 HTTP/3 client

### Data-bin classes

- 0 — precinct (JPP-stream)
- 1 — extended precinct (JPP-stream, has Aux)
- 2 — tile header
- 4 — tile (JPT-stream only; not emitted by this server)
- 5 — extended tile (JPT-stream only, has Aux; not emitted by this server)
- 6 — main header
- 8 — metadata

The End-of-Response (EOR) message is deliberately not a class. Per
§D.3 it sits outside Annex A and uses a special 0x00 identifier byte
rather than a Bin-ID VBAS; see the dedicated EOR handling below.

### JPIP sequence number / identifier

Per §A.3.2.1:

s = Σ_{r' < r} npw[r'] · nph[r'] + p_rc I = t + (c + s · num_components) · num_tiles

`I` is the in-class identifier used in every precinct data-bin
message header. `CodestreamIndex::I(t, c, r, p_rc)` computes it;
the client reassembler decomposes `I` back to `(t, c, r, p_rc)` by
inverting the formula and linear-scanning `s_offset[]`.

### Zero-skip IDWT

Absent precincts appear in the sparse codestream as one-byte empty
packet headers (`0x00`). Rows that entirely derive from absent
precincts carry zero subband data, and `idwt_2d_state::adv_step()`
short-circuits the lifting pass when both its neighbour rows are
zero. The cost drops to ~10 % of a populated row (the counter
increments and zero checks stay).

### Viewport-region decode

`jpip_end_frame_region(handle, rgb, w, h, x, y, rw, rh)`:

1. **Precinct filter** — the server already sends only the precincts
   that overlap the region; absent rows naturally zero-skip.
2. **Row limit** — `set_row_limit(ry1)` tells the decoder to stop
   iterating after the last row of the region (`ry1 = ceil((y+rh) /
   2^reduce)`). Rows before `ry0` still iterate but zero-skip.
3. **Column range** — `set_col_range(rx0, rx1)` makes the horizontal
   1D IDWT process only the columns inside the region. Samples
   outside the range aren't stored or lifted.

Result: a centred viewport on a 42K × 10K image decodes in ~400 ms
in WASM at `reduce=0` instead of ~3000 ms for the full canvas.

### Cache-model field (§C.9)

`CacheModel` tracks the data-bins a client holds — completely, or up
to a byte offset. Its `format()` method emits the §C.9 model field
body used as `&model=` — for example `Hm,Ht0,Ht1-5,M0,P7:987`. Range
compression (`Ht1-5` instead of `Ht1,Ht2,Ht3,Ht4,Ht5`) keeps the
query short on multi-tile images; partial holdings carry the §C.9.2
`:bytes` qualifier and are never range-compressed.

| Class | Prefix | Example |
|---|---|---|
| Main header (6) | `Hm` | `Hm` (id always 0) |
| Tile header (2) | `Ht` | `Ht0`, `Ht1-5` |
| Precinct (0) | `P` | `P7`, `P7:987` (first 987 bytes) |
| Metadata (8) | `M` | `M0` |

`parse()`/`apply()` accept the legacy `Hp` precinct prefix, the
`:bytes` partial qualifier (the server resumes delivery from that
offset), and `-`-prefixed subtractive statements (`-P5` — the client
discarded the bin; the server forgets it was sent). The same model
statements work in stateless requests and as session-mode updates.

The browser demos track only headers — precincts are intentionally
excluded so the foveation demo's periphery decays when the gaze
moves.

### WASM C API

Defined in `web/src/jpip_wrapper.cpp`, exported via
`-sEXPORTED_FUNCTIONS` in `web/CMakeLists.txt`:

| Function | Purpose |
|---|---|
| `jpip_create_context(jpp, len)` | parse initial JPP-stream, build `CodestreamIndex` |
| `jpip_get_canvas_{width,height}` | canvas dimensions |
| `jpip_get_num_components`, `jpip_get_total_precincts` | asset metadata |
| `jpip_set_reduce(n)` | discard N DWT levels |
| `jpip_get_response_buffer(ctx, min_size)` | grow-only staging pointer; avoid per-frame `_malloc` |
| `jpip_add_response(ctx, buf, len)` | one-shot: parse a complete JPP-stream buffer and merge into the `DataBinSet` |
| `jpip_feed_stream_begin(ctx)` | progressive path: reset the per-context streaming parser before the first chunk of a new response (v0.17.0) |
| `jpip_feed_stream(ctx, buf, len)` | feed the next HTTP chunk of a chunked response; incomplete messages are buffered internally until the next call supplies the rest |
| `jpip_feed_stream_end(ctx)` | finalize the response; returns 0 if the stream ended at a clean JPP message boundary, nonzero if a partial message was still pending |
| `jpip_track_precincts_in_cache(ctx, enabled)` | opt-in LRU precinct cache (v0.18.0) — when on, received precincts are tracked in the client cache model so the server skips redelivery on subsequent pans |
| `jpip_set_precinct_cache_budget(ctx, lo, hi)` | LRU cache budget in bytes (64-bit value split lo/hi for Emscripten i32 calling convention); default 64 MB |
| `jpip_get_precinct_cache_count(ctx)` | diagnostic: number of precinct bins currently in the LRU |
| `jpip_get_cache_model(ctx)` | C-string for `&model=` advertisement |
| `jpip_reset_cache(ctx)` | soft reset — drops precincts, keeps headers; also clears the LRU when precinct tracking is on |
| `jpip_end_frame(ctx, rgb, w, h)` | full-canvas decode |
| `jpip_end_frame_region(ctx, rgb, w, h, x, y, rw, rh)` | viewport-region decode |
| `jpip_destroy_context(ctx)` | release |

The browser demos use the `jpip_feed_stream*` trio (not
`jpip_add_response`) so they can feed HTTP chunks into the decoder
as `response.body.getReader()` yields them; `jpip_add_response` is
preserved for one-shot callers that have the full body in hand.

## Deployment

For production-scale hosting (Docker + Cloudflare Tunnel + TLS
certificates for HTTP/3), see [`deploy/README.md`](../deploy/README.md).

## Known limitations

- **Progression order**: the core reassembler patches the COD marker
  to LRCP. When using `--server` mode against natively-LRCP or
  natively-RLCP codestreams, the demo falls back to `--use-filter`.
- **Layers**: v1 supports single-layer streams. Multi-layer bins are
  re-emitted per layer as-is; per-layer byte offsets within a bin
  are not parsed.
- **SOP / EPH markers**: refused by the reassembler (v1 scope).
- **TLS for HTTP/3**: the server requires `--cert` + `--key`; no
  automatic certificate management.

## References

- ISO/IEC 15444-9 (3rd edition) — JPIP.
- RFC 9114 — HTTP/3.
- [Deployment guide](../deploy/README.md) — Docker + Cloudflare Tunnel for the server.