stdlib/stubs

The method-signature "shape" catalog and dispatch-stub pools that let synthesized rtypes route interpreted method calls back into the interpreter.

Overview

stdlib/stubs is the upper half of mvm's native method-dispatch mechanism (see ADR-021); runtype is the lower half. A synthesized rtype needs each method's Ifn/Tfn to point at a real Go function PC. stubs pre-generates pools of such functions, one pool per method-signature shape, and pairs each pool with a dispatcher that forwards the call to a per-slot handler closure (which re-enters the interpreter). Two kinds of shape coexist: the per-signature typed shapes below, and the ABI word-class shapes (ADR-022) that let one pool serve many signatures.

The package exists separately from runtype to break an import cycle: runtype.Attach* need a resolved stub PC, while the stub pools need runtype.FuncPC. Inverting the attach API (PC-based runtype.MethodSpec) lets stubs depend on runtype one-directionally instead.

Key types and functions

• Shape + ShapeS1 ... ShapeS38 -- the catalog of supported method signatures. A few representatives:

  Shape	Signature	Covers
  S1	func() string	Stringer, error, GoStringer, flag.Value.String
  S2 / S3	func() ([]byte, error) / func([]byte) error	json/text/binary Marshal/Unmarshal
  S4 / S5 / S6 / S7	Is/As/Unwrap shapes	errors.Is/As/Unwrap
  S8 / S9 / S10	Len/Less/Swap	sort.Interface
  S11 / S12	Push/Pop	heap.Interface
  S13	func([]byte) (int, error)	io.Reader/io.Writer
  S14	func(fmt.State, rune)	fmt.Formatter
  S15 / S16	MarshalXML/UnmarshalXML	xml.Marshaler/Unmarshaler
  S22 ... S31	fs method shapes	fs.FileInfo, fs.DirEntry, fs.FS and friends
  S32 ... S36	slog method shapes	slog.Handler, slog.LogValuer
  S37	func() (rune, int, error)	io.RuneReader
  S38	func()	niladic marker methods

• Method -- {Name, Exported, Sig, Shape, Handler, WordKey, Core}. The shape-carrying input the vm builds. For a typed shape, Handler is the matching HandlerS* closure; for the word-class path, a non-empty WordKey names the generated word-shape pool and Core is the marshaling closure (Shape/Handler ignored).

• HandlerS1 ... HandlerS38 -- per-shape handler function types.

• CoreFunc -- the word-class marshaling closure type, func(recv unsafe.Pointer, pw []unsafe.Pointer, sw []uint64, rpw []unsafe.Pointer, rsw []uint64); the vm supplies it, the generated dispatcher calls it.

• Attach{Methods,StructMethods,PrimitiveMethods,SliceMethods,ArrayMethods, MapMethods,PtrMethods} -- mirror the runtype.Attach* entry points but accept []Method; each resolves every method's shape (typed or word-class) to a free stub slot, then calls runtype.

• HasWordShape -- reports whether a generated word-shape pool exists for a key, so the vm drops (not errors on) an unsupported word-shape.

• SlotsUsedS1 ... SlotsUsedS38 -- slot-pool usage counters (for tests / metrics).

• HighWater -- per-pool slot high-water vs capacity (typed and word shapes). mvm dumps it on exit when MVM_POOLSTATS is set; use it to right-size pools, since each slot is one generated function and the whole package compiles ~50k of them (under -race that roughly triples compiler memory, so make fast lowers GOGC for the build).

Internal design

Each shape SN has two files:

• pool_sN.go (generated by gen_pools.go, //go:build !wasm) -- the stub functions stubSN_k(recv, ...) { dispatchSN(k, recv, ...) } and a stubsSN array of their PCs (runtype.FuncPC). One stub per slot; the Go compiler emits the correct ABI, so no assembly is needed. The pool size poolSizeSN lives in the generated sizes.go. The whole pool is native-only (see "wasm" below).
• registry_sN.go (hand-written) -- a slot pool of HandlerSN, an acquireSlotSN that claims the next free slot, and dispatchSN(slot, ...) that looks up and invokes the handler.

flowchart LR
    m["[]Method<br/>(shape + handler)"] --> acq["acquireSlots<br/>(claim a slot per method)"]
    acq --> spec["[]runtype.MethodSpec<br/>(StubPC = stubsSN slot)"]
    spec --> g["runtype.Attach*"]
    g --> rt["synthesized rtype"]
    rt -. "native call hits Ifn" .-> stub["stubSN_k"]
    stub --> disp["dispatchSN(k, recv)"]
    disp --> h["HandlerSN closure<br/>(re-enters interpreter)"]

Slots are claimed monotonically and never reclaimed (the per-shape counter has no safe decrement under concurrent attaches); release only nils a slot's handler to free its closure captures. S1 carries 3072 slots (Stringer/Error are the most-attached shape) and S38 5120 (protobuf markers); the rest default to 256.

wasm carries no pools (shared-PC dispatch)

The ~53k stub functions were about half the wasm binary; the wasm target carries none (all pool_*.go are //go:build !wasm). This is sound because no native caller dispatches an interpreted method on wasm: interpreted code uses IfaceCall, and interpreted reflect is intercepted by the vm (reflectValueShim/reflectTypeShim). The method entry is never invoked; it exists only so the synth rtype carries a method set for reflect introspection, which reads name/signature metadata, not the entry.

So wasm wires every Ifn/Tfn to the runtime's -1 unreachable TextOff sentinel (fill_wasm.go leaves MethodSpec.StubPC zero; runtype.makeMethod maps that to -1, which reflect/runtime resolve to runtime.unreachableMethod). A real PC is deliberately not wired: it would enter reflect's GC-scanned reflectOffs table, where on wasm a code PC aliases a heap span and crashes the GC. vm.synthSharedPC makes resolveDispatch attach every method without building handlers. arrays_wasm.go declares empty stubsSN arrays so registry_sN.go still compile (unreachable; the linker drops them). Native dispatch is unchanged. See ADR-022.

Interpreting dispatchers from the mirror

The trap is only avoided once dispatching packages are interpreted, not bridged. cmd/extract's BuildTags[pkg] = "!wasm" drops a bridge on wasm, so the package loads from the embedded mirror (github.com/mvm-sh/std) -- it must be in the mirror or it is absent on wasm. A bridge never loads mirror source on native, so native keeps the reflect bindings.

Phase 3 grows this from fmt toward all dispatchers: the mirror also ships strconv, strings, bytes, bufio, sort, unicode*, io, io/fs, context, the encoding/* family (json, gob, xml, binary, csv, base64, hex, ...), and net/url, net/netip. Their internal floor is mirrored with pure-Go overlays where upstream is asm/linkname/runtime-coupled: internal/bytealg, internal/godebug (reports defaults), unique (a map-based interner, no weak GC). Floor that can't be mirrored is re-exported under its import path in stdlib/internal_stubs.go (internal/abi.NoEscape, internal/reflectlite).

Keep-native (bridged, never interpreted): the floor (reflect/runtime/sync/unsafe/syscall + asm internals) plus the pure-compute families math*, crypto*, hash*, compress*, archive* (for speed; they never dispatch an interpreted method, so they need no stubs) and the host floor os/syscall/internal/poll/internal/syscall/* (their raw-syscall internals use go:linkname, which mvm does not interpret; bridging also lets native compress/archive wrap an os.File without hitting the trap). net sockets stay likewise bridged/absent; only the pure-Go net/url/net/netip are interpreted.

MVM_INTERP=<list> drops those bridges on a native build, to validate interpretation without a wasm rebuild. After editing the mirror, regenerate stdlib/src.zip (go run stdlib/gen_stdzip.go) and touch stdlib/srcfs.go so the embed refreshes.

Dropping bridges to shrink the wasm binary

A bridge reflect.ValueOfs every exported symbol, pinning it against dead-code elimination -- even in packages mvm's own code keeps (crypto, net/http, archive/zip via modfs), so most bridge weight is unused functions. cmd/extract's WasmDropPrefixes/WasmDropExact tag heavy, rarely-needed bridges !wasm (crypto, net, image, debug, go, compress, archive, database/sql, ...); they then interpret from the mirror or error until mirrored. WasmKeepExact exempts a package from a dropped prefix: net, and go/build because the mirror ships only go/build/constraint, so x/text/internal/gen would not load (+0.5 MB). That buys name resolution, not a working call: build.Import shells out to go list, which fails on wasip1, and no -short run reaches it. This cut the binary 39.1 -> 24.2 MB (bridge-free floor ~19 MB). The keep-native set stays bridged: reflect/runtime/sync/unsafe/syscall plus the pure-compute families (math*, crypto*, hash*, compress*, archive*); the rest is interpreted from the mirror as it is filled in (phase 3).

Word-class shapes

The typed shapes above need one hand-written registry_sN.go per Go signature. The word-class shapes (ADR-022) key on the method's ABI register words instead, so one pool serves every signature that classifies the same way (Equal(StructA) bool over an interpreted two-word struct and any other func(ptr-word, int-word) int-word share pi_i). The catalog is wordShapes in gen_pools.go, and emitWord generates a pool_w*.go per shape -- the same stub-pool layout as the typed shapes, but with one generic dispatcher: it scatters the native register words into pw (pointer words, a typed []unsafe.Pointer so the GC scans them) and sw (integer words), calls the per-slot CoreFunc, then gathers the result words back out. The CoreFunc -- built by the vm (Machine.makeWordCore) -- does the reflect-driven value<->word marshaling and owns the error policy. registerWordPool records each generated pool in a sync.Map keyed by the word-shape (acquireWordSlot/HasWordShape read it); the vm computes the same key independently via detectWordShape. The path is gated to 64-bit little-endian targets; on 32-bit or big-endian targets only the typed shapes attach. On wasm (also 64-bit LE but a stack ABI, not registers) the vm classifies and marshals by 8-byte stack slots instead of register words, packing sub-word struct fields; the generated pools are arch-agnostic source and shared (see ADR-022).

The vm-side glue is not here -- vm/synth_bridge.go owns detectShape (signature -> Shape) and the generic makeHandlerS*, plus detectWordShape (signature -> word-shape key) and makeWordCore for the word path, because those need Machine/Iface/Type. The shapes whose signatures name a specific stdlib package (io/fs, log/slog, encoding/xml) live in stdlib/synth_method_shapes.go, registered into vm via vm.RegisterExtendedShapes, so vm core need not import those packages; their handlers re-enter the interpreter through the exported vm.SynthCall. A method is matched to a typed shape first and only falls back to the word path, so the faster, error-aware typed handlers win where they apply.

Dependencies

• runtype -- FuncPC (pools) and the Attach* synthesizers (wrappers).
• Standard library: reflect, unsafe, sync/atomic, plus the packages the shaped signatures mention (fmt, encoding/xml, io/fs, time, context, log/slog).
• Consumed by vm/synth_bridge.go (Shape/Method/HandlerS*/Attach*).

Regenerate the pools with go generate ./stdlib/stubs/ (or make generate) after editing the shape catalog in gen_pools.go.

Open questions / TODOs

• Pools are finite; a process attaching more distinct methods of one shape than its pool holds errors out (stubs: shape SN stub pool exhausted). Sizes are a static guess tuned to the test suite (wasm carries no pools; see above).
• A new typed shape is append-only edits to gen_pools.go + a hand-written registry_sN.go + a detectShape/handler case (in vm/synth_bridge.go for a generic shape, or stdlib/synth_method_shapes.go for a stdlib-specific one); an ABI-compatible signature needs none of that and rides an existing word-shape.
• Under the register ABI the word path carries float64 (f), complex128 (ff), float32 (g, a single-precision FP-register word, distinct stub from f), complex64 (gg), and sub-word-packed structs, but drops arrays of length

  1 and signatures over the arch's register budget. The wasm/ABI0 path has no budget and carries every type as stack bytes (float32 as raw i bytes, so its key differs from the register g), dropping only on a missing pool. The path is disabled entirely on non-64-bit or big-endian targets.

• The word-shape catalog is hand-curated (full enumeration would explode), so growing it is guided by telemetry: run with MVM_WORDDROPS=1 and the process reports, at exit, every signature detectWordShape dropped -- a "missing pools" list of word-shapes to add, plus an "unsupported" list (arrays / over budget).