stdlib

Standard library wrappers and the synthetic-module redirect for importing native Go packages into mvm.

Overview

The stdlib package is the bridge between mvm programs and the Go standard library. It has three responsibilities:

1. Wrap native Go symbols so interpreted code can import "fmt" and call fmt.Println as if it were local.
2. Seed the synthetic std module so generic-first and pure-Go packages can be interpreted from upstream source instead of bridged (see ADR-017).
3. Host package patchers that overlay native symbols at first Eval -- now used only by the runtime-introspection bridge.

Interface satisfaction -- letting interpreted values implement fmt.Stringer, error, json.Marshaler, etc. at native call boundaries -- used to live here as bridge structs, but now happens by attaching methods to a synthesized rtype in runtype + stdlib/stubs. See ADR-021.

Key types and functions

• Values (map[string]map[string]reflect.Value) -- the native-symbol registry. Outer key: package path ("fmt"). Inner key: symbol name ("Println"). Each value is a reflect.Value wrapping the Go function, variable, or type.
• PackagePatcher (func(*vm.Machine, map[string]vm.Value)) -- a callback that mutates a package's exported symbol map. Used to splice mvm-native types in place of stdlib ones.
• RegisterPackagePatcher(importPath, fn) -- append-only registration. Called from package init() (now only stdlib/runtime_virt.go).
• PackagePatchers() map[string][]PackagePatcher -- patcher list, consulted once by Interp.patchStdlibOverrides on the first Eval.
• EmbeddedStd() []byte -- bytes of stdlib/src.zip, a Go-proxy-format zip of the synthetic github.com/mvm-sh/std module baked into the binary via //go:embed. Consumed by stdmod to seed the parser's stdlib FS without touching the network. The zip contains only *.go implementation files plus go.mod and LICENSE -- tests, examples, and repo scaffolding (Makefile, patches/, .git) are stripped. See ADR-017.

Internal design

Package layout: core, ext, all

Generated bindings split across two sub-packages so that consumers can control the transitive-dependency footprint of an embedded mvm:

• stdlib/core/: pure-compute, browser-safe packages with modest transitive footprint (fmt, bytes, strings, time, encoding/json, regexp, ...). One file per import path, e.g. stdlib/core/bytes.go. ~40 files.
• stdlib/ext/: host-coupled or transitively heavy packages (net/*, os/*, crypto/*, image/*, runtime/*, syscall/*, ...). Each syscall binding is platform-specific (syscall_<os>_<arch>.go). ~170 files.
• stdlib/all/: convenience aggregator. Blank-importing this package pulls in core + ext for the full set. Consumers who want a smaller link footprint import stdlib/core directly.

The split lives in cmd/extract/categories.go (the Core map). See ADR-013.

Generated native bindings

Most bindings are produced by cmd/extract, driven by //go:generate directives in stdlib/gen.go. A binding contains only an init() that inserts entries into Values. Files are written under stdlib/core/<pkg>.go or stdlib/ext/<pkg>.go depending on the Core map; cmd/extract runs go/format.Source on its output before writing, so generated files are gofmt-clean.

Generated files carry a // Code generated by cmd/extract; DO NOT EDIT. marker. make clean_generate deletes any file matching that marker; hand-written package-level files (stdlib/patcher.go, stdlib/srcfs.go, stdlib/stdlib.go) must not carry it.

Exclusions (packages intentionally not bound) are documented in the file header of stdlib/gen.go: unsafe, plugin, runtime/race, time/tzdata, the generic-first set (cmp, iter, maps, slices), and syscall (handled per-platform by the Makefile loop, not by go generate).

Interpreted-source packages: the synthetic std module

Generic-first stdlib packages (cmp, iter, maps, slices) cannot be extracted as reflect.Value entries because their generic functions never materialize until instantiation. A growing set of other pure-Go packages (errors, path, ...) is also better served by interpreting upstream source than by maintaining hand-written native bridges.

These packages live in a separate Go module, github.com/mvm-sh/std, sourced from $GOROOT/src and patched locally where mvm cannot interpret upstream as-is (iter.Pull/Pull2 needing runtime coroutines, etc.). At generation time (make generate) stdlib/gen_stdzip.go walks ../../std and emits stdlib/src.zip in proxy layout; that zip is committed to the repo and //go:embed-ed by stdlib/srcfs.go.

At runtime stdmod wraps the embedded bytes (or a network-fetched override) in a redirecting fs.FS that the parser sees in its stdlibFS slot. Stdlib-shaped imports get rewritten to github.com/mvm-sh/std/<pkg> and resolved through modfs. Native bridges still take precedence -- the parser checks Packages[importPath] before any FS lookup -- so packages registered in core/ext continue to use their pre-compiled bindings even if the std module also publishes them.

The std module is a deliberately partial mirror; performance-critical or hard-to-interpret packages (fmt, runtime, reflect, sync, time, crypto/* with assembly fast paths, ...) stay as bridges and are intentionally not added to the std module's package list.

See ADR-017 for the complete design.

Hand-written bindings

• stdlib/core/unsafe.go -- the unsafe pseudo-package cannot be extracted. It registers Pointer, Sizeof, Alignof, Offsetof, Add, Slice, String, SliceData, StringData. Sizeof/Alignof/Offsetof are intercepted at compile time in goparser.evalConstExpr (const contexts) and comp.compileBuiltin (runtime). Slice/SliceData are also intercepted to compute pointer-element-dependent result types. The stub implementations panic if reached at runtime.

Interface satisfaction (synthesized rtypes)

Interpreted values satisfy native Go interfaces (fmt.Stringer, error, json.Marshaler, sort.Interface, fmt.Formatter, ...) because mvm attaches their methods to a synthesized rtype that native dispatch reads directly. That machinery lives in runtype and stdlib/stubs, not here -- stdlib no longer defines per-call bridge structs, and the former jsonx/xmlx/gobx shadow packages are gone. See ADR-021.

Package patchers (patcher.go)

stdlib.RegisterPackagePatcher("runtime", patchRuntime)

Patchers are consulted by Interp.patchStdlibOverrides on the first Eval. Each registered patcher for an import path is called with the live *vm.Machine and the package's vm.Value symbol map, which it may overlay with replacement symbols.

Introduced for the mvm-native stdlib shadows (ADR-012), the sole remaining patcher is now stdlib/runtime_virt.go, which overlays the runtime introspection entry points for the runtime bridge (see ADR-016). Interface dispatch no longer needs patchers.

The registry is populated only from init() functions, so no locking is required.

Dependencies

• reflect -- wrapping Go values.
• vm/ -- Machine, Value, Type, RegisterPackagePatcher consumers, and the bridge seams (RegisterNativeMethodHook, RegisterMethodValueShim, RegisterExtendedShapes, RegisterSentinelHooks, WalkCallStack, RegisterSynthIfaceTargetFunc). The *runtime.Func sentinel registry and the io.EOF / synth-shape stub behavior that used to sit in vm now live here.
• symbol/ -- BinPkg creates package descriptors (called by the parser via ImportPackageValues).

Open questions / TODOs

• Reflect-walking packages (encoding/json, encoding/xml, encoding/gob, text/template) now see interpreted methods via synthesized rtypes, so the old per-package shadows are gone. Coverage is bounded by the shape catalog; a method whose signature has no shape still won't be found by native reflection.
• fmt.Print* route to the Machine's writer via interp.patchFmtBindings.