Public Owned UTF-8 API v1

Status: implemented bounded Project-v10 API; generated packages remain unpublished and explicit v9/v10 promotion decisions remain open. Release evidence is HOSTED GREEN under the v0.4.0 release baseline.

Audience: compiler contributors, generated-package integrators, and promotion reviewers.

The v0.2.0 tag commit 5f6fb9655fdec92c57ab71615cfd7bfa8cc76051 passed the complete blocking release run, including the Windows v10 generated-Rust consumer and the tagged Rust/Project matrices. Authoring-time “unrun” statements below are superseded only for nonignored cases selected by that workflow. No package publication, separate provisioned/ignored gate, or formal v9/v10 promotion is inferred.

Closed identity

• Project schema: semaprax.project.v10
• Project profile: owned-utf8-api.v1
• Descriptor schema: semaprax.public-owned-utf8-api.v1
• npm metadata schema: semaprax.owned-utf8-api.v1
• Native Rust manifest schema: semaprax.native-rust-owned-utf8-sdk.v1
• Native Rust manifest file: semaprax.native-rust-owned-utf8-sdk.json
• Descriptor digest domain: semaprax.public-owned-utf8-api.digest.v1\0

All v8 descriptor, carrier, metadata, generated JavaScript, Wasm, native-provider, and Rust-package bytes remain selected by their v8 identities. A v10 descriptor cannot replay as v8, and a v8 descriptor cannot contain owned-utf8.

Descriptor result

owned-utf8 is a distinct result type. It is not an alias for owned-bytes. The v10 profile admits all v8 result types and adds a SEMAPRAX string result. Borrowed input mappings are unchanged.

The authored v10 input correction explicitly reuses the existing v8 captured-intrinsic whole-tuple preflight. It checks the cumulative 65,536-byte borrowed-input bound, including exact UTF-8 lengths, before payload snapshots; module bytes have a separate 16 MiB pre-copy/hash bound. V10 JavaScript and dependent artifact bindings intentionally change, not descriptor, Wasm, TypeScript, or v8 JavaScript bytes. Raw Bytes are still never decoded as text.

Physical boundary

The physical value is (opaque provider handle, exact byte length). Neither the provider nor a target adapter searches for a NUL terminator. Embedded NUL bytes are data. The fixed maximum result length is 65,536 bytes.

The native provider uses an allocation representation with an explicit length header, also reused by the later ordinary String contents correction without changing v10 output. Before it attaches an opaque handle or publishes result fields, it validates the exact bytes as Unicode scalar-value UTF-8. An invalid value returns the adapter-failure status and publishes no result.

render_owned_data_c_header now exposes the same descriptor-derived low-level C11 integration boundary used by the owned-data provider. It declares only fixed-width values, an opaque context, opaque byte handles, and the closed status/tag result. A C integrator obtains the exact length, copies the bytes, and drops the handle before closing the context; no native String layout or NUL-terminated spelling crosses the boundary. Focused local evidence compiles the header consumer and actual v10 provider separately, links and executes at O0/O2, and preserves the embedded NUL plus multibyte UTF-8 by exact length.

The Wasm adapter transports the same length-delimited owned carrier. The npm facade consumes the carrier once and decodes only an owned-utf8 result with a fatal UTF-8 decoder. An owned-bytes result remains Uint8Array; it is never silently decoded. The generated Rust facade copies and settles the opaque handle before String::from_utf8, so valid and hostile invalid byte sequences both settle provider ownership exactly once.

The root compiler alone authenticates provider semantics from replayed HIR. The unpublished lower package authenticates the closed descriptor, provider byte integrity, compiler-declared textual binding, held tools, and filesystem publication facts; it verifies the renamed stage through retained stage authority and receives no HIR or independent semantic-proof authority.

Settlement

The v10 Rust package shares the Cargo build-script path boundary. Missing, non-Unicode, and CR/LF package paths reject before Cargo instructions; the target guard retains precedence. Only generated build.rs and its manifest integrity bindings change, not UTF-8 descriptors, provider archives, safe/FFI Rust, or package schemas. These regressions are implemented.

The existing closure admission remains narrow: a String-returning function must have a literal or direct retained-function call as its body (optionally inside an empty block). A non-String-returning body may not stage String expressions, and compiler-owned String intrinsics remain rejected. A direct call's argument expressions can nevertheless contain String temporaries, bindings, nested calls, and blocks. Unused owned String parameters also require settlement; the narrow outer-body rule does not remove those obligations.

The v10 Wasm correction tracks those physical String owners in deterministic local cells derived from validated HIR and the emitter's exact function plan. This is the existing inline String-cleanup convention, not a reinterpretation of resource CleanupPlan slots or a new cleanup schema. Place reads clone their value, preserving the source binding; temporary handoffs move and clear their source cell. Expression/scope exit settles nonescaping cells before reuse or loop backedges. Calls evaluate all arguments left to right before atomically transferring staged String arguments to the callee. Call-out memory is only transport, not a second cleanup owner.

Success settles non-result owners before result publication. A provisional String result is cleared only after the caller store succeeds; recoverable failure sweeps remaining cells without changing the primary status. Host exceptions, allocation failures, or failing finalizers do not acquire a new recoverable cleanup guarantee: the instance remains poisoned/fail-stop.

For v10 packages selecting an owned UTF-8 result, the npm arena bound is derived from the selected acyclic call closure: one transient handoff slot plus the maximum call-path sum of authenticated Bytes cleanup leaves and String owner cells. Checked arithmetic rejects token space overflow. This is conservative simultaneous-owner accounting, not a heap byte or exact-liveness claim. V8/v9 retain their fixed 16-owner runtime bytes; their separate Bytes-copy admission is unchanged. V10 Wasm/JavaScript and dependent integrity bindings intentionally change; descriptors and public signatures do not. Scalar/Bytes-only v10 selections retain their existing memory layout and fixed arena bound.

Copy does not transfer ownership. Consume/drop settles exactly one live handle. Every successful string result is copied and settled before JavaScript or safe Rust publication. Invalid UTF-8 cannot be returned as a host string, and a conversion failure cannot leave a live provider handle. Stale, foreign, wrong-length, repeated, or exhausted handles retain the v8 fail-closed rules.

The owned npm invocation correction makes a post-consumption JavaScript UTF-8 decoding failure poison the instance even when its arena is empty. Imported UTF-8 validation returns zero only for authenticated malformed text; memory/carrier faults and unexpected host exceptions cannot masquerade as malformed user bytes. The shared v8/v9/v10 failure-state correction changes runtime JavaScript and dependent integrity bindings, not Wasm, descriptors or public signatures. Its regressions are implemented.

The authored shared Rust invocation guard additionally proves the complete provider context settled before any outward value or recoverable error, including a UTF-8 conversion failure. An inner owner guard precedes context closure on Rust unwind; uncertain settlement remains fail-stop. Only a proven-closed context may be reinitialized on a later call. Its private invocation counter resets while the linked provider's handle issuer remains nonreused. Generated safe/private Rust and integrity bindings intentionally change, not provider C/ABI, public signatures, descriptor or manifest schemas. These regressions are implemented.

The later descriptor-selected Rust helper correction omits unused owner-only FFI declarations for admitted scalar-only v10 selections, while keeping the complete context protocol. Any selected Bytes/owned-UTF8 result retains copying, malformed-carrier discard and owner guards byte-for-byte. Only scalar-only private FFI and dependent integrity bindings change, not v10 admission, public signatures, providers or schemas. Warning-denied scalar SDK metadata/consumer checks pass with Rust 1.85.1, 1.88 and 1.98; the documented real minimum-version Cargo handoff covers v8/v9, not an actual v10 provider package.

The native correction is confined to the existing v10 owned-UTF8 provider projection. Its per-function physical String owner cells are declared and initialized before any recoverable failure branch. Emission records the exact cells in bounded staged output; no heap-allocated runtime registry or resource CleanupPlan change is introduced. Temporary-to-binding, branch, call, and provisional-result handoffs transfer ownership rather than creating a second cleanup owner. Every argument is evaluated before the complete String argument group transfers to the callee. Normal scope exit settles nonescaping live owners before loop reuse; the common failure exit settles every remaining owner and preserves the primary status and caller result storage. Success settles non-result owners before publication and relinquishes the result only after its store.

Native generation still emits the supplied program, not only the selected public closure. The same physical bookkeeping must therefore cover emitted String intrinsics and contracts in unselected functions; this does not admit them in the public v10 closure. Allocation failure remains fail-stop. These rules add no unwind, signal, or longjmp recovery guarantee.

The v10 correction changes only v10 native provider C and its dependent integrity bindings. The subsequent ordinary native String correction reuses the ledger under a separate selection and leaves v10 provider bytes unchanged. The separately amended owned-data provider correction also reuses that ledger and length-header representation for v8/v9 emitted Strings, without changing v10 output or widening selected closure admission. Frozen command/callable projections retain their separate String limitations. Context-handle closure alone is not proof that pre-handle String allocations were freed. The implemented cross-backend settlement and sanitizer regressions use the hosted-green release baseline; broader support and public promotion retain their owning gates.

Authored evidence

Project v10 follows the separate Windows owned npm publication contract: full-toolchain held-handle publication with an existing parent and an admitted output leaf; standalone Windows publication rejects safely. This changes filesystem routing, not UTF-8 semantics, artifacts or Unix behavior. Its admitted publication and consumer-route regressions have hosted-green release evidence.

The shared descriptor cross-replay cases include authentic v10 Bytes and owned-UTF8 counterparts under the same profile and synthetic subject facts. Self-replay succeeds; each correctly digested descriptor must reject against the other's retained HIR. This is signature binding regression evidence in the v0.4.0 baseline, not cross-schema rejection, source provenance, behavioral equivalence or target execution.

tests/project/v10_recipe_consumer.rs uses a real four-source Project, with two imported owned-String helpers sharing a display name but retaining different stable identities, including source-escaped control characters. It replays the actual inline npm carrier and reopens all six published artifacts for exact equality. Its Node consumer uses the published bindings and Wasm, covering empty text, leading BOM, embedded NUL, multibyte text, raw malformed UTF-8 remaining Bytes, late-argument arithmetic failure and subsequent reuse. A helper display-only rename must preserve every descriptor fact except the three Project revision/graph bindings. Native coverage in this fixture stops at the compiler-replayed package passed to an intentionally rejecting publisher; it does not compile, publish, or consume a native SDK. The v0.4.0 release regression corpus is HOSTED GREEN.

cargo test --locked -p semaprax --test project v10_recipe_consumer::

The separate provisioned gate crates/semaprax-toolchain/tests/project_owned_utf8_sdk_v1.rs consumes the same four-source fixture through the real private native publisher. It regenerates the provider from retained checked Project HIR, reopens the exact seven-file SDK, and compares its canonical manifest against that descriptor/provider binding and the reopened file hashes. This test-specific consistency check is not an independent proof of archive provenance or provider semantics. An unchanged external Rust consumer then builds in an isolated workspace with --locked --offline and the existing short Cargo target-directory guard. Its safe API forbids unsafe code and covers the same primary Node corpus: exact 17-byte BOM/NUL/Unicode text, empty text, raw malformed UTF-8, 65,536-byte Bytes, capacity-plus-one input rejection, and repeated checked failure and recovery through two SDK objects. Retained String and Vec<u8> values remain independent after later calls, mutation and SDK destruction. These observations do not replace physical allocation accounting or prove reuse of one initialized native context; the maximum-byte case here is not a maximum String result. The gate also checks exact revision-only descriptor changes after helper rename, SPX-I234 no-clobber rejection, unchanged consumer inputs, SDK artifacts and Project source bytes.

This gate is explicitly ignored until selected on a provisioned host with absolute CLANG and SEMAPRAX_ARCHIVER paths and native Cargo. Windows also requires the existing SEMAPRAX_VCTOOLS, INCLUDE, LIB and SEMAPRAX_LINKER configuration. Authoring or skipping it does not count as a successful SDK gate, and it requires that explicit selection:

cargo test --locked -p semaprax-toolchain --test project_owned_utf8_sdk_v1 -- --ignored

The separate tests/support/owned_utf8_capacity.rs subject isolates the String result boundary in a two-source Project. A single selected literal contains 65,535 or 65,536 UTF-8 bytes, including repeated BOM, NUL, multibyte and astral characters. No other String literal consumes the shared 65,536-byte Wasm literal pool. The real npm and native Rust publication gates compare returned strings against independently spelled byte oracles, exercise repeated calls through two instances, and retain earlier host values. Native host values are also checked after SDK destruction. Package and source inputs remain exact. The native gate reuses the same test-only manifest consistency oracle as the multi-module SDK fixture; neither oracle establishes archive provenance.

A separate 65,537-byte source must fail ordinary Project admission with SPX-W110 (owned UTF-8 literal table exceeds 65536 bytes) before the Project callback or publication. This is a compile-time literal-pool boundary, not evidence of a native runtime over-limit rejection. These cases do not replace physical allocation accounting, failure-path settlement, or maximum input coverage. Both gates are implemented; the native one is explicitly ignored and requires the same provisioned tools described above:

cargo test --locked -p semaprax --test project owned_utf8_capacity::
cargo test --locked -p semaprax-toolchain --test project owned_utf8_capacity:: -- --ignored

Retained reference evaluation

The additive retained-Project evaluator replays the exact v10 descriptor and then independently authenticates the selected HIR closure before evaluating it without filesystem, process, publication, target, or cache authority. The public result vocabulary is v10-only: the six v8 scalar/owned-byte shapes plus one owned UTF-8 String. It does not extend the exhaustive v8 result enum.

The selected export must be an explicit, non-entry stable identity and must match its descriptor parameter identities, names, types, order, and result. Automatic-identity monomorphic helpers remain valid inside the exact selected closure, as they are in descriptor admission. Every reached function is rechecked as import-, generic-, effect-, and contract-free, the closure is acyclic and contains at most 256 functions, and the v10-only closure-shape predicate is applied independently of the broader internal-String interpreter. Malformed selectors produce the fixed non-echoing SPX-F102 diagnostic.

Each invocation has two fixed, non-caller-widenable logical materialization limits:

• 65,536 cumulative owned UTF-8 payload bytes; and
• 4,096 cumulative owned UTF-8 allocations, including empty strings.

These are logical interpreter payload limits, not claims about allocator metadata or physical heap size. One atomic precharge occurs before every interpreter-owned String allocation. Rejection neither mutates the committed counters nor allocates, and checked-arithmetic overflow fails as the same closed capacity outcome. There are no refunds. Literal materialization and the existing source-preserving String place read both charge; authenticated let/result handoff, staged argument-to-callee transfer, drop, and final carrier handoff do not. The runtime value carrier has no implicit deep-clone route, so future String copies must pass through the same accounting seam. Compiler-owned String intrinsics remain rejected by v10 admission, but their allocation paths also precharge defensively so admission drift cannot bypass the meter.

Borrowed str and Slice<u8> snapshots retain their separate cumulative 65,536-byte input bound and are admitted completely before any snapshot. The successful host copy-out retains its separate 65,536-byte output bound. Neither host boundary is charged again as an interpreter-owned materialization; doing so would make an otherwise valid maximum-size result impossible. A successful active Bytes or UTF-8 result emits exactly one distinct copy-out-and-settle event. Scalars, None, Result::Err, and every failure emit none. Empty UTF-8 is still an active owner and therefore emits one UTF-8 settlement event.

Fuel is charged before a node's materialization precharge, the first selected failure remains sticky, and the closed quota outcome is not a guard error or a language status. Its attempted counters identify the rejecting precharge; ordinary used counters report only committed charges, including when a later language, fuel, or depth failure wins. Counters are created afresh for every evaluation. This reference lane is not target-execution or promotion evidence, and v8, v9, legacy, and internal-String evaluator behavior remains selected by their existing APIs.

The shared lower v8/v10 descriptor reader also rejects repeated parameter identities within one export, even under a freshly computed descriptor digest; see the owned-data descriptor contract. This tightens malformed-input rejection only, without changing emitted package bytes, public signatures, or schema identities.

Focused authored lifetime evidence is in tests/project/owned_utf8_lifetimes.rs and its raw-arena/real-facade Node consumer. It derives and replays the real descriptor before target generation; locals and Copy-only loops occur inside admitted direct-call arguments. The loop case retains one String before the loop and settles it on exit; it does not allocate Strings per iteration. Direct String loop storage remains SPX-T252, and an otherwise legal scalar helper staging Strings remains outside the selected v10 closure (SPX-J113). Negative cases preserve the rejected outer-body String forms. The native O0/O2 fixture is success-value evidence only, not native failure-path allocation evidence.

cargo test --locked -p semaprax --test project owned_utf8_lifetimes::

tests/native/owned_utf8_settlement.rs independently derives/replays the descriptor and emits the actual v10 length-header native provider. Its fixed test-only allocation table observes provider allocations and frees at O0/O2: local, late-argument, nested-call, callee, and loop failures must leave no allocation live and must preserve poisoned result fields. Subsequent calls reuse the same context. Success covers clones, branches, mixed Bytes/String, more than sixteen owners, empty text, embedded NUL, and multibyte UTF-8. Separately labeled raw semantic calls exercise emitted-but-unselected String intrinsics, equality, guarded matching, and failed postconditions without widening admission. The generated Rust package consumer exercises safe API reuse after failure; it is not a substitute for the physical allocation counter.

cargo test --locked -p semaprax --test native owned_utf8_settlement::
cargo test --locked -p semaprax --test project_native_rust_owned_utf8_v1

The separate sanitizer gate requires an installed absolute SEMAPRAX_STRING_SANITIZER_CLANG path with ASan/UBSan support:

cargo test --locked -p semaprax --test native owned_utf8_settlement::provisioned_native_string_provider_asan_ubsan -- --ignored --exact

Allocation counters establish the leak assertion even on hosts without LeakSanitizer; ASan/UBSan are additional memory/undefined-behavior checks, not a claim of LeakSanitizer coverage. None of these new gates was run in this batch.

Non-claims

The implementation's nonignored regression evidence ran in the v0.2.0 tag matrix; separately provisioned and ignored gates remain outside that claim. No generated package is published and formal promotion is not claimed. Project v10 remains blocked on an explicit Project v9 promotion decision; the v0.2.0 tag supplies hosted regression evidence for selected cases, not that decision. The local C11 integration test is not package publication, cross-platform support, or a promotion decision.

This profile adds no command, filesystem, process, network, daemon, recovery, arbitrary publication, or general text-streaming authority. It does not decode raw Bytes, expose a public aggregate ABI, or weaken Project v1-v9 behavior.