Chimera

Introduction

Chimera is a runtime for executing programs in a zero-setup sandbox. The command-line program chimera loads a program on the host, translates its instructions one basic block at a time into a private code cache, intercepts system calls, and either virtualizes them or forwards them to the OS kernel. Chimera also provides a Rust library API for programs that want to build their own sandboxes. For example, you can implement a syscall handler that audits system calls, or one that translates filesystem operations into a virtual filesystem.

Chimera is built on same-ISA dynamic binary translation. In other words, Chimera translates program code before executing it, but unlike many other binary translators, Chimera expects the program to share the host's instruction set. For example, you can run ARM64 programs on ARM64 and x86 programs on x86, but not ARM64 programs on x86. The purpose of dynamic binary translation is to intercepts instructions that require virtualization, such as system calls, but execute other instructions natively without overhead. The Chimera runtime is co-located in the program's virtual memory address space, which allows, for example, fast system call interception. The approach also requires no additional setup, as everything runs on the host using the host operating system, but with Chimera providing virtualization where needed.

Background

Same-ISA dynamic binary translation

A dynamic binary translator reads the guest's instructions on demand and emits an equivalent stream of host instructions into a private region called the code cache. The unit of translation is the basic block: a straight-line sequence that ends at the first instruction whose behavior cannot be reproduced by an isolated copy — any branch, any call, any return, any system call. The non-terminator instructions of a block run from the cache; the terminator is rewritten into a sequence that computes the next guest program counter and exits the cache.

A second-level structure called the translation map records, for each guest program counter that has been translated, the host program counter at which its block begins. A small piece of runtime code called the dispatcher consults the map after every block exit. If the next guest PC is already in the map, the dispatcher jumps to the corresponding host PC; if not, it invokes the translator on the guest PC, installs the new mapping, and then jumps. The two ways guest code can re-enter Chimera are exits to the dispatcher and rewritten system-call instructions.

Same-ISA dynamic binary translation, the design Chimera uses, exploits the symmetry between host and guest to reduce the body of a block to a memory copy. The decoder still has to walk each instruction to find its length and to flag the few that cannot be copied unchanged, but the per-instruction cost collapses to bytes-copied rather than re-encoding work. The relevant prior art is DynamoRIO (Bruening, Zhao, & Amarasinghe), the Arm-side MAMBO (Callaghan, Gorgovan, & Luján), and Apple's Rosetta 2 (described publicly by Johnson and reverse-engineered by Nakagawa). The Rosetta 2 work is cross-ISA, but its approach to indirect-branch handling and code-cache layout informs the present design.

Program loading on Linux

A Linux process is built by the kernel from an ELF image. The kernel parses the ELF header, walks the program headers to find every PT_LOAD segment, maps each segment at the virtual address its p_vaddr requests with the protection bits its p_flags carry, and copies in p_filesz bytes of file content (the remainder of p_memsz is zero-filled). A position-independent executable (ET_DYN) is mapped at a randomized base the kernel picks under its ELF_ET_DYN_BASE rule (approximately 0x5555_5555_0000 on x86-64 with 4-level page tables, plus ASLR jitter); a non-position-independent executable (ET_EXEC) is mapped exactly where it asks to live.

If the executable carries a PT_INTERP segment, it names a dynamic interpreter — almost always /lib64/ld-linux-x86-64.so.2. The kernel maps the interpreter as a second image at a base of its own choosing and hands control not to the executable's entry point but to the interpreter's. The interpreter is itself an ET_DYN ELF, and is responsible for walking the executable's PT_DYNAMIC segment, mapping every shared library the executable depends on (libc.so.6 chief among them), resolving symbols and applying relocations, running constructor functions in .init_array, and finally jumping to the executable's _start.

The kernel hands the interpreter the information it needs through the initial stack, which it builds before transferring control. From low to high address the stack carries: argc, then the argv pointer array terminated with a NULL, then the envp pointer array terminated with a NULL, then the auxiliary vector (a sequence of (type, value) pairs terminated with AT_NULL), then the bytes of the strings themselves. The auxiliary vector is the kernel's per-process metadata channel and the interpreter reads it to learn the executable's program-header address (AT_PHDR), the number and size of program headers (AT_PHNUM, AT_PHENT), the executable's entry point (AT_ENTRY), the interpreter's own base (AT_BASE), the page size (AT_PAGESZ), CPU feature bits (AT_HWCAP, AT_HWCAP2), the vDSO's address (AT_SYSINFO_EHDR), and a few others. The Linux ABI specifies the layout precisely because user-space code reads it directly off the stack at process entry.

Program loading on Darwin

A Darwin process is built from a Mach-O image. The Mach-O header is followed by a sequence of load commands; the loader walks them in order. LC_SEGMENT_64 commands play the role of PT_LOAD: each names a segment (__TEXT, __DATA, __LINKEDIT, …), gives its virtual address, size, protection, and the file region whose contents back it. The lowest segment, __PAGEZERO, is conventionally a 4 GB unreadable mapping at virtual address 0; it has no file backing and exists only to trap null-pointer dereferences.

LC_LOAD_DYLINKER names the dynamic linker, conventionally /usr/lib/dyld. Unlike Linux's ld.so, dyld is not loaded as a separate fresh image. Apple delivers it as part of the dyld shared cache, a single file the kernel maps at a fixed system-wide virtual address that contains dyld together with every system library — libsystem, CoreFoundation, the Objective-C runtime, AppKit, and on. Both __TEXT and a writable __DATA for each cached image live in this single mapping. The cache is initialized once per boot and shared across every process on the system.

The entry point is named by either LC_MAIN or the older LC_UNIXTHREAD. LC_MAIN gives the file offset of the executable's main function; the convention is that dyld arranges for main to be called with (argc, argv, envp, apple) in x0..x3 and lr set so that main returning lands in libsystem's exit. LC_UNIXTHREAD gives a saved register state to load; this form is used by dyld itself, which has no main. The apple array is dyld's per-process metadata channel, analogous to Linux's auxiliary vector. Its entries are NUL-terminated key=value strings; _NSGetExecutablePath and a handful of other libsystem queries consult it.

Static binding metadata is carried separately from the executable's segments. Modern Mach-Os (clang ≥ 13 or so) use LC_DYLD_CHAINED_FIXUPS: rebases and binds are encoded as a chain of pointers walked at load time, with each pointer's high bits naming either an imported symbol or a slide-relative rebase target. Older binaries (and what rustc currently emits) use LC_DYLD_INFO_ONLY, an opcode stream of rebase/bind/lazy-bind instructions. Thread-local variables use a third layout: S_THREAD_LOCAL_VARIABLES sections hold descriptors of the form {thunk, key, offset} that the compiler walks on every TLV access, and S_THREAD_LOCAL_REGULAR / S_THREAD_LOCAL_ZEROFILL sections hold the initial-data template for fresh threads.

Loading the guest

Loading the executable

On Linux, Chimera follows the kernel's procedure. It reads the ELF, walks PT_LOAD segments, and mmaps each at the address its p_vaddr requests with the requested protections. For an ET_EXEC binary the mapping uses MAP_FIXED_NOREPLACE, which fails rather than overwriting an existing mapping; for an ET_DYN binary Chimera first reserves a contiguous PROT_NONE region with anonymous mmap, computes the slide that places the binary's vmaddr range inside that reservation, and maps each segment with MAP_FIXED at the slid address. If the binary carries PT_INTERP, Chimera loads the interpreter as a second image with the same procedure and arranges for dispatch to begin at the interpreter's entry, not the executable's. From that point on, the dynamic linker is just translated guest code: every mmap of libc.so.6 and every other library reaches Chimera's syscall handler and ends up where the host kernel chooses.

On Darwin, Chimera reads the Mach-O, picks the arm64 (or arm64e) slice if the file is a fat binary, walks LC_SEGMENT_64 commands, and mmaps each segment as an anonymous mapping at the segment's vmaddr plus a slide. The slide is zero for non-PIE binaries that asked for a fixed address; it is the offset returned by an upfront PROT_NONE reservation for PIE binaries and for dyld itself. The 16-kilobyte page size that Darwin/arm64 uses is honored in alignment.

The critical departure on Darwin is that Chimera does not hand control to /usr/lib/dyld. The cached dyld is bound to the same system-wide virtual address in every process on the system, and Chimera's own process has already initialized that cached dyld for itself: the dyld in the shared cache reads "initialized," its __DATA already names Chimera's heap, and bringing up a second guest-side instance of the same dyld lands on its own assertions. Even if the assertions were sidestepped, dyld would attempt to call into a libsystem whose initializers have already run for the runtime. The runtime therefore links the guest in process. It reads the guest's LC_DYLD_CHAINED_FIXUPS or LC_DYLD_INFO_ONLY metadata, applies rebases against the slid runtime address, resolves every imported symbol against the host's loaded libraries via dlsym(RTLD_DEFAULT, ...), and writes the resulting addresses into the guest's binding slots. LC_LOAD_DYLIB paths are dlopen'd in the host process beforehand to make their symbols visible to the lookup. Thread-local variable descriptors are rewritten so the thunk slot points at a Chimera-provided routine that allocates per-thread storage from the in-image template on first access through pthread_key_create / pthread_setspecific. The guest's entry, named by LC_MAIN, is then dispatched directly with the calling convention dyld would have used.

Setting up the stack

On Linux, Chimera allocates an 8 MB anonymous mapping with MAP_STACK and writes the System V AMD64 process-entry frame from the top down: the string bodies for argv and envp, the AT_PLATFORM and AT_EXECFN strings, a 16-byte AT_RANDOM block, and then the fixed structure of argc, the argv pointer array with its NULL terminator, the envp pointer array with its NULL terminator, and the auxiliary vector. The vector carries AT_PHDR / AT_PHENT / AT_PHNUM pointing at the executable's program-header table in its mapped image, AT_BASE set to the interpreter's load base, AT_ENTRY set to the executable's entry, the page size, the real/effective uid and gid, AT_HWCAP and AT_HWCAP2 passed through from the host's own getauxval, and AT_SYSINFO_EHDR when the host has a vDSO. The frame is aligned to 16 bytes; rsp on first translated instruction points at argc.

On Darwin, the frame layout reflects LC_MAIN semantics. There is no argc cell on the stack: instead the runtime allocates an 8 MB anonymous stack, writes the argument and environment strings and an executable_path=<path> apple-entry string near the top, lays out three pointer arrays (argv, envp, apple — each NULL-terminated) below them, and seeds x0..x3 with argc, the argv pointer, the envp pointer, and the apple pointer. x30 is initialized to zero. A guest main that returns through ret thus jumps to PC zero, which the dispatcher recognizes as a clean exit with x0 as the status code.

Address space layout

After all of the above, the Chimera process holds the runtime's own image, the guest's executable image (and its dynamic interpreter on Linux), every shared library the loader has brought in, the code cache that holds translated instructions, and two stacks. There is no kernel boundary between any of these regions: everything is one Linux or Darwin process and one virtual address space. The regions are kept disjoint by construction. Chimera's own text and data live where the host kernel placed the PIE Rust executable. The guest's executable lives where its load address (or MAP_FIXED_NOREPLACE reservation) put it. The dynamic interpreter, on Linux, lives at a base Chimera picks well clear of both. Every shared library the interpreter (Linux) or the in-process linker (Darwin) brings in goes through anonymous mmap and ends up in the conventional mmap region the host kernel hands out to ordinary anonymous mappings. The code cache is a separate 16 MB anonymous mmap, allocated when the dispatcher first starts. On Linux x86-64 its page-table permission remains read, write, and execute so the runtime can patch translated branches and self-modifying-code deopts, but the mapping is assigned an x86 protection key and every guest-running thread enters the cache with that key write-disabled in PKRU. The runtime clears the write-disable bit only while it is back on the runtime side emitting or patching code. On Darwin the cache mapping carries MAP_JIT and writes go through pthread_jit_write_protect_np toggles.

high address  ──────────────────────────────────────────────────
              guest stack          (mmap RW, built by Chimera)
              ...
              mmap region:
                guest shared libraries  (loaded by ld.so on Linux,
                                          by Chimera's in-process
                                          linker on Darwin)
                guest interpreter       (Linux only)
                code cache              (mmap RWX, pkey write-disabled in guest)
                Chimera runtime heap
                other anonymous mappings
              ...
              Chimera image        (PIE)
              ...
              guest executable     (fixed address on Linux ET_EXEC;
                                     slid PIE elsewhere)
              ...
              __PAGEZERO           (Darwin only, lowest 4 GB)
low  address  ──────────────────────────────────────────────────

Two stacks coexist. The host thread that ran Chimera's main retains its original kernel-allocated stack, and that is the stack the dispatcher and every helper function in the Rust runtime use. Chimera allocates a second stack for the guest during program loading. Every transition between the two worlds swaps the stack pointer: when translated code exits a block, the trampoline restores the runtime's stack pointer from the per-thread context; when the dispatcher re-enters the cache, the trampoline restores the guest's stack pointer.

Two heaps also coexist, and keeping them apart is a correctness requirement rather than a convenience. On Linux the guest's dynamic interpreter loads a second libc.so.6 into the process, and that libc keeps its own main_arena that would, naively, share the one process-wide program break with whatever libc the runtime itself uses. Two allocators handing out chunks from the same brk segment without knowing about each other's bookkeeping is silent corruption: a chunk one side believes it owns is handed back to the other, and the fault surfaces minutes later as a malloc abort or a wild-pointer dereference. An allocator choice on the runtime side cannot close the hazard by itself — a Rust #[global_allocator] never covers the host libc's internal allocations (opendir, getaddrinfo, and friends call __libc_malloc directly), and those chunks live in the brk-backed arena no matter what the embedder installs. Chimera therefore removes the sharing at the source: the guest's brk is virtualized. At image load the runtime reserves a private break arena — a PROT_NONE, MAP_NORESERVE mapping — and services every guest brk against it, growing pages writable on extension and discarding them on shrink, with the kernel's own contract (an unsatisfiable request returns the current break unchanged). The process's real program break is never moved on the guest's behalf and belongs to the host libc's allocator alone, so guest heap pages can be unmapped at teardown without tearing memory out from under it. The final binaries — the chimera CLI and the example embedders — still install mimalloc as the Rust #[global_allocator], whose mmap-backed segments keep the runtime's own heap traffic off the process's break as well; the global allocator is per-binary and cannot be set from the library crate, so an embedder that wants the same separation installs one of its own. The guest's mmap continues to pass through Chimera's syscall path to the host kernel.

Translated execution

Translation

The unit of work is a single basic block. translate(guest_pc) reads guest bytes starting at that address, emits a contiguous run of host instructions into the code cache, and returns the host program counter at which the block begins. The decoder walks the guest one instruction at a time. The end of a block is the first instruction whose flow_control is not Next: any branch, any call, any return, any system call (which the x86 decoder reports as a Call, recognized separately by opcode).

The body of a block is, with few exceptions, a verbatim copy. On x86-64, Chimera builds the block as an iced_x86::InstructionBlock and lets the encoder lay it out at the cache's next free address; the encoder takes care of RIP-relative displacements that need adjusting to refer to their original guest target. On arm64, Chimera reads guest instruction words directly, copies them through unchanged, and intercepts the three families that carry a PC-relative reference — ADR, ADRP, and load-literal LDR — to rewrite them into a short movz/movk sequence that materializes the original guest target into the destination register. Everything else, including every arithmetic, logical, and memory instruction, is a single word copy.

The terminator is rewritten unconditionally. A direct branch, executed unchanged, would reach the guest's original image, which translated code never enters. The translator replaces it with a stub that computes the next guest program counter (the literal target for an unconditional branch; one of two values selected by the original condition for a conditional branch; the popped return address for RET; the operand for an indirect branch or call), writes that program counter into the context structure's rip/pc slot, and jumps to the common exit trampoline. A syscall or SVC is rewritten the same way except that the exit trampoline sets the context's exit_kind to SYSCALL, signaling the dispatcher loop that the syscall handler must run before the next block is entered. The instruction's architectural side effects on Linux (rcx ← next_rip, r11 ← rflags) are synthesized into the context slots so the guest sees them on resume; on Darwin the carry flag in NZCV carries the success/error distinction and the translator captures it from the handler's return.

Dispatcher and control flow

The dispatcher is not a piece of assembly: it is the Rust run loop. After every block exit it reads the context's pc slot, looks the value up in a HashMap<u64, u64> keyed by guest program counter and yielding host program counter, calls translate and inserts a new entry on a miss, and re-enters the cache. The block transition through the dispatcher involves a context switch in both directions (save the guest GPRs, restore the runtime's; later, save the runtime's, restore the guest's), a hash-table probe, and the assembly trampoline that performs the register-file shuffle. That round-trip is paid only when an edge is first taken or an indirect branch misses, not at every boundary. Once a block's successor is translated, the direct branch that ends the block is patched in place — a single aligned store of its rel32 displacement — to jump straight into the successor; and every indirect branch ends in an inline, cache-resident probe that hashes the guest target, reads its slot in a direct-mapped table, and jumps to the cached host PC on a match or falls through to the dispatcher on a miss. Returns, virtual-call dispatch, and computed branches therefore stay in the cache in the common case.

The trampoline that brackets each cache entry lives in trampoline.S and is shared across all blocks. Its entry path saves Chimera's callee-saved registers on the runtime stack, stashes the runtime's stack pointer in the context, loads the guest's FPU/SIMD state through XRSTOR (x86) or paired LDP q?, q? (arm64), loads the guest GPRs from the context, switches the stack pointer to the guest's, restores the guest's flags, and jumps to the host program counter the dispatcher passed. The reverse path runs at every block exit: each per-block exit stub has already written the next guest program counter and the live rax/x16 into the context, the trampoline saves the rest of the GPRs, the FPU state, and the guest's flags, restores the runtime's flags and stack pointer, pops the callee-saved registers, and returns to the Rust caller of dispatch.

On x86-64 the save and restore use XSAVE/XRSTOR rather than FXSAVE/FXRSTOR. The legacy FXSAVE area covers only x87 and the low 128 bits of the XMM registers; it silently drops the upper 128 bits of every YMM register and all of the ZMM and opmask state. Because every basic-block boundary round-trips through the dispatcher, and the runtime's own Rust code clobbers vector registers in between, an FXSAVE-only trampoline corrupts any guest that keeps live state in the upper lane across a block edge — which glibc's AVX2 string routines do on every loop back-edge, returning wrong results that crash the guest. The trampoline therefore saves the extended state with an XSAVE component mask of 0xe7 (x87, SSE, AVX, and the three AVX-512 components), intersected with the host's XCR0 by the processor, into a 64-byte-aligned area in the context. The first entry restores from a zeroed area, which initializes every component to its architectural reset state with one exception: XRSTOR always reloads MXCSR from the legacy region regardless of the state-bitmap, so the runtime seeds that field with the ABI default 0x1f80 (all SSE exceptions masked) before the first entry, matching the value the kernel gives a fresh process. Thereafter the guest's own MXCSR round-trips through the save area like the rest of its FP state.

Reaching the context structure from translated code requires a base, and on each architecture the base comes from a register the guest is conventionally forbidden to touch. On Linux x86-64, Chimera installs the context pointer as the GS-segment base through arch_prctl(ARCH_SET_GS, ...) at the start of start_thread. The choice is safe because mainstream x86-64 Linux toolchains leave GS unused — thread-local storage lives on FS — and because ARCH_SET_GS/ARCH_GET_GS are intercepted in the passthrough handler and ARCH_SET_FS/ARCH_GET_FS are virtualized through a guest_fs_base slot in the context. The kernel's real FS base stays bound to Chimera's own TLS, so Rust code in the runtime continues to work after the guest has configured FS for itself; the trampoline reloads the guest's FS base on every entry and the runtime's on every exit. All 16 general-purpose registers stay available to translated code.

On Darwin arm64, Chimera uses x18. Apple's platform ABI reserves x18 for the operating system, and guest code emitted by every supported toolchain treats it as unusable. Translated code reaches the context with plain ldr/str instructions whose base is x18 and whose displacement is the field offset; no syscall-shaped mechanism is required to install the pointer because the trampoline writes x18 from the CHIMERA_CTX_PTR global on every entry. The per-block prologue re-syncs the guest's x16 from the context, since x16 is the register the exit-trampoline machinery uses to carry its branch target and is therefore clobbered across every cache entry.

Runtime services

System calls

Every guest system call passes through Chimera. The translator rewrites each syscall / svc instruction into a sequence that writes the next guest program counter into the context, materializes the side effects the instruction would have had (rcx and r11 on x86, NZCV on arm64), and jumps to a syscall-exit trampoline whose only difference from the ordinary block-exit path is that it sets exit_kind = SYSCALL. The Rust run loop sees that exit_kind, packages the syscall number and the six argument registers into a SystemCall value, hands it to the active handler, writes the handler's reported return value into the context's rax/x0 slot, reflects the handler's error flag into the carry bit of the context's NZCV slot on Darwin (where libc consults it), and resumes the guest at the instruction immediately following the rewritten syscall.

The handler the CLI uses, and the default an chimera::Sandbox starts with, is Passthrough. It forwards every call to the host kernel verbatim with five classes of exception that the runtime needs for its own correctness. On Linux, arch_prctl(ARCH_SET_FS, ...) records the guest's requested FS base into the context and returns success without touching the kernel; ARCH_GET_FS reads it back from the context; ARCH_SET_GS and ARCH_GET_GS return -EINVAL because GS is the runtime's context register. exit and exit_group are intercepted: forwarding them would terminate Chimera itself, so they no-op and the run loop captures the requested code from the syscall's first argument and ends the run cleanly after the handler returns. execve and execveat are refused with -EPERM and the attempt is logged: forwarding either would have the host kernel discard the entire process image — runtime, code cache, and translation map included — and resume at the new program's entry point natively, so the replacement program would run with no translation and no sandbox at all. Denying the call is a stop-gap; a complete implementation would intercept the exec and re-enter Chimera on the new image rather than let the kernel hand control to untranslated code. On Darwin, BSD exit (syscall #1) is intercepted for the same reason; thread_set_tsd_base (the per-thread TPIDRRO_EL0 update) is intercepted because forwarding it would clobber the same register Chimera's Rust runtime uses to find its own pthread state. The pc == 0 sentinel that follows a main returning to a NULL link register is treated by the run loop as a clean exit with x0 as the status code.

A second group of calls is runtime-owned in a different sense: the runtime forwards them to the host kernel itself rather than handing them to the handler, because their effects must stay consistent with state the runtime maintains. The memory-management calls mmap, munmap, and mremap mutate the table of guest mappings Chimera keeps so it can tear every one of them down when the address space is reset or dropped, so the runtime services them and updates the table from the kernel's own result. The protection calls mprotect and pkey_mprotect join them for a related reason: Chimera holds a write-xor-execute invariant over the guest, which never executes its own pages natively — the dispatcher reads guest bytes and runs translated blocks from the code cache instead — so a guest page must never carry PROT_EXEC in the host page tables. Before forwarding any call in this group the runtime clears PROT_EXEC from the prot argument, substituting PROT_READ so the translator can still read the bytes it will translate and an execute-only request does not collapse to PROT_NONE. Forwarding the rewritten call from the runtime rather than the handler is what makes the invariant hold unconditionally: the guest can never reach the kernel with PROT_EXEC set on its own pages, and a handler cannot deny a memory-management call the guest needs to run. The same invariant is why personality requests carrying READ_IMPLIES_EXEC are refused — the persona would have the kernel add PROT_EXEC to every readable mapping, undoing the stripping — and why calls that could change the bytes behind an already-translated guest program counter or queue kernel work outside this path (remap_file_pages, userfaultfd, process_vm_writev, ptrace, the io_uring family, shmat/shmdt) are refused outright. Handlers still observe every call in the runtime-owned group, but they observe it rather than service it.

A program using the library can replace Passthrough with anything implementing the SystemCalls trait (or its C function-pointer equivalent): a handler that returns -EPERM for every call produces a sealed sandbox, a handler that wraps each call for logging produces a system-call tracer (the bundled strace example), a handler can synthesize answers without ever touching the host kernel. Whatever the handler does, the runtime's intercepts above still apply — they belong to the runtime, not to the handler.

Threads

A guest thread is a host operating-system thread. When the guest issues clone with CLONE_VM — the flag pthread_create sets to share an address space — Chimera cannot forward it: the host kernel would start a task that runs the guest's code natively, with no context register bound, no per-thread ThreadState, and no path through the translator. The runtime intercepts the call and spawns the host thread itself, handing it a fresh ThreadState cloned from the parent's register file (with rax zeroed and the child stack installed) and a clone of the Arc that names the shared process state. That host thread runs the same translate-execute loop as the initial thread, over the same code cache and address space. The model is NPTL's, mirrored directly: every guest thread is a kernel scheduling entity in one thread group, and the calls that act on threads — futex, gettid, getpid, tgkill, set_robust_list, sched_* — forward to the host kernel unchanged, because Chimera and the guest are one process sharing one address space.

The state a thread does not share lives in its ThreadState: the guest register file, the 64-byte-aligned extended FP/SIMD save area, and the scratch slots the trampoline and the inline indirect-branch lookup borrow. Each host thread reaches its own ThreadState through the context register, which on Linux x86-64 is the GS base, bound per host thread by an arch_prctl(ARCH_SET_GS, ...) the run loop issues before it enters the cache. The guest's own thread pointer is virtualized alongside it: CLONE_SETTLS seeds the child's guest_fs_base, and the trampoline reloads the FS base on every entry and the runtime's on every exit, so __thread storage and errno are genuinely per-thread while the kernel's real FS base stays bound to the runtime's TLS. Everything else — the guest mappings, the translated-code cache, and the embedder's syscall handler — is shared through one Process, the analogue of the kernel's mm_struct, that every thread holds by Arc.

Replicating the kernel's thread-creation side effects in the right order is load-bearing. The kernel writes the new task's thread ID into the CLONE_PARENT_SETTID/CLONE_CHILD_SETTID words before the child runs a single instruction; glibc points those words at its thread-control-block tid field and reads it early, using it as the thread's identity for, among other things, pthread_rwlock writer ownership. Chimera therefore performs these writes from the child, with the child's own gettid, before it enters the cache — not from the parent after the child is already running, which would let a fresh thread store a stale identity and later mistake itself for another. The matching teardown is CLONE_CHILD_CLEARTID: when a thread's run loop ends, the runtime zeroes the registered clear-tid word and issues a non-private FUTEX_WAKE on it, exactly as the kernel does, which is the primitive pthread_join blocks on.

Synchronization needs no per-primitive emulation. A guest mutex, condition variable, barrier, pthread_once, or semaphore reduces to a lock word at a real virtual address and a futex wait or wake on it, and both forward verbatim — the waiters are real host threads parked in the kernel. The atomic operations on those words are same-ISA: the translator copies a lock-prefixed cmpxchg or xadd through unchanged, LOCK prefix intact, so it is as atomic in the cache as in the guest. What the shared cache does require is that its own bookkeeping be safe for concurrent readers. Translation and insertion are serialized under the address-space lock, so only one thread emits a block at a time and the hot path of already-translated blocks runs lock-free. A direct branch is linked to its successor by a single aligned atomic store of the rel32 displacement, so a sibling executing the site reads the old target or the new one but never a torn splice. The inline indirect-branch table, the one lock-free structure a reader touches on every indirect branch, publishes each {guest_pc, host_pc} slot torn-read-safe — the writer invalidates the key, writes the host PC, then writes the key, and the reader re-checks the key after loading the host PC — so a slot republished concurrently can never send a branch into the wrong block.

Thread and process exit diverge the way the kernel's do. A thread-local exit ends only the calling thread: its run loop returns and its host thread terminates while the rest of the process runs on, which is the pthread_exit path. exit_group ends the whole thread group from whichever thread issues it; the runtime publishes the request and the status code on the shared Process, every thread observes it at its next block or syscall boundary and stops, and a thread parked in a blocking host syscall is interrupted with a reserved host signal so it reaches that boundary rather than waiting on a call that may never return. The initial thread is special only in that its run returning ends the process, so when it exits on its own through pthread_exit while siblings are still alive it waits for the last of them before returning — the process lives until the final thread terminates, as POSIX requires.

Signals

Signal virtualization is not yet implemented. The runtime installs no host-side signal handler of its own. A synchronous fault inside translated code — a guest SIGSEGV from a wild pointer, a SIGFPE from a divide-by-zero — propagates as a host process error against host execution state, with the program counter pointing into the code cache rather than the guest's image, and is not routed through any handler the guest may have registered through rt_sigaction or Darwin's Mach-exception interface. Asynchronous signals are not delivered to guest handlers either. The translator and dispatcher leave the structural room for a signal subsystem (per-block metadata that lets a faulting host PC be mapped back to a guest PC, a context structure ready to be repopulated for a synthesized ucontext_t), but the subsystem itself remains to be written. Future work lists what that work entails.

Future work

Two improvements not in the current design would meaningfully change Chimera's performance profile.

The most consequential is preserving the host CPU's return-address-stack prediction. The translator treats ret as a special case of an indirect branch, routing every return through the inline indirect-branch probe instead of a native ret — which keeps it in the cache but still defeats the processor's return-address-stack predictor, since the host instruction is an indirect jmp rather than a ret. Same-ISA does not require the rewrite: if every guest instruction maps to a host instruction at a known offset, the address a translated call pushes is already the host PC of the post-call instruction, and a native ret jumps right back into the cache. The cost is transparency. Guest code that reads its own return address off the stack — backtracers, setjmp/longjmp, custom unwinders — sees host PCs instead of guest PCs and requires translation-aware tooling. With direct-branch patching and the inline indirect-branch probe already in place, this is the remaining lever that brings overhead from "tens of times native" toward the 10–30% range mature implementations report.

The second is hybrid AOT translation. The current design is pure JIT: translate is invoked on a dispatcher miss, one block at a time. The executable's text is fully known at load time, however, and could be translated up front. AOT eliminates the first-execution dispatcher round-trip for every block reachable from _start and lets translations be laid out in original source order, friendlier to the instruction cache than the dispatcher-miss order a pure JIT produces. The interpreter (on Linux), the C library, anything dlopen'd, and any code the guest generates itself would still need JIT, so AOT is an addition rather than a replacement.

Signal virtualization, the omission noted in Runtime services, is the third open item. The minimum useful subsystem installs a Chimera host-side handler for every signal the guest might want, records the guest's rt_sigaction choices in a per-process table, synthesizes a guest ucontext_t from the host one by mapping the interrupted host PC back to a guest PC through per-block translation metadata, defers asynchronous signals to the next block boundary, and rewrites rt_sigreturn (and Darwin's _sigtramp) as a system call the runtime recognizes. The Rosetta 2 design that motivates the return-address-stack item also informs this one and is described by Johnson.

References

Derek Bruening, Qin Zhao, & Saman Amarasinghe (2012). Transparent dynamic instrumentation. In VEE '12.

Guillermo Callaghan, Cosmin Gorgovan, & Mikel Luján (2020). Optimising dynamic binary modification across 64-bit Arm microarchitectures. In VEE '20. https://doi.org/10.1145/3381052.3381322

Dougall Johnson (2022). Why is Rosetta 2 fast? https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/

Koh M. Nakagawa (2021). Project Champollion: Reverse engineering Rosetta 2 (Version 0.1.0) [Computer software]. https://github.com/FFRI/ProjectChampollion