Merge the x86-64 jit, phase 1; todo: tick it, add phase 2

1 files changed, 7 insertions, 5 deletions

diff --git a/README.md b/README.md
index ed16c42..4537188 100644
--- a/README.md
+++ b/README.md
@@ -1,12 +1,12 @@
 # tinyvm zygote
 
-a small stack cpu began as a C89 code golf entry and became a self-hosting runtime. a native core runs vm code with an arm64 jit, through llvm on supported builds, or in the interpreter. the genome contains a compiler for zc, a C-like language with forth influences. the compiler resides in vm memory alongside the program; code can call its functions, modify its tables, add syntax, and compile new code at runtime. heap objects carry an ownership tag (u, m or d) checked by a static analyzer, with deterministic frees at scope ends and a mark-sweep collector underneath. types are a compiler primitive, with structs, generics, reflection and an optional typechecker. programs can install up to 512 custom opcodes (virtual enhancements), which the jits inline as native code. code marked `o3` bypasses the stack machine: the core compiles the region as a unit and keeps its variables in registers.
+a small stack cpu began as a C89 code golf entry and became a self-hosting runtime. a native core runs vm code with an arm64 or x86-64 jit, through llvm on supported builds, or in the interpreter. the genome contains a compiler for zc, a C-like language with forth influences. the compiler resides in vm memory alongside the program; code can call its functions, modify its tables, add syntax, and compile new code at runtime. heap objects carry an ownership tag (u, m or d) checked by a static analyzer, with deterministic frees at scope ends and a mark-sweep collector underneath. types are a compiler primitive, with structs, generics, reflection and an optional typechecker. programs can install up to 512 custom opcodes (virtual enhancements), which the jits inline as native code. code marked `o3` bypasses the stack machine: the core compiles the region as a unit and keeps its variables in registers.
 
 this folder has the latest code. `docs/SPEC.md` is the language reference, and `docs/LEARN.md` is a tour of the language. `docs/history/` is the scrapbook: notes for every stage, plus frozen code from the stages that still have it, archived as `docs/history/code-snapshots.tar.gz`.
 
 ## files
 
-`src/core/core.c` is the native core. it contains the arm64 jit, the llvm backend (behind `-DZYLLVM`), the portable interpreter, the enhancement unit (recipe parser, vm-code lifter, optimizer and lowering), and the o3 region compiler. `src/runtime/rt.zc` provides i/o, native division enhancements, the heap, and the collector; the compiler is the files of `src/compiler/`, named in order by `src/compiler/parts` (the lexer, types, expressions, statements, analyzer, declarations, check mode and image building each have a file; `base.zc` begins with the overview). all are written in zc. together they make up the genome. `src/tools/zc0.c` is stage 0 of the bootstrap, a c compiler for the subset of zc used by the compiler. `src/tools/build.zc` has the resident compiler compile its own source into a fresh image, and `src/tools/mkgenome.c` checks that image against the memory map before it becomes `build/core/genome.bin`. an image is a file of little-endian 64-bit words, and every core takes its genome in at compile time with c23 `#embed` (`src/core/genome.h`). `examples/` contains small programs (hello, factorial, echo, self-patching code, an owning list, a repl, a typed program, and an enhancement demo) and larger proofs of concept:
+`src/core/core.c` is the native core. it contains the arm64 jit, the x86-64 jit, the llvm backend (behind `-DZYLLVM`), the portable interpreter, the enhancement unit (recipe parser, vm-code lifter, optimizer and lowering), and the o3 region compiler. `src/runtime/rt.zc` provides i/o, native division enhancements, the heap, and the collector; the compiler is the files of `src/compiler/`, named in order by `src/compiler/parts` (the lexer, types, expressions, statements, analyzer, declarations, check mode and image building each have a file; `base.zc` begins with the overview). all are written in zc. together they make up the genome. `src/tools/zc0.c` is stage 0 of the bootstrap, a c compiler for the subset of zc used by the compiler. `src/tools/build.zc` has the resident compiler compile its own source into a fresh image, and `src/tools/mkgenome.c` checks that image against the memory map before it becomes `build/core/genome.bin`. an image is a file of little-endian 64-bit words, and every core takes its genome in at compile time with c23 `#embed` (`src/core/genome.h`). `examples/` contains small programs (hello, factorial, echo, self-patching code, an owning list, a repl, a typed program, and an enhancement demo) and larger proofs of concept:
 
 - `mandel.zc`: the mandelbrot set in 16.16 fixed point, with the fixed-point multiply wired in as a custom opcode.
 - `bf.zc`: a brainfuck compiler. it turns brainfuck into zc source, has the resident compiler compile it, and runs it (native code on the jit). it takes a program after a backtick, or runs hello world.
@@ -72,7 +72,7 @@ windows needs clang or gcc and powershell for the tests. the batch file is the s
 
 ### native backends
 
-on arm64, `src/core/core.c` uses its jit by default. other hosts use the interpreter. `-DZYINTERP` forces interpretation; `-DZYLLVM` enables the llvm backend when its headers and libraries are available. direct builds from the root are:
+on arm64, `src/core/core.c` uses its arm64 jit by default, and on x86-64 linux its x86-64 jit. other hosts use the interpreter. `-DZYINTERP` forces interpretation (`make build/bin/zyi` builds that core beside `zy`); `-DZYLLVM` enables the llvm backend when its headers and libraries are available. direct builds from the root are:
 
 ```sh
 make genome
@@ -434,7 +434,7 @@ going into o3 is free: o3 code reads normal objects the same way and ignores the
 
 ## the llvm backend
 
-a tiered jit for hosts the arm64 backend doesn't cover. the interpreter counts how often each block start gets reached, and a hot block is translated to llvm ir using the same block rules as the arm64 jit: it ends at call, jump, ret or halt, keeps going past a jz, and literal targets are fused. the ir goes through the O2 pipeline and gets compiled by orc's lljit. stores into compiled words drop the blocks covering them, like on arm64. it pays off for long-running code: a loop-and-recursion benchmark runs about 3.3x faster than the interpreter. short bursty work like the self-compile is slower, since compile time dominates there. `ZYHOT=1` compiles every block, which is a good stress test, and the test suite runs that way too.
+a tiered jit for hosts the native jits don't cover, or where it is asked for explicitly (`-DZYLLVM` replaces the x86-64 jit). the interpreter counts how often each block start gets reached, and a hot block is translated to llvm ir using the same block rules as the arm64 jit: it ends at call, jump, ret or halt, keeps going past a jz, and literal targets are fused. the ir goes through the O2 pipeline and gets compiled by orc's lljit. stores into compiled words drop the blocks covering them, like on arm64. it pays off for long-running code: a loop-and-recursion benchmark runs about 3.3x faster than the interpreter. short bursty work like the self-compile is slower, since compile time dominates there. `ZYHOT=1` compiles every block, which is a good stress test, and the test suite runs that way too.
 
 ## the heap
 
@@ -444,7 +444,7 @@ the heap is a buddy allocator over [HEAP_LO, top), where top starts at HEAP_HI a
 
 ## c abi
 
-zc can call native c functions and hand zc functions to native code. it is built on the syscall gate (ops 14 to 17), so it needs a host with `dlsym`: the interpreter, the llvm backend and the arm64 jit on linux and macos (the jit has to be a dynamic binary, since a static one has no `dlsym`). the microcore, the javascript vm, the web build and the windows scripts answer -1, and a native call there gives -1 and nothing else.
+zc can call native c functions and hand zc functions to native code. it is built on the syscall gate (ops 14 to 17), so it needs a host with `dlsym`: the interpreter, the llvm backend and the arm64 and x86-64 jits on linux and macos (the jit has to be a dynamic binary, since a static one has no `dlsym`). the microcore, the javascript vm, the web build and the windows scripts answer -1, and a native call there gives -1 and nothing else.
 
 ```
 include rt;
@@ -569,6 +569,8 @@ calls push the arguments left to right and the callee's prologue pops them into
 
 registers: x19 holds the memory base, x20 the data stack pointer, x21 the return stack pointer, x22 the translation table and x23 the state block (memory, s, r, table, coverage map, Q, G, &J, store hook, install hook, enhancement fallback). o3 regions also use x26..x28, which the entry stub saves.
 
+the x86-64 jit (linux hosts, `ZYJIT_X64`) keeps the same state in callee-saved registers: r12 is the memory base, r13 the data stack pointer, r14 the return stack pointer, r15 the translation table and rbx the state block, with rax, rcx, rdx, rsi, rdi and r8..r11 as scratch and rsp 16-byte aligned inside translated code so calls into c are plain system v calls. it shares everything but the emitter with the arm64 jit: the translation table, the coverage map, the store hook, the dropped-translation bookkeeping, the flush, the block rules below and the dispatch stubs' policy. inside a block it does not move r13 for every push and pop: the translator counts the words pushed and popped and addresses stack slots from r13 with that offset, and brings r13 up to date before a call into c and at every exit. it translates the base opcodes, the float and byte opcodes (the byte loads inline, the byte stores through a c helper), install, and the c abi call; every installed enhancement is a call to the interpreter's evaluator for its slot (`xrun`). o3 regions are not compiled to x86-64: a region entry is never chained to, so control reaches it through the c loop, which runs the region on the region interpreter exactly as the interpreter build does, and a store from either side into the other's code drops what it should (the regions mark their words in the same coverage map). the region interpreter is slower than the jit's own code for the same words (the `bench.sh` o3 kernels take 2 to 4 times longer than their plain versions on this jit, where they are 3 to 8 times faster on the interpreter), so `ZYNOO3=1` leaves regions alone and runs their vm code through the jit, as the microcore does, until a region compiler for x86-64 exists.
+
 compilation is lazy. the first time control reaches an address, that block is translated from live memory. a block ends at call, jump, ret or halt. a `z` doesn't end it: the not-taken side keeps compiling, which makes superblocks. a literal followed by `c`, `j` or `z` is fused into a constant target. every exit goes through a dispatch stub that checks both stack pointers against their limits (kept in x24 and x25), bounds-checks the target, loads `tab[t]`, and branches straight to it when it's already compiled. otherwise it returns to c, which compiles the block and re-enters. enhancements are inlined: inputs load from the data stack into x0..x15, the optimized micro-ops run in registers with x16 and x17 as scratch, and the outputs go back. an install calls into c and invalidates translations and o3 regions that depend on its slot. invalidated code is rebuilt on its next entry. when the loop reaches an o3 entry point, it compiles the region (or reuses it) instead of translating a block.
 
 self-modification is handled with a counted coverage map: one byte per word, the number of live translations that cover it, sticky at 255. a store checks that byte inline: an uncovered target, the usual case, gets a plain store. a covered target calls the hook, which looks the word up in a page index (64-word pages, each with a list of the blocks that touch it) and drops only the translations containing that word, decrementing the coverage of just their words. the jit code then returns to c at the next instruction, so even patches to the very next word take effect. writing back the same value is a no-op. installing an enhancement does not flush: every translation records which enhancement slots it inlined, and an install drops only those (and the o3 regions that depend on the slot). if the 16 mb code buffer or the block index fills, everything is flushed. on llvm each block has its own resource tracker, so dropped code is freed once no compiled code is running.