x86-64 jit: inline new_k, recycle_leaf and own fast paths, in o3 regions and plain blocks

a call to rt.new_k / rt.recycle_leaf with a constant class, or to rt.own,
does its class-list pop or push (or owner store) inline and goes on after
the call; an empty list, memdebug, a held range, a block outside memory, a
translated word or a type/class mismatch takes the ordinary call. in a
region this replaces the o3_invoke round trip. o3 frees of leaf objects now
use recycle_leaf too (new_k stamps the type inside o3 since b86db60).
ZYNOFASTALLOC=1 turns it off; ZYPERFMAP=1 writes a perf map of translated code.
tests/programs/o3_freelist.zc: reuse, zeroing, an empty list, untyped
neighbours, memdebug; jit vs ZYNOFASTALLOC vs ZYO3INTERP.

1 files changed, 1 insertions, 1 deletions

diff --git a/README.md b/README.md
index 839eac2..6a56945 100644
--- a/README.md
+++ b/README.md
@@ -590,7 +590,7 @@ calls push the arguments left to right and the callee's prologue pops them into
 
 registers: x19 holds the memory base, x20 the data stack pointer, x21 the return stack pointer, x22 the translation table and x23 the state block (memory, s, r, table, coverage map, Q, G, &J, store hook, install hook, enhancement fallback). o3 regions also use x26..x28, which the entry stub saves.
 
-the x86-64 jit (linux hosts, `ZYJIT_X64`) keeps the same state in callee-saved registers: r12 is the memory base, r13 the data stack pointer, r14 the return stack pointer, r15 the translation table and rbx the state block, with rax, rcx, rdx, rsi, rdi and r8..r11 as scratch and rsp 16-byte aligned inside translated code so calls into c are plain system v calls. it shares everything but the emitter with the arm64 jit: the translation table, the coverage map, the store hook, the dropped-translation bookkeeping, the flush, the block rules below and the dispatch stubs' policy. inside a block it does not move r13 for every push and pop: the translator counts the words pushed and popped and addresses stack slots from r13 with that offset, and brings r13 up to date before a call into c and at every exit. it translates the base opcodes, the float and byte opcodes (the byte loads inline, the byte stores through a c helper), install, and the c abi call; every installed enhancement is a call to the interpreter's evaluator for its slot (`xrun`). o3 regions are compiled to x86-64 too, but not as on arm64: there are too few registers to keep the vars in them, so each var lives in its home frame slot (rbp points at the frame, reloaded at every entry), and a block's values get the six registers rsi, rdi, r8..r11. the vars a block changes are written home at its end, and the other slots that share a var get it at every exit. calls are plain vm calls through the dispatch stub, and ops the lowering lacks (floats, popcount) call the evaluator. a region that needs more than six live values, or makes native calls, runs on the region interpreter, entered from the c loop; `ZYO3INTERP=1` puts every region there, and `ZYNOO3=1` leaves regions alone and runs their vm code through the jit, as the microcore does.
+the x86-64 jit (linux hosts, `ZYJIT_X64`) keeps the same state in callee-saved registers: r12 is the memory base, r13 the data stack pointer, r14 the return stack pointer, r15 the translation table and rbx the state block, with rax, rcx, rdx, rsi, rdi and r8..r11 as scratch and rsp 16-byte aligned inside translated code so calls into c are plain system v calls. it shares everything but the emitter with the arm64 jit: the translation table, the coverage map, the store hook, the dropped-translation bookkeeping, the flush, the block rules below and the dispatch stubs' policy. inside a block it does not move r13 for every push and pop: the translator counts the words pushed and popped and addresses stack slots from r13 with that offset, and brings r13 up to date before a call into c and at every exit. it translates the base opcodes, the float and byte opcodes (the byte loads inline, the byte stores through a c helper), install, and the c abi call; every installed enhancement is a call to the interpreter's evaluator for its slot (`xrun`). o3 regions are compiled to x86-64 too, but not as on arm64: there are too few registers to keep the vars in them, so each var lives in its home frame slot (rbp points at the frame, reloaded at every entry), and a block's values get the six registers rsi, rdi, r8..r11. the vars a block changes are written home at its end, and the other slots that share a var get it at every exit. calls are plain vm calls through the dispatch stub, and ops the lowering lacks (floats, popcount) call the evaluator. a region that needs more than six live values, or makes native calls, runs on the region interpreter, entered from the c loop; `ZYO3INTERP=1` puts every region there, and `ZYNOO3=1` leaves regions alone and runs their vm code through the jit, as the microcore does. calls to `rt.new_k` and `rt.recycle_leaf` with a constant size class, and to `rt.own`, are inlined in both plain blocks and compiled regions: the jit pops or pushes the class list itself and goes on after the call, and takes the ordinary call for anything unusual (an empty list, memdebug, a held range, a word it has translated, a type or class that does not match). the runtime hands over the addresses it needs with its stack registration (`frame_control[17..21]`, `SC_FAST` in `src/core/stack.h`); `ZYNOFASTALLOC=1` turns the inlining off. `ZYPERFMAP=1` writes `/tmp/perf-<pid>.map`, so `perf report` names translated code by its vm address range (`zc_<first>_<last>`, `zc_o3_<lo>_<hi>`).
 
 compilation is lazy. the first time control reaches an address, that block is translated from live memory. a block ends at call, jump, ret or halt. a `z` doesn't end it: the not-taken side keeps compiling, which makes superblocks. a literal followed by `c`, `j` or `z` is fused into a constant target. every exit goes through a dispatch stub that checks both stack pointers against their limits (kept in x24 and x25), bounds-checks the target, loads `tab[t]`, and branches straight to it when it's already compiled. otherwise it returns to c, which compiles the block and re-enters. enhancements are inlined: inputs load from the data stack into x0..x15, the optimized micro-ops run in registers with x16 and x17 as scratch, and the outputs go back. an install calls into c and invalidates translations and o3 regions that depend on its slot. invalidated code is rebuilt on its next entry. when the loop reaches an o3 entry point, it compiles the region (or reuses it) instead of translating a block.