line tables and backtraces: port agent/line-tables onto main, op 56 on the x86-64 jit, docs, measurements, tests; default genome reservation 0x68000

1 files changed, 23 insertions, 1 deletions

diff --git a/TODO.md b/TODO.md
index 210b4a3..3b6e62d 100644
--- a/TODO.md
+++ b/TODO.md
@@ -324,6 +324,28 @@ of the open list.
   the frame window (top 24 plus last 4) prints "... N more frames" with a raw
   word count. merge note: main has the x86-64 jit since this branched; op 56
   must also work there (it reads `cb_r`, check the x64 hook sets it).
+  resumed on work/line-tables (ported onto main fa35813): done. the x86-64 jit
+  needed no change: its store hook publishes s and r into S before a gate
+  store, so `cb_r` is exact there (comment at `hook` in core.c);
+  `tests/programs/trace_jit.zc` traces from translated code and agrees on zy,
+  zyi and zym, also with ZYNOO3=1. docs: README "line tables and backtraces"
+  (after memory debug), SPEC 5.15, 8.3 (`rt.trace`), 8.4 and appendix a.
+  measurements: genome 388100 words at fa35813, 395477 with this work (+7377,
+  +1.9%) with lines off, 410937 (+3.9% more) with lines on; run-only images
+  grow by their tables (`trace_run.zc` 31051 to 31119 words, `gc.zc` 30822 to
+  30857); `examples/mandel.zc` compiles and runs in about 22 ms on the x86-64
+  jit with lines on, off, and on main, and the genome compiles itself in
+  about 1.2 s either way (no measurable difference). the genome no longer fit
+  the default reservation (0x60000 words), so `preset_default` in genmap.c now
+  reserves 0x68000 and the map constants in rt.zc moved by 0x8000 (README map
+  table updated). side effect: the heap starts higher, where many genome code
+  words look like heap addresses to the conservative collector (it scans
+  GORG..GORG[6], code included), so `tests/programs/gc.zc` freed 969 of 1000
+  instead of 998; its threshold is now 950. `make parity` puts `//:lines
+  <stdin>` before each program, so zc0 images trace like zy. not checked: the
+  javascript vm on the final build (no tsc on the box), the full suite.
+  open: scan only the genome's globals as roots, not its code; whether the
+  genome should ever carry lines (+15460 words).
 - bounds checking with `unbounded` (`agents/bounds-checking`, 2 commits, last
   46aa911). done, builds to the fixpoint: zc0 accepts and ignores `unbounded`
   in all three positions; the compiler has the prefix (any order with
@@ -470,7 +492,7 @@ brief someone without re-reading the code.
 - [ ] (partly done, see "agent work at the end of the session") bounds checking on by default, `unbounded` to turn it off (function prefix, block, top-level `unbounded;`, mirroring `unchecked`), implied by `o3`. architecture: indexing is compiled in the postfix `[` handling of the expressions part; growable arrays already check (`dat`, `range_jz`, the `outside` enhancement, `proven()` removing checks a loop test already did). fixed arrays `T[n]` with a variable index get the same inline check against the constant `n`. pointers get a runtime call (for example `rt.bchk(p, i)`) that checks only when `p` is the start of a live heap block (header check word `p ^ MAGIC`, `paylen`), skipping constant index 0 and sizes the analyzer knows (`lnsz`). reports go through `oob`. the runtime, core, compiler parts, server, `run.zc` and `build.zc` start with `unbounded;`, which zc0 accepts and ignores (genome bytes unchanged). keep std, icon and actor checked unless a function touches raw memory on purpose. measure interpreter time before and after.
 - [ ] after the fuzzing work, add more general functionality to the standard library.
 - [ ] (partly done, see "agent work at the end of the session") struct destructors: a struct may declare `fn drop()`, called wherever the compiler frees an owned value of that type, before the memory is released. architecture: the compiler frees owned locals in `freevar` (analyzer part), called from `exit_vars`, reassignment and explicit `rt.free`/`rt.recycle` calls (`s_free`, `s_recycle` handling in the call code); when the local's static type is a pointer to a struct with a `Name.drop` method, emit a call to it first. objects freed by the collector or inside unchecked code do not run `drop`: document that. first users: file descriptors in `std.Host`, processes in `actor.zc`.
-- [ ] (partly done, see "agent work at the end of the session") runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing `  at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it.
+- [x] (done on work/line-tables, see "agent work at the end of the session") runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing `  at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it.
 - [ ] borrow returns tied to an argument. inference already works for a callee defined before the call: `char* skip(char* s) { ...; return s + i; }` makes `let r = skip(t); rt.free(t); r[0]` an error. the gap: when the call comes before the callee's definition, nothing is reported and the program reads freed memory. extend the deferred effect requirements (`effect_require`, `effect_finish` at unit end) so a forward-called function whose return turns out to alias a parameter produces the same error. for bodyless declarations, externs and function types, add an unobtrusive spelling, for example a `borrows s` clause after the parameter list (`char* skip(char* s) borrows s;`), checked against the body when one exists.
 - [ ] variadic arguments and `std.print`. design: a pack is passed as one hidden argument, a caller-built read-only growable-array handle (`fixed T[?]` view: [len, cap, items, element size] in the caller's frame, so `len(a)` and `a[i]` work unchanged and nothing is allocated). `...int a` is a typed pack; several packs per function when their element types differ (`void f(...int a, ...char b)`): each extra argument goes to the pack whose element type matches its static type, an ambiguous match is an error, packs come after the fixed parameters. `...any args` packs each value with a kind code (int, float, char, string, bstr, pointer, ...) and its type record: `any` is a small built-in record { kind, value, type }. `...*any args` makes the call comma-free: after the fixed arguments, the parser reads unary expressions until `)`, commas optional, so `std.print("x = " name ", age " age "\n");` works and binary expressions must be parenthesized. `std.print(...*any args)` prints each argument by its kind (`put_n`, the float printer, `put_s`, `put_bs`, a character for `char`, hex for pointers).
 - [ ] x86-64 jit phase 2: compile o3 regions to x86-64 (port `rcompile` and the region register allocation; regions now run on the region interpreter and are 2 to 4 times slower than their plain versions on this jit) and inline enhancements (lower the optimized micro-ops to x86 registers as the arm64 path does, instead of calling `xrun`). validate with `tests/drivers/jit_x64_test.sh` and `tests/bench/bench.sh`, which now has an x86-64 jit column. add the driver to `tests/test.sh` on x86-64 linux.