Merge branch 'main' into work/spec

1 files changed, 29 insertions, 2 deletions

diff --git a/TODO.md b/TODO.md
index 7e21881..4635ef0 100644
--- a/TODO.md
+++ b/TODO.md
@@ -378,6 +378,33 @@ of the open list.
   scratchpad `base/`, not in the zip); `make parity`. the 35% fixed-array cost
   is worth a look before merging (hoist checks out of loops whose bound is the
   array length, extend `proven`).
+  resumed on work/bounds (ported onto main f75ce6a, conflicts in base, decl
+  and expr; the context save is 50 words, `fbnd` at 49 after `dfr_base`).
+  done: o3call's main is `unbounded`; `o3_bounds` and `bounds_hi` now expect
+  no check (o3 implies unbounded); new boundsrules, bounds_fixed, bounds_heap,
+  zc0unbounded (also a run-only image) with test.sh lines; bounds_fixed and
+  bounds_heap are in parity.known; docs in README ("bounds checks"), SPEC
+  2.6.1, 3.7.4, 3.13.4, 4.10.1, new 4.15, 5.1.1 and the keyword table, the
+  server's `kwtext` and `iskeyword`, the vscode grammar. genome 395477 to
+  399590 words (+4113, +1.0%); the self-compile takes about 1.1 to 1.2 s
+  before and after. the fixed-array cost: the `for (i < K)` proof never fired
+  (the loop shape needed a local on the right of `<`), fixed now, so a
+  constant-bound loop costs nothing; the inline check is now plain ops
+  (`i | (n - 1 - i)` < 0) instead of the `outside` enhancement, which the
+  x86-64 jit calls rather than inlines. best of 3, ms, main / port / final:
+  hash-index kernel jit 37 / 72 / 44, interpreter 412 / 504 / 542; loop with a
+  variable bound jit 55 / 113 / 69, interpreter 741 / 907 / 865; loop with a
+  constant bound jit 46 / 100 / 53, interpreter 663 / 822 / 690. the plain-op
+  check is slightly slower than the enhancement on the interpreter for an
+  unprovable index, and probably on arm64, where enhancements are inlined (not
+  measured here). still open: pointers, jit 52 to 279 ms and interpreter 710
+  to 1141 ms on a 256-word sweep (`rt.bnd` is an enhancement call on x86-64);
+  a variable loop bound is not hoisted (needs a loop-entry test of `n <= N`, or
+  of the heap block's payload for a pointer, and a flag the body tests).
+  inlining enhancements in the x86-64 jit would help both. also fixed here:
+  four conformance expectations on main lacked the backtrace line that line
+  tables print (3-expressions/dyn_array_*), and test.sh's run-only gc sed
+  still looked for `f >= 990`.
 - spec conformance (`agents/spec-conformance`, last 808886b; the first,
   interrupted attempt is `agents/spec-conformance-first-attempt`, already
   folded in). done: sections 1 source (114 programs, zy 103 ok), 2 types
@@ -487,14 +514,14 @@ brief someone without re-reading the code.
 - [ ] continue removing `unchecked` from the runtime. a reverted trial showed: `copy_w`/`fill_w` pass the analyzer when their gate stores go through `syscall(SYS_COPY, ...)`, `dgrow` with `own(w, it)` instead of `w[-2] = it`, `dpush` with `own` before the store, `dnew` as is; but the checked versions changed behavior (7 program checks failed, `tests/programs/share.zc` hung with a checked `cons`), so checked code inserts cleanup these functions can't afford. each needs its contract taught to the analyzer. `free`, `recycle`, `recycle_leaf`, `gc` are the allocator and collector the analyzer's own cleanup calls: document them as the irreducible escapes. `src/lib/actor.zc` has about 20 more.
 - [ ] revise diagnostics, comments and learning/reference prose using professional compiler documentation guidance, preserving lowercase and adding message regressions. touches messages everywhere, so do it when few other compiler changes are in flight.
 - [ ] (partly done, see "agent work at the end of the session") test blocks: `test "name" { ... }` with `expect(cond)`, compiled only by `zy --test file.zc`, which runs each block and prints one line per test with the position of the first failed `expect`; skipped at parse time otherwise. build it as a library through `run.syntax` and `run.statement_hook`, like `src/lib/actor.zc`.
-- [ ] (partly done, see "agent work at the end of the session") bounds checking on by default, `unbounded` to turn it off (function prefix, block, top-level `unbounded;`, mirroring `unchecked`), implied by `o3`. architecture: indexing is compiled in the postfix `[` handling of the expressions part; growable arrays already check (`dat`, `range_jz`, the `outside` enhancement, `proven()` removing checks a loop test already did). fixed arrays `T[n]` with a variable index get the same inline check against the constant `n`. pointers get a runtime call (for example `rt.bchk(p, i)`) that checks only when `p` is the start of a live heap block (header check word `p ^ MAGIC`, `paylen`), skipping constant index 0 and sizes the analyzer knows (`lnsz`). reports go through `oob`. the runtime, core, compiler parts, server, `run.zc` and `build.zc` start with `unbounded;`, which zc0 accepts and ignores (genome bytes unchanged). keep std, icon and actor checked unless a function touches raw memory on purpose. measure interpreter time before and after.
+- [x] (done on work/bounds, see "agent work at the end of the session") bounds checking on by default, `unbounded` to turn it off (function prefix, block, top-level `unbounded;`, mirroring `unchecked`), implied by `o3`. architecture: indexing is compiled in the postfix `[` handling of the expressions part; growable arrays already check (`dat`, `range_jz`, the `outside` enhancement, `proven()` removing checks a loop test already did). fixed arrays `T[n]` with a variable index get the same inline check against the constant `n`. pointers get a runtime call (for example `rt.bchk(p, i)`) that checks only when `p` is the start of a live heap block (header check word `p ^ MAGIC`, `paylen`), skipping constant index 0 and sizes the analyzer knows (`lnsz`). reports go through `oob`. the runtime, core, compiler parts, server, `run.zc` and `build.zc` start with `unbounded;`, which zc0 accepts and ignores (genome bytes unchanged). keep std, icon and actor checked unless a function touches raw memory on purpose. measure interpreter time before and after.
 - [ ] after the fuzzing work, add more general functionality to the standard library.
 - [ ] (partly done, see "agent work at the end of the session") struct destructors: a struct may declare `fn drop()`, called wherever the compiler frees an owned value of that type, before the memory is released. architecture: the compiler frees owned locals in `freevar` (analyzer part), called from `exit_vars`, reassignment and explicit `rt.free`/`rt.recycle` calls (`s_free`, `s_recycle` handling in the call code); when the local's static type is a pointer to a struct with a `Name.drop` method, emit a call to it first. objects freed by the collector or inside unchecked code do not run `drop`: document that. first users: file descriptors in `std.Host`, processes in `actor.zc`.
 - [x] (done on work/line-tables, see "agent work at the end of the session") runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing `  at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it.
 - [ ] borrow returns tied to an argument. inference already works for a callee defined before the call: `char* skip(char* s) { ...; return s + i; }` makes `let r = skip(t); rt.free(t); r[0]` an error. the gap: when the call comes before the callee's definition, nothing is reported and the program reads freed memory. extend the deferred effect requirements (`effect_require`, `effect_finish` at unit end) so a forward-called function whose return turns out to alias a parameter produces the same error. for bodyless declarations, externs and function types, add an unobtrusive spelling, for example a `borrows s` clause after the parameter list (`char* skip(char* s) borrows s;`), checked against the body when one exists.
 - [ ] variadic arguments and `std.print`. design: a pack is passed as one hidden argument, a caller-built read-only growable-array handle (`fixed T[?]` view: [len, cap, items, element size] in the caller's frame, so `len(a)` and `a[i]` work unchanged and nothing is allocated). `...int a` is a typed pack; several packs per function when their element types differ (`void f(...int a, ...char b)`): each extra argument goes to the pack whose element type matches its static type, an ambiguous match is an error, packs come after the fixed parameters. `...any args` packs each value with a kind code (int, float, char, string, bstr, pointer, ...) and its type record: `any` is a small built-in record { kind, value, type }. `...*any args` makes the call comma-free: after the fixed arguments, the parser reads unary expressions until `)`, commas optional, so `std.print("x = " name ", age " age "\n");` works and binary expressions must be parenthesized. `std.print(...*any args)` prints each argument by its kind (`put_n`, the float printer, `put_s`, `put_bs`, a character for `char`, hex for pointers).
 - [ ] x86-64 jit phase 2: compile o3 regions to x86-64 (port `rcompile` and the region register allocation; regions now run on the region interpreter and are 2 to 4 times slower than their plain versions on this jit) and inline enhancements (lower the optimized micro-ops to x86 registers as the arm64 path does, instead of calling `xrun`). validate with `tests/drivers/jit_x64_test.sh` and `tests/bench/bench.sh`, which now has an x86-64 jit column. add the driver to `tests/test.sh` on x86-64 linux.
-- [ ] `memdebug`'s `obj_base` searches only 64 words back for a block header, so under `memdebug on` an interior access more than 64 words into a block (`buf[255]` on a 256-word block) reports "bad heap pointer". find the header from the buddy allocator's size classes instead of a bounded scan, with a regression program.
+- [x] `memdebug`'s `obj_base` used to search only 64 words back for a block header, so `buf[255]` on a 256-word block reported "bad heap pointer". it now rounds the pointer down to each size class's alignment (largest first) and takes the first header whose check word, size and class agree and whose payload holds the pointer, so interiors at any depth work, a deep pointer into a freed block reports use after free, and header words and free space still report "bad heap pointer". regression: `tests/programs/memdebug_interior.zc` (in `tests/test.sh` beside the other memdebug checks). note that a stray pointer that lands inside another live block is accepted, since memdebug cannot know which block it came from.
 - [ ] (partly done, see "agent work at the end of the session") `defer` follow-ups (small): accept `defer if (...) ...;` and other single statements besides an expression; decide whether deferred code may name an owner declared after the `defer` (now rejected by the lexical rule; one line in `defer_one`).
 - [ ] build the llvm core by default when `llvm-config` is available (the makefile falls back to the interpreter otherwise), so x86 hosts get a native tier and the language server stops being interpreted; keep `ZYNOLLVM=1` and `-DZYINTERP` as the opt-outs.
 - [ ] stack-effect checking: every function and `asm` block declares or is inferred to have a fixed data-stack effect, and the compiler reports a block whose paths leave the stack unbalanced. the first step of a proving scaffold.