bounds checking: constant loop bound proof works, plain-op fixed-array check, docs (readme, spec 4.15 and friends, server, vscode), conformance and test.sh fixes, todo

1 files changed, 28 insertions, 1 deletions

diff --git a/TODO.md b/TODO.md
index e601b10..9a2f52f 100644
--- a/TODO.md
+++ b/TODO.md
@@ -371,6 +371,33 @@ of the open list.
   scratchpad `base/`, not in the zip); `make parity`. the 35% fixed-array cost
   is worth a look before merging (hoist checks out of loops whose bound is the
   array length, extend `proven`).
+  resumed on work/bounds (ported onto main f75ce6a, conflicts in base, decl
+  and expr; the context save is 50 words, `fbnd` at 49 after `dfr_base`).
+  done: o3call's main is `unbounded`; `o3_bounds` and `bounds_hi` now expect
+  no check (o3 implies unbounded); new boundsrules, bounds_fixed, bounds_heap,
+  zc0unbounded (also a run-only image) with test.sh lines; bounds_fixed and
+  bounds_heap are in parity.known; docs in README ("bounds checks"), SPEC
+  2.6.1, 3.7.4, 3.13.4, 4.10.1, new 4.15, 5.1.1 and the keyword table, the
+  server's `kwtext` and `iskeyword`, the vscode grammar. genome 395477 to
+  399590 words (+4113, +1.0%); the self-compile takes about 1.1 to 1.2 s
+  before and after. the fixed-array cost: the `for (i < K)` proof never fired
+  (the loop shape needed a local on the right of `<`), fixed now, so a
+  constant-bound loop costs nothing; the inline check is now plain ops
+  (`i | (n - 1 - i)` < 0) instead of the `outside` enhancement, which the
+  x86-64 jit calls rather than inlines. best of 3, ms, main / port / final:
+  hash-index kernel jit 37 / 72 / 44, interpreter 412 / 504 / 542; loop with a
+  variable bound jit 55 / 113 / 69, interpreter 741 / 907 / 865; loop with a
+  constant bound jit 46 / 100 / 53, interpreter 663 / 822 / 690. the plain-op
+  check is slightly slower than the enhancement on the interpreter for an
+  unprovable index, and probably on arm64, where enhancements are inlined (not
+  measured here). still open: pointers, jit 52 to 279 ms and interpreter 710
+  to 1141 ms on a 256-word sweep (`rt.bnd` is an enhancement call on x86-64);
+  a variable loop bound is not hoisted (needs a loop-entry test of `n <= N`, or
+  of the heap block's payload for a pointer, and a flag the body tests).
+  inlining enhancements in the x86-64 jit would help both. also fixed here:
+  four conformance expectations on main lacked the backtrace line that line
+  tables print (3-expressions/dyn_array_*), and test.sh's run-only gc sed
+  still looked for `f >= 990`.
 - spec conformance (`agents/spec-conformance`, last 808886b; the first,
   interrupted attempt is `agents/spec-conformance-first-attempt`, already
   folded in). done: sections 1 source (114 programs, zy 103 ok), 2 types
@@ -480,7 +507,7 @@ brief someone without re-reading the code.
 - [ ] continue removing `unchecked` from the runtime. a reverted trial showed: `copy_w`/`fill_w` pass the analyzer when their gate stores go through `syscall(SYS_COPY, ...)`, `dgrow` with `own(w, it)` instead of `w[-2] = it`, `dpush` with `own` before the store, `dnew` as is; but the checked versions changed behavior (7 program checks failed, `tests/programs/share.zc` hung with a checked `cons`), so checked code inserts cleanup these functions can't afford. each needs its contract taught to the analyzer. `free`, `recycle`, `recycle_leaf`, `gc` are the allocator and collector the analyzer's own cleanup calls: document them as the irreducible escapes. `src/lib/actor.zc` has about 20 more.
 - [ ] revise diagnostics, comments and learning/reference prose using professional compiler documentation guidance, preserving lowercase and adding message regressions. touches messages everywhere, so do it when few other compiler changes are in flight.
 - [ ] (partly done, see "agent work at the end of the session") test blocks: `test "name" { ... }` with `expect(cond)`, compiled only by `zy --test file.zc`, which runs each block and prints one line per test with the position of the first failed `expect`; skipped at parse time otherwise. build it as a library through `run.syntax` and `run.statement_hook`, like `src/lib/actor.zc`.
-- [ ] (partly done, see "agent work at the end of the session") bounds checking on by default, `unbounded` to turn it off (function prefix, block, top-level `unbounded;`, mirroring `unchecked`), implied by `o3`. architecture: indexing is compiled in the postfix `[` handling of the expressions part; growable arrays already check (`dat`, `range_jz`, the `outside` enhancement, `proven()` removing checks a loop test already did). fixed arrays `T[n]` with a variable index get the same inline check against the constant `n`. pointers get a runtime call (for example `rt.bchk(p, i)`) that checks only when `p` is the start of a live heap block (header check word `p ^ MAGIC`, `paylen`), skipping constant index 0 and sizes the analyzer knows (`lnsz`). reports go through `oob`. the runtime, core, compiler parts, server, `run.zc` and `build.zc` start with `unbounded;`, which zc0 accepts and ignores (genome bytes unchanged). keep std, icon and actor checked unless a function touches raw memory on purpose. measure interpreter time before and after.
+- [x] (done on work/bounds, see "agent work at the end of the session") bounds checking on by default, `unbounded` to turn it off (function prefix, block, top-level `unbounded;`, mirroring `unchecked`), implied by `o3`. architecture: indexing is compiled in the postfix `[` handling of the expressions part; growable arrays already check (`dat`, `range_jz`, the `outside` enhancement, `proven()` removing checks a loop test already did). fixed arrays `T[n]` with a variable index get the same inline check against the constant `n`. pointers get a runtime call (for example `rt.bchk(p, i)`) that checks only when `p` is the start of a live heap block (header check word `p ^ MAGIC`, `paylen`), skipping constant index 0 and sizes the analyzer knows (`lnsz`). reports go through `oob`. the runtime, core, compiler parts, server, `run.zc` and `build.zc` start with `unbounded;`, which zc0 accepts and ignores (genome bytes unchanged). keep std, icon and actor checked unless a function touches raw memory on purpose. measure interpreter time before and after.
 - [ ] after the fuzzing work, add more general functionality to the standard library.
 - [ ] (partly done, see "agent work at the end of the session") struct destructors: a struct may declare `fn drop()`, called wherever the compiler frees an owned value of that type, before the memory is released. architecture: the compiler frees owned locals in `freevar` (analyzer part), called from `exit_vars`, reassignment and explicit `rt.free`/`rt.recycle` calls (`s_free`, `s_recycle` handling in the call code); when the local's static type is a pointer to a struct with a `Name.drop` method, emit a call to it first. objects freed by the collector or inside unchecked code do not run `drop`: document that. first users: file descriptors in `std.Host`, processes in `actor.zc`.
 - [x] (done on work/line-tables, see "agent work at the end of the session") runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing `  at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it.