merge main into work/line-tables

1 files changed, 22 insertions, 31 deletions

diff --git a/TODO.md b/TODO.md
index 3b6e62d..e601b10 100644
--- a/TODO.md
+++ b/TODO.md
@@ -392,8 +392,7 @@ of the open list.
   107, memdebug_report_line line 9, asm_float_literal uses `word w`); real:
   `memdebug scrub;` rejected (b.1.1/b.2.1 confirmed), top-level `o3 { }`
   rejected (c), `case rect():` with fields possibly rejected (unresolved).
-  left: sections 5 to 8, the appendix b confirm/refute list, `known.txt`, the
-  `--zc0` runs, final counts, then add the runner to tests/test.sh.
+  resumed on work/spec and finished; see the closed item in the list below.
 - record and replay (no code; agent stopped during research). findings: every
   syscall goes through `hostcall(op, a, b, c)` (core.c ~264) with three callers
   (interpreter ~6324, jit `hook()` ~3617 which the x86-64 helpers also reach,
@@ -438,34 +437,26 @@ of the open list.
   text); return; }`. open: whether the keyword handler can decline when the
   next token is not a string (so `test` stays usable as a name). diagnostics
   print `zc: line N:` with no file name.
-- compiler cross-reference report (`agents/xref-report`, one wip commit
-  e5d5136). done: `src/tools/xref.js` and `src/tools/xref.sh` (`--help`,
-  unknown option exits 2). the language server has no documentSymbol, so the
-  globals come from the tool's own parse of the joined compiler source (652
-  globals, 628 functions, nothing left unparsed); writers and readers come from
-  one zclsp references request per global over 4 parallel servers. one full run
-  finished in 11m53s (about 2.5 s per request, about 30 s per server to check
-  the document); its output is in the zip at `agents/xref-report/xref-out/`
-  (report.txt, report.json), not yet reviewed by a person. top 15 globals by
-  number of writer functions, read off that json by the coordinator: xty 29,
-  xbor 23, xfloat 22, lv 21, xaf 21, lvk 20, xvk 19, ltag 17, xzero 17,
-  lstate 16, sflag 16, effect_value 15, fslots 13, panic 13, chkabort 12 (these
-  are the first candidates for the "group the parallel arrays" item). 60
-  globals show no writer and 198 no reader (many are `_cap` twins); some of the
-  "no writer" ones are the known limits below, so check before trusting them.
-  written but never run: the `make xref` target, the README and ONBOARD.md
-  sections, the xref.sh entry in `tests/drivers/cli_help_test.py`. not done:
-  `tests/xref.baseline` (derive it from report.json, one line per global with
-  its sorted writers, or run `xref.sh --update` in the background), `--check`
-  and `--update` never run, the self-test (copy src/compiler to a scratch dir,
-  add one write to a global in another function, `xref.sh --compiler-dir DIR
-  --out SCRATCH --check` must exit 1), README says "minutes" where it should
-  say about 12 minutes. known limits: a store through a pointer (`*p = ...`),
-  an array written by a callee (`copy_w`, `memset`) and a global reached only
-  through its `_initial` array are not counted as writes. next: baseline,
-  `--check` (expect 0), self-test, `python3 tests/drivers/cli_help_test.py
-  --group scripts`, commit. a 12 minute run is too slow for every ci job; run
-  it nightly or only when `src/compiler/` changes.
+- compiler cross-reference report (`agent/xref-report`, cherry-picked onto
+  work/xref without conflicts). done: `src/tools/xref.js` and
+  `src/tools/xref.sh` (`--help`, unknown option exits 2), the `make xref`
+  target, the readme and onboard.md sections, and the xref.sh entry in
+  `tests/drivers/cli_help_test.py` (passes with `--group scripts`).
+  `tests/xref.baseline` is checked in, written by `xref.sh --update` (658
+  globals, 633 functions, one line per global with its sorted writers); a run
+  took about 10 s with 8 servers, not 12 minutes, so the readme now says tens
+  of seconds. `--check` against it exits 0, and exits 1 when one writer is
+  dropped from one baseline line. `src/tools/xref_selftest.sh [server]` does
+  both cases in a scratch directory and runs in the lsp section of
+  `tests/test.sh` next to the other node drivers (not in -q). known limits: a
+  store through a pointer (`*p = ...`), an array written by a callee
+  (`copy_w`, `memset`) and a global reached only through its `_initial` array
+  are not counted as writes. the baseline is not yet reviewed by a person;
+  the 60 "no writer" globals still need checking against those limits. the self-test also
+  copies src/compiler to a scratch dir, adds a new function writing
+  `namecount`, and checks that `--compiler-dir ... --check` exits 1 naming it;
+  it rejects unknown options and is in cli_help_test.py. open: deciding whether ci runs it on every job or only
+  when `src/compiler/` changes.
 
 ### decisions
 - no structs in zc0 (owner's call): zc0 stays the small trusted root of the
@@ -481,7 +472,7 @@ items with an architecture note carry the design worked out so far, enough to
 brief someone without re-reading the code.
 
 - [ ] remaining zc0 gaps: string literals over 4095 bytes, `embed` of an array type inside a function, `x as T` with more than a single name (`x as int*`), zc0 silently accepting `.u` on a parameter, and integer literals (`010` is 8 in zc0, 10 in the compiler; zc0 saturates on overflow). `docs/history/43-stdlib/notes.md` still says zc0 reserves u/m/d: add a pointer forward rather than rewriting history.
-- [ ] (partly done, see "agent work at the end of the session") verify both compilers against `docs/SPEC.md` with a fresh agent that did not write it: expectations derived from the spec text before running anything, one program per rule under `tests/conformance/<section>/`, a runner `tests/conformance/run.sh`, each failure classified as self-hosted bug, zc0 bug, or spec error, and appendix b confirmed item by item. then add the runner to `tests/test.sh`.
+- [x] verify both compilers against `docs/SPEC.md`: `tests/conformance/<section>/` (sections 1 to 8 plus `9-appendix-b`, 657 programs), runner `tests/conformance/run.sh` (`--zc0`, `--zym`, `--zy`, `-w`), understood failures in `tests/conformance/known.txt` (class a self-hosted bug, b zc0, c spec issue), wired into the interp section of `tests/test.sh`. zy 626 ok, 31 known; zc0 221 ok, 38 known, 398 skipped (outside appendix a). appendix b.2 items 1 to 16 confirmed except b.2.8, which contradicts 5.3.2 and 3.14.2 (the spec documents the address). new class a bugs: `case member():` on a member with fields is rejected (4.7.3); top-level `o3 { }` is rejected (4.10.2, class c). open: `rt.memreport()` reports buddy capacity (`rt.new(2)` is 3 words), whether that is the spec's W; whether top-level `unchecked;` is in the zc0 subset.
 - [ ] fix the implementation bugs listed in `docs/SPEC.md` appendix b, once confirmed: `memdebug scrub;` rejected; enum members can't reference earlier members; a plain function redefinition silently shadows the first; a prototype after the definition hides it; `?:` with one int and one float branch gives garbage; `bool` not normalised; float global initializers rejected; a global initializer naming another global uses its address; `include std;` inside a function body; syntax-error recovery hiding later errors; unterminated `/*` and string literals; `x as int * y`; prototypes needing parameter names; `len(char*)` with typecheck off; duplicate `case` labels; local `const` in constant expressions; constant expressions without precedence (`1+2*3` is 9). fix the prose items the appendix lists too.
 - [ ] (partly done, see "agent work at the end of the session") a cross-reference report: for every global of the compiler, the functions that write it and the ones that only read it. build it from the language server's references (`textDocument/references` on `build/bin/zclsp`, so scoping is exact) and classify each reference as a write when an assignment operator, `++` or `--` follows it. output under `build/xref/`; a committed baseline and a `--check` mode that fails when a new writer of a global appears, so it can run in ci. 
 - [ ] optional, low priority: plain enums and `?:` in zc0 (both are small and produce ordinary code), so parity covers more programs. no structs in zc0 (decided).