From 074a95a5e3aec808bbc6f05d865775d44a3938af Mon Sep 17 00:00:00 2001 From: zinc Date: Wed, 14 Oct 2026 18:03:01 +0330 Subject: todo: sync open list with 2026-10-14 work --- TODO.md | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/TODO.md b/TODO.md index 426555b..711340a 100644 --- a/TODO.md +++ b/TODO.md @@ -507,8 +507,7 @@ brief someone without re-reading the code. - [x] remaining zc0 gaps, one commit each on work/xref: string literals have no length limit (the token text grows; `1-source/lit_string_over_4095_bytes.zc`); `.u` on a typed parameter is rejected with the compiler's message, "a parameter tag is .m or .d" (`5-declarations/fn_param_tag_u_rejected.zc`); `x as T` reads a whole type, so `x as int*` and `x as char**` work (`3-expressions/cast_as_pointer_type.zc`); `embed` of an array type inside a function puts the elements in the code and the local holds their address (`tests/programs/embed_local_array.zc`, a parity program); integer literals as the compiler reads them, so `010` is 10, `0x` is 0 and values wrap modulo 2^64 (the eleven class b literal entries left `known.txt`). `docs/history/43-stdlib/notes.md` has a forward note that zc0 no longer reserves u/m/d. the genome is byte-identical to main's. - [x] verify both compilers against `docs/SPEC.md`: `tests/conformance/
/` (sections 1 to 8 plus `9-appendix-b`, 657 programs), runner `tests/conformance/run.sh` (`--zc0`, `--zym`, `--zy`, `-w`), understood failures in `tests/conformance/known.txt` (class a self-hosted bug, b zc0, c spec issue), wired into the interp section of `tests/test.sh`. zy 626 ok, 31 known; zc0 221 ok, 38 known, 398 skipped (outside appendix a). appendix b.2 items 1 to 16 confirmed except b.2.8, which is documented behavior (5.3.2, 3.14.2; the spec wins). top-level `o3 { }` is rejected (4.10.2, class c, left alone). the seven class a bugs are fixed, one commit each: a bad literal in a function's first statement (capture_scan left panic set), `u*d` with visible `u`/`d`, the bstr flag surviving a cast, the 24-parameter limit, `return e;` in a void function (now dropped, yields 0), `memdebug scrub;`, and `case member():` on a member with fields. the conformance run is in `tests/test.sh` but not in `-q`. open: `rt.memreport()` reports buddy capacity (`rt.new(2)` is 3 words), whether that is the spec's W; whether top-level `unchecked;` is in the zc0 subset. -- [ ] fix the implementation bugs listed in `docs/SPEC.md` appendix b, once confirmed: enum members can't reference earlier members; a plain function redefinition silently shadows the first; a prototype after the definition hides it; `?:` with one int and one float branch gives garbage; `bool` not normalised; float global initializers rejected; `include std;` inside a function body; syntax-error recovery hiding later errors; unterminated `/*` and string literals; `x as int * y`; prototypes needing parameter names; `len(char*)` with typecheck off; duplicate `case` labels; local `const` in constant expressions; constant expressions without precedence (`1+2*3` is 9). (`memdebug scrub;` is fixed; a global initializer naming another global giving its address is documented behavior, 5.3.2.) fix the prose items the appendix lists too. -- [ ] (partly done, see "agent work at the end of the session") a cross-reference report: for every global of the compiler, the functions that write it and the ones that only read it. build it from the language server's references (`textDocument/references` on `build/bin/zclsp`, so scoping is exact) and classify each reference as a write when an assignment operator, `++` or `--` follows it. output under `build/xref/`; a committed baseline and a `--check` mode that fails when a new writer of a global appears, so it can run in ci. +- [ ] fix the implementation bugs still open in `docs/SPEC.md` appendix b: unterminated string literals are only reported as a later syntax error (b.2.11 fixed the `/*` half); duplicate `case` labels are not diagnosed (b.2.15); a local `const` cannot be used in a constant expression (b.2.16, b.1.7); constant expressions have no precedence, so `1+2*3` is 9 (b.1.3); syntax-error recovery hides errors in the skipped statements (b.2.10, by design; decide whether to keep it). fixed and merged from work/spec: enum members naming earlier members (b.2.2), function redefinition (b.2.3), prototype after the definition (b.2.4), `?:` with int and float branches (b.2.5), `bool` normalisation (b.2.6), float global initializers (b.2.7, integer constants too via 46825f2), `include std;` in a function body (b.2.9), unterminated `/*` (b.2.11), `x as int * y` settled as a syntax error (b.2.12, doc only), unnamed prototype parameters (b.2.13), `len(char*)` with typecheck off (b.2.14). fix the prose items appendix b.1 lists too (verify which are still open). - [ ] optional, low priority: plain enums and `?:` in zc0 (both are small and produce ordinary code), so parity covers more programs. no structs in zc0 (decided). - [ ] group the compiler's parallel per-local arrays (`lname`, `ltype`, `lstate`, `lhome`, `lown`, `lslot`, `ltag`, `lpos`, `lnsz`, `lleaf`, ...) without structs: declare each family together in one place, with one comment block stating each array's meaning and invariant (for example when `lhome` is meaningful), route snapshot and restore of the family through one pair of functions, and use the cross-reference report to confirm nothing else writes them. then symbols and types the same way. fixpoint and analyzer behavior unchanged. - [ ] continue removing `unchecked` from the runtime. a reverted trial showed: `copy_w`/`fill_w` pass the analyzer when their gate stores go through `syscall(SYS_COPY, ...)`, `dgrow` with `own(w, it)` instead of `w[-2] = it`, `dpush` with `own` before the store, `dnew` as is; but the checked versions changed behavior (7 program checks failed, `tests/programs/share.zc` hung with a checked `cons`), so checked code inserts cleanup these functions can't afford. each needs its contract taught to the analyzer. `free`, `recycle`, `recycle_leaf`, `gc` are the allocator and collector the analyzer's own cleanup calls: document them as the irreducible escapes. `src/lib/actor.zc` has about 20 more. @@ -520,22 +519,36 @@ brief someone without re-reading the code. - [x] (done on work/line-tables, see "agent work at the end of the session") runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing ` at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it. - [ ] borrow returns tied to an argument. inference already works for a callee defined before the call: `char* skip(char* s) { ...; return s + i; }` makes `let r = skip(t); rt.free(t); r[0]` an error. the gap: when the call comes before the callee's definition, nothing is reported and the program reads freed memory. extend the deferred effect requirements (`effect_require`, `effect_finish` at unit end) so a forward-called function whose return turns out to alias a parameter produces the same error. for bodyless declarations, externs and function types, add an unobtrusive spelling, for example a `borrows s` clause after the parameter list (`char* skip(char* s) borrows s;`), checked against the body when one exists. - [ ] variadic arguments and `std.print`. design: a pack is passed as one hidden argument, a caller-built read-only growable-array handle (`fixed T[?]` view: [len, cap, items, element size] in the caller's frame, so `len(a)` and `a[i]` work unchanged and nothing is allocated). `...int a` is a typed pack; several packs per function when their element types differ (`void f(...int a, ...char b)`): each extra argument goes to the pack whose element type matches its static type, an ambiguous match is an error, packs come after the fixed parameters. `...any args` packs each value with a kind code (int, float, char, string, bstr, pointer, ...) and its type record: `any` is a small built-in record { kind, value, type }. `...*any args` makes the call comma-free: after the fixed arguments, the parser reads unary expressions until `)`, commas optional, so `std.print("x = " name ", age " age "\n");` works and binary expressions must be parenthesized. `std.print(...*any args)` prints each argument by its kind (`put_n`, the float printer, `put_s`, `put_bs`, a character for `char`, hex for pointers). -- [ ] (inlining and o3 regions done on work/jit2) x86-64 jit phase 2: compile o3 regions to x86-64 (port `rcompile` and the region register allocation; regions now run on the region interpreter and are 2 to 4 times slower than their plain versions on this jit) and inline enhancements (lower the optimized micro-ops to x86 registers as the arm64 path does, instead of calling `xrun`). validate with `tests/drivers/jit_x64_test.sh` and `tests/bench/bench.sh`, which now has an x86-64 jit column. add the driver to `tests/test.sh` on x86-64 linux. - done so far: enhancements are inlined (`x64_spec_enh`, `x64_enh`, `x64_uop` in core.c; seven value registers rsi, rdi, r8..r11, rbp; rax, rcx, rdx scratch; floats, byte loads and stores, native calls, popcnt and f2s/s2f still go through `xrun`; `ZYNOENH=1` calls everything as before). bench.sh plain kernels on the x86-64 jit, ms before/after: collatz 49.1/32.1, sieve 4.4/4.1, digits 41.9/14.7, fib 0.83/0.77; o3 unchanged (regions still interpreted). a 256-word pointer sweep with bounds checks 291 to 72 ms. `tests/programs/enh_ops.zc` covers every lowered op on edge values; jit_x64_test runs in test.sh's jit section (not in the quick path). o3 regions now compile (`rjit_x64`, `x64_ralloc`, `remit_x64` in core.c): vars stay in their frame slots, values in six registers, calls go through x_disp; `ZYO3INTERP=1` keeps the region interpreter. bench.sh o3 kernels on the x86-64 jit, ms interpreted/compiled: collatz 289/10.4, sieve 8.7/0.40, digits 89/3.0, fib 3.0/0.92 (fib is still slower than its plain version: calls and returns go through the dispatch stub). still interpreted: regions with more than six live values (one in tests/programs/o3.zc) or native calls. later: vars in registers across a loop, direct o3 calls, floats in xmm registers, spilling instead of giving up. next: the loop-entry flag. - [x] `memdebug`'s `obj_base` used to search only 64 words back for a block header, so `buf[255]` on a 256-word block reported "bad heap pointer". it now rounds the pointer down to each size class's alignment (largest first) and takes the first header whose check word, size and class agree and whose payload holds the pointer, so interiors at any depth work, a deep pointer into a freed block reports use after free, and header words and free space still report "bad heap pointer". regression: `tests/programs/memdebug_interior.zc` (in `tests/test.sh` beside the other memdebug checks). note that a stray pointer that lands inside another live block is accepted, since memdebug cannot know which block it came from. - [ ] (partly done, see "agent work at the end of the session") `defer` follow-ups (small): accept `defer if (...) ...;` and other single statements besides an expression; decide whether deferred code may name an owner declared after the `defer` (now rejected by the lexical rule; one line in `defer_one`). -- [ ] build the llvm core by default when `llvm-config` is available (the makefile falls back to the interpreter otherwise), so x86 hosts get a native tier and the language server stops being interpreted; keep `ZYNOLLVM=1` and `-DZYINTERP` as the opt-outs. - [ ] stack-effect checking: every function and `asm` block declares or is inferred to have a fixed data-stack effect, and the compiler reports a block whose paths leave the stack unbalanced. the first step of a proving scaffold. - [ ] declared side effects at module boundaries (globals touched, syscalls or not), checked against the effect masks the compiler already computes. - [ ] `requires`/`ensures` contracts on functions, compiled to run-time checks under a switch, with the cases the value facts settle (constants, simple ranges) checked statically. - [ ] (partly done, see "agent work at the end of the session") record and replay: a core option that logs every syscall gate result to a file and replays a run from the log exactly, for reproducing failures and adding tracing after the fact. -- [ ] a debug adapter for vscode (breakpoints, stepping, locals) on the interpreter, built on the line tables. +- [ ] a debug adapter for vscode (breakpoints, stepping, locals) on the interpreter, built on the line tables. goes to whoever finishes the bounds hints. - [ ] reduce the cost of word-per-character strings: measure, then decide between making `bstr` the default string, unifying the two apis behind one name, or both. - [ ] continue fuzzing the compiler, analyzer, runtime, standard library, lsp, build tools and generated artifacts, including analyzer properties, memory-limit edges, compiler image mutation and long-running lsp memory use; feed generated programs to `make parity` for differential fuzzing between zc0 and the self-hosted compiler; fix discovered bugs. - [ ] keep improving the borrow checker: fewer annotations in ordinary code, clearer messages naming both the owner and the conflicting use, and new fixtures for each rule that changes. +- [ ] x86-64 jit, phase 2 follow-ups (phase 2 is in the done list). we are not chasing llvm parity on the jit (owner's decision). + - region 474..692 in `tests/programs/o3.zc` falls back to the region interpreter; find out why and compile it. + - fib is slightly slower under o3 than plain (0.86 against 0.78 ms), likely region entry and exit on every recursive call; related to direct o3-to-o3 calls. + - the self-modifying-region caveat: compiled regions don't see code that rewrites itself inside a region; document it or invalidate the compiled region. +- [ ] (in progress on work/exactsize) exact-size bounds checks: n is stored in p[-4] bits 32..62 and the check is 2 loads. +- [ ] (in progress on work/checks) loop-entry check hoisting, dropping checks proven safe by `len()` or a known size, and `--explain-checks` v1 json: header `{"version":1}`, then one line per kept check with file, line, col, end_line, end_col, fn, kind, loop_depth, reason, fix and an optional edit. +- [ ] (in progress on work/bench) `make bench`, `bench-check` and `bench-selftest`: baseline from ec49c6d, jit and llvm rows, interp rows only with `INTERP=1`. follow-up: fill the checks column from `--explain-checks` and re-save the baseline; the rule that fails when the check count rises is already in bench-check. +- [ ] lsp and vscode bounds hints: diagnostics for kept checks with an all / loops / off setting, quick fixes, and a hot-checks command; the vscode extension supports all of it. later: `o3-candidate` hints for hot loops outside o3. +- [ ] hoisting v2: use the u/m/d ownership tags to prove a loop can't free a pointer (v1 refuses to hoist when the loop body has any free, release or realloc, or any call that might free). +- [ ] gc roots change: its own branch, memdebug scrub, full suite, bench comparison, and the owner reviews it before merge. +- [ ] shrink the 500 genome words added by b.2.13. +- [ ] file-level `o3;` and `unchecked;` directives that apply to every function after them, like `memdebug scrub;`. they replace the deferred top-level `o3 {}` and `unchecked {}` blocks. - [ ] run the full suite once everything above is done (`make test JOBS=4`; `make check` is the quick one), fix what it finds, and only then tick this item. ## done in the sorted list +- [x] a cross-reference report: for every global of the compiler, the functions that write it and the ones that only read it. build it from the language server's references (`textDocument/references` on `build/bin/zclsp`, so scoping is exact) and classify each reference as a write when an assignment operator, `++` or `--` follows it. output under `build/xref/`; a committed baseline and a `--check` mode that fails when a new writer of a global appears, so it can run in ci. done: `--check` runs in `tests/test.sh` through `xref_selftest.sh` against `tests/xref.baseline`. +- [x] x86-64 jit phase 2: compile o3 regions to x86-64 (port `rcompile` and the region register allocation; regions now run on the region interpreter and are 2 to 4 times slower than their plain versions on this jit) and inline enhancements (lower the optimized micro-ops to x86 registers as the arm64 path does, instead of calling `xrun`). validate with `tests/drivers/jit_x64_test.sh` and `tests/bench/bench.sh`, which now has an x86-64 jit column. add the driver to `tests/test.sh` on x86-64 linux. done: inline enhancements merged as 1f4aceb, compiled o3 regions merged as ec49c6d. + done so far: enhancements are inlined (`x64_spec_enh`, `x64_enh`, `x64_uop` in core.c; seven value registers rsi, rdi, r8..r11, rbp; rax, rcx, rdx scratch; floats, byte loads and stores, native calls, popcnt and f2s/s2f still go through `xrun`; `ZYNOENH=1` calls everything as before). bench.sh plain kernels on the x86-64 jit, ms before/after: collatz 49.1/32.1, sieve 4.4/4.1, digits 41.9/14.7, fib 0.83/0.77; o3 unchanged (regions still interpreted). a 256-word pointer sweep with bounds checks 291 to 72 ms. `tests/programs/enh_ops.zc` covers every lowered op on edge values; jit_x64_test runs in test.sh's jit section (not in the quick path). o3 regions now compile (`rjit_x64`, `x64_ralloc`, `remit_x64` in core.c): vars stay in their frame slots, values in six registers, calls go through x_disp; `ZYO3INTERP=1` keeps the region interpreter. bench.sh o3 kernels on the x86-64 jit, ms interpreted/compiled: collatz 289/10.4, sieve 8.7/0.40, digits 89/3.0, fib 3.0/0.92 (fib is still slower than its plain version: calls and returns go through the dispatch stub). still interpreted: regions with more than six live values (one in tests/programs/o3.zc) or native calls. later: vars in registers across a loop, direct o3 calls, floats in xmm registers, spilling instead of giving up. next: the loop-entry flag. +- [x] llvm by default (decided): llvm is turned on automatically on non-x86-64 hosts; on x86-64 the jit stays the default and llvm is opt-in with `LLVM=1`. +- [x] stale known-line check: conformance and parity fail when a known entry passes again (merged in c1072ab). - [x] `defer expr;` and `defer { ... }`: tokens grabbed at the `defer`, recompiled inline at every exit (block end through `block_close`, `return` after its value, `break`/`continue` for entries deeper than the loop), newest first; only defers reached so far register. names are lexical (a local declared after the `defer` is hidden in the replay). a deferred `rt.free(p)` is `p`'s transfer to the analyzer. each entry replays in the analyzer mode it was written in; closures get their own base through `ctx_push`. `return`, `break` and `continue` that would leave a deferred statement are rejected. a failed entry reports once. limit 256 pending per function (`DFR_MAX`). spec 4.14, learn section 34, `tests/programs/defer.zc`, seven reject fixtures. - [x] four small fixes: hover on standard-library members (`hover_embedded_sym` sends std/icon symbols to the library lookup; `bs_hover` renders a member hit), `capture.end`'s terminator (the buffer has `cap + 1` words; latent, since the allocator's power-of-two slack hid it; `tests/programs/capture_long.zc`), bare sibling static calls inside a struct's methods (`mstruct` set for every non-generic struct; the struct's own constant or static beats a top-level name, a local beats both; `tests/programs/struct_static_call.zc`, spec 5.5.5), and `void main()` in run-only images (`run.zc` declares an untyped `fn main();`; spec 5.2.5). - [x] a hand-written x86-64 jit, phase 1 (`ZYJIT_X64` in `src/core/core.c`, default on x86-64 linux; `-DZYINTERP` and `make build/bin/zyi` give the interpreter). r12 memory, r13 data stack, r14 return stack, r15 table, rbx state; stack slots addressed with a virtual offset so r13 moves only at calls and exits; shares the table, coverage map, store hook and invalidation with the arm64 jit. enhancements are called through `xrun`, o3 regions run on the region interpreter (`ZYNOO3=1` runs their vm code through the jit instead). 179 of 179 programs agree with the interpreter (`tests/drivers/jit_x64_test.sh`), the self-compile is byte-identical and about 7x faster (1.5 s against 11 s), and the rejection, learn, icon, actor, lsp and hint suites pass on it. -- cgit v1.2.3