diff options
| author | 2026-10-14 09:28:59 +0000 | |
|---|---|---|
| committer | 2026-10-14 09:29:04 +0000 | |
| commit | f13f84044be1bd18160f4845a739592219638862 (patch) | |
| tree | de51ba649bf1f32adc64b890d5b8bb882b12a718 /docs/history | |
| parent | todo: record the reverted unchecked trial and what it showed (diff) | |
docs: remove temporary notes and fix onboarding drift
- delete TASKLIST.md and docs/design/handoff.md; the handoff content lives in README.md
- move docs/BOOTSTRAP-OWNERSHIP.md to docs/history/ and index it in docs/history/README.md
- point ONBOARD.md, docs/BORROW.md and the ticked TODO.md item at the new path
- ONBOARD.md section 9 no longer mentions TASKLIST.md
- ONBOARD.md section 3 and 4 markers are now `// ...` (line comments lost their colon)
verified from a clean build: make clean, make (fixpoint 376903 words), the
section 0-7 commands (section 4 after the marker fix), and the rejection suites. make check was
started but did not complete (stopped before it finished).
Diffstat (limited to 'docs/history')
| -rw-r--r-- | docs/history/BOOTSTRAP-OWNERSHIP.md | 183 | ||||
| -rw-r--r-- | docs/history/README.md | 1 |
2 files changed, 184 insertions, 0 deletions
diff --git a/docs/history/BOOTSTRAP-OWNERSHIP.md b/docs/history/BOOTSTRAP-OWNERSHIP.md new file mode 100644 index 0000000..6cc61ce --- /dev/null +++ b/docs/history/BOOTSTRAP-OWNERSHIP.md @@ -0,0 +1,183 @@ +# the bootstrap did not compile: owned values could not be moved between locals + +status: resolved. the analyzer gained the two rules proposed below and a clearer +recast diagnostic, and `make genome` reaches its fixpoint from a clean checkout. +see "resolution" at the end; the rest of this note is the record of the problem. +affects: `make genome` and the `fixpoint` check +see also: `docs/BORROW.md` + +## summary + +`make` bootstraps the compiler with `zc0` compiling `src/runtime/rt.zc`, +`src/core/core.zc` and `src/compiler/compiler.zc` into a stage-0 image, stage-0 +becoming `build/zy0`, and `zy0` compiling the same three files again into the +genome. the second compilation runs with lifetime checking enabled and rejects +the compiler's own source. `mkgenome` then read the diagnostics as if they +were an image: + +``` +mkgenome: input is not a genome for this map +``` + +the real output of `zy0` is a diagnostic list, not an image. nothing after the +fixpoint can run, so a checkout with no prebuilt genome cannot be built. + +this is a completeness gap in the ownership rules, not a soundness failure. the +checker is refusing correct programs; it is not accepting unsafe ones. the +practical cost is that the compiler can no longer compile itself. + +## the affected code + +`methdef` in `src/compiler/compiler.zc` reassembles a typed generic method's +token list. it saves the output stream state, redirects the stream at a scratch +buffer, renders the return type into it, splits the result into tokens, builds a +replacement buffer that interleaves the new tokens with the old ones, and hands +the replacement back in place of the original. + +```zc +word output = rt.outbuf, grow = rt.outgrow, buffer = rt.new(4096), return_tokens, joined; +rt.outmode = 2; rt.outbuf = m*buffer; rt.outn = 0; rt.outcap = 4095; rt.outgrow = 0; +tprint(rty); if (rtag) rt.put_s(".u"); buffer[rt.outn] = 0; +rt.outmode = mode; rt.outbuf = output; rt.outn = length; rt.outcap = capacity; rt.outgrow = grow; +return_tokens = tokens(buffer); +... +joined = rt.new((r[0] + return_tokens[0]) * 3 + 1); +... +word previous = m*r; r = d*joined; rt.free(return_tokens); rt.free(buffer); rt.free(previous); +``` + +three separate moves are needed here, and each one hits a different limit: + +1. lend the local scratch buffer to the output stream, then keep using it +2. replace `r`, an owned local, with `joined`, another owned local +3. store the local block into the global code-block list + +## what the checker reports + +``` +zc: line 5571: buffer: used after it was given away; owner buffer was transferred on line 5570 (declared on line 5569) +zc: line 5573: buffer: used after it was given away; owner buffer was transferred on line 5570 (declared on line 5569) +zc: line 5589: r: is an owned object and can't be copied into another variable +zc: line 5589: r: still owns an object; overwriting it would leak it +zc: line 5589: buffer: used after it was given away; owner buffer was transferred on line 5570 (declared on line 5569) +zc: line 5590: joined: d object was never given away +zc: line 10572: only a named owned object can be recast +zc: line 10575: only a named owned object can be recast +zc: 8 errors +``` + +the line numbers are into the concatenation, not into `compiler.zc`. subtract +1401 lines: 5570 is `rt.outbuf = m*buffer;`, 5589 is the `previous` line, and +10572 and 10575 are `rt.code_blocks = m*block;` in `syntax_resident`. + +## what was tried + +| change | result | +|---|---| +| `rt.outbuf = buffer;` — drop the `m*` | still reported as a transfer; any owned value assigned into a global transfers, and a second diagnostic appeared at the following line | +| move the scratch buffer to a module global `rbuf`, assign it plainly, borrow it into `rt.outbuf` | accepted; removes five of the eight errors. this is the existing idiom, see `sgb` at `sig_attach` and `ltxt` in the literal pool | +| `word previous = m*r; r = d*joined; rt.free(previous);` unchanged | `initial declarations still cannot copy an owned local`, per BORROW.md line 30 | +| declare `previous` in the block's declaration list, then `previous = m*r;` | `r: is an owned object and can't be copied into another variable` | +| `rt.free(r); r = joined;` — the documented free-then-replace idiom | `joined: is an owned object and can't be copied into another variable`, and `r` is then reported as given away at its next use | + +## is this a borrow checker issue + +yes, and it is worth separating two kinds of gap. + +**a move between two locals has no syntax.** the accepted moves in the +compiler's own source all originate at a call: + +```zc +word p = m*rt.new(capacity); // src/compiler/compiler.zc:644 +buf = m*inc_open(full); // src/compiler/compiler.zc:4632 +dgbuf = chk_resize(dgbuf, dgpos, cap) // src/compiler/compiler.zc:401 +rt.free(d*buf); // src/compiler/compiler.zc:4447 +``` + +an owned value already held in a local can be disposed with `d*`, but it cannot +be handed to another local, and it cannot replace one. the free-then-replace rule +in BORROW.md line 30 covers `rt.free(buffer); buffer = replacement;` — but only +when `replacement` arrives from a call. with a local on the right-hand side the +same rule rejects the assignment and then reports the destination as given away. + +the replacement built here is a genuine local: it is written field by field and +then has to become `r`. no current spelling expresses that. the shortest +workaround is to route the swap through a call that takes both pointers and +returns the survivor, so the move originates at a call result. + +**a local cannot be lent to a global.** the output stream is global state. +redirecting it at a local scratch buffer is a borrow, not a transfer, but every +spelling is a transfer: + +- `rt.outbuf = m*buffer;` transfers, so every later use of `buffer` is a use + after transfer +- `rt.outbuf = buffer;` also transfers +- the three working redirects in the compiler all assign a global or a pointer + derived from one — `rt.outbuf = dgbuf + dgpos + 5;`, `rt.outbuf = sgb;`, + `rt.outbuf = ift + at;`, `rt.outbuf = ltxt;` + +so the scratch buffer has to live at module scope. that is workable here, and it +is what the module-global `rbuf` change adopts, but it costs something the +local version did not: the buffer is shared state for the duration of the call, +so any reentrancy through `methdef` would collide, and the checker cannot see +that the borrow ends at the restore. + +## proposal + +in rough order of value: + +1. allow a move into an existing local when the source is an owned local and the + destination's previous owner is released in the same statement list, generalizing + the free-then-replace rule beyond call results. this alone resolves the + `methdef` reassembly. +2. allow a global to hold a borrow of a local for a bounded region, so an output + stream can be pointed at a scratch buffer without a module-level variable. a + scoped-borrow rule on assignment into a global field would do it, and it removes + the reentrancy hazard that the module-global workaround introduces. +3. give `rt.code_blocks = m*block;` a supported spelling. the error, `only a named + owned object can be recast`, does not describe what is wrong: `block` is named and + owned. it is used again at `here = block + 3;` two lines later, which is the real + constraint, and the diagnostic should say so and point at the later use. + +none of these accept a program that is currently accepted unsafely. each widens +the set of programs the checker can describe. item 3 is also a message bug +independent of the rule change: a wrong diagnostic on a correct-looking transfer. + +## reproducing + +`src/tools/bootstrap-check.sh` rebuilds the two bootstrap stages and prints the +compiler's diagnostics on its own source instead of the word dump. it exits +through `mkgenome` when the image is malformed, which is the same failure `make` +reports. + +``` +bash src/tools/bootstrap-check.sh +``` + +a passing run prints `image ok: <n> bytes`. + +## resolution + +the analyzer (`src/compiler/compiler.zc`) now accepts the three spellings: + +1. **moves into a released owner.** `rt.free(r); r = joined;` moves `joined` into `r` + when `r` was released first and `joined` is an owner of the same block or a deeper + one. `joined` is retired by the move. a destination that still owns something is + still rejected (`still owns an object; overwriting it would leak it`), and so is a + copy of an owner into a fresh declaration. +2. **bounded lends to globals.** `g = m*x;` with `x` an owned local lends `x` to `g`: + `x` keeps ownership and stays usable, and the lend ends when the same block assigns + `g` again. giving `x` away, returning, breaking or leaving the block while the lend + is open is an error (`is lent to global g; assign the global again before ...`). the + lend list is per function and per block depth (`lend_add`, `lend_clear`, + `lend_exit`), so a restore inside a nested branch does not end a lend made outside it. +3. **recasts of plain locals.** `rt.code_blocks = m*block;` failed because `block` came + from `chk_resize`, which returns collector-managed storage, so `block` never owned + anything. the compiler now writes `rt.code_blocks = block;`, and a recast of a plain + local reports `block: isn't an owned object, so there is no ownership to recast` + instead of the old message. + +`methdef` uses the lend for its scratch buffer and the move for the joined token list. +the regressions are `tests/programs/ownmove.zc` and `tests/errors/lend_*.zc`, +`recast_plain.zc` and `move_live_owner.zc`. + diff --git a/docs/history/README.md b/docs/history/README.md index ead4aa0..1a91d19 100644 --- a/docs/history/README.md +++ b/docs/history/README.md @@ -54,3 +54,4 @@ the code folders are snapshots. they're not built or tested from here, the top l - 51 llvm-inline-stdlib: llvm inlining of o3 calls, stdlib text and num helpers, host syscalls through SYS_RAW, and the fuzz campaign's findings - 52 closures: closure calls work (the block is never run; a call into a heap closure goes through its entry word), captures chain through enclosing closures, and the closure test joins the suite - 53 typed source: every function, local and global in the compiler, the runtime and the language server is typed (zc0's image does not change), and the extension caches diagnostics by a sha256 of the tree they cover +- BOOTSTRAP-OWNERSHIP.md: notes only (the two ownership rules that let the compiler compile itself, the partial fix before them, and the diagnostic that was reworded) |
