From d8da21a406f449d277c42b91d4c9bf8b20bd0dcc Mon Sep 17 00:00:00 2001 From: zinc Date: Wed, 14 Oct 2026 07:55:08 +0000 Subject: embed: tests, editor support and docs; js vm file access; test fixes embed declarations get accept and reject fixtures, a run-only image check through zc0, an lsp keyword hover and the grammar keyword. errors point at the declaration and still declare the name, so its uses are not reported again. zc0 tracks byte-string globals and lays embedded arrays out one word per element, the way its own indexing reads them. the command-line javascript vm reaches node's file system for the file syscalls, so include and embed work there; other embedders opt in. the web build is self-contained in build/web with its examples. README, BORROW and BOOTSTRAP-OWNERSHIP describe the new build and ownership rules, and ONBOARD.md is a reading and verification order for the code. the suite passes the map header to mklayout, the wasm fuzz program uses the include format, and lsp drivers scale their deadlines on hosts without the arm64 jit (ZC_TEST_TIMEOUT_SCALE). LLVM_CONFIG selects an llvm with headers for the llvm section. --- docs/ACTOR.md | 2 +- docs/BOOTSTRAP-OWNERSHIP.md | 38 ++++++++++++++++--- docs/BORROW.md | 8 +++- docs/ICON.md | 2 +- docs/design/memory.md | 10 ++--- docs/design/storage-analysis.md | 2 +- docs/design/test-coverage.md | 82 ++++++++++++++++++++--------------------- 7 files changed, 88 insertions(+), 56 deletions(-) (limited to 'docs') diff --git a/docs/ACTOR.md b/docs/ACTOR.md index 913c6bc..6ee5627 100644 --- a/docs/ACTOR.md +++ b/docs/ACTOR.md @@ -56,4 +56,4 @@ messages copy their typed object graphs into the receiving context. supported va ## checks -`python3 tests/drivers/actor_test.py bin/zy` and `python3 tests/drivers/actor_test.py bin/zym` exercise library-only syntax installation, declared actors, independent captures, two-instance parallelism, callback registration before send, statement delivery, fifo ordering, typed graph copies, closure messages, lifecycle behavior and rejection fixtures. the parallel fixture measures worker time inside the vm: two 300 ms tasks must complete in less than 580 ms. +`python3 tests/drivers/actor_test.py build/bin/zy` and `python3 tests/drivers/actor_test.py build/bin/zym` exercise library-only syntax installation, declared actors, independent captures, two-instance parallelism, callback registration before send, statement delivery, fifo ordering, typed graph copies, closure messages, lifecycle behavior and rejection fixtures. the parallel fixture measures worker time inside the vm: two 300 ms tasks must complete in less than 580 ms. diff --git a/docs/BOOTSTRAP-OWNERSHIP.md b/docs/BOOTSTRAP-OWNERSHIP.md index 509db2f..6cc61ce 100644 --- a/docs/BOOTSTRAP-OWNERSHIP.md +++ b/docs/BOOTSTRAP-OWNERSHIP.md @@ -1,7 +1,9 @@ -# the bootstrap does not compile: owned values cannot be moved between locals +# the bootstrap did not compile: owned values could not be moved between locals -status: open, blocks `make` -affects: `make genome`, the `fixpoint` check, the `genome.h is up to date` check +status: resolved. the analyzer gained the two rules proposed below and a clearer +recast diagnostic, and `make genome` reaches its fixpoint from a clean checkout. +see "resolution" at the end; the rest of this note is the record of the problem. +affects: `make genome` and the `fixpoint` check see also: `docs/BORROW.md` ## summary @@ -10,7 +12,7 @@ see also: `docs/BORROW.md` `src/core/core.zc` and `src/compiler/compiler.zc` into a stage-0 image, stage-0 becoming `build/zy0`, and `zy0` compiling the same three files again into the genome. the second compilation runs with lifetime checking enabled and rejects -the compiler's own source. `bin/mkgenome` then reads the diagnostics as if they +the compiler's own source. `mkgenome` then read the diagnostics as if they were an image: ``` @@ -152,4 +154,30 @@ reports. bash src/tools/bootstrap-check.sh ``` -a passing run prints `image ok: bytes`. \ No newline at end of file +a passing run prints `image ok: bytes`. + +## resolution + +the analyzer (`src/compiler/compiler.zc`) now accepts the three spellings: + +1. **moves into a released owner.** `rt.free(r); r = joined;` moves `joined` into `r` + when `r` was released first and `joined` is an owner of the same block or a deeper + one. `joined` is retired by the move. a destination that still owns something is + still rejected (`still owns an object; overwriting it would leak it`), and so is a + copy of an owner into a fresh declaration. +2. **bounded lends to globals.** `g = m*x;` with `x` an owned local lends `x` to `g`: + `x` keeps ownership and stays usable, and the lend ends when the same block assigns + `g` again. giving `x` away, returning, breaking or leaving the block while the lend + is open is an error (`is lent to global g; assign the global again before ...`). the + lend list is per function and per block depth (`lend_add`, `lend_clear`, + `lend_exit`), so a restore inside a nested branch does not end a lend made outside it. +3. **recasts of plain locals.** `rt.code_blocks = m*block;` failed because `block` came + from `chk_resize`, which returns collector-managed storage, so `block` never owned + anything. the compiler now writes `rt.code_blocks = block;`, and a recast of a plain + local reports `block: isn't an owned object, so there is no ownership to recast` + instead of the old message. + +`methdef` uses the lend for its scratch buffer and the move for the joined token list. +the regressions are `tests/programs/ownmove.zc` and `tests/errors/lend_*.zc`, +`recast_plain.zc` and `move_live_owner.zc`. + diff --git a/docs/BORROW.md b/docs/BORROW.md index 10beb84..f270574 100644 --- a/docs/BORROW.md +++ b/docs/BORROW.md @@ -27,7 +27,11 @@ effects are cached for the current compiler analysis and participate in the lang aliases are tied to their owner. when a tracked alias has no later use, a subsequent move or free can proceed; uses after a definite transfer or free produce an error that points to both the transfer and the later use. independent struct fields and constant array elements can be used when their projections prove they do not overlap. unknown indices and overlapping projections are not treated as disjoint. -after an owned local is released, assignment from another owner in the same declaring scope moves the replacement into that empty slot. the source and its aliases expire, the destination receives its cleanup responsibility, and loop paths still have to agree. this supports buffer-growth code such as `rt.free(buffer); buffer = replacement;` without a cast. initial declarations still cannot copy an owned local, overwriting a live owner remains an error, and captured owners are excluded from this replacement rule. +after an owned local is released, assignment from another owner declared in the same scope or a deeper one moves the replacement into that empty slot. the source and its aliases expire, the destination receives its cleanup responsibility, and loop paths still have to agree. this supports buffer-growth code such as `rt.free(buffer); buffer = replacement;` without a cast, including a replacement built inside a branch. initial declarations still cannot copy an owned local, overwriting a live owner remains an error, and captured owners are excluded from this replacement rule. + +## lends to globals + +`g = m*x;` lends an owned local to a global for the rest of the block: `x` keeps ownership and stays usable while code reached through `g` reads it, and assigning `g` again in the same block ends the lend. giving `x` away, returning, breaking out, or reaching the end of the block while the lend is open is an error, so a global never holds a borrow that outlives its owner. a restore inside a nested branch does not end a lend made outside it; the restore has to be unconditional. this is the idiom for pointing the output stream at a scratch buffer. an element borrow records its base array and known range. a constant index outside a known length gets a bounds diagnostic. mutations such as `push`, `pop`, and `resize` advance the array generation and invalidate element borrows; a later use reports the mutation and where the borrow was made. uncertain branch and loop facts are not promoted to definite errors. runtime checks remain for dynamic indices. @@ -41,7 +45,7 @@ the compiler functions, standard library, and language server currently run with the analysis runs with the existing parser, ownership checker, and lifetime records; it does not have a separate source parser or a complete path-sensitive proof. snapshots conservatively join branches and loops. low-level and unknown boundaries can hide behavior, so a quiet diagnostic set is not a whole-program safety proof. zc is single-threaded; shared closure cells do not define concurrent mutation rules. -two completeness limits block the compiler from compiling itself, and are written up in `docs/BOOTSTRAP-OWNERSHIP.md`. an owned value already held in a local has no move into another local; every accepted move originates at a call, so the free-then-replace rule above applies only when the replacement comes from a call result. and a local cannot be lent to a global, so an output stream has to be redirected at a module-level buffer rather than a scratch local. neither limit accepts an unsafe program; both refuse correct ones. +two completeness limits once stopped the compiler from compiling itself: an owner held in a local could not move into another local, and a local could not be lent to a global. the replacement and lend rules above close them; `docs/BOOTSTRAP-OWNERSHIP.md` keeps the record. the maintained regressions cover these cases: diff --git a/docs/ICON.md b/docs/ICON.md index 9c222c8..3f6ec8b 100644 --- a/docs/ICON.md +++ b/docs/ICON.md @@ -109,4 +109,4 @@ the status applies to the most recent transformation or binding operation. succe ## focused tests -the acceptance, rejection and regression fixtures live under `tests/icon`. run them with `python3 tests/drivers/icon_test.py bin/zy`; use `--js` for the javascript vm. select one fixture or category with `--case`, for example `python3 tests/drivers/icon_test.py bin/zym --case loop_step`. add `--source` to test the working library source before rebuilding the embedded compiler module; omit it when checking the built image. +the acceptance, rejection and regression fixtures live under `tests/icon`. run them with `python3 tests/drivers/icon_test.py build/bin/zy`; use `--js` for the javascript vm. select one fixture or category with `--case`, for example `python3 tests/drivers/icon_test.py build/bin/zym --case loop_step`. add `--source` to test the working library source before rebuilding the embedded compiler module; omit it when checking the built image. diff --git a/docs/design/memory.md b/docs/design/memory.md index c282df5..8875c72 100644 --- a/docs/design/memory.md +++ b/docs/design/memory.md @@ -5,12 +5,12 @@ the compiler, runtime, and host must agree on their addresses. `genmap` writes a matching `memmap.h`, runtime source, and compiler source; `src/tools/automap.sh` builds an image against those files. rebuild the host against the generated headers. changing a header alone does not relocate an existing genome. compact compiler and run-only maps also put their heap above the fixed regions, so compiler storage can grow after startup. ```sh -bin/genmap --preset default --check +build/bin/genmap --preset default --check bash src/tools/automap.sh small build/small -bin/genmap --preset default --code 65536 --symbols 256 --locals 64 --fwd 512 --pool 8192 --hash 1024 -o build/custom +build/bin/genmap --preset default --code 65536 --symbols 256 --locals 64 --fwd 512 --pool 8192 --hash 1024 -o build/custom ``` -run `bin/genmap --help` for the table, heap, genome, and stack options. sizes are in vm words unless an option says otherwise; one word is eight bytes. `--snap-cap` selects the initial snapshot reservation and `--quote-cap` selects an initial token count, with three words per token. both regions grow after startup. +run `build/bin/genmap --help` for the table, heap, genome, and stack options. sizes are in vm words unless an option says otherwise; one word is eight bytes. `--snap-cap` selects the initial snapshot reservation and `--quote-cap` selects an initial token count, with three words per token. both regions grow after startup. ## layout inventory @@ -79,8 +79,8 @@ the stub and board ports reserve `MICRO_PORT_WORDS` cells at build time; it defa the default native map places a growing heap above the fixed regions. hosted runners accept `TINYVM_MEMORY_WORDS` for initial memory and `TINYVM_MAX_MEMORY_WORDS` for its growth limit. javascript also accepts `memoryWords` and `maxMemoryWords` when constructing a vm. invalid configurations are rejected before execution. ```sh -TINYVM_MAX_MEMORY_WORDS=16777216 bin/zy < examples/hello.zc -TINYVM_MEMORY_WORDS=4194304 bin/zym < examples/hello.zc +TINYVM_MAX_MEMORY_WORDS=16777216 build/bin/zy < examples/hello.zc +TINYVM_MEMORY_WORDS=4194304 build/bin/zym < examples/hello.zc ``` the native core reserves a stable backing allocation up to its startup maximum; increasing the available size keeps addresses and native code pointers valid. the microcore and javascript vm replace backing allocations when they grow, preserving existing contents. native bounds use the reservation; portable bounds use the current allocation. memory syscall `9` with argument `4` requests growth, and argument `5` changes the growth limit during execution. a limit cannot fall below the current size. the native limit cannot exceed its startup reservation. diff --git a/docs/design/storage-analysis.md b/docs/design/storage-analysis.md index cff713a..432139a 100644 --- a/docs/design/storage-analysis.md +++ b/docs/design/storage-analysis.md @@ -30,4 +30,4 @@ facts are copied into each reference's editor record while the compiler parses i ## focused checks -`node tests/drivers/storage_analysis_test.js bin/zclsp` checks known bounds, packed elements, direct aliases, free and scope lifetimes, hover metadata, and safe or uncertain neighboring cases. +`node tests/drivers/storage_analysis_test.js build/bin/zclsp` checks known bounds, packed elements, direct aliases, free and scope lifetimes, hover metadata, and safe or uncertain neighboring cases. diff --git a/docs/design/test-coverage.md b/docs/design/test-coverage.md index fae9356..e916df4 100644 --- a/docs/design/test-coverage.md +++ b/docs/design/test-coverage.md @@ -35,7 +35,7 @@ the `qualifiers` group in `toolchain_lsp_contract_test.js` checks both qualifier ## public standard-library documentation -`src/lib/std.zc` documents public operations for output capture, numeric helpers, packed byte strings, host access, arenas, growable byte buffers, and array algorithms, including parameter and return behavior. the array algorithms document their callback roles, result types, traversal order, and reference sharing. `src/tools/embed.sh all --check` verifies that the compiler's embedded source matches the module. `hover_surface_test.js` and `hover_features_test.js` check selected embedded library documentation; they do not assert that every public function has a description. edge-case behavior for the broader standard-library API still needs dedicated acceptance tests. +`src/lib/std.zc` documents public operations for output capture, numeric helpers, packed byte strings, host access, arenas, growable byte buffers, and array algorithms, including parameter and return behavior. the array algorithms document their callback roles, result types, traversal order, and reference sharing. the compiler carries the module through an `embed` declaration, so its embedded text is the module file as of the build. `hover_surface_test.js` and `hover_features_test.js` check selected embedded library documentation; they do not assert that every public function has a description. edge-case behavior for the broader standard-library API still needs dedicated acceptance tests. ## unused-argument hints @@ -69,53 +69,53 @@ the formatter indents case labels one level inside their switch by default (`Cas ## focused commands -run from the repository root. native binaries and the language server must already be built; the wasm command needs a built `src/web/zc.wasm`. +run from the repository root. native binaries and the language server must already be built; the wasm command needs a built `build/web/zc.wasm` (`make web`). ```sh -python3 tests/drivers/fuzz_compiler.py bin/zy bin/zym --js -python3 tests/drivers/fuzz_malformed.py bin/zy bin/zym -node tests/drivers/fuzz_wasm.js src/web/zc.wasm -python3 tests/drivers/image_growth_test.py bin/zy -python3 tests/drivers/code_emission_growth_test.py bin/zy -python3 tests/drivers/code_address_growth_test.py bin/zy +python3 tests/drivers/fuzz_compiler.py build/bin/zy build/bin/zym --js +python3 tests/drivers/fuzz_malformed.py build/bin/zy build/bin/zym +node tests/drivers/fuzz_wasm.js build/web/zc.wasm +python3 tests/drivers/image_growth_test.py build/bin/zy +python3 tests/drivers/code_emission_growth_test.py build/bin/zy +python3 tests/drivers/code_address_growth_test.py build/bin/zy python3 tests/drivers/seed_growth_test.py python3 tests/drivers/memory_limits_test.py -python3 tests/drivers/stack_growth_test.py bin/zy +python3 tests/drivers/stack_growth_test.py build/bin/zy python3 tests/drivers/layout_config_test.py -python3 tests/drivers/compiler_growth_test.py bin/zy -python3 tests/drivers/parser_growth_test.py bin/zy -node tests/drivers/fuzz_lsp.js bin/zclsp -node tests/drivers/lsp_growth_test.js bin/zclsp -node tests/drivers/protocol_growth_test.js bin/zclsp -node tests/drivers/core_lsp_test.js bin/zclsp -node tests/drivers/hover_values_test.js bin/zclsp -node tests/drivers/hover_features_test.js bin/zclsp -node tests/drivers/hover_surface_test.js bin/zclsp -CONTRACT_GROUP=inference node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -CONTRACT_GROUP=qualifiers node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -python3 tests/drivers/compiler_contract_test.py bin/zy --group qualifiers -python3 tests/drivers/compiler_contract_test.py bin/zym --group qualifiers -python3 tests/drivers/inference_growth_test.py bin/zclsp -node tests/drivers/storage_analysis_test.js bin/zclsp -node tests/drivers/hints_test.js bin/zclsp -node tests/drivers/include_test.js bin/zclsp -CONTRACT_GROUP=quickfix node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -CONTRACT_GROUP=modulehover node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -CONTRACT_GROUP=namespacecompletion node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -CONTRACT_GROUP=navigation node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp +python3 tests/drivers/compiler_growth_test.py build/bin/zy +python3 tests/drivers/parser_growth_test.py build/bin/zy +node tests/drivers/fuzz_lsp.js build/bin/zclsp +node tests/drivers/lsp_growth_test.js build/bin/zclsp +node tests/drivers/protocol_growth_test.js build/bin/zclsp +node tests/drivers/core_lsp_test.js build/bin/zclsp +node tests/drivers/hover_values_test.js build/bin/zclsp +node tests/drivers/hover_features_test.js build/bin/zclsp +node tests/drivers/hover_surface_test.js build/bin/zclsp +CONTRACT_GROUP=inference node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +CONTRACT_GROUP=qualifiers node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +python3 tests/drivers/compiler_contract_test.py build/bin/zy --group qualifiers +python3 tests/drivers/compiler_contract_test.py build/bin/zym --group qualifiers +python3 tests/drivers/inference_growth_test.py build/bin/zclsp +node tests/drivers/storage_analysis_test.js build/bin/zclsp +node tests/drivers/hints_test.js build/bin/zclsp +node tests/drivers/include_test.js build/bin/zclsp +CONTRACT_GROUP=quickfix node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +CONTRACT_GROUP=modulehover node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +CONTRACT_GROUP=namespacecompletion node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +CONTRACT_GROUP=navigation node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp node tests/drivers/codeactions_client_test.js node tests/drivers/ext_cache_test.js -python3 tests/drivers/array_methods_test.py bin/zy -python3 tests/drivers/array_methods_test.py bin/zy --case map -python3 tests/drivers/array_methods_test.py bin/zy --case filter -python3 tests/drivers/array_methods_test.py bin/zy --case search -python3 tests/drivers/array_methods_test.py bin/zy --case reduce +python3 tests/drivers/array_methods_test.py build/bin/zy +python3 tests/drivers/array_methods_test.py build/bin/zy --case map +python3 tests/drivers/array_methods_test.py build/bin/zy --case filter +python3 tests/drivers/array_methods_test.py build/bin/zy --case search +python3 tests/drivers/array_methods_test.py build/bin/zy --case reduce python3 tests/drivers/cli_help_test.py python3 tests/drivers/runtime_image_test.py -CONTRACT_GROUP=magic node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -CONTRACT_GROUP=strings node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -CONTRACT_GROUP=arguments node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -CONTRACT_GROUP=modules node tests/drivers/toolchain_lsp_contract_test.js bin/zclsp -node tests/drivers/nav_test.js bin/zclsp -tests/runerr.sh bin/zy tests/errors +CONTRACT_GROUP=magic node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +CONTRACT_GROUP=strings node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +CONTRACT_GROUP=arguments node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +CONTRACT_GROUP=modules node tests/drivers/toolchain_lsp_contract_test.js build/bin/zclsp +node tests/drivers/nav_test.js build/bin/zclsp +tests/runerr.sh build/bin/zy tests/errors ``` -- cgit v1.2.3