diff options
Diffstat (limited to 'TODO.md')
| -rw-r--r-- | TODO.md | 22 |
1 files changed, 14 insertions, 8 deletions
@@ -163,8 +163,11 @@ and pushed there. `ZC_TEST_TIMEOUT_SCALE=4 node tests/drivers/lsp_test.js build/bin/zclsp`, `ZC_TEST_TIMEOUT_SCALE=4 node tests/drivers/hint_fixes_test.js build/bin/zclsp`, `make parity` (zc0 against the self-hosted compiler; by - design differences live in `tests/parity.known`). on x86 the language server - is interpreted and slow; `ZC_TEST_TIMEOUT_SCALE` stretches the deadlines. + design differences live in `tests/parity.known`). on x86-64 linux `zy` and + the language server now run on the x86-64 jit (o3 regions still interpreted); + `ZC_TEST_TIMEOUT_SCALE` stretches the deadlines on slower hosts. + `bash tests/drivers/jit_x64_test.sh build/bin/zyi build/bin/zy` compares the + jit with the interpreter (`make build/bin/zyi`) on every program. - work was done by delegating each item to a separate agent in its own git worktree (outside the repo, under `/workspace/tinyvm-wt/`) and merging its branch. every brief started with "read ONBOARD.md first" plus the sections relevant @@ -196,8 +199,8 @@ the reasoning behind the open items survives the session. friction: the toolchain's own sources must stay in zc0's subset; word-per- character strings cost eight times the memory of `bstr` and split every api in two (`put_s` and `put_bs`); the compiler's state is global parallel - arrays; on x86 the language server is interpreted (about 18 s to check the - compiler). + arrays; on x86 the language server was interpreted (about 18 s to check the + compiler) until the x86-64 jit landed. - priorities, in order: things that turn wrong behavior into a located, reproducible failure come first, because everything else (fuzzing, a proving scaffold, outside users) depends on them: runtime line tables, bounds @@ -236,7 +239,8 @@ the reasoning behind the open items survives the session. outside the self-compiling loop (a prover inside the genome can approve its own mistakes, as the fixpoint approved the precedence bug); prefer translation validation of each compile over proving the compiler whole. -- native code: the hand-written jit exists only for arm64; the llvm backend +- native code: the hand-written jit exists for arm64 and (phase 1, no o3 + regions or inlined enhancements yet) x86-64 linux; the llvm backend (`-DZYLLVM`) is the portable native tier but the default build leaves it out. cheap win: build the llvm core by default whenever `llvm-config` is found. the x86-64 port of the hand-written jit is worth it for compile @@ -283,8 +287,9 @@ these were running as agents in worktrees. if their branches were not merged before the session ended, the work is lost and each item below must be started again from its description in the open list. a zip of the repository with every worktree is uploaded to the owner after each update. -- spec verification (`tests/conformance/`), see "verify both compilers". -- x86-64 jit phase 1, see "a hand-written x86-64 jit". +- spec verification: the first agent was stopped by a session interruption + and its partial `tests/conformance/` was not merged; start it again from + "verify both compilers". - runtime line tables and backtraces, see "runtime source positions". - bounds checking with `unbounded`, see "bounds checking". - the cross-reference report, see "a cross-reference report". @@ -320,11 +325,11 @@ brief someone without re-reading the code. - [ ] test blocks: `test "name" { ... }` with `expect(cond)`, compiled only by `zy --test file.zc`, which runs each block and prints one line per test with the position of the first failed `expect`; skipped at parse time otherwise. build it as a library through `run.syntax` and `run.statement_hook`, like `src/lib/actor.zc`. - [ ] (in flight) bounds checking on by default, `unbounded` to turn it off (function prefix, block, top-level `unbounded;`, mirroring `unchecked`), implied by `o3`. architecture: indexing is compiled in the postfix `[` handling of the expressions part; growable arrays already check (`dat`, `range_jz`, the `outside` enhancement, `proven()` removing checks a loop test already did). fixed arrays `T[n]` with a variable index get the same inline check against the constant `n`. pointers get a runtime call (for example `rt.bchk(p, i)`) that checks only when `p` is the start of a live heap block (header check word `p ^ MAGIC`, `paylen`), skipping constant index 0 and sizes the analyzer knows (`lnsz`). reports go through `oob`. the runtime, core, compiler parts, server, `run.zc` and `build.zc` start with `unbounded;`, which zc0 accepts and ignores (genome bytes unchanged). keep std, icon and actor checked unless a function touches raw memory on purpose. measure interpreter time before and after. - [ ] after the fuzzing work, add more general functionality to the standard library. -- [ ] (in flight) a hand-written x86-64 jit, ported from the arm64 one in `src/core/core.c` (the `#ifdef ZYJIT` section; `ZYJIT` is only defined for `__aarch64__`). phase 1: a separate macro (for example `ZYJIT_X64`), shared architecture-neutral parts (translation table `tab`, coverage `cov`, invalidation `be_drop`, block boundaries, dispatcher) and an x86 emitter. registers: r12 memory base, r13 data stack, r14 return stack, r15 translation table, rbx state block (the arm64 x19..x23); system v calls to c helpers. semantics identical to the interpreter (wrapping, 0/1 comparisons, literal encoding, stack and `MW` checks, retranslation after stores into code). o3 regions take the portable path in phase 1. validate every `tests/programs`, `examples` and `examples/learn` program on interpreter and jit with identical output (a driver under `tests/drivers/`), the rejection suites, icon and actor fixtures; report timings for a compute example and for compiling the compiler. make it the default x86-64 linux core only when everything matches; `-DZYINTERP` still forces the interpreter. phase 2: o3 regions and inlined enhancements. note the llvm backend (`-DZYLLVM`) is the other native path; the default build could also enable it whenever `llvm-config` is found. - [ ] struct destructors: a struct may declare `fn drop()`, called wherever the compiler frees an owned value of that type, before the memory is released. architecture: the compiler frees owned locals in `freevar` (analyzer part), called from `exit_vars`, reassignment and explicit `rt.free`/`rt.recycle` calls (`s_free`, `s_recycle` handling in the call code); when the local's static type is a pointer to a struct with a `Name.drop` method, emit a call to it first. objects freed by the collector or inside unchecked code do not run `drop`: document that. first users: file descriptors in `std.Host`, processes in `actor.zc`. - [ ] (in flight) runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing ` at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it. - [ ] borrow returns tied to an argument. inference already works for a callee defined before the call: `char* skip(char* s) { ...; return s + i; }` makes `let r = skip(t); rt.free(t); r[0]` an error. the gap: when the call comes before the callee's definition, nothing is reported and the program reads freed memory. extend the deferred effect requirements (`effect_require`, `effect_finish` at unit end) so a forward-called function whose return turns out to alias a parameter produces the same error. for bodyless declarations, externs and function types, add an unobtrusive spelling, for example a `borrows s` clause after the parameter list (`char* skip(char* s) borrows s;`), checked against the body when one exists. - [ ] variadic arguments and `std.print`. design: a pack is passed as one hidden argument, a caller-built read-only growable-array handle (`fixed T[?]` view: [len, cap, items, element size] in the caller's frame, so `len(a)` and `a[i]` work unchanged and nothing is allocated). `...int a` is a typed pack; several packs per function when their element types differ (`void f(...int a, ...char b)`): each extra argument goes to the pack whose element type matches its static type, an ambiguous match is an error, packs come after the fixed parameters. `...any args` packs each value with a kind code (int, float, char, string, bstr, pointer, ...) and its type record: `any` is a small built-in record { kind, value, type }. `...*any args` makes the call comma-free: after the fixed arguments, the parser reads unary expressions until `)`, commas optional, so `std.print("x = " name ", age " age "\n");` works and binary expressions must be parenthesized. `std.print(...*any args)` prints each argument by its kind (`put_n`, the float printer, `put_s`, `put_bs`, a character for `char`, hex for pointers). +- [ ] x86-64 jit phase 2: compile o3 regions to x86-64 (port `rcompile` and the region register allocation; regions now run on the region interpreter and are 2 to 4 times slower than their plain versions on this jit) and inline enhancements (lower the optimized micro-ops to x86 registers as the arm64 path does, instead of calling `xrun`). validate with `tests/drivers/jit_x64_test.sh` and `tests/bench/bench.sh`, which now has an x86-64 jit column. add the driver to `tests/test.sh` on x86-64 linux. - [ ] build the llvm core by default when `llvm-config` is available (the makefile falls back to the interpreter otherwise), so x86 hosts get a native tier and the language server stops being interpreted; keep `ZYNOLLVM=1` and `-DZYINTERP` as the opt-outs. - [ ] stack-effect checking: every function and `asm` block declares or is inferred to have a fixed data-stack effect, and the compiler reports a block whose paths leave the stack unbalanced. the first step of a proving scaffold. - [ ] declared side effects at module boundaries (globals touched, syscalls or not), checked against the effect masks the compiler already computes. @@ -337,6 +342,7 @@ brief someone without re-reading the code. - [ ] run the full suite once everything above is done (`make test JOBS=4`; `make check` is the quick one), fix what it finds, and only then tick this item. ## done in the sorted list +- [x] a hand-written x86-64 jit, phase 1 (`ZYJIT_X64` in `src/core/core.c`, default on x86-64 linux; `-DZYINTERP` and `make build/bin/zyi` give the interpreter). r12 memory, r13 data stack, r14 return stack, r15 table, rbx state; stack slots addressed with a virtual offset so r13 moves only at calls and exits; shares the table, coverage map, store hook and invalidation with the arm64 jit. enhancements are called through `xrun`, o3 regions run on the region interpreter (`ZYNOO3=1` runs their vm code through the jit instead). 179 of 179 programs agree with the interpreter (`tests/drivers/jit_x64_test.sh`), the self-compile is byte-identical and about 7x faster (1.5 s against 11 s), and the rejection, learn, icon, actor, lsp and hint suites pass on it. - [x] remove temporary markdown files (`TASKLIST.md`, `docs/design/handoff.md` deleted; `BOOTSTRAP-OWNERSHIP.md` moved to `docs/history/`). - [x] verify every command in `ONBOARD.md` from a clean checkout and fix the ones that drifted. |