diff options
| -rw-r--r-- | TODO.md | 94 |
1 files changed, 94 insertions, 0 deletions
@@ -174,6 +174,93 @@ and pushed there. - prose everywhere is lowercase with plain punctuation; `//` line comments, `/* */` only between code; a comment above a declaration is its hover text. +### vision and direction + +the coordinator's view of where zc stands and where it should go, recorded so +the reasoning behind the open items survives the session. + +- what zc gets right: the compiler lives in memory next to the program, so + `eval`, parse words, immediate functions and library-defined syntax (actors + with no actor code in the compiler) fall out of the architecture: forth's + strength with c's syntax. the ownership analyzer infers most of what other + languages make you annotate (it already infers that `skip(s)` returns a + borrow of `s`). unchecked code falls back to a conservative collector instead + of undefined behavior. the language server is the real compiler in check + mode, so diagnostics cannot drift from compilation. one image runs on the + interpreter, microcore, javascript, wasm, arm64 jit and llvm. +- what holds it back: trust and visibility. until `docs/SPEC.md` there was no + spec, and the two implementations disagreed (`&&`/`||`) without anyone + noticing. run-time failures carry no position and a wild read can exit 0. + "checked" is not purely static: turning the checker on inserts cleanup that + can change behavior (the reverted `unchecked` trial hung `share.zc`). + friction: the toolchain's own sources must stay in zc0's subset; word-per- + character strings cost eight times the memory of `bstr` and split every api + in two (`put_s` and `put_bs`); the compiler's state is global parallel + arrays; on x86 the language server is interpreted (about 18 s to check the + compiler). +- priorities, in order: things that turn wrong behavior into a located, + reproducible failure come first, because everything else (fuzzing, a proving + scaffold, outside users) depends on them: runtime line tables, bounds + checking, and the spec with its parity runner and conformance suite. then the + language features that make zc pleasant (`defer`, destructors, variadics and + `std.print`, test blocks, borrow returns for declarations). then speed (the + x86-64 jit). +- bootstrap philosophy: zc0 is an independent implementation in c that never + retires (like go 1.4, unlike rust's downloaded stage 0 or ocaml's and zig's + committed binaries), so the system builds from source with only a c compiler + and no committed blob. its value is being small, so it grows only where a + feature is cheap and produces no code of its own (tags, recasts, byte + strings, maybe enums and `?:`), never structs. the parity runner is a small + form of wheeler's diverse double-compiling: the two front ends check each + other. +- managing the compiler's global state: (1) a cross-reference report of + writers and readers per global, checked in ci; (2) group each family of + parallel arrays in one place with its invariants written once, and route + snapshot and restore through one pair of functions; (3) the split into phase + files (done). do not try to make the state non-global: the single-pass design + is fast because of it; the goal is visible ownership of state, not removal. +- debugging, beyond line tables and bounds checks: every effect a program has + goes through one syscall gate, so record and replay is cheap here (log each + gate result, replay a run exactly, then add tracing or watchpoints at will). + once line tables exist, a debug adapter for vscode (breakpoints, stepping, + locals) on top of the interpreter is the next step. +- a proving scaffold (the owner asked for an opinion): yes, but check before + prove. a written semantics comes first (the spec, then a small executable + reference evaluator). grow it out of the analyzer that already exists rather + than beside it. order: stack-effect checking for every function and `asm` + block (the commonest invisible bug on a stack machine); declared side + effects at module boundaries (globals touched, syscalls or not) checked + against the effect masks the compiler already computes; `requires`/`ensures` + contracts compiled to run-time checks under a switch, with the easy ones + discharged statically from the value facts. keep the trusted base small and + outside the self-compiling loop (a prover inside the genome can approve its + own mistakes, as the fixpoint approved the precedence bug); prefer + translation validation of each compile over proving the compiler whole. +- native code: the hand-written jit exists only for arm64; the llvm backend + (`-DZYLLVM`) is the portable native tier but the default build leaves it + out. cheap win: build the llvm core by default whenever `llvm-config` is + found. the x86-64 port of the hand-written jit is worth it for compile + latency (microseconds per block against milliseconds for llvm), which + matters for short programs and the language server. + +### feature decisions (owner's requirements, settled) +- bounds checking is on by default; `unbounded` (used like `unchecked`: + function prefix, block, top-level statement) turns it off; `o3` implies + `unbounded`. +- `defer` runs at the end of the current block on every exit path, like the c + defer proposal, plus ownership awareness; destructors (`fn drop()`) exist as + well. +- borrow returns: inferred from the body wherever possible; any spelling for + declarations must stay unobtrusive (proposed: `borrows s` after the + parameter list). +- `string` is a built-in alias of `char*` (done); c typedefs (done). +- variadics: count and per-argument types available, several packs per + function when element types differ (`void f(...int a, ...char b)`), a mixed + `...any` pack, and `...*any` for comma-free calls; `std.print` is a global, + javascript-like print built on it. +- struct-level untyped constants (done), typo suggestions for members (done). +- every new agent brief starts with "read ONBOARD.md first". + ### done in this session (all merged) - build from source with c23 `#embed`, everything generated under `build/`, `ONBOARD.md`, the embed declaration (`embed "f" as [type] name;`). @@ -238,6 +325,13 @@ brief someone without re-reading the code. - [ ] (in flight) runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing ` at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it. - [ ] borrow returns tied to an argument. inference already works for a callee defined before the call: `char* skip(char* s) { ...; return s + i; }` makes `let r = skip(t); rt.free(t); r[0]` an error. the gap: when the call comes before the callee's definition, nothing is reported and the program reads freed memory. extend the deferred effect requirements (`effect_require`, `effect_finish` at unit end) so a forward-called function whose return turns out to alias a parameter produces the same error. for bodyless declarations, externs and function types, add an unobtrusive spelling, for example a `borrows s` clause after the parameter list (`char* skip(char* s) borrows s;`), checked against the body when one exists. - [ ] variadic arguments and `std.print`. design: a pack is passed as one hidden argument, a caller-built read-only growable-array handle (`fixed T[?]` view: [len, cap, items, element size] in the caller's frame, so `len(a)` and `a[i]` work unchanged and nothing is allocated). `...int a` is a typed pack; several packs per function when their element types differ (`void f(...int a, ...char b)`): each extra argument goes to the pack whose element type matches its static type, an ambiguous match is an error, packs come after the fixed parameters. `...any args` packs each value with a kind code (int, float, char, string, bstr, pointer, ...) and its type record: `any` is a small built-in record { kind, value, type }. `...*any args` makes the call comma-free: after the fixed arguments, the parser reads unary expressions until `)`, commas optional, so `std.print("x = " name ", age " age "\n");` works and binary expressions must be parenthesized. `std.print(...*any args)` prints each argument by its kind (`put_n`, the float printer, `put_s`, `put_bs`, a character for `char`, hex for pointers). +- [ ] build the llvm core by default when `llvm-config` is available (the makefile falls back to the interpreter otherwise), so x86 hosts get a native tier and the language server stops being interpreted; keep `ZYNOLLVM=1` and `-DZYINTERP` as the opt-outs. +- [ ] stack-effect checking: every function and `asm` block declares or is inferred to have a fixed data-stack effect, and the compiler reports a block whose paths leave the stack unbalanced. the first step of a proving scaffold. +- [ ] declared side effects at module boundaries (globals touched, syscalls or not), checked against the effect masks the compiler already computes. +- [ ] `requires`/`ensures` contracts on functions, compiled to run-time checks under a switch, with the cases the value facts settle (constants, simple ranges) checked statically. +- [ ] record and replay: a core option that logs every syscall gate result to a file and replays a run from the log exactly, for reproducing failures and adding tracing after the fact. +- [ ] a debug adapter for vscode (breakpoints, stepping, locals) on the interpreter, built on the line tables. +- [ ] reduce the cost of word-per-character strings: measure, then decide between making `bstr` the default string, unifying the two apis behind one name, or both. - [ ] continue fuzzing the compiler, analyzer, runtime, standard library, lsp, build tools and generated artifacts, including analyzer properties, memory-limit edges, compiler image mutation and long-running lsp memory use; feed generated programs to `make parity` for differential fuzzing between zc0 and the self-hosted compiler; fix discovered bugs. - [ ] keep improving the borrow checker: fewer annotations in ordinary code, clearer messages naming both the owner and the conflicting use, and new fixtures for each rule that changes. - [ ] run the full suite once everything above is done (`make test JOBS=4`; `make check` is the quick one), fix what it finds, and only then tick this item. |