1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
# historical test results and maintenance notes
this material records earlier builds. current work and verification are tracked in TODO.md.
## test results
the whole of `test.sh` is 614 checks on a machine with gcc, qemu-user, llvm, node and clang's wasm32 target (572 passed, 0 failed; the baremetal boards need cross compilers and are skipped without them). on the interpreter alone it starts with the fixpoint (236652 words), an up-to-date `genome.h`, and then every example and feature test, including a shift past bit 31 and equality of a value above 2^31. it also covers joins and fixup frees (with a leak count), `switch` and `?:`, the buddy allocator, aliases used while their owner is alive, quarantine (a freed block stays intact for a stale alias, a second free does nothing, and a collection keeps a block a word still points at), `do`/`while` with `enum` and a cast, `include` and the string helpers, syscalls, memdebug leaks, use-after-free (including through an interior pointer), double free, payload poison and interior pinning, several errors in one run, all three stack overflows, growable and fixed arrays, and `tests/o3.zc` (loops, recursion, calls inside loops, `switch`, `&& ||`, structs through pointers, arrays, division by constants, and a function with more variables than registers). it checks that every o3 region in that test lifts, that the o3 bounds check fires, and that all 26 programs in `tests/errors/` (including derived aliases used after their owner dies), all 10 in `tests/typeerrors/` and all 9 in `tests/langerrors/` get rejected with the expected message. the same script then runs that list on the microcore, checks that the microcore's self-compile matches the interpreter's image, and checks that `micro.c` and `port_stub.c` compile with `-ffreestanding`. that is 113 checks with the jit and llvm skipped. it then builds the 339968-word compiler and runs 22 programs on the small microcore and again on `core.c` at that size, builds the 32768-word run-only image for six programs, and checks that both reduced microcores compile freestanding. that is 165 checks with the jit, llvm and bare-metal boots skipped. when the cross gcc and `qemu-system-*` for a target exist, `test.sh` adds 7 boots on that target (run-only hello, fact, wide and gc, then the resident compiler fed hello, fact and wide). both virt boards were present here, which is 14 more. the jit needs `qemu-aarch64`, the linux-user emulator, which `qemu-system-aarch64` is not. the llvm backend needs `llvm-config`. those two were not executed in the environment that cut the word to 64 bits. they were executed later (stage 24), on this host, and the full script then passed 292 checks. stage 25 added three more (the default map matches the checked-in sources, and the flag form of `genmap` reproduces the small and run maps), 295 passed, 0 failed. stage 26 added eight more (echo until eof, the clock and RAM-file line, the exit text and the exit code, on each virt board), 303 passed, 0 failed. stage 27 added twelve more: the float line on the interpreter, the microcore, the arm64 jit and the llvm backend, the same line on the small microcore and on `core.c` at that size, the run-only image, both virt boards running the program and compiling it, and one llvm run with `ZYHOT=1` so every float block is compiled. 315 passed, 0 failed. stage 28 added twelve more, the same shape of check for byte loads and stores, including a one-byte patch of a live literal. 327 passed, 0 failed. stage 29 added twenty-three more for packed byte strings: the string line, the example, and the hosted byte-file check on the interpreter, the microcore, the arm64 jit and the llvm backend; the string line and the example on the small microcore and on `core.c` at that size; the run-only image of both; both virt boards running the program and compiling it; and one llvm run with `ZYHOT=1`. 350 passed, 0 failed. stage 30 added eighteen more for shared cells: the sharing line and the memdebug line on the interpreter, the microcore, the arm64 jit and the llvm backend; both of those on the small microcore and on `core.c` at that size; the run-only sharing line; both virt boards running it and compiling it; and one llvm run with `ZYHOT=1`. 368 passed, 0 failed. stage 31 added one more: a soft-float interpreter matches the float line. 369 passed, 0 failed. stage 32 added twenty-two more: negative zero is false, and a float stays a float through calls, globals and fields, on the interpreter, the microcore, the arm64 jit and the llvm backend; both of those on the small microcore and on `core.c` at that size; the run-only image of the negative-zero program; both virt boards running that program and compiling it and the float-type program; a soft-float check of negative zero; and llvm runs with `ZYHOT=1` for both. 391 passed, 0 failed. stage 33 added ten more for o3 floats: the lifted kernel matches the plain one, a constant expression folds, a NaN minus itself stays a NaN, and negative zero is false inside the region, on the interpreter, the microcore, the arm64 jit and the llvm backend; the same line on the small microcore and on `core.c` at that size; both virt boards compiling it; a soft-float interpreter; and one llvm run with `ZYHOT=1`. 401 passed, 0 failed. stage 34 added thirteen more for the float text: shortest, fixed and scientific forms, a 17-digit tie, and the full finite range, on the interpreter, the microcore, the arm64 jit and the llvm backend; the same line on the small microcore and on `core.c` at that size; the run-only image; both virt boards running it and compiling it; a soft-float interpreter; and one llvm run with `ZYHOT=1`. 414 passed, 0 failed. the jit binary is `aarch64-linux-gnu-gcc -O2 -w -DJTYPE=int -static core.c io_linux.c` (gcc 13.3.0, static, against the aarch64 sysroot) run by `qemu-aarch64` 8.2.2. the llvm binary is host gcc 13.3.0 with `-DZYLLVM` and LLVM 18.1.3 (`llvm-config --cflags --ldflags --libs`). both ran the same program list as the interpreter and self-compiled to the same image. `ZYTRACE=1` on `tests/o3.zc` printed 8 `bytes of arm64` lines and 8 `compiled by llvm` lines. `ZYHOT=1` on the llvm binary made `tests/types.zc` match the interpreter byte for byte. the first jit run failed one check: `tests/enhance.zc` printed `100 0 343 3,2` and then halted. opcode 21 calls a C function that returns a 32-bit `int`. a write to `w0` zeroes the high half of `x0`, so the failure code `-4` (the `noisy` enhancement, whose body does i/o and does not lift) was stored as a large positive cell. the compiler treated the install as a success, emitted the unwired opcode, and the jit halted. the translator now does `sxtw x0, w0` before storing that cell. the Darwin jit (`MAP_JIT` and the write-protect around each flush) was not run. this host is x86_64 Linux, and qemu-user is the arm64 cpu the jit actually executed on. the 64-bit image is not binary-compatible with a 32-bit genome.
## open ends
status after stage 48: done from the older lists below are native calls that the optimizer sees, a heap that is no longer limited by the initial map, host-assisted fill, copy and scan, sized integers and f32 with packed arrays and packed struct fields, growable `[]sized`, f32 in `extern` and `cfunc`, the micro-ops `ctz popcnt bswap rotl rotr f2s s2f`, page-indexed invalidation, per-slot install invalidation and per-block llvm resource trackers (the self-modifying benchmark in `bench/selfmod.zc` runs about 7 times faster on the jit). o3 calls on llvm take a shorter path into the callee (about 25 percent off a call-bound loop). not done: struct by value and struct returns on apple arm64, a fully inline llvm-to-llvm o3 call (calls still go through a c helper), loop-invariant motion and cross-block cse in the core's own optimizer, automatic tiering without the `o3` keyword, floats and the f2s and s2f ops in the javascript vm, and re-running the windows scripts and the bare metal boards after the map change.
fixed in stage 13 (details in `history/13-gap-fixes`): branch and loop joins, free lists that never merged (now a buddy allocator), missing stack checks, stopping at the first error, macro line numbers, required prototypes, `d*` for storing into borrowed objects, `**` declarations, and `?:` and `switch`. fixed in stage 18 (details in `history/18-aliases`): untracked aliases. a pointer computed from a tracked variable (`p + k`, a pointer field, an index whose element is a pointer) is stored as a borrow of that variable, on the same child link the analyzer already keeps, so using it after the owner dies is an error. `free` quarantines the block; the next sweep only reclaims quarantined blocks nothing points at, so a stale alias reads a dead but intact block and a second free is a no-op. fixed in stage 19 (details in `history/19-tools`): memory debug (header checks, allocation lines, `memreport`, poison, interior pinning), `do`/`while`, `enum`, C casts, `include`, a small syscall surface and `lib/std.zc`. fixed in stage 20 (details in `history/20-words`): every cell is 64 bits, on the interpreter, the arm64 jit and the llvm path. fixed in stage 21 (details in `history/21-micro`): `micro.c` is a portable interpreter for the same genome, with the host behind `micro_port.h`. fixed in stage 22 (details in `history/22-size`): the memory map is chosen at build time, with a 221184-word compiler and a 32768-word run-only image. fixed in stage 23 (details in `history/23-baremetal`): the microcore boots with no OS on QEMU's RISC-V and AArch64 `virt` boards and prints on the emulated UART. fixed in stage 24 (details in `history/24-backends`): the arm64 jit and the llvm backend were actually executed, and the jit's install return is sign-extended so a failed enhancement takes its fallback. fixed in stage 25 (details in `history/25-genmap`): one layout generates every memory size. the small compiler, the run-only image and the historical 1m map are presets of it. fixed in stage 26 (details in `history/26-bareapi`): the virt boards answer the syscall gate (console EOF, a hardware clock, exit status, a RAM file store, random, memory info) and `port_stub.c` still returns -1. fixed in stage 27 (details in `history/27-float`): IEEE-754 binary64 on the interpreter, the arm64 jit, the llvm backend, the microcore and both virt boards. the literal is opcode 600 plus the raw bits, and `fops.h` prints and parses the same text everywhere. fixed in stage 28 (details in `history/28-bytes`): byte addresses beside word addresses, opcodes 612..617. fixed in stage 29 (details in `history/29-strings`): packed byte strings (`b"..."`) beside the word strings, with length, index, slice, concat, comparison, `ston`/`ntos`, and syscall ops 10..13. fixed in stage 30 (details in `history/30-sharing`): shared cons cells and strings on the existing heap. bit 8 of the header is the share bit, `free` does not quarantine one, and the sweep collects it when nothing points at it. `LEARN.md` walks through the language. the c abi came later (see "c abi"). a word string is still one character per word; a byte string is eight characters per word. the microcore does not run o3 as a fast path. the small compiler does not self-host, and a 60000-word array does not fit in its heap. still open:
**the conservative collector.** opcode 619 pushes the data-stack index, and the collector marks from there to the top, so a word the program has already dropped is not a root. `scrub` zeroes the unused region from the bottom up to the live top. the frame stack is still scanned whole. zeroing a frame in the prologue would do the same there, at a few words per call.
**invalidation.** done in stage 48; see the self-modification paragraph in the jit section. what remains is only that a covered word whose count saturates at 255 is never decremented back to zero until a full flush, which costs a hook call on stores to it and nothing else.
**automatic tiering.** dropped. the o3 keyword stays the way to get a native region for a function. the reasons (the region record, the calling convention for dynamic arrays, the compile-time o3 checks) are in design/tiering.md.
**o3's own gaps.** on the jit, an o3 call to another compiled region is a direct call. on llvm a call to a compiled region is now inlined as a spill, a check, a frame setup and an indirect call through the callee's native entry, with no reload on the fast path (fib went from 0.69 s to 0.39 s), and the c helper is only the slow path. it is not yet a register-passing direct call, which costs about 12 ns where c costs 3 to 4. the core now has a global pass after the per block one: a constant lattice per variable across the region's blocks, refolded through the same emitter, so a branch on a known value is folded away. push and pop on a growable array are inline. a value that stays on the vm stack across a branch goes through memory, since blocks only share variables. there is still no loop-invariant motion or cross-block cse in the core (llvm does it on its build), no automatic tiering, and struct by value and struct returns in native calls are done for x86-64 and arm64 (see the extern note above), not for apple arm64. regions aren't built into images, so `rt.zc` and the compiler can't use o3, and o3 code that rewrites its own region sees the change at the next entry. `ZYNOGLOBAL=1` turns the global pass off.
**generics are checked at their instances.** a generic function's body is not compiled where it is written: its tokens are kept and each set of type arguments gets its own copy, compiled after the item that asked for it, so an error inside a template body shows up when something instantiates it (like a c++ template), and a generic name used as a value has the template's type and halts if it is ever called. a struct passed to a generic parameter by value is passed by pointer, as everywhere else in zc. the template has to be written before the first call that gives it type arguments, since a call to a name that is not yet known as generic is read as an ordinary forward call. an unbound parameter is `word`.
**check mode and closures.** a closure copies what it uses when it is made, so a closure that changes a captured variable changes its own copy, not the function's. the hints are heuristics on one function at a time: they don't follow a value through a call, and a stack buffer hint doesn't check that the size is what you wanted. after a fatal error in check mode (running out of locals, say) the compiler unwinds and can print one follow-on error. if a document makes the check run for 3 million tokens without unwinding the server stops, which hasn't happened outside of a deliberately broken parse word. the windows scripts were updated by reading, not run.
**the rest.** the prescan only knows tags, not types, so a call to a typed function that's defined further down gets `word` as its result type until the definition shows up. the llvm backend compiles one module per block, so it warms up slowly. stores done by enhancements don't invalidate compiled code. a parse word that changes the syntax can confuse the prescan. it only records signatures, though, and a real definition always overrides what the prescan guessed.
---
|
