# tinyvm zygote a small stack cpu began as a C89 code golf entry and became a self-hosting runtime. a native core runs vm code with an arm64 jit, through llvm on supported builds, or in the interpreter. the genome contains a compiler for zc, a C-like language with forth influences. the compiler resides in vm memory alongside the program; code can call its functions, modify its tables, add syntax, and compile new code at runtime. heap objects carry an ownership tag (u, m or d) checked by a static analyzer, with deterministic frees at scope ends and a mark-sweep collector underneath. types are a compiler primitive, with structs, generics, reflection and an optional typechecker. programs can install up to 512 custom opcodes (virtual enhancements), which the jits inline as native code. code marked `o3` bypasses the stack machine: the core compiles the region as a unit and keeps its variables in registers. this folder has the latest code. `docs/LEARN.md` is a tour of the language. `docs/history/` is the scrapbook: notes for every stage, plus frozen code from the stages that still have it, archived as `docs/history/code-snapshots.tar.gz`. ## files `src/core/core.c` is the native core. it contains the arm64 jit, the llvm backend (behind `-DZYLLVM`), the portable interpreter, the enhancement unit (recipe parser, vm-code lifter, optimizer and lowering), and the o3 region compiler. `src/runtime/rt.zc` provides i/o, native division enhancements, the heap, and the collector; `src/compiler/compiler.zc` is the compiler. both are written in zc and begin with an overview. together they make up the genome. `src/tools/zc0.c` is stage 0 of the bootstrap, a c compiler for the subset of zc used by `src/compiler/compiler.zc`. `src/tools/build.zc` has the resident compiler compile its own source into a fresh image, and `src/tools/mkgenome.c` checks that image against the memory map before it becomes `build/core/genome.bin`. an image is a file of little-endian 64-bit words, and every core takes its genome in at compile time with c23 `#embed` (`src/core/genome.h`). `examples/` contains small programs (hello, factorial, echo, self-patching code, an owning list, a repl, a typed program, and an enhancement demo) and larger proofs of concept: - `mandel.zc`: the mandelbrot set in 16.16 fixed point, with the fixed-point multiply wired in as a custom opcode. - `bf.zc`: a brainfuck compiler. it turns brainfuck into zc source, has the resident compiler compile it, and runs it (native code on the jit). it takes a program after a backtick, or runs hello world. - `dump.zc`: prints any heap struct, nested ones included, with no printing code, by walking its type through reflection. - `cli2.zc`: reads the command line and the environment, then captures its own output into a string. - `tree.zc`: a binary search tree whose nodes own their children. there's no `rt.free` anywhere, and the live object count drops back to 0 when the root's block ends. - `lang.zc`: extends the language at compile time with a `|>` pipe operator (one table entry, since a stack machine implements it with `call`), `unless`, an `assert` that reports its source line, and `times n { }`. - `staged.zc`: writes a polynomial as source with its coefficients baked in, compiles it while running, and finds it by name. - `quine.zc`: prints its own source. - `life.zc`: conway's life on a typed 2d board, with a glider wrapping around a torus. - `primes.zc`: an o3 sieve over a fixed array, the primes gathered into a growable one, then twin primes and the widest gap. `tests/` contains feature tests, `tests/errors/` contains programs the lifetime analyzer must reject, `tests/typeerrors/` contains programs the typechecker must reject, and `tests/langerrors/` covers o3 and array misuse. each error test begins with its expected message; `tests/runerr.sh` runs a test folder. `tests/test.sh` bootstraps the compiler and runs the suite on the interpreter, microcore, and available jit and llvm backends, then repeats a subset on smaller maps. `tests/bench/bench.sh` compares o3 with plain code on each backend. `src/lib/std.zc` is embedded as source in the compiler and explicitly compiled by programs with `include std;`. each file has its own namespace and imports; including a dependency does not import it into the caller. use `include rt, std;` when a file needs both runtime APIs and standard-library helpers. use `include core;` for the core gate (`core.syscall`), and `include run;` for compiler APIs such as `run.evals`. `docs/LEARN.md` is a language tour, and `docs/history/` keeps earlier versions. `src/micro/micro.c` is a portable interpreter for the same genome; `src/micro/micro_port.h` defines its host interface, with `src/micro/port_linux.c` as the hosted port and `src/micro/port_stub.c` as a bare-metal starting point. `src/baremetal/` boots the microcore without an operating system under `qemu-system-riscv64` and `qemu-system-aarch64`. `src/core/memmap.h` defines the default map and initial reservations. `src/tools/genmap.c` writes smaller maps and matching source files; `src/runtime/run.zc` is the runtime-only boot stub. `namespace app;` gives declarations in a file the `app` namespace. imports are file-local: a dependency's `include` statements do not import their namespaces into the caller. the learning examples [27](examples/learn/27-namespaces.zc) and [28](examples/learn/28-string-inference.zc) show a namespaced file with multiple imports and the inferred types of both string literal forms. ## build and run run these commands from the repository root. everything built or generated goes under `build/`: executables in `build/bin/`, the default memory map in `build/core/map/`, the compiler genome in `build/core/genome.bin`, and the javascript, web and editor builds beside them. the tree holds sources only, `make clean` removes `build/`, and every target builds from an empty `build/`: ```sh make build/bin/zy < examples/hello.zc make test JOBS=4 make genome make web make js make run PROG=examples/hello.zc ``` the cores embed their genome with c23 `#embed`, so the c compiler has to be clang 19 or gcc 15 or later. `src/tools/findcc.sh` finds one (`CC` set on the command line or in the environment is used as given, and must support it too). `make genome` bootstraps the compiler, compares two self-compiled images, and writes `build/core/genome.bin` only after they agree; `make` reruns it whenever a source of the genome changes, and the cores that embed it are rebuilt after it. `make check` runs the smaller suite while iterating. `ONBOARD.md` is a reading and verification order for getting to know the code. ### command-line help the native vm, portable vm, language server, bootstrap compiler, image checker and map generator accept `--help` or `-h`. help exits before reading input or allocating vm memory, so it also works with stdin left open. `build/bin/zy` reads source from stdin and runs `main`; `build/bin/zym` runs the embedded image, and compiler images can read source from stdin. run-only images read program input directly. `build/bin/zclsp --format`, `--annotate` and `--types` remain available for source on stdin; without one of these options it reads language-server protocol messages. `build/bin/zc0` compiles bootstrap source from stdin to a genome image on stdout; `build/bin/mkgenome` checks an image from stdin against its memory map and copies it to stdout, and when the input is a compiler's error report instead of an image it prints the report. `build/bin/genmap` accepts a preset or compiler/run mode, optional output directory, and build-time map sizing options. the help for each tool lists its input/output format and memory environment variables where applicable. the same early help option is available in `tinyvm.sh`, `tinyvm.bat`, `src/tools/zcc`, the public `src/tools/*.sh` build scripts, and the javascript vm and extraction utilities. `src/tools/zcc [-o OUT] [-k] FILE.zc` builds a standalone run-only binary. `src/tools/automap.sh run|small|default|lsp OUTDIR FILE...` generates a sized map (`OUTDIR/memmap.h`) and image (`OUTDIR/genome.bin`). `src/tools/mklsp.sh [OUTDIR]`, `src/tools/mklspwasm.sh [OUTFILE.wasm]` and `src/tools/mkvsix.sh [OUTFILE.vsix]` build the language server and extension. `src/tools/findcc.sh [--wasm]` prints a compiler with `#embed` (and the wasm32 target), and `src/tools/bootstrap-check.sh` runs the first two bootstrap stages and prints the compiler's diagnostics on its own source. the javascript vm accepts an optional source file; without one it reads source from stdin. `src/js/mklayout.js` and `src/js/extract_lsp.js` write the javascript layout and the language server's image from the maps and images under `build/`. `python3 tests/drivers/cli_help_test.py` checks the native, micro, LSP, bootstrap and image tools, shell scripts, and javascript utilities with open stdin and invalid memory settings, rejects unknown options, and confirms the language server source options still work. `make js` writes the javascript vm into `build/js/`: `tinyvm.js`, its `layout.js` and a copy of `genome.bin` that it reads at start-up. it needs node.js and the typescript compiler (`tsc`). running the built `build/js/tinyvm.js` directly needs node.js only. ### interactive runners `./tinyvm.sh` opens the linux menu; `tinyvm.bat` opens the windows menu. each runner builds, runs, checks source, starts the repl, and runs tests. direct commands use the same entry point: ```sh ./tinyvm.sh build ./tinyvm.sh run examples/hello.zc ./tinyvm.sh run-micro examples/hello.zc ./tinyvm.sh lsp examples/opt_test.zc ./tinyvm.sh js examples/primes.zc ./tinyvm.sh test ``` ```cmd tinyvm.bat build tinyvm.bat run examples\hello.zc tinyvm.bat lsp examples\opt_test.zc tinyvm.bat test ``` windows needs clang or gcc and powershell for the tests. the batch file is the single windows entry point; its menu also offers individual build targets and cleanup. ### native backends on arm64, `src/core/core.c` uses its jit by default. other hosts use the interpreter. `-DZYINTERP` forces interpretation; `-DZYLLVM` enables the llvm backend when its headers and libraries are available. direct builds from the root are: ```sh make genome clang -O2 -fwrapv -DJTYPE=int src/core/core.c src/core/io_linux.c -Isrc/core -Ibuild/core/map --embed-dir=build/core -o build/bin/zy clang -O2 -fwrapv src/micro/micro.c src/micro/port_linux.c -Isrc/core -Isrc/micro -Ibuild/core/map --embed-dir=build/core -o build/bin/zym clang -O2 -fwrapv -DJTYPE=int -DZYLLVM $(llvm-config --cflags) src/core/core.c src/core/io_linux.c -Isrc/core -Ibuild/core/map --embed-dir=build/core -o build/bin/zy-llvm $(llvm-config --ldflags --libs) ``` `-I` finds the generated `memmap.h` and `--embed-dir` finds `genome.bin`; pointing them at another map and image (a `src/tools/automap.sh` output directory) builds a core for that machine. `J` must match the host port: the supplied ports use `int`. a source unit ends at eof or a backtick outside a comment or literal; bytes after that backtick remain available to the program. `ZYTRACE=1` logs translations and installations to stderr. with tracing, `ZYO3DUMP=1` also prints region micro-ops. on llvm builds, `ZYHOT=n` sets the compilation threshold, `ZYPASSES` selects the pass pipeline, and `ZYNOLLVM=1` disables llvm compilation. ### language server `src/lsp/server.zc` is a language server written in zc. it runs the compiler (`src/compiler/compiler.zc`) in check mode and provides a json-rpc protocol. diagnostics use the compiler's messages. parse words such as `immediate fn`, `run.add_binop` and `quote` execute during checks, so editor parsing and highlighting reflect syntax added by the program. hover shows declaration signatures and documentation. a top-level function, global or struct shows its declaration line; a method or static function shows its indented declaration. comments directly above a declaration provide its documentation, and a declaration without comments still shows its signature. parameters show their declarations, while `let` locals show the compiler-assigned type. untyped locals, parameters and function returns show inferred types; the server obtains these by checking the document again with unresolved names open. it caches the result until the document changes, so the first hover after an edit may take longer. hovering an imported module name shows its module overview and include form; source libraries can provide the same overview in a comment after their namespace and include headers. embedded `core.zc` is provided alongside runtime, compiler and standard-library sources, so `core.syscall` and `rt.syscall` navigate to and reference their distinct declarations. keywords such as `static`, `micro` and `unchecked` show their form and meaning from one table in `src/lsp/server.zc`. check mode records errors instead of printing them and continues after an error, up to 20 diagnostics per document. each record contains a line and token offset; the server maps these to utf-16 ranges. check mode also reports non-error findings as hints. - **diagnostics** on open, change and save, with an id echoed on every reply, `MethodNotFound` for unknown requests and `-32002` before `initialize`. - **definition and references** (`textDocument/definition`, `textDocument/references`): a local, parameter or field goes to its declaration by the offset the compiler recorded, and a function, global, constant, struct or type goes to where the compiler saw it defined, in an included file too (the file is found from the document's directory upward, then as written in the `include`). references lists the uses in the document, with the declaration when asked. built-in names navigate to their embedded source when one is available. - **module completions**: after an imported namespace and a dot, completion lists that module's exported members with signatures and documentation; it does not add unrelated local or module names. module hovers and completions cover `rt`, `run`, `core`, `lsp`, `std` and `icon`. - **import quick fixes**: compiler diagnostics for missing imports or unqualified exports offer a workspace edit. a bare name is looked up in every module: the runtime and compiler modules, loaded files, and the libraries under `src/lib` read from source even when they are not included, including static groups (`itoa` becomes `std.Text.itoa` with `include std;`). a misspelled name gets the closest defined name or module export (`putc` becomes `rt.put_c`). the edit adds the needed `include` without duplicating an existing import, qualifies bare exports, and preserves namespace declarations, comments, and crlf line endings. the editor client applies only edits for the current document version, while keeping protocol version epochs separate from editor document versions. - **semantic tokens** (`textDocument/semanticTokens/full`) in 12 types: keyword, type, function, variable, string, number, operator, comment, parameter, property, macro, enumMember. token kinds come from the compiler's symbol tables and declarations. check mode records each resolved name and source location, following the compiler's lookup order: locals and parameters precede globals, and built-ins are checked last. a local named `cap` is a variable wherever it is in scope, and the built-in `cap` only applies to uses outside that scope. - **formatting** (`textDocument/formatting`, and `zclsp --format < file.zc` on the command line): a pretty-printer in the spirit of clang-format with `ColumnLimit: 0`. it keeps the author's line breaks and sets the indent of every line (from braces, open parentheses and unfinished statements; `case` labels are indented one level inside their `switch` by default), the spaces after commas and around `=`, `==`, `&&` and the like, the blank lines between lines, and the final line break. it never moves a token to another line, and it leaves strings, comments, `asm` and `vm` blocks and everything after a backtick as written, so the tokens of a program are the same before and after. options come from a `.zcformat` file in the document's directory or the nearest directory above it, one `Key: value` per line (`#` starts a comment), then from the editor's tab size and `insertSpaces`, then from the defaults: `UseTab: true`, `IndentWidth: 4` (the spaces of a level when `UseTab` is false), `MaxEmptyLines: 1`, `ContinuationIndent: 1`, `NormalizeSpacing: true`, `CaseIndent: 1`, `FinalNewline: true`. `CaseIndent: 0` aligns each `case` label with its `switch`. `tests/drivers/format_test.js` formats every `.zc` file in the repository and checks that the tokens are unchanged and that formatting twice gives the same text. - **streaming and cancelling**: a check of a document with 100 or more top-level items sends the diagnostics found so far each time the item count doubles, so errors near the top appear before a large file is done. before each of those points the server queries the host (`core.syscall(42)`) for pending input. if input is waiting, the check is dropped without publishing because newer text may replace the current document. the check publishes once no newer input is waiting. smaller documents are checked in one go. - **cli mode**: source on stdin prints one line per finding, `line:col: error: ...` or `line:col: hint: ...`, or `no errors`. the server runs on its own memory map (`genmap --preset lsp`), which has a bigger genome slot and a smaller frame stack than the default, and it keeps its documents in a store of its own, 4 chars to a word. ```cmd tinyvm.bat build lsp # build\bin\zclsp.exe src/tools/mklsp.sh build/lsp # elsewhere: build/lsp/zclsp tinyvm.bat lsp examples\opt_test.zc ``` `tests/drivers/lsp_test.js` drives the server over the protocol (unicode columns, a 50 kb document, state that must not leak between documents, parse words) and checks that every valid program in `examples` and `tests` comes back clean and every bad one in `tests/errors`, `typeerrors` and `langerrors` is rejected with the message in its first line. ### vscode extension the `editors/vscode` directory contains the complete VS Code extension for ZC: - **syntax grammar**: full TextMate grammar (`syntaxes/zc.tmLanguage.json`) highlighting keywords, types, lifetime tags/recasts (`.u`, `.m`, `.d`, `u*`, `m*`, `d*`), literals, and operators. - **language configuration**: bracket matching, indentation rules, auto-closing pairs, and comments (`//`, `/* */`). - **language client**: a small node client (`client/extension.js`) with no packages. it speaks the protocol properly (byte lengths, `textDocument` wrappers, `didClose`), restarts the server if it exits and sends the open documents again. build a package with `make vsix` (or `bash src/tools/mkvsix.sh`), which writes `build/zc-language-support.vsix`, then use "extensions: install from vsix" in vscode. the server files are staged in `build/editors/vscode/server/` and the extension is packed from a copy in `build/vsix/`, so the tree is never written to; to run the extension from source, open that copy in an extension development host. the bundled wasm server needs no native executable. `zc.lsp.serverPath` optionally selects a native server. see [the extension guide](editors/vscode/README.md) for settings and logs. ### javascript & typescript runtime a complete, self-contained 64-bit TinyVM core implemented in JavaScript and TypeScript is provided in `src/js/`: - `src/js/tinyvm.ts` (compiled to `build/js/tinyvm.js`): Implements the 64-bit word machine on `BigInt64Array` with byte addressing (`Uint8Array` / `DataView`), dual stacks, all 22 base opcodes, byte/float opcodes (612–619), system calls, and custom virtual enhancement execution (`xrun`). - `build/js/genome.bin`: the compiler genome, the same image the native cores embed, which `tinyvm.js` reads from beside itself. `build/js/layout.js` holds the memory map's constants, written by `node src/js/mklayout.js` from `build/core/map/memmap.h`. - file syscalls (open, read, write, close) reach node's file system when the vm is run from the command line, so `include "path"` and `embed` work there; a vm created by other code (the editor's) has no file access unless it passes `files: true`. run any `.zc` program in Node.js: ```cmd node build/js/tinyvm.js examples/hello.zc node build/js/tinyvm.js examples/fact.zc node build/js/tinyvm.js examples/mandel.zc # or via the Windows runner: tinyvm.bat js examples/primes.zc ``` ### web (webassembly) `src/web/` runs the compiler in a browser. `make web` compiles the microcore and genome to `build/web/zc.wasm` with clang's wasm32 target (through `src/web/build.sh`), without emscripten or libc, and copies the page and its loader beside it. `src/web/port_wasm.c` provides the port and the 64-by-64-to-128-bit multiply used by the core. `src/web/zc.js` loads the module and exposes `machine.run(source)`, which returns the output produced by `zy < file.zc` from a fresh instance. `src/web/index.html` provides an editor, examples list, and output pane. serve `build/web` with `python3 -m http.server -d build/web` and open `http://localhost:8000/`. ``` make web node tests/drivers/wasm_test.js build/web/zc.wasm ``` ## zc, the language it reads like c. underneath every value is a 64 bit word and memory is word addressed, so `p + 1` is the next word whatever `p` points at, and strings are one char per word with a 0 at the end. types are optional: untyped code is typed `word`, which goes with anything. ``` include rt; const N = 10; int[N] table; // a global array int fib(int n) { if (n < 2) return n; return fib(n - 1) + fib(n - 2); } fn main() { for (int i = 0; i < N; i++) table[i] = fib(i); let sq = { let t = table[5]; t * t }; // a block is an expression too, its last line is its value let x = asm { 2 3 + }; // raw vm code, leaves one value rt.put_n(sq + x); rt.put_c(10); return 0; } ``` declarations are C-style, type first: `int a = 1, b;`, `node* p = rt.new(node);`, `char[4] buf;`, `point origin;`. `let` declares without a type: it infers `char*` from a word-string literal and `bstr` from a packed byte-string literal, infers other pointers and function types from the initializer, uses a pointer alias for an in-place array or struct, and keeps numbers as plain words. `let a[10];` is an untyped array. `const k = 5;` inside a function is a local constant with no type: it takes the initializer's type, needs an initializer, and can't be assigned. statements are declarations, `if`/`else`, `while`, `do`/`while`, `for`, `switch`, `break`, `continue`, `return`, `{ }` blocks, `asm { }` and expressions. `do` body `while (` cond `);` runs the body once before the test, and `continue` jumps to the test. `enum color { RED, GREEN = 5, BLUE };` makes global constants, and `color` is a type of its own in typed code (`typecheck on;`): a member takes the enum, a plain int does not become a `color`, and a `color` still becomes an `int`. untyped code treats the members as plain ints. each member is also reached as `color.BLUE`, in an expression or a `case` label. a member can carry fields too: `enum shape { circle { int r; }, rect { int w; int h; }, dot };` makes the constructors `shape.circle(3)` and `shape.rect(4, 5)`, which return an owned `shape*`, and `shape.dot()`. the tag is the word at offset 0 of the object. a `switch` over a `shape*` matches a member with `case circle(r):`, which binds its fields in that arm only. a switch on a tagged enum that leaves out a member and has no `default` gets a warning on its line (not an error). `(T)e` is a cast with the same meaning as `e as T`: the bits stay, the type changes. it has to be a simple type the parser can see in a few tokens, `int` or `*node`, not a nested array. `include "path";` reads a file and compiles it at that point, signatures first, so a library can live in its own file. each file has its own namespace and imports; dependencies included by a file do not import into the caller. `u`, `m` and `d` are ordinary names. a tag is `.u`, `.m` or `.d`, and `u*x` with no space before the star is still a recast. `switch (e) { case 1, 2: ... default: ... }` takes constant labels, and its cases don't fall through: each ends where the next begins, and `break` leaves early. operators follow C precedence, including `c ? a : b`, plus `p.x` and `p->x` (the same thing: `.` goes through a pointer on its own), `e as T` to change a type without changing the bits, and the casts below. comparisons and `&& ||` give 0 or 1. `/`, `%` and `<<` compile to native enhancement opcodes. character literals, `0x` hex and the usual escapes are there. functions are declared like C, return type first: `int f(int x)`, `node*.u make()`, `.u grab()` (a tag on its own), `void log(char* s)`. `fn f(a, b)` starts without explicit parameter or return types; constraints from calls and returned values propagate through callers and callees until inference reaches a fixed point, including recursive calls anchored by typed callees. conflicting argument types or an unknown reassignment leave the affected type unresolved instead of choosing a type. the learning example [32](examples/learn/32-inferred-calls.zc) shows this propagation and an owned raw-list return. a generic function lists its type parameters after its name, `T first(*list l)`, the same place a call gives explicit ones, `first(l)`. the return type can use them before they're declared: while it's read, an unknown name is taken as a type parameter, and the list after the name has to confirm it. function types are written the same way, `int(int, *char)`, and `fn(int)` for one that returns a word. the earlier `fn f(...) -> T` spelling is gone. array types put the element type first. `T[n]` has its size in the type and lives in place. `T[]` also lives in place; its length is inferred from an initializer such as `int[] a = {1, 2, 3};` and cannot grow. `T[?]` can also be initialized in a function with a list, such as `int[?] a = {1, 2};`. it is a growable array: a small handle object that owns its items, made with `rt.new(T[?])` or `rt.new(T[?], n)`, indexed with a bounds check, and changed with `push(a, x)`, `pop(a)` and `resize(a, n)`. `len(a)` works on both (for `T[n]` it's the constant) and `cap(a)` gives the room before the next grow. `fixed T[?]` is a `T[?]` that can't change size: `rt.new(fixed T[?], n)` needs the length, and `push`, `pop` and `resize` on it are compile errors. a `fixed T[?]` can't be passed where a `T[?]` is expected, the other way round is fine, and a constant index outside a `T[n]` is caught at compile time. the items belong to the array, so freeing it frees them, and objects stored into a `T*[?]` are its children like any container's. the array intrinsics also accept receiver syntax: `a.len()`, `a.cap()`, `a.push(x)`, `a.pop()` and `a.resize(n)`. fixed arrays provide `len()` and `cap()` but reject `push()`, `pop()` and `resize()`. [the learning example](examples/learn/19-growable.zc) shows both receiver forms. `fixed const T[n]` and `const fixed T[n]` both make the array cells and aliases read-only. element writes, increments, compound assignments, and rebinding a `const fixed T[?]` handle are rejected even when `typecheck off;` is active. a read-only array reference cannot become a mutable reference. a formal parameter such as `const fixed int[2] values` accepts either read-only or mutable arrays. element values can still be copied into ordinary mutable scalars. [the learning example](examples/learn/33-const-fixed.zc) shows both qualifier orders and a read-only formal parameter. `include std;` also registers read-only array algorithms: `find` and `find_if` return a matching index, `contains`, `any`, and `all` return a boolean word, `count` counts matches, `map` transforms elements into a new handle, `filter` copies matching elements into a new handle, and `reduce` folds them from left to right. `any` is false and `all` is true for an empty input; `reduce` returns its initial value. they accept mutable fixed or growable arrays; support for const-qualified arrays is unverified. lambda parameter types follow the element type, map's result type follows the callback return, and filter preserves the input element type. algorithms visit the input's initial length in index order, so callbacks must not change its length. map/filter results are independent handles that shallow-share reference elements with the collector. aggregate-by-value elements are not supported by `push`, `pop`, `map`, or `filter`. [the runnable example](examples/learn/31-array-algorithms.zc) shows inferred callbacks and search. `embed "path" as [type] name;` declares `name` with the bytes of a file, read when the unit is compiled; the path is found the way an `include` path is (next to the including file, then in the workspace folder, then from the working directory). with no type, or `bstr`, `name` is a byte string of the whole file. `char*` gives a word string, one character per word. a number type reads the bytes little-endian: an integer type takes at most its size and zero-extends a shorter file (sign-extends for `i8`, `i16` and `i32`), and `float` or `f32` needs exactly 8 or 4 bytes. `T[]` and `T[n]` lay the bytes out as array elements of a number type, packed for the sized types and one byte per element for `char[]`; a file that is not a whole number of elements, or that does not fit `T[n]`, is an error. at the top level `name` is a global and the bytes sit in the image; in a function it is a local, and an array is copied into the frame each time the declaration runs. the compiler uses it for the libraries it carries (`embed "src/lib/std.zc" as bstr std_embedded;`). ``` include rt; embed "tests/data/embed/text.txt" as greeting; // a bstr embed "tests/data/embed/answer.bin" as int answer; embed "tests/data/embed/bytes.bin" as u8[] table; fn main() { rt.put_s(greeting); rt.put_n(answer + table[0]); return 0; } ``` a unit is read whole and scanned for function signatures before it compiles, so calls can come before definitions, tags included, and prototypes are optional. `main` runs after the whole program has compiled. use `void f()` for a function with no result and `return;` to leave it; value-returning functions use an explicit result type or the inferred `fn` form. the runtime api lives in the `rt` namespace and requires `include rt;`: `rt.put_c`, `rt.get_c`, `rt.eof`, `rt.put_s`, `rt.put_n`, `rt.halt`, `rt.len_w`, `rt.eq_w`, `rt.copy_w`, `rt.fill_w`, the packed-string helpers `rt.len_s`, `rt.at_s`, `rt.slice_s`, `rt.cat_s`, `rt.eq_s`, `rt.s_to_n`, `rt.n_to_s` and `rt.put_bs`, allocation and ownership helpers such as `rt.new`, `rt.new_t`, `rt.free`, `rt.gc`, `rt.memreport`, and host functions including `rt.syscall` and `rt.readfile`. their signatures and descriptions are in the runtime source comments and built-in hover table. `src/lib/std.zc` declares the `std` namespace; its static groups are called as `std.Group.name(args)`: `std.Text`, `std.Num`, `std.Host`, `std.Bytes`, `std.Arena`, and `std.Buf`. the constructors `std.arena(n)` and `std.bnew(n)` and globals such as `std.argc` and `std.argv` are plain declarations in that namespace. capture uses static functions: `std.capture.start()` begins output capture and `std.capture.end(buf)` ends it and returns the buffer. `std.argc` and `std.argv` provide command-line arguments to `main`; `std.envc` and `std.envp` provide environment entries in `NAME=value` form. `std.argc` is an `int`, and `std.argv` is a `char*[?]`; the environment fields have matching types. `std.argv[0]` is the program path, or `` when source comes from stdin. `std.libinit()` fills all four before `main` when the program includes `std`. Since include dependencies do not import into the caller, runtime calls also require `include rt;`: ``` include rt, std; fn main() { for (int i = 0; i < std.argc; i++) { rt.put_n(i); rt.put_c(32); rt.put_s(std.argv[i]); rt.put_c(10); } return 0; } ``` `std.capture.start()` redirects program output into a string, and `std.capture.end(buf)` stops capture and returns the string. these are static functions in the `std.capture` group; output redirection is also used by the language server (`outmode`): ``` namespace; include rt, std; fn main() { let buf = std.capture.start(); rt.put_s("hello world!"); // nothing on the terminal std.capture.end(buf); rt.put_s(buf); // hello world! } ``` ## memory model: u, m and d `rt.new(n)` or `rt.new(T)` gives back a fresh object. bind it to a local and the local owns it, tagged **m**. tags live on local variables and parameters, the analyzer follows them through the code, and the compiler inserts `rt.free` calls where owned objects die. a tag is written after the type, `*node.u p`, or alone, `.d x`. `u`, `m` and `d` are ordinary names; the tag is the dot. **m** means the object belongs to this block and must be used in it. it may be passed to callees as a borrow; scope exit frees it. an m object that is never used in its block is an error. **d** transfers the object to a lower scope. passing it to a `.d` parameter or accessing it in an inner block transfers ownership; the receiving scope owns it as m and frees it when that block ends. using it after transfer is an error, as is ending the current block before the transfer occurs. **u** transfers the object to the enclosing scope, where it becomes m. transfer occurs through `return`, a block expression value, or assignment to a variable in the directly enclosing block. returning an owned local transfers it automatically; a `.u` result declaration provides the contract to callers before the function body is available. an owned u object cannot fall out of scope or have its result discarded. bind the result to an owner and call `rt.free`, or pass it to a consuming parameter. explicit recasts use `u*x`, `d*x`, or `m*x`, with no space before the star. a space is ordinary multiplication by a variable of that name. recasts remain available for low-level code, while checked returns, `.d` calls, frees, and owner assignments infer common transfers. a declaration can carry a tag too, `.u r;`, and then whatever lands in it gets that tag. recasting a local that owns nothing is an error that names it. an owner can move into another local whose object was released first: `rt.free(r); r = replacement;` gives `r` the replacement's object and retires `replacement`, when `replacement` lives in the same block as `r` or a deeper one. an owner cannot be copied into a local that still owns something. a global can borrow a local owner for a bounded stretch: `g = m*x;` lends `x` to `g` while `x` keeps ownership and can still be used, and the lend ends when the same block assigns `g` again (usually restoring its old value). the analyzer rejects giving `x` away, returning, or leaving that block while the lend is open, so `g` never points at a released object. this is how an output stream is pointed at a local scratch buffer: ``` word saved = rt.outbuf; rt.outbuf = m*buffer; render(); rt.outbuf = saved; ``` parameters: an untagged parameter is a lend, and the compiler infers how a checked body uses it. `.m` states a borrow contract and `.d` receives ownership. Passing an owned value to `.d` transfers it automatically, so `take(p)` can replace `take(d*p)`. A checked borrow can return an alias of its parameter, but it cannot return an alias of a local owner. A fresh result passed to a proved non-retaining function, as in `rt.put_s(message())`, is held in a compiler slot and released after the call. Unknown or retaining callees need an explicit contract or a named owner. ``` include rt; int*.u box(int v) { int* b = rt.new(int); *b = v; return b; } void add(list* l, int*.d item) { l.items[l.len] = item; l.len++; return; } ``` containers own what you store in them. `p[i] = c` or `p.f = c` needs `c` to carry the same tag as `p` (or be a fresh object), and from then on `c` follows `p`: if `p` is recast, so is `c`, if `p` is given away `c` goes with it, and when `p` is freed its children are freed too. a child can't be recast or given away on its own. storing into an object you only borrowed gives the child away: it's dead here afterwards, and the caller's object owns it. the runtime records the owner in the child's header, so freeing a container frees exactly the objects it owns. globals, plain pointers and arrays aren't tracked. storing an owned object there needs `d*`, and the mark-sweep collector takes it over. a fresh object can go straight in. a pointer computed from a tracked variable is tracked too. `p + k`, `p.field` and `p[i]` (when that field or element is a pointer, or when `p + k` is untyped pointer arithmetic on an owned object) stores a borrow of `p` in the local it lands in, the same child link a real child uses. using that local after `p` dies is an error. the pointer itself is not an owner: it can be copied, incremented and passed around, and it is not freed on its own. ### memory debug off unless a program asks. `memdebug on;` at the top level turns on header checks and allocation sites for the rest of the file, and the runtime enables them before `main`. the same line inside a function does it from there on. `memdebug scrub;` also fills a freed payload with 57005 (`0xDEAD`). `memdebug off;` stops it. `rt.mem_set(n)` is the same switch as a call: bit 1 checks and sites, bit 2 poison, bit 4 interior pinning. with checks on, every load and store through a heap address looks for the object header, walking back at most 64 words so `p + k` and a field still find it. a dead header stops the program: ``` zy: use after free from line 12 zy: double free from line 12 zy: bad heap pointer ``` the line is the `new` that made the block. a double free prints and returns; it does not smash the block. `rt.memreport()` prints one line per recorded object that is still live, `leak: line 12 3 words`, and returns how many. the word count is the block's payload, which is a power-of-two chunk minus the 5 word header, not the size you asked for. the table holds 64 sites. freed blocks stay in it until the storage is reused, which is how a double free still knows the line. nothing here changes programs that never say `memdebug`. bit 4 makes the collector treat a word that falls inside a quarantined payload as a pointer, up to 64 words back to the header. an exact pointer already did this. turn it on when you want `p + k` to keep the block. it can also pin a block that a stale integer happens to land in, so it stays off unless you set the bit. where paths join (after an `if`, a loop, a `switch` or `?:`) every variable has to be in the same state on every path that reaches the join, with one exception. a variable that became d, or was given away, on some path is dead afterwards, and the compiler frees it on the paths where it's still alive, on their way in. a loop's exit state is the join of its `break`s and its normal exit, so giving something away and breaking is fine. what a loop can't do is give something away on a path that comes back around to the top, since the next pass would use it dead. a u object doesn't get the dead treatment: it still has to leave. ### the analyzer it runs inside the compiler on every function, including code compiled at runtime by `eval` and friends, and reports each violation with a line number and the variable's name. the compiler keeps going after errors, up to 20: a syntax error skips to the next `;` or `}` (or the next top-level declaration), and analyzer and type errors just get reported, since the parse is still in step. a unit with errors doesn't run. `tests/errors/` has one small program per rule. to turn it off: `unchecked fn f() { }` for one function, `unchecked { }` for a block, and a top-level `unchecked;` for the rest of the unit (a parse word can also set `check_on = 0`). unchecked code doesn't track anything, so what it allocates belongs to the collector, and checked variables can't be touched from inside an unchecked block. ## types types are a compiler primitive. the compiler keeps them as records in a type area at 0x80000 (65536 words), hash-consed so the same type is always the same pointer. they stay in memory after compiling, so programs and parse words can look at them. ``` include rt; struct point { int x, y; }; struct list { int len; *T items; }; type num = int; T first(list* l) { return l.items[0]; } list*.u mk(int cap) { list* l = rt.new(list); l.items = rt.new(T, cap); return l; } ... list* ps = mk(8); // explicit type arguments, or inferred from the arguments ``` the kinds are `word`, `int`, `char`, `bool`, `void`, pointers `T*`, arrays `T[n]`, `T[?]` and `fixed T[?]`, function types `int(int, char*)`, structs, generic structs and generic functions. structs and arrays live in place (in a frame, a global or inside another struct), and their value is their address, so they're passed around by pointer. `sizeof` counts words. a generic function is compiled once for each set of type arguments it is called with, so `T` can be a number, a pointer or a struct of any size, `sizeof(T)` and `rt.new(T)` mean the real type, and a struct can take several parameters (`struct record { K key; V val; };`). indexing scales by the element's size, while plain pointer arithmetic stays in words, like b. `rt.new(T)` and `rt.new(T, n)` stamp the type into the object's header. ### pointer types a pointer type places its star after the type, like c: `int*`, `point*`, `vec*`, `vec*`, `point**`. a tag follows the stars (`vec*.u`, `int*.d x`). there is no prefix form: `*int` as a type is an error, and a star in front of an expression is only ever a dereference. a pointer to an array or to a function type is spelled with a `type` alias (`type row = int[3];` then `row*`), because `int*[3]` reads as an array of pointers. `tprint`, and so error messages and reflection, print the same form. ### methods and closures a struct can hold functions. they get a hidden first parameter called `this`, and `p.name(args)` calls them with `p` in that place: ``` struct point { let x, y; fn dist(other) { return (this.x - other.x) * (this.x - other.x) + (this.y - other.y) * (this.y - other.y); } } ... p.dist(q); ``` underneath, a method is an ordinary function called `point.dist` whose first parameter is `point* this`, compiled once the struct is complete, so a method can call the ones declared after it. a field with the same name wins over a method. a method of a generic struct is a generic function over the struct's parameters, compiled for each set of arguments it is called with (the receiver says which), so `this.items[i]` and `sizeof(K)` inside it are exact. `fn(a, b) { ... }` in an expression is a closure. the value is a heap block: word 0 is the body's entry, and its captured cell pointers start at word 3. the vm only runs code in the code area, so the block is never executed. a call through a value checks whether the callee is at or above `HEAP_LO`. a plain function is called as it is. a closure is called with the block plus 3 as its environment and jumps to the entry in word 0. captured locals and parameters share cells with the enclosing function and sibling closures, so mutations stay visible after the closure is made and after its creator returns. captured owned objects remain alive with those cells. the block is an owned object; a fresh closure can be passed directly to a proved borrower, and returning one transfers it to the caller. `examples/learn/25-closures.zc` and `examples/learn/26-closure-state.zc` cover these. `fn:type(args)` declares a closure's result and parameter types. an owned closure can be returned from a factory with `.u`; its captured cells remain alive for the returned value. [the typed closure example](examples/learn/30-typed-closure.zc) covers this form. ### hints check mode (the language server) reports things that are legal but probably not what you meant. they don't change what the compiler emits, and a normal build doesn't compute them. - a variable that is never read - a `let` with a constant initializer that is never assigned again: make it a `const` - multiplying by a power of two: write a shift - a number of 100 or more with no name (powers of two, masks and character literals are left alone) - `rt.new` inside a loop: it allocates on every pass - a `rt.new(n)` of 64 words or less that never leaves its function: a stack buffer, `let buf[n]`, avoids the heap - an owned object last used four or more lines above the end of its block by default, hints use blue informational squiggles (diagnostic severity 3). `hints on;` restores this presentation for following code, `hints small;` keeps the same hints but uses compact dots (severity 4), and `hints off;` suppresses hints for following code. each directive applies by source position to the code after it; `on` is the default. an inline block comment immediately before a numeric literal marks the literal as intentional and suppresses its magic-number hint: ```zc return n + /* offset in memory */ 0x40ff; ``` the comment must precede the literal itself. a comment before another operand does not suppress the hint for the literal. ordinary comments immediately above a declaration provide its hover documentation; `//:` marks an implementation note instead. [the learning example](examples/learn/29-block-comments.zc) combines declaration documentation with the inline magic-number escape. `examples/opt_test.zc` illustrates these cases and still runs. ### reflection `typeof(e)` and `typeof(T)` give a type record and `sizeof(...)` its size. both work on any expression, which is compiled dry and never runs. the readers are compiler functions, so they need `include run;`: `run.tkind(t)` (one of the `run.TY_` constants), `run.tsize`, `run.tname`, `run.tlen`, `run.tnfields`, `run.tfname(t, i)`, `run.tftype`, `run.tfoff`, `run.tret`, `run.tnparams`, `run.tparam`, `run.tparamtag` (the u/m/d tag of a parameter), `run.tretu`, `run.tnargs` and `run.targ` for generic instances, and `run.tprint(t)`. `run.sigof(f)` gives the type of a function by address (untyped ones get a signature built from their arity and tags), `run.sigofs("name")` by name, and `rt.typeof_obj(p)` reads the type stamped on a heap object (0 when the header has no type yet: `rt.new(n)` or `rt.new_raw`). ### the typechecker off by default. `typecheck on;` turns it on for what follows and `typecheck off;` turns it back off (a parse word can flip `tc_on`). it checks initializers, assignments, arguments (unifying generic parameters), return values, arithmetic, comparisons, dereferencing, indexing and calls. `word` matches anything, numbers mix with each other, a literal 0 is a null pointer, and arrays decay to pointers. it stops at the first error, printing what it expected and what it got. `tests/typeerrors/` has one per rule. some things are checked whatever the setting, because the compiler needs them to generate code: field names, and field access on something that isn't a struct. ### sized integers and f32 `i8 u8 i16 u16 i32 u32` (and `byte`, which is `u8`) are integer types that keep their size. a variable, parameter, field or return value of one is still a word, but it is cut to size every time it is stored: unsigned values are masked, signed ones are masked and then sign extended, so `u8 a = 250; a = a + 10;` leaves 4, and `i8 s = 100; s = s + 100;` leaves -56. arrays and pointers of them are packed, which is where the memory goes: a `u8[100]` takes 13 words and `rt.new(u16, n)` takes n/4 words, rounded up. indexing a packed array or pointer reads and writes the bytes with the byte ops (612 to 617), so the same code runs on every engine and o3 lifts it like any other load. `f32` is single precision: a variable holds the value already rounded to a float (arithmetic is done in double, which gives the correctly rounded float result for `+ - * /`), and an array or pointer of f32 is packed two to a word. `f64` is a second name for `float`. the rounding is the `f2s` and `s2f` micro-ops, one instruction on arm64. ``` include rt; u8 gbuf[100]; struct pix { u8 r, g, b, a; i16 dx; }; f32* p = rt.new(f32, 1000); // 500 words p[3] = 0.1; // stored as the nearest float ``` two limits. `sizeof` counts words, so `sizeof(gbuf)` is 13 and a struct of four u8s has `sizeof` 1: the fields of a non-generic struct are packed into bytes, and a field read or write goes through the same byte ops as an array element. growable `u8[?]`, `i16[?]`, `f32[?]` and the rest work too (the handle's element size is stored as minus the byte size, and `push`, `pop`, `resize`, indexing and `for` use byte accesses for them), in o3 as well. the address of one element of a packed array cannot be taken, because it is a byte inside a word, and that is on purpose: pointer arithmetic moves in words, so such an address would not mean the same thing as `&a[i]` does elsewhere. take the array's own address and index it. `f32` is a type for `extern` and `cfunc` signatures: it goes to c as a 32-bit float and comes back rounded. a zc struct whose fields are all one word (`int`, `float`, pointers) can be passed by value to an `extern`: `extern long f(struct point p);`. it goes to c as its words, classed per eightbyte by the sysv x86-64 rules (a float word in an sse register, any other in an integer register, the whole struct on the stack when it does not fit, and a struct over 16 bytes always on the stack). arm64 (aapcs64) follows the same idea: a homogeneous float struct of up to four words goes in the v registers, a struct of two words or less in the x registers, and a bigger one is passed by reference to a copy. this is checked on x86-64 and on arm64 under qemu-user, on the interpreter, in o3 code, and on the llvm backend and jit (`tests/programs/structval.zc`). a struct can also be returned: `extern struct point mk(long x);` gives a new `point*` owned by the caller (the stub makes it with `new` and the native call fills it in). the return is at most 7 words, and its floats are classed like the arguments'. returns are checked on x86-64 and on arm64 under qemu-user, on the interpreter, and on llvm (`tests/programs/structret.zc`). apple arm64 is not checked. ## virtual enhancements the cpu has room for 512 custom opcodes (22 to 533), and opcode 21 (`%` in asm) installs one at runtime. it's the closest a soft cpu gets to rewiring itself. an enhancement takes up to 8 inputs off the data stack and leaves up to 8 outputs, and it's written one of two ways: ``` enhance clamp(x, lo, hi) micro "x = max x lo; x = min x hi"; enhance 2 divmod(a, b) micro "r2 = sdiv a b; r1 = srem a b; r0 = r2"; // the number is how many outputs enhance cube(x) vm { 0 $ 0 $ * * }; enhance noisy(x) vm { 0 $ . } else soft_noisy; // can't be lifted (it does i/o), so calls use the fallback ``` a micro recipe works on registers `r0`..`r15`. inputs arrive in those registers, also under their parameter names, and outputs leave from `r0` upward. the micro-ops are `li mov add sub mul sdiv udiv srem urem and or xor shl shr sar neg not lt ltu eq ne min max minu maxu mulhs mulhu clz abs sel ld st`, followed by the bit operations `ctz popcnt bswap rotl rotr` and the single-precision conversions `f2s s2f` (rounding a double's bits to 32-bit float and back exactly). float operations and byte loads and stores are available to enhancements on the full cores. `src/runtime/rt.zc` defines callable enhancements for `clz ctz popcnt bswap rotl rotr mulhi umulhi f2s s2f`. they compile to one instruction on arm64 (popcnt uses the vector unit), an llvm intrinsic on the llvm backend, or plain c in the interpreter. the optimizer folds them when their inputs are constant. a `vm` body is straight-line vm code lifted by symbolic execution. both recipe and vm enhancements become ssa; the core optimizer folds constants, simplifies algebra, removes common subexpressions and dead code, and lowers constant division to multiply-high sequences (`ZYTRACE=1` prints the result). the arm64 jit allocates registers and inlines each use; llvm inlines the ir; the interpreter runs the optimized list. if an operation does not fit in registers, the jit calls the interpreter for that operation. when literals feed the last inputs of an enhancement (`x / 7` is `x 7 idiv`), the arm64 jit specializes that use: it substitutes the literals, optimizes again, and turns `x / 7` into multiply-high and shifts instead of division. an enhancement is called like a function when it has one output, or by name from asm. `enhance` installs it at compile time, so the wiring is available before `main` runs and parse words can use it immediately. an image installs its enhancements from its symbol table when it boots. that's how `/`, `%`, and `<<` work: `src/runtime/rt.zc` declares `idiv`, `imod`, and `ishl` as one-instruction recipes. `rt.install(desc)` is the raw interface; its descriptor format is documented in `src/core/core.c`. stores performed by an enhancement do not invalidate compiled code, so use enhancements for data operations. ## o3 `o3` marks code for the optimizing tier. it goes on a function, a block or a single variable: ``` o3 int sum(fixed int[?] a) { int s = 0; for (int i = 0; i < len(a); i++) s += a[i]; return s; } fn main() { o3 int hits = 0; // one o3 variable o3 { for (int i = 0; i < 1000; i++) if (i % 7 == 3) hits++; } ... } ``` what it gives up, as asked: variables declared in o3 code must be typed (`let` and untyped parameters are errors), they have no address (`&x` and naming one from `asm` are errors), an untyped `rt.new(n)` in o3 code is `new_oz` (no type stamp, and the words asked for are zero), and an o3 function skips the frame stack check in its prologue. typed `rt.new(T)` and `rt.new(T, n)` stamp the header and zero the payload, in o3 the same as outside it. what it gets: the compiler hands each o3 function body and each o3 block to the core as a region (install kind 2: the range of vm words plus its typed scalar variables' frame slots, each with a register), and the core compiles the region right away. the cpu has 7 registers for variables. a variable takes the one most recently freed by a scope that closed, or the next unused one, so loop counters in sibling loops share a register, and a variable declared when all 7 are busy stays in memory. a register shared by several slots gets written back to all of them when the region exits, and they never hold different live values, since their scopes don't overlap. inside the core, a region is cut into basic blocks at jump targets, branch fallthroughs and call return points. an empty jump is then folded into its target, a block with one predecessor is pasted onto that predecessor, and a straight loop is rotated so one iteration is a single block that branches to itself. a call, a return, a value left on the stack, or a second branch stays where it is, and an entry is never pasted over. each block is lifted to ssa the way enhancements are: the vm stack becomes values, and only what's still on it at the end of the block is written to memory. the frame pointer and the promoted slots become variables that live in registers across the whole region, so `i++` on an o3 int is one add. enhancements are spliced into the block, so `/`, `%` and `<<` become one instruction or a magic-number sequence. a call to `ld8`, `st8`, `ld16`, `st16`, `ld32` or `st32` from o3 code emits opcodes 612..617, and the region lifts those instead of calling out. every block then goes through the same optimizer (folding, algebra, cse, dce, division by constants). in o3 code `a[i]` on a `T[?]` is compiled inline with its bounds check (`outside(i, len)`, one enhancement), and `len(a)` is a load. a `fixed T[?]` in an o3 variable does better: its items pointer and length can't change, so the compiler gives it two hidden o3 variables holding them. they're loaded when the region starts and whenever the array variable is assigned, so indexing is a compare against a register and an indexed load, with no loads through the handle. the backends: on arm64 the region becomes one piece of native code. variables sit in w0..w7, block values in w8..w14 and w26..w28, constants are folded into instructions where they fit, loads from a fixed address are one `ldr`, and blocks branch to each other directly with no dispatch and no stack checks. on the llvm build the region becomes one function, with the variables in allocas that mem2reg turns into ssa across blocks, and it goes through llvm's O3 pipeline. on the plain interpreter, a small region interpreter runs the optimized blocks with the variables in locals. calls, returns, jumps out of the region, i/o, `install`, a store to the frame pointer and anything else a block can't lift all leave the region: the variables are written back to the frame first, so outside code sees ordinary memory. a call's return point is an entry, which loads them again. stores inside a region still check the coverage map, so writing to compiled code works, and a store into the region's own code drops it (it lifts again at its next entry). to keep variable addresses from waiting on the stack across a branch, assignment to an o3 variable in o3 code evaluates the value first and the address after. the lifter refuses a block that would leave such an address on the stack, rather than miscompile it. single o3 variables outside o3 code are reached through `lfa(k)`, an enhancement for `fp + k`, which the arm64 jit specializes into a load and an add. `bench.sh` runs four kernels from `tests/programs/bench.zc` with and without `o3`. milliseconds per run, with start-up and compile time removed by timing 1 run against enough runs to take more than a second (arm64 is under qemu on an x86 host, so only its ratios mean much, and the machine was noisy enough that the small numbers move by 20% between runs): ``` backend kernel plain ms o3 ms speedup interp collatz 324.90 240.61 1.4x interp sieve 47.08 21.71 2.2x interp digits 160.89 100.82 1.6x interp fib 3.48 4.09 0.8x llvm collatz 47.03 3.34 14.1x llvm sieve 10.77 0.28 38.9x llvm digits 18.93 1.45 13.1x llvm fib 0.87 1.01 0.9x arm64 (qemu) collatz 323.89 13.04 24.8x arm64 (qemu) sieve 56.84 1.74 32.6x arm64 (qemu) digits 147.06 4.17 35.3x arm64 (qemu) fib 5.19 5.71 0.9x ``` fib gains nothing and can lose a little, since every call leaves the region through the dispatcher and comes back in through an entry stub, which costs about what the body saves. `tests/bench/run.sh` times a word sieve and a binary tree against the same programs in C; the machine, the medians, and the gaps are in `docs/history/35-bench`. ### leaving o3: fattening going into o3 is free: o3 code reads normal objects the same way and ignores their metadata. typed `rt.new(T)` stamps the type into the header and zeroes the payload, so a node is born with the type it will keep. a child store in o3 code writes `p[-2] = owner` in the region when memdebug is off (a shared child keeps the owner it has, and a null store does not land) and does not call `own`. it also skips `fatten` when that value was just allocated, or returned from an o3 function whose type has a pointer. `rt.new(n)` with no type inside o3 calls `rt.new_oz`: the type word stays -(n + 1), and the words asked for are zero. `rt.new_raw` still leaves those words uninitialized and zeroes only the slack past them. `fatten` runs where o3 data reaches code that isn't o3, using the static type known there: storing a pointer into a normal local or a global, passing one to a function that isn't o3 (`rt.free` and `rt.recycle` excepted), the call site where ordinary code calls an o3 function whose result has pointers, `push` into a growable array, and, at the end of an o3 block, whatever the o3 variables declared outside it hold. a call or a return from one o3 function to another does not fatten, does not call `own`, and does not run the callee's prologue: the caller enters the callee's region directly. `run.fatten(p, t)` (in the compiler, so it can read type records) stamps the type, as `T[0]` when more than one element was asked for, zeroes the slack, and then follows every typed pointer, field, array element and growable array element inside. an object whose header already holds a type stops the walk, and a depth limit of 32 stops cycles through growable arrays. untyped words carry no hint, so pointers kept in `word` variables stay raw until typed code sees them. ## the llvm backend a tiered jit for hosts the arm64 backend doesn't cover. the interpreter counts how often each block start gets reached, and a hot block is translated to llvm ir using the same block rules as the arm64 jit: it ends at call, jump, ret or halt, keeps going past a jz, and literal targets are fused. the ir goes through the O2 pipeline and gets compiled by orc's lljit. stores into compiled words drop the blocks covering them, like on arm64. it pays off for long-running code: a loop-and-recursion benchmark runs about 3.3x faster than the interpreter. short bursty work like the self-compile is slower, since compile time dominates there. `ZYHOT=1` compiles every block, which is a good stress test, and the test suite runs that way too. ## the heap **constant-size allocations.** when an `rt.new` has a size the compiler knows (`rt.new(T)` of a type, or `rt.new(6)`), it works out the size class itself and calls `rt.new_k(n, t, k)`, which pops a block straight off the class's free list and skips the class search. a typed `rt.new` of a struct made only of numbers (no pointer fields, no `word`, no generics) is also freed through `recycle_leaf(p, t, k)` when its owner goes out of scope: it checks the type word and the class still match what the compiler assumed (the variable may have been reassigned), and if so pushes the block back on the head of its list with no walk over children. a loop that makes such an object each pass and frees it at the end of the block therefore gets the same block back every time: the pair costs a dozen steps, where the general path searched classes, split blocks and walked the object. memdebug, an empty list, or a block inside the cut-off type area use the general path, so the checks and the messages stay what they were. in the interpreter this cut a 200000-pass loop of `point* p = rt.new(point)` from about 1.9 s to 0.8 s. the heap is a buddy allocator over [HEAP_LO, top), where top starts at HEAP_HI and moves up. when no block is big enough, `rt.new_raw` asks `heap_grow` to append a piece (a few times the failed request, at least 0x40000 words, capped by MW). the new range enters the same free lists, so it behaves like part of the heap from startup. growth stops when `heap_reset` cuts the allocator off below the type area; the language server also runs with this cut in place. with a host that answers `core.syscall(9, 0)`, the heap can grow until the host denies more memory. `tests/programs/growheap.zc` allocates well past the initial heap, frees it, then takes a block larger than any part of the original heap. when available, host operations 18, 19, and 20 fill, copy, and scan memory in one call, avoiding long zc loops. blocks are aligned powers of two, and a freed block merges with its free buddy so space can be reused at any size. an earlier bug let a child merge into a container while the container was still being freed; growable arrays exposed it because their handles and item blocks are often buddies. allocation finds the smallest free block that fits and splits it. each size class has a doubly linked free list, so removing a buddy during a merge takes constant time. checked code frees deterministically. `rt.free` quarantines a block: it marks the header free but leaves the bytes intact, so a stale alias reads a dead block and a second free through it does nothing. the next collection returns a quarantined block to the free lists only when no word points to it. `rt.free(x)` and compiler-generated cleanup for an owned local call `rt.recycle` instead, returning the block to its size-class list after freeing its children. `rt.arena_new(n)` creates an owned block for bump allocation; `rt.arena_alloc` returns interior words and `rt.arena_reset` rewinds the bump. the mark recognizes exact pointers by their check word. with memdebug bit 4, it also accepts interior pointers up to 64 words into the payload. `memdebug scrub` fills the payload with 57005 after releasing its children. the mark-sweep collector is conservative and non-moving; it handles globals, unchecked code, and values passed to it. `rt.gc()` returns the number of objects freed, and collection also runs when an allocation does not fit. the sweep frees through the same merging path. a shared object (header flag bit 8) is not quarantined: `rt.free` leaves it in place, and collection reclaims it when unmarked. the low three flag bits are the live, mark, and pin state; the share bit stays set across a mark. ## c abi zc can call native c functions and hand zc functions to native code. it is built on the syscall gate (ops 14 to 17), so it needs a host with `dlsym`: the interpreter, the llvm backend and the arm64 jit on linux and macos (the jit has to be a dynamic binary, since a static one has no `dlsym`). the microcore, the javascript vm, the web build and the windows scripts answer -1, and a native call there gives -1 and nothing else. ``` include rt; extern double pow(double x, double y); extern long c_strlen(str s) = "len_w"; // zc name = "c name" extern void qsort(word *base, long n, long size, void *cmp); int cmp(int *a, int *b) { return *a - *b; } o3 int cmp3(int *a, int *b) { return *a - *b; } fn main() { let v[4]; v[0] = 3; v[1] = 1; v[2] = 2; v[3] = 0; qsort(v, 4, 8, rt.cfunc(cmp3)); // native code calls back into o3 rt.put_n(c_strlen(b"hello")); } ``` `extern` reads a c prototype and defines an ordinary zc function with that name, so a call is a normal call. types are spelled the c way. the integer types (`long`, `size_t`, `int64_t` and the other 64-bit names) pass a whole word; `int`, `short`, `char` and their unsigned forms pass and come back as that width, sign or zero extended; `double` and `float` are a zc float (a `float` goes to the c side as a 32-bit float); any `T *` is a byte address, the same number `addr_b` gives, except `word *`, which is a word address (a zc object or array, what `fixed int[?]` or `new` hands you), and `str`, which is a zc byte string (`b"..."`) passed as the address of its bytes. a result that points inside vm memory comes back as a byte address or word address by the same rule, and any other pointer (a `malloc` result, a `FILE *`) is an opaque number you can hand back to c. up to 15 arguments in all. the first six integers and eight floats go in registers and the rest on the c stack, in order, so a function like `snprintf` with many arguments works; more than 15 is a compile error. from o3 code the call is a call instruction in the generated code: the arm64 jit emits it, and the llvm backend emits a direct call to the symbol's address, so llvm can see through it, and `sqrt`, `fabs`, `floor`, `ceil`, `trunc`, `round`, `sin`, `cos`, `exp`, `exp2`, `log`, `log2`, `log10`, `pow`, `copysign` and `fma` on doubles become llvm's own intrinsics and fold or vectorize like any other arithmetic. the darwin host adapts the known fixed prefixes of `printf`, `fprintf`, `sprintf`, and `snprintf`; other darwin variadic exports still need an explicit ABI adapter. a variadic function is declared once per shape you call it with (`extern int snprintf2(word *buf, long cap, str fmt, long a, double b) = "snprintf";`), and the call sets up the registers a variadic call needs. a missing symbol prints `zy: no native symbol name` and the call gives -1. `rt.cload("libfoo.so")` (a byte string) opens a library so its symbols resolve, `rt.cfind(name)` asks whether one does, and the math library is opened on demand. `rt.cfunc(f)` gives a native function pointer that runs the zc function `f`. the c signature comes from `f`'s declared types: `float` is a double, a pointer (`int *`, `T[?]`) is a word address (so `int cmp(int *a, int *b)` reads the elements c points at), and everything else is a long. an `o3` function is entered at its compiled region, so a comparator or a worker in o3 runs as native code both ways. the pointer is only good while a native call from zc is running (it runs vm code on top of the stacks as they were at that call), so it belongs in an argument of the call that uses it or in a callback registration that is done again each time. there are 16 distinct callbacks per result class (long, double, float); the same function and signature reuse their slot. a callback that makes native calls of its own works, to any depth the stacks allow. how it works: the compiler writes the symbol's name, a signature word (4 bits per kind: the result first, then each argument) and room for the arguments as data in the code stream, and the stub it compiles stores its arguments there and does `core.syscall(14, block)`. the host looks the name up with `dlsym` (and remembers the answer in the block), passes the integer arguments in the first six integer registers and the floats in the first eight fp registers by calling through a prototype of six longs and eight doubles, and converts the result. `rt.cfunc` asks gate op 16 for a trampoline: one of 16 prebuilt c functions per result class, which pushes its arguments on the vm data stack, pushes a return address that stops the nested run, runs the vm function with the same `vm_until` an o3 call uses, and pops the result. the stack pointers it needs are published by the gate store (a global in the interpreter, an alloca store in llvm blocks, and the same two words the jit saves on exit). a call from o3 code to an extern leaves the region like any call to ordinary code, and the stub's store to the gate goes through the host, so a native call costs a gate round trip. lowering it to a direct call inside a region (so llvm sees it) is not done. ## syscalls the host is reached by storing an op number at word `0x84002`. the three words after it are the arguments, and the store leaves the result in place. `core.syscall(op, a, b, c)` gates core vm requests; `rt.syscall(op, a, b, c)` is the runtime wrapper used by runtime helpers. ops 1 to 5 still pass word strings, one character per word. ops 10 to 13 pass the byte address of a packed string's payload (`rt.addr_b(s + 1)`), which is NUL-terminated. ``` SYS_OPEN 1 open(path, 0 read, or 1 create/truncate for write) -> fd or -1 SYS_CLOSE 2 close(fd) SYS_READ 3 read(fd, buf, n) -> bytes stored, one per word SYS_WRITE 4 write(fd, buf, n) -> bytes written SYS_GETENV 5 getenv(name, buf, cap) -> length, or -1 if unset SYS_TIME 6 unix seconds SYS_BOPEN 10 open a packed path SYS_BREAD 11 read into packed bytes SYS_BWRITE 12 write packed bytes SYS_BGETENV 13 getenv into packed bytes SYS_RAW 40 any linux syscall: the block at `rawblk` holds the number and six arguments; the result is the kernel's, or -errno. `std.Host.raw(nr, a, b, c, d, e, f)` in src/lib/std.zc builds the block. linux only, -1 elsewhere SYS_POLL 42 1 when input can be read without waiting, else 0 (-1 where the host cannot tell). a streaming host (the wasm and javascript servers) suspends the run once at the call so its event loop can deliver input that is waiting, then answers; the language server uses it to drop a stale check SYS_MEMINFO 9 memory facts: 0 total words, 1 heap start, 2 heap end, 3 total bytes SYS_FILL 18 fill n words at an address with a value SYS_COPY 19 copy n words (overlap safe) SYS_SCAN 20 does any word of a block look like a heap pointer (used by release) SYS_ARGC 21 how many entries argv has SYS_ARG 22 argv(index, buf, cap) -> length, or -1 past the end SYS_ENVC 23 how many entries envp has SYS_ENV 24 envp(index, buf, cap) -> length, or -1 past the end ``` ops 21 to 24 are the command line and the environment. the string ops take an index and copy one entry across one character per word, like `SYS_GETENV`: a zero buffer asks for the length alone, so the caller sizes the string before it fetches it. `argv[0]` is the program path, and `` when the source was read from stdin, so `argc` is 1 even with nothing on the command line. the javascript and wasm vms are given `args` and `env` when they are constructed, and a bare-metal port has neither, so it answers `` on its own. `rt.readfile(path)` returns a fresh string, or 0 if the file can't be opened. `include` uses it. ops 14 to 17 are the c abi. a host that doesn't know an op answers -1, and the zc side then does the work itself with a loop, so the microcore and the javascript vm run the same programs. these are not available from o3 code, whose stores don't go through the gate. ## code as data the compiler resides in vm memory with the program; its functions and globals can be called by compiled code through the `run` namespace, imported with `include run;`. `asm { }` emits raw vm words: numbers push, `-5` pushes a negative, `#n` emits a raw word, one-char tokens from the mnemonic table are opcodes, `:name` defines a label local to the block, enhancement names emit their opcode, and other names push the address of a variable or function. `quote { ... }` is a token list as a fresh object: `[n, kind, value, line, kind, value, line, ...]`, so errors inside a macro expansion point at the line the tokens came from, where names, strings and operators are interned pointers (`run.intern("while")` gets one). `run.tokens(s)` makes one from a string, and `run.qdup(q)` copies one. `run.eval(q)` compiles a token list as a function body and runs it. `run.compile(q)` returns its address instead, `run.load(q)` compiles top-level declarations, and `run.evals(s)` runs `run.eval` on a string. parse words: `immediate fn name() { }` runs at compile time whenever `name` shows up as a statement, inside an expression or at the top level. it starts with the current token on its own name and works the compiler directly: `run.next()`, `run.peek()` and `run.peekat(k)` (up to 4 tokens ahead), `run.tk`/`run.tv` (current token kind and value), `run.expect`, `run.accept`, `run.expr()`, `run.stmt()`, `run.substmt()`, `run.ptype()`, `run.emit`, `run.lit`, `run.op`, `run.hole`/`run.setlit` for jumps, and `run.here`. `run.grab()` reads a `{ }` group into a token list, and `run.feed(q)` pushes a list back into the input, which is all a macro needs. parse words can call `run.eval` partway through a compile: the compiler saves its state and emits the new function in place with a jump around it. the built-in keywords are entries in the same table, so a parse word called `while` replaces `while`. tables: `run.add_binop("**", 11, "ipow c")` adds or replaces a binary operator with a precedence and its code, written like asm (names resolve when it's used). `run.add_unop` does the same for prefix operators, and the lexer learns new spellings from both. `run.mnem` is the asm mnemonic table, `q@!$_~cjzr,?.+-*&|^><%`, and can be rewritten. ## the bootstrap `src/tools/zc0.c` compiles the subset of zc used by `src/compiler/compiler.zc` (without the analyzer, types, quote, parse words, or block expressions, but with `enhance` and top-level `embed` declarations) into the first genome, `build/s0/genome.bin`. a core built around that image compiles `src/compiler/compiler.zc` again in the VM using the full compiler, analyzer, and native division code, and `mkgenome` accepts the result only if it is an image (a compiler that rejects its own source prints diagnostics instead, which `mkgenome` shows). a core built around the second image compiles the source a third time; the two self-compiled images must be identical, and the first becomes `build/core/genome.bin`. the test suite checks this fixpoint on the interpreter, microcore, jit and llvm backends when available. in the zc0 subset `u`, `m` and `d` are recast tags, not names. an image build (`build_image(org, dst)`) compiles with only the image's names, keywords, and built-in types visible, writes code for the requested address, and appends a symbol table containing its names and enhancements. an image build (`build_image(org, dst)`) compiles with only the image's names, keywords, and built-in types visible, writes code for the requested address, and appends a symbol table containing its names and enhancements. ## the instruction set every word is 64 bits. an odd word is a literal that pushes `w>>1` (63 bits, sign extended). an even word is an opcode, `w/2`. there's a data stack and a return stack, both stored inside memory. a literal cannot hold the minimum word (`-2^63`); `rt.put_n` and `std.Text.itoa` special-case it. shift counts wrap at 64. `mulhs` and `mulhu` are the high half of a 128-bit product, and `clz` of 0 is 64. byte addresses name the same memory as word addresses. word `w` is the eight bytes `w << 3` through `(w << 3) + 7`, and byte 0 of a word is its low 8 bits, so the layout does not depend on the host's endianness. `rt.addr_b(w)` and `rt.addr_w(b)` convert. a byte address is an ordinary integer: the odd/even tag is only for code words, and `u`/`m`/`d` stay on the word pointer of a heap object. opcodes 612..617 load and store 8, 16 and 32 bits, little-endian, and a 16- or 32-bit access may cross into the next word. an access that does not lie entirely in memory loads 0 and stores nothing. `rt.ld8`, `rt.st8`, `rt.ld16`, `rt.st16`, `rt.ld32` and `rt.st32` in the runtime are those opcodes. a byte store of a word some translation covers is dropped the same way a word store is. it does not fire the syscall gate; that is still a word store at `SYSGATE`. `bops.h` is the shared implementation. a byte string is that layout as a value. `b"..."` (the `b` against the quote) has type `bstr` and packs the characters into the code stream: word 0 is the length, the bytes start at the next word, eight per word, and a 0 byte follows the last character. `"..."` has type `char*` and stays one character per word. `len` on a byte string uses the packed representation. `s[i]` loads one byte and is not a place to store. `s[i:j]` copies a slice into a fresh object, end exclusive, and clamps the bounds. `+` concatenates two byte strings. the six comparisons require a byte string on both sides and order unsigned bytes. `rt.s_to_n` parses a signed integer and `rt.n_to_s` builds a fresh string; both special-case the minimum word. `rt.put_s` of one byte string writes its bytes. a result of `+`, a slice, `rt.n_to_s` or `rt.new_s` is a shared heap object. the compiler records that a local holds a byte string in bit 6 of its tag. `rt.cons(a, b)` and `rt.triple(a, b, c)` are shared cells on the same heap; `rt.car`, `rt.cdr` and `rt.tget` read them, and `rt.share(p)` publishes an existing object. bit 8 of the header flags is the share bit. it is orthogonal to live, marked and releasing. `rt.free` of a shared object does nothing, `rt.own` and `rt.release` do not follow one, and the sweep collects it when no word points at it, cycles included. a unique object still quarantines on `rt.free`. putting a heap object into a cell publishes it, so the old owner's later `rt.free` does not clear a cell that still points at it. a float is an IEEE-754 binary64 value in one whole word, not an odd literal: 63 bits is not enough, and bit 0 of a double is data. opcode 600 (`flit`, the word 1200) is followed by those raw bits, and the decoder consumes that word itself. 601..604 are add, sub, mul, div. 605..607 are `==`, `<` and `<=` (the other relations are those plus a swap or a xor). 608 and 609 convert int to float and float to int, truncating toward zero. 610 prints six digits after the dot and 611 parses a word-string into bits. 618 prints with a mode: 0 is the shortest spelling that parses back to the same bits, 1 is fixed (digits after the dot) and 2 is scientific (significant digits), and the precision is capped at 17. they sit above the enhancement range, so slots 22..533 do not move. where the processor has an FPU the arithmetic is that FPU: the arm64 jit emits `fadd`, `fmul`, `fcmp`, `scvtf` and `fcvtzs`; llvm lowers the same opcodes to native `double` operations; the interpreter and the microcore call `fops.h`, which is a volatile host `double` so gcc emits SSE2, AArch64 FP or RV64 `fadd.d`. both virt boards turn the FPU on before `main` (RISC-V `mstatus.FS`, AArch64 `CPACR_EL1.FPEN`) and compile with a hard-float ABI. a port with no FPU builds with `-DZY_SOFTFLOAT`: the same `fops.h` entry points are then an integer IEEE implementation, round ties to even, and the float line is unchanged. `fops.h` is also the shared parser and the printer, and both are integer arithmetic, so the text is the same on every platform and does not call libc. opcode 610 is an optional minus, the integer digits, a dot, and six fractional digits, with a tie in the next digit rounded away from zero, for the whole finite range. opcode 618 is the shortest round-trip spelling (a plain decimal when the exponent is between -4 and 15, otherwise scientific), or a fixed or scientific form. the parser keeps the digits it is given, rounds ties to even, and accepts an exponent, `inf` and `nan`. `nan`, `inf` and `-inf` are the non-finite texts, and `-0` is distinct from `0`. `float` is a type. the compiler tracks it through locals (bit 5 of the tag), globals, fields, parameters, call results and returns, and promotes an integer operand of `+ - * /` and the comparisons, so a use does not need its own annotation. `++` is still an integer add. `o3` lifts 600..609: two constant operands fold by the IEEE operation itself, float values stay in FP registers, and `x - x` is left alone so a NaN stays a NaN. `0.0` and `-0.0` are both false in `if`, `while`, `for` and `&&` `||`: a float is compared with `+0` before the integer test. `1.field` is a field access. a dot after digits is a fraction only when a digit or an exponent follows it, and an integer address uses a field name that belongs to one struct. ``` 0 q halt 5 ~ swap 10 , in (Q) 15 * mul 20 < less 1 @ load 6 c call 11 ? empty (J) 16 & and 21 % install 2 ! store 7 j jump 12 . out (G) 17 | or 22..533 enhancements 600 flit 601 fadd 602 fsub 603 fmul 604 fdiv 605 feq 606 flt 607 fle 608 i2f 609 f2i 610 fout 611 fparse 618 ftext 612 ld8 613 st8 614 ld16 615 st16 616 ld32 617 st32 3 $ pick 8 z jz 13 + add 18 ^ xor 4 _ drop 9 r ret 14 - sub 19 > shr ``` stack effects: `!` is `(v addr --)`, `$` is `(n -- copy of the nth item)` so `0 $` is dup, `z` is `(cond addr --)` and jumps when cond is 0, `c` and `j` take their target from the stack, and `%` is `(descriptor -- status)`. there's still no div or mod in the base set, because a divider can't finish in one cycle in hardware. an enhancement can add one. ## memory map and abi see [memory configuration](docs/design/memory.md) for build controls, runtime reservations, and host limits. the initial layout reserves code, compiler tables, the genome, and stacks below `PW`. the heap starts at `PW` on hosted growing maps. tables and execution stacks grow when needed, and emitted code uses stable heap segments after its initial region fills. native code-address metadata grows with it. `MW` selects the default host memory limit (1 GiB, `0x8000000` words, on the default build); `TINYVM_MEMORY_WORDS` and `TINYVM_MAX_MEMORY_WORDS` configure startup memory and its growth limit. the native core reserves stable backing up to the startup maximum, while portable hosts replace their backing allocation when they grow. a build can select another default with `-DMW=...`; the generated `memmap.h` rejects a value smaller than the initial heap. the default map's header is generated by `genmap --preset default` into `build/core/map/` from the `@map` const blocks in `src/runtime/rt.zc` and `src/compiler/compiler.zc`, which stay the source of truth. ``` 0x000000..0x07FFFF code: the program, with globals between functions (CODE_END) 0x080000..0x08FFFF type records (65536 words) 0x090000 frame pointer word (ZFP); 0x090002 is the syscall gate 0x090100..0x113FFF compiler data: interned names, symbol table (16384), locals, fixups, tables 0x114100..0x163FFF genome: 0x114100 entry, then the symbol table address and the image end (327680 words reserved) 0x164100..0x1A3FFF frame stack, grows down from 0x1A4100 (262144 words) 0x1A4100..0x1B40FF return stack, grows up 0x1B4100..0x1BC0FF mark stack 0x1BC100..0x1C4FF0 data stack, grows down 0x1C5000..0x224FFF the heap to start with (0x60000 words), then it grows up to MW ``` the numbers describe the default build's initial reservations. compact compiler and run-only maps start with smaller reservations and also support growth. `build/bin/genmap` writes matching layouts (see "smaller machines"); `--preset fixed` retains the older bounded heap for compatibility. the host answers `core.syscall(9, k)` with facts about the map (0 total words, 1 heap start, 2 heap end for now, 3 bytes). argument 4 requests memory growth, and argument 5 changes the growth limit within the host's backing constraints. calls push the arguments left to right and the callee's prologue pops them into its frame. locals live in the frame at `*0x90000 + slot`. every function leaves exactly one word on the data stack. heap objects have a 5 word header: size, flags with the size class in the high bits, a check word, owner, type. ## how the core works registers: x19 holds the memory base, x20 the data stack pointer, x21 the return stack pointer, x22 the translation table and x23 the state block (memory, s, r, table, coverage map, Q, G, &J, store hook, install hook, enhancement fallback). o3 regions also use x26..x28, which the entry stub saves. compilation is lazy. the first time control reaches an address, that block is translated from live memory. a block ends at call, jump, ret or halt. a `z` doesn't end it: the not-taken side keeps compiling, which makes superblocks. a literal followed by `c`, `j` or `z` is fused into a constant target. every exit goes through a dispatch stub that checks both stack pointers against their limits (kept in x24 and x25), bounds-checks the target, loads `tab[t]`, and branches straight to it when it's already compiled. otherwise it returns to c, which compiles the block and re-enters. enhancements are inlined: inputs load from the data stack into x0..x15, the optimized micro-ops run in registers with x16 and x17 as scratch, and the outputs go back. an install calls into c and invalidates translations and o3 regions that depend on its slot. invalidated code is rebuilt on its next entry. when the loop reaches an o3 entry point, it compiles the region (or reuses it) instead of translating a block. self-modification is handled with a counted coverage map: one byte per word, the number of live translations that cover it, sticky at 255. a store checks that byte inline: an uncovered target, the usual case, gets a plain store. a covered target calls the hook, which looks the word up in a page index (64-word pages, each with a list of the blocks that touch it) and drops only the translations containing that word, decrementing the coverage of just their words. the jit code then returns to c at the next instruction, so even patches to the very next word take effect. writing back the same value is a no-op. installing an enhancement does not flush: every translation records which enhancement slots it inlined, and an install drops only those (and the o3 regions that depend on the slot). if the 16 mb code buffer or the block index fills, everything is flushed. on llvm each block has its own resource tracker, so dropped code is freed once no compiled code is running. the data and return stacks are checked at block boundaries and grow before their guard space is exhausted. `reserve_data(n)` and `reserve_returns(n)` request capacities explicitly. the frame stack grows in stable segments, preserving pointers into older frames; `reserve_frames(n)` controls the next segment's minimum size. when the host or heap cannot provide more storage, execution stops with a controlled overflow or allocation diagnostic. platform: the darwin path uses `MAP_JIT`, `pthread_jit_write_protect_np` and `sys_icache_invalidate`; linux uses rwx mmap and `__builtin___clear_cache`. this session verified the macos arm64 native path for dispatch, frames, stacks, and code segments, along with the FFI learning example and the LLVM 23 backend. the shell runner was exercised with bash on macos. no linux-host runtime tests were run in this session, and the windows batch runner was not run on a windows host. under the hardened runtime, the darwin path also needs the allow-jit entitlement. every host shares the same semantics: shift counts wrap at 64, stores outside memory are ignored, loads outside memory read 0, and jumping outside memory or to an unwired opcode halts. cells are 8 bytes on the host; the word addresses in the map above did not move. ## the microcore `src/micro/micro.c` is a portable interpreter for the same genome, small enough to move to new targets, including bare metal. it complements `src/core/core.c` with the memory map, two stacks, base opcodes, call and return, and genome loading at `GORG`. it has no jit, llvm backend, o3 lifter, or enhancement optimizer. it uses `stdint.h` and the shared headers `src/micro/micro_port.h`, `src/core/fops.h`, `src/core/bops.h`, `src/core/stack.h` and `src/core/genome.h`, which embeds the image; `micro_port.h` includes the generated `memmap.h`. its five host functions are declared in `src/micro/micro_port.h`. bss must be zero before `micro_run`. on a cpu without a 64-bit divider, the compiler may link its own division helpers. the microcore's initial memory comes from the generated map, and hosted ports can configure the initial size and growth limit. the host interface is `src/micro/micro_port.h`. `src/micro/port_linux.c` provides stdin, stdout, syscalls, and stderr diagnostics, matching `src/core/io_linux.c` plus the syscall gate in `src/core/core.c`. syscalls 1 to 6 handle files, environment variables, and unix time; 7 exits, 8 reads a counter, and 9 reports or configures memory. `src/micro/port_stub.c` is the freestanding template: connect `uart_putc` to a transmit register, leave `micro_getc` at end of input, and have `micro_syscall` return -1 for every operation. the runtime then sees failed file opens and no clock. `src/baremetal/port.c` implements the same five functions for the virt boards, where the operations are available. link the stub instead of `port_linux.c` and call `micro_run` from reset. `/`, `%`, and `<<` are not base opcodes. `src/runtime/rt.zc` installs them, along with `lfa` and `outside`, through opcode 21 when the genome boots; programs can install more with `enhance`. the microcore supports install because examples such as `wiring.zc`, `mandel.zc`, and o3 programs call `lfa` and `outside` from ordinary bytecode. recipes and straight-line `vm` bodies are lowered to micro-ops and executed, without constant folding or multiply-high rewrites, so `x / 7` remains a divide. a `vm` body that performs i/o or branches fails at install time with the same error as the full core, and any `else` fallback remains available. install kind 2, an o3 region, returns success but is ignored; the bytecode still runs, so these programs keep their output without the o3 fast path. lowering is limited to 128 ops and 256 values per enhancement, and 4096 ops in the pool. enhancement stores do not pass through the syscall gate, matching `src/core/core.c`. what it leaves out, on purpose: native code, the optimizer, o3's register interpreter, coverage maps, and threads. it is the speed of the plain interpreter, and a bit slower where the full core turned a constant divide into a multiply. it is still not a C ABI, and a character is still one word. details are in `docs/history/21-micro`. ## smaller machines `src/tools/genmap.c` computes the layout for each build. the `small` preset carries the compiler; `run` carries the runtime and a precompiled program. `src/tools/automap.sh` sizes the genome reservation to the actual image and rebuilds the bootstrap compiler for that map. `make run PROG=...` and `src/tools/zcc` build run-only images through stage 0, so their source must fit the bootstrap compiler's supported subset. `src/tools/build.zc` is the full-language image builder: it compiles the source after its backtick into a little-endian genome image. the run-only runtime installs enhancement descriptors before heap initialization, so enhanced operations and full-language array algorithms are available in self-hosted images. ```sh bash src/tools/automap.sh small build/small clang -O2 -fwrapv src/micro/micro.c src/micro/port_linux.c -Isrc/core -Isrc/micro -Ibuild/small --embed-dir=build/small -o build/small/zym build/small/zym < examples/hello.zc make run PROG=examples/hello.zc build/run/zymr build/bin/genmap --preset default --check ``` use `genmap` options to adjust heap, table, and stack sizes at build time. compile the core against the generated headers and the genome for that same layout. `--preset fixed` keeps the historical layout; `--preset default` places the heap above the fixed regions so it can grow. see [memory configuration](docs/design/memory.md) for controls and current limits. ## bare metal `src/baremetal/run.sh` builds the microcore without an os or libc and boots it on a qemu `virt` board. the port provides a uart, clock, ram files, and exit handling. startup enables the fpu, clears bss, and calls `micro_run`. ```sh src/baremetal/run.sh ready riscv src/baremetal/run.sh riscv run examples/hello.zc src/baremetal/run.sh riscv io examples/echo.zc src/baremetal/run.sh riscv compile examples/fact.zc src/baremetal/run.sh aarch64 run tests/programs/gc.zc ``` `run` embeds the compiled program. `io` also supplies stdin through the uart. `compile` embeds the resident compiler and sends it source followed by a backtick. the scripts check for the required cross compiler and qemu before running; unavailable targets are skipped by the test suite. board ram remains a physical limit. ## maintenance the compiler also reports [definite bounds and lifetime findings](docs/design/storage-analysis.md), with known sizes and lifetimes in editor hovers. the maintenance checklist and remaining memory work are tracked in [TODO.md](TODO.md). the [development transcript](docs/history/transcript.md) and [earlier test results](docs/history/maintenance-notes.md) are historical records.