aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorGravatar Moon Flower <moonflower@elhenney.local> 2026-10-14 08:29:46 +0330
committerGravatar Moon Flower <moonflower@elhenney.local> 2026-10-14 08:29:46 +0330
commit1c53304d03414890af0da7432d1aea87e754a14a (patch)
tree97190d86cf548c9bc63f68c86a4d2a4f09809713 /README.md
parenta (diff)
feat: grow memory limits, add icon/actor libraries, overhaul LSP diagnostics
Compiler and VM - make every memory map limit dynamic and adjustable at both build time and runtime, including compiler tables, stacks, and every execution region - reserve and grow tables, operand and return stacks, frame segments, and image-building output without relocating live code or globals - rename i/o and string helpers consistently (puts -> put_s, putn -> put_n, scat -> cat_s) across compiler, runtime, examples, tests, and docs - recast and consume values assigned to globals from inside functions - allow fixed and const together, add typed closure returns (fn:t(...)), multi-include syntax (include rt, run, core;), and recurse type inference - never infer string literals as word, anywhere, as argument, variable, or return type - draft docs/BORROW.md: a borrow checker less intrusive than Rust's but more intelligent LSP and editor integration - require explicit includes and give every toolchain area its own namespace (rt, lsp, run, core) - hover for built-in types, inferred types, values known at hover time, array and pointer lifetime/size, library names in include statements, and inferred lifetimes with explanations - honor /* */ comments as documentation for variables and arguments, and stop misattributing section headers or trailing comments as variable hints - stream diagnostics and cancel stale requests on change - jump to definition and references, including cross-file - quick fixes with automatic library import, and namespace completion Hints and diagnostics - render hints as blue squigglies by default, with hints on/small/off per file - suppress magic-number hints via a preceding /* */ comment, and say so - warn on unused arguments; allow multiple _ discards in parameters, pattern matching, and tagged enums, warning if a _ is actually used - rewrite hint and comment prose in professional compiler-doc register Libraries - add icon.zc: run-time self-modification, find/replace syntax, type and block edits, and step-through inspection of locals and globals - add actor.zc: async actors with Actor<I,O>, implicit this, spawn/recv/send, injected into the compiler machinery at include time rather than built in Tooling and docs - reorganize sources, tests, and designs into subdirectories - unify windows entry points in tinyvm.bat and add a matching tinyvm.sh - add --help across the toolchain - polish the learning guide, stdlib docs, and design notes Tests - add accept, reject, and regression coverage across the compiler, LSP, - runtime, icon, actors, threads, formatting, and the toolchain contracts, plus fuzzers and property tests
Diffstat (limited to 'README.md')
-rw-r--r--README.md193
1 files changed, 119 insertions, 74 deletions
diff --git a/README.md b/README.md
index 711247b..bcad133 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,6 @@
# tinyvm zygote
-a tiny stack cpu that started as a c89 code golf entry and ended up as a self-hosting runtime. a small native core runs vm code: jit-compiled to arm64, compiled through llvm on other hosts if you build with it, or interpreted. the genome it boots is a compiler for zc, a c-like language that keeps the forth habits. the compiler lives in vm memory next to your program, and your program can call into it, patch its tables, add syntax to it and compile new code while it runs. heap objects carry an ownership tag (u, m or d) that a static analyzer inside the compiler checks, with deterministic frees at scope ends and a mark-sweep collector underneath. types are a compiler primitive, with structs, generics, reflection and an optional typechecker. a program can also wire up to 512 custom opcodes into the cpu (virtual enhancements), which the jits inline as native code. and code marked `o3` skips the stack machine: the core compiles it as a whole region, with its variables in registers.
+a small stack cpu began as a C89 code golf entry and became a self-hosting runtime. a native core runs vm code with an arm64 jit, through llvm on supported builds, or in the interpreter. the genome contains a compiler for zc, a C-like language with forth influences. the compiler resides in vm memory alongside the program; code can call its functions, modify its tables, add syntax, and compile new code at runtime. heap objects carry an ownership tag (u, m or d) checked by a static analyzer, with deterministic frees at scope ends and a mark-sweep collector underneath. types are a compiler primitive, with structs, generics, reflection and an optional typechecker. programs can install up to 512 custom opcodes (virtual enhancements), which the jits inline as native code. code marked `o3` bypasses the stack machine: the core compiles the region as a unit and keeps its variables in registers.
this folder has the latest code. `docs/LEARN.md` is a tour of the language. `docs/history/` is the scrapbook: notes for every stage, plus frozen code from the stages that still have it.
@@ -12,14 +12,16 @@ this folder has the latest code. `docs/LEARN.md` is a tour of the language. `doc
- `bf.zc`: a brainfuck compiler. it turns brainfuck into zc source, has the resident compiler compile it, and runs it (native code on the jit). it takes a program after a backtick, or runs hello world.
- `dump.zc`: prints any heap struct, nested ones included, with no printing code, by walking its type through reflection.
- `cli2.zc`: reads the command line and the environment, then captures its own output into a string.
-- `tree.zc`: a binary search tree whose nodes own their children. there's no `free` anywhere, and the live object count drops back to 0 when the root's block ends.
-- `lang.zc`: grows the language while it compiles: a `|>` pipe operator (one table entry, since on a stack machine it's just `call`), `unless`, an `assert` that knows its source line, and `times n { }`.
+- `tree.zc`: a binary search tree whose nodes own their children. there's no `rt.free` anywhere, and the live object count drops back to 0 when the root's block ends.
+- `lang.zc`: extends the language at compile time with a `|>` pipe operator (one table entry, since a stack machine implements it with `call`), `unless`, an `assert` that reports its source line, and `times n { }`.
- `staged.zc`: writes a polynomial as source with its coefficients baked in, compiles it while running, and finds it by name.
- `quine.zc`: prints its own source.
- `life.zc`: conway's life on a typed 2d board, with a glider wrapping around a torus.
- `primes.zc`: an o3 sieve over a fixed array, the primes gathered into a growable one, then twin primes and the widest gap.
-`tests/` contains feature tests, `tests/errors/` contains programs the lifetime analyzer must reject, `tests/typeerrors/` contains programs the typechecker must reject, and `tests/langerrors/` covers o3 and array misuse. each error test begins with its expected message; `tests/runerr.sh` runs a test folder. `tests/test.sh` bootstraps the compiler and runs the suite on the interpreter, microcore, and available jit and llvm backends, then repeats a subset on smaller maps. `tests/bench/bench.sh` compares o3 with plain code on each backend. `src/lib/std.zc` is a library included by programs, not part of the genome. `docs/LEARN.md` is a language tour, and `docs/history/` keeps earlier versions. `src/micro/micro.c` is a portable interpreter for the same genome; `src/micro/micro_port.h` defines its host interface, with `src/micro/port_linux.c` as the hosted port and `src/micro/port_stub.c` as a bare-metal starting point. `src/baremetal/` boots the microcore without an operating system under `qemu-system-riscv64` and `qemu-system-aarch64`. `src/core/memmap.h` defines the default map and initial reservations. `src/tools/genmap.c` writes smaller maps and matching source files; `src/runtime/run.zc` is the runtime-only boot stub.
+`tests/` contains feature tests, `tests/errors/` contains programs the lifetime analyzer must reject, `tests/typeerrors/` contains programs the typechecker must reject, and `tests/langerrors/` covers o3 and array misuse. each error test begins with its expected message; `tests/runerr.sh` runs a test folder. `tests/test.sh` bootstraps the compiler and runs the suite on the interpreter, microcore, and available jit and llvm backends, then repeats a subset on smaller maps. `tests/bench/bench.sh` compares o3 with plain code on each backend. `src/lib/std.zc` is embedded as source in the compiler and explicitly compiled by programs with `include std;`. each file has its own namespace and imports; including a dependency does not import it into the caller. use `include rt, std;` when a file needs both runtime APIs and standard-library helpers. use `include core;` for the core gate (`core.syscall`), and `include run;` for compiler APIs such as `run.evals`. `docs/LEARN.md` is a language tour, and `docs/history/` keeps earlier versions. `src/micro/micro.c` is a portable interpreter for the same genome; `src/micro/micro_port.h` defines its host interface, with `src/micro/port_linux.c` as the hosted port and `src/micro/port_stub.c` as a bare-metal starting point. `src/baremetal/` boots the microcore without an operating system under `qemu-system-riscv64` and `qemu-system-aarch64`. `src/core/memmap.h` defines the default map and initial reservations. `src/tools/genmap.c` writes smaller maps and matching source files; `src/runtime/run.zc` is the runtime-only boot stub.
+
+`namespace app;` gives declarations in a file the `app` namespace. imports are file-local: a dependency's `include` statements do not import their namespaces into the caller. the learning examples [27](examples/learn/27-namespaces.zc) and [28](examples/learn/28-string-inference.zc) show a namespaced file with multiple imports and the inferred types of both string literal forms.
## build and run
@@ -37,6 +39,12 @@ make run PROG=examples/hello.zc
`make genome` bootstraps the compiler, compares two self-compiled images, and updates `src/core/genome.h` only after they agree. rebuild the native, javascript, and web runtimes after changing the genome. `make check` runs the smaller suite while iterating.
+### command-line help
+
+the native vm, portable vm, language server, bootstrap compiler, image converter and map generator accept `--help` or `-h`. help exits before reading input or allocating vm memory, so it also works with stdin left open. `bin/zy` reads source from stdin and runs `main`; `bin/zym` runs the embedded image, and compiler images can read source from stdin. run-only images read program input directly. `bin/zclsp --format`, `--annotate` and `--types` remain available for source on stdin; without one of these options it reads language-server protocol messages. `bin/zc0` compiles bootstrap source from stdin to a C genome header on stdout; `bin/mkgenome` converts little-endian 64-bit image words from stdin into that header format. `bin/genmap` accepts a preset or compiler/run mode, optional output directory, and build-time map sizing options. the help for each tool lists its input/output format and memory environment variables where applicable.
+
+the same early help option is available in `tinyvm.sh`, `tinyvm.bat`, `src/tools/zcc`, the public `src/tools/*.sh` build scripts, and the javascript vm and extraction utilities. `src/tools/zcc [-o OUT] [-k] FILE.zc` builds a standalone run-only binary. `src/tools/automap.sh run|small|default|lsp OUTDIR FILE...` generates a sized map and image; `src/tools/embed.sh std|icon|all [--check]` refreshes embedded modules, while its `lsp OUT` form writes source providers. `src/tools/mklsp.sh [OUTDIR]`, `src/tools/mklspwasm.sh [OUTFILE.wasm]` and `src/tools/mkvsix.sh [OUTFILE.vsix]` build the language server and extension. the javascript vm accepts an optional source file; without one it reads source from stdin. the extraction utilities take no input arguments and read the generated headers from their documented paths. `python3 tests/drivers/cli_help_test.py` checks the native, micro, LSP, bootstrap and image tools, shell scripts, and javascript utilities with open stdin and invalid memory settings, rejects unknown options, and confirms the language server source options still work.
+
`make js` regenerates the JavaScript bundle and needs Node.js and the TypeScript compiler (`tsc`). running the committed `src/js/tinyvm.js` directly needs Node.js only.
### interactive runners
@@ -78,14 +86,19 @@ cc -O2 -fwrapv -DJTYPE=int -DZYLLVM $(llvm-config --cflags) src/core/core.c src/
### language server
-`src/lsp/server.zc` is a language server written in zc. it doesn't have a parser of its own: it is the real compiler (`src/compiler/compiler.zc`) running in check mode, with a json-rpc server on top. so the errors are the compiler's errors, with the same words, and a program that grows the language with parse words (`immediate fn`, `add_binop`, `quote`) is checked and highlighted with the syntax it added, because those words run during the check exactly as they do in a build.
+`src/lsp/server.zc` is a language server written in zc. it runs the compiler (`src/compiler/compiler.zc`) in check mode and provides a json-rpc protocol. diagnostics use the compiler's messages. parse words such as `immediate fn`, `run.add_binop` and `quote` execute during checks, so editor parsing and highlighting reflect syntax added by the program.
-hover shows what a name is, from the text of its declaration. a top-level function, global or struct shows its declaration line, and a method or static function of a struct shows its indented line the same way. the comment lines directly above a declaration are its doc, whether or not it has a signature, so a declaration with no doc still shows its signature. a parameter shows its declaration, and a `let` local shows the type the compiler gave it (recorded during the check, by byte offset). an untyped local, parameter or function return shows the type the compiler infers for it. to get that, the server checks the document once more with the untyped names left open and reads back the type each one ends up with. the result is kept until the document changes, so the first hover after an edit takes longer than the rest. a built-in function shows the signature stored in the genome, and a keyword such as `micro` or `unchecked` shows a short line on its form and what it does, from one table in `src/lsp/server.zc`. calls that come through an `include` are not followed yet, so a helper like `Text.itoa` from `src/lib/std.zc` shows nothing from the server.
+hover shows declaration signatures and documentation. a top-level function, global or struct shows its declaration line; a method or static function shows its indented declaration. comments directly above a declaration provide its documentation, and a declaration without comments still shows its signature. parameters show their declarations, while `let` locals show the compiler-assigned type. untyped locals, parameters and function returns show inferred types; the server obtains these by checking the document again with unresolved names open. it caches the result until the document changes, so the first hover after an edit may take longer. hovering an imported module name shows its module overview and include form; source libraries can provide the same overview in a comment after their namespace and include headers. embedded `core.zc` is provided alongside runtime, compiler and standard-library sources, so `core.syscall` and `rt.syscall` navigate to and reference their distinct declarations. keywords such as `static`, `micro` and `unchecked` show their form and meaning from one table in `src/lsp/server.zc`.
-check mode is the compiler with errors that write records instead of printing. it doesn't halt: it keeps going after an error, up to 20 of them, so a document with several mistakes shows all of them. each record carries its line and the token's place in the source, and the server turns that into a utf-16 range. check mode also notices things that aren't errors, and reports them as hints (see below).
+check mode records errors instead of printing them and continues after an error, up to 20 diagnostics per document. each record contains a line and token offset; the server maps these to utf-16 ranges. check mode also reports non-error findings as hints.
- **diagnostics** on open, change and save, with an id echoed on every reply, `MethodNotFound` for unknown requests and `-32002` before `initialize`.
-- **semantic tokens** (`textDocument/semanticTokens/full`) in 12 types: keyword, type, function, variable, string, number, operator, comment, parameter, property, macro, enumMember. what a name is comes from the compiler's own symbol tables and the names it saw declared, not from guessing. the check records each name it resolves and where, and a name resolves the way the compiler resolves it, so locals and parameters are looked up ahead of globals, and built-ins come last. a local named `cap` is a variable wherever it is in scope, and the built-in `cap` only applies to uses outside that scope.
+- **definition and references** (`textDocument/definition`, `textDocument/references`): a local, parameter or field goes to its declaration by the offset the compiler recorded, and a function, global, constant, struct or type goes to where the compiler saw it defined, in an included file too (the file is found from the document's directory upward, then as written in the `include`). references lists the uses in the document, with the declaration when asked. built-in names navigate to their embedded source when one is available.
+- **module completions**: after an imported namespace and a dot, completion lists that module's exported members with signatures and documentation; it does not add unrelated local or module names. module hovers and completions cover `rt`, `run`, `core`, `lsp`, `std` and `icon`.
+- **import quick fixes**: compiler diagnostics for missing imports or unqualified exports offer a workspace edit. the edit adds the needed `include` without duplicating an existing import, qualifies bare exports, and preserves namespace declarations, comments, and crlf line endings. the editor client applies only edits for the current document version, while keeping protocol version epochs separate from editor document versions.
+- **semantic tokens** (`textDocument/semanticTokens/full`) in 12 types: keyword, type, function, variable, string, number, operator, comment, parameter, property, macro, enumMember. token kinds come from the compiler's symbol tables and declarations. check mode records each resolved name and source location, following the compiler's lookup order: locals and parameters precede globals, and built-ins are checked last. a local named `cap` is a variable wherever it is in scope, and the built-in `cap` only applies to uses outside that scope.
+- **formatting** (`textDocument/formatting`, and `zclsp --format < file.zc` on the command line): a pretty-printer in the spirit of clang-format with `ColumnLimit: 0`. it keeps the author's line breaks and sets the indent of every line (from braces, open parentheses and unfinished statements; `case` labels are indented one level inside their `switch` by default), the spaces after commas and around `=`, `==`, `&&` and the like, the blank lines between lines, and the final line break. it never moves a token to another line, and it leaves strings, comments, `asm` and `vm` blocks and everything after a backtick as written, so the tokens of a program are the same before and after. options come from a `.zcformat` file in the document's directory or the nearest directory above it, one `Key: value` per line (`#` starts a comment), then from the editor's tab size and `insertSpaces`, then from the defaults: `UseTab: true`, `IndentWidth: 4` (the spaces of a level when `UseTab` is false), `MaxEmptyLines: 1`, `ContinuationIndent: 1`, `NormalizeSpacing: true`, `CaseIndent: 1`, `FinalNewline: true`. `CaseIndent: 0` aligns each `case` label with its `switch`. `tests/drivers/format_test.js` formats every `.zc` file in the repository and checks that the tokens are unchanged and that formatting twice gives the same text.
+- **streaming and cancelling**: a check of a document with 100 or more top-level items sends the diagnostics found so far each time the item count doubles, so errors near the top appear before a large file is done. before each of those points the server queries the host (`core.syscall(42)`) for pending input. if input is waiting, the check is dropped without publishing because newer text may replace the current document. the check publishes once no newer input is waiting. smaller documents are checked in one go.
- **cli mode**: source on stdin prints one line per finding, `line:col: error: ...` or `line:col: hint: ...`, or `no errors`.
the server runs on its own memory map (`genmap --preset lsp`), which has a bigger genome slot and a smaller frame stack than the default, and it keeps its documents in a store of its own, 4 chars to a word.
@@ -138,64 +151,76 @@ node tests/drivers/wasm_test.js
it reads like c. underneath every value is a 64 bit word and memory is word addressed, so `p + 1` is the next word whatever `p` points at, and strings are one char per word with a 0 at the end. types are optional: untyped code is typed `word`, which goes with anything.
```
+include rt;
const N = 10;
-[N]int table; // a global array
+int[N] table; // a global array
int fib(int n) { if (n < 2) return n; return fib(n - 1) + fib(n - 2); }
fn main() {
for (int i = 0; i < N; i++) table[i] = fib(i);
let sq = { let t = table[5]; t * t }; // a block is an expression too, its last line is its value
let x = asm { 2 3 + }; // raw vm code, leaves one value
- put_n(sq + x); put_c(10);
+ rt.put_n(sq + x); rt.put_c(10);
return 0;
}
```
-declarations are c-style, type first: `int a = 1, b;`, `node* p = new(node);`, `[4]char buf;`, `point origin;`. `let` declares without a type: it infers pointers and function types from the initializer, uses a pointer alias for an in-place array or struct, and keeps numbers as plain words, and `let a[10];` is an untyped array. statements are declarations, `if`/`else`, `while`, `do`/`while`, `for`, `switch`, `break`, `continue`, `return`, `{ }` blocks, `asm { }` and expressions. `do` body `while (` cond `);` runs the body once before the test, and `continue` jumps to the test. `enum color { RED, GREEN = 5, BLUE };` makes global constants, and `color` is a type of its own in typed code (`typecheck on;`): a member takes the enum, a plain int does not become a `color`, and a `color` still becomes an `int`. untyped code treats the members as plain ints. each member is also reached as `color.BLUE`, in an expression or a `case` label. a member can carry fields too: `enum shape { circle { int r; }, rect { int w; int h; }, dot };` makes the constructors `shape.circle(3)` and `shape.rect(4, 5)`, which return an owned `shape*`, and `shape.dot()`. the tag is the word at offset 0 of the object. a `switch` over a `shape*` matches a member with `case circle(r):`, which binds its fields in that arm only. a switch on a tagged enum that leaves out a member and has no `default` gets a warning on its line (not an error). `(T)e` is a cast with the same meaning as `e as T`: the bits stay, the type changes. it has to be a simple type the parser can see in a few tokens, `int` or `*node`, not a nested array. `include "path";` reads a file and compiles it at that point, signatures first, so a library can live in its own file. `u`, `m` and `d` are ordinary names. a tag is `.u`, `.m` or `.d`, and `u*x` with no space before the star is still a recast. `switch (e) { case 1, 2: ... default: ... }` takes constant labels, and its cases don't fall through: each ends where the next begins, and `break` leaves early. operators are c's with c's precedence, including `c ? a : b`, plus `p.x` and `p->x` (the same thing: `.` goes through a pointer on its own), `e as T` to change a type without changing the bits, and the casts below. comparisons and `&& ||` give 0 or 1. `/`, `%` and `<<` compile to native enhancement opcodes. character literals, `0x` hex and the usual escapes are there.
+declarations are C-style, type first: `int a = 1, b;`, `node* p = rt.new(node);`, `char[4] buf;`, `point origin;`. `let` declares without a type: it infers `char*` from a word-string literal and `bstr` from a packed byte-string literal, infers other pointers and function types from the initializer, uses a pointer alias for an in-place array or struct, and keeps numbers as plain words. `let a[10];` is an untyped array. statements are declarations, `if`/`else`, `while`, `do`/`while`, `for`, `switch`, `break`, `continue`, `return`, `{ }` blocks, `asm { }` and expressions. `do` body `while (` cond `);` runs the body once before the test, and `continue` jumps to the test. `enum color { RED, GREEN = 5, BLUE };` makes global constants, and `color` is a type of its own in typed code (`typecheck on;`): a member takes the enum, a plain int does not become a `color`, and a `color` still becomes an `int`. untyped code treats the members as plain ints. each member is also reached as `color.BLUE`, in an expression or a `case` label. a member can carry fields too: `enum shape { circle { int r; }, rect { int w; int h; }, dot };` makes the constructors `shape.circle(3)` and `shape.rect(4, 5)`, which return an owned `shape*`, and `shape.dot()`. the tag is the word at offset 0 of the object. a `switch` over a `shape*` matches a member with `case circle(r):`, which binds its fields in that arm only. a switch on a tagged enum that leaves out a member and has no `default` gets a warning on its line (not an error). `(T)e` is a cast with the same meaning as `e as T`: the bits stay, the type changes. it has to be a simple type the parser can see in a few tokens, `int` or `*node`, not a nested array. `include "path";` reads a file and compiles it at that point, signatures first, so a library can live in its own file. each file has its own namespace and imports; dependencies included by a file do not import into the caller. `u`, `m` and `d` are ordinary names. a tag is `.u`, `.m` or `.d`, and `u*x` with no space before the star is still a recast. `switch (e) { case 1, 2: ... default: ... }` takes constant labels, and its cases don't fall through: each ends where the next begins, and `break` leaves early. operators follow C precedence, including `c ? a : b`, plus `p.x` and `p->x` (the same thing: `.` goes through a pointer on its own), `e as T` to change a type without changing the bits, and the casts below. comparisons and `&& ||` give 0 or 1. `/`, `%` and `<<` compile to native enhancement opcodes. character literals, `0x` hex and the usual escapes are there.
+
+functions are declared like C, return type first: `int f(int x)`, `node*.u make()`, `.u grab()` (a tag on its own), `void log(char* s)`. `fn f(a, b)` starts without explicit parameter or return types; constraints from calls and returned values propagate through callers and callees until inference reaches a fixed point, including recursive calls anchored by typed callees. conflicting argument types or an unknown reassignment leave the affected type unresolved instead of choosing a type. the learning example [32](examples/learn/32-inferred-calls.zc) shows this propagation and an owned raw-list return. a generic function lists its type parameters after its name, `T first<T>(*list<T> l)`, the same place a call gives explicit ones, `first<int>(l)`. the return type can use them before they're declared: while it's read, an unknown name is taken as a type parameter, and the list after the name has to confirm it. function types are written the same way, `int(int, *char)`, and `fn(int)` for one that returns a word. the earlier `fn f(...) -> T` spelling is gone.
-functions are declared like c, return type first: `int f(int x)`, `node*.u make()`, `.u grab()` (a tag on its own), `void log(char* s)`. `fn f(a, b)` is the untyped form, returning a word. a generic function lists its type parameters after its name, `T first<T>(*list<T> l)`, the same place a call gives explicit ones, `first<int>(l)`. the return type can use them before they're declared: while it's read, an unknown name is taken as a type parameter, and the list after the name has to confirm it. function types are written the same way, `int(int, *char)`, and `fn(int)` for one that returns a word. the earlier `fn f(...) -> T` spelling is gone.
+array types put the element type first. `T[n]` has its size in the type and lives in place. `T[]` also lives in place; its length is inferred from an initializer such as `int[] a = {1, 2, 3};` and cannot grow. `T[?]` can also be initialized in a function with a list, such as `int[?] a = {1, 2};`. it is a growable array: a small handle object that owns its items, made with `rt.new(T[?])` or `rt.new(T[?], n)`, indexed with a bounds check, and changed with `push(a, x)`, `pop(a)` and `resize(a, n)`. `len(a)` works on both (for `T[n]` it's the constant) and `cap(a)` gives the room before the next grow. `fixed T[?]` is a `T[?]` that can't change size: `rt.new(fixed T[?], n)` needs the length, and `push`, `pop` and `resize` on it are compile errors. a `fixed T[?]` can't be passed where a `T[?]` is expected, the other way round is fine, and a constant index outside a `T[n]` is caught at compile time. the items belong to the array, so freeing it frees them, and objects stored into a `T*[?]` are its children like any container's.
-arrays come in three kinds. `[n]T` has its size in the type and lives in place. `[]T` is a growable array: a small handle object that owns its items, made with `new([]T)` or `new([]T, n)`, indexed with a bounds check, and changed with `push(a, x)`, `pop(a)` and `resize(a, n)`. `len(a)` works on both (for `[n]T` it's the constant) and `cap(a)` gives the room before the next grow. `fixed []T` is a `[]T` that can't change size: `new(fixed []T, n)` needs the length, and `push`, `pop` and `resize` on it are compile errors. a `fixed []T` can't be passed where a `[]T` is expected, the other way round is fine, and a constant index outside a `[n]T` is caught at compile time. the items belong to the array, so freeing it frees them, and objects stored into a `[]*T` are its children like any container's.
+the array intrinsics also accept receiver syntax: `a.len()`, `a.cap()`, `a.push(x)`, `a.pop()` and `a.resize(n)`. fixed arrays provide `len()` and `cap()` but reject `push()`, `pop()` and `resize()`. [the learning example](examples/learn/19-growable.zc) shows both receiver forms.
-a unit is read whole and scanned for function signatures before it compiles, so calls can come before definitions, tags included, and prototypes are optional. `main` runs after the whole program has compiled. every function returns one word, 0 if it falls off the end.
+`fixed const T[n]` and `const fixed T[n]` both make the array cells and aliases read-only. element writes, increments, compound assignments, and rebinding a `const fixed T[?]` handle are rejected even when `typecheck off;` is active. a read-only array reference cannot become a mutable reference. a formal parameter such as `const fixed int[2] values` accepts either read-only or mutable arrays. element values can still be copied into ordinary mutable scalars. [the learning example](examples/learn/33-const-fixed.zc) shows both qualifier orders and a read-only formal parameter.
-builtins from the runtime: `put_c`, `get_c`, `eof`, `put_s`, `put_n`, `halt`, `len_w`, `eq_w`, `copy_w`, `fill_w`, `new`, `new_t`, `new_raw`, `new_oz`, `free`, `recycle`, `arena_new`, `arena_alloc`, `arena_reset`, `gc`, `isobj`, `osize`, `typeof_obj`, `live_objects`, `install`, `zdiv`, `zmod`, `zshl`, `mem_set`, `memreport`, `checkptr`, `syscall`, `readfile`. `src/lib/std.zc` groups its helpers in structs of static functions, called as `Group.name(args)`, with typed signatures: `Text` (`atoi`, `itoa`, `cat`, `starts`), `Host` (`env`, `now`, `writefile`), `Bytes` (the same helpers for packed byte strings), `Arena` (`alloc`, `reset`), `Buf` (`length`, `capacity`, `get`, `set`) and `capture` (`start`, and `end` on its buffer). the helpers are typed like any function: a method declares its return type and parameters the same way (`int atoi(char* s)`, `static word start()`). the constructors `arena(n)` and `bnew(n)` stay plain functions, because an owned result has to carry its `.u` tag, and a method with a `.u` return is not supported yet. the globals `argc`, `argv`, `envc` and `envp` are set by `libinit`.
+`include std;` also registers read-only array algorithms: `find` and `find_if` return a matching index, `contains`, `any`, and `all` return a boolean word, `count` counts matches, `map` transforms elements into a new handle, `filter` copies matching elements into a new handle, and `reduce` folds them from left to right. `any` is false and `all` is true for an empty input; `reduce` returns its initial value. they accept mutable fixed or growable arrays; support for const-qualified arrays is unverified. lambda parameter types follow the element type, map's result type follows the callback return, and filter preserves the input element type. algorithms visit the input's initial length in index order, so callbacks must not change its length. map/filter results are independent handles that shallow-share reference elements with the collector. aggregate-by-value elements are not supported by `push`, `pop`, `map`, or `filter`. [the runnable example](examples/learn/31-array-algorithms.zc) shows inferred callbacks and search.
-the command line, the way c hands it to `main`: `argc` is an `int` and `argv` an `[]word` (a zc string is one character per word, so there is no `string` type), and the same for `envc` and `envp`, whose entries are `NAME=value`. `argv[0]` is the program path, and `<eval>` when the source came from stdin. `libinit()` fills all four and the runtime calls it before `main`, if the program has one, so `main` keeps its `fn main()` shape and every example is unchanged:
+a unit is read whole and scanned for function signatures before it compiles, so calls can come before definitions, tags included, and prototypes are optional. `main` runs after the whole program has compiled. use `void f()` for a function with no result and `return;` to leave it; value-returning functions use an explicit result type or the inferred `fn` form.
+
+the runtime api lives in the `rt` namespace and requires `include rt;`: `rt.put_c`, `rt.get_c`, `rt.eof`, `rt.put_s`, `rt.put_n`, `rt.halt`, `rt.len_w`, `rt.eq_w`, `rt.copy_w`, `rt.fill_w`, the packed-string helpers `rt.len_s`, `rt.at_s`, `rt.slice_s`, `rt.cat_s`, `rt.eq_s`, `rt.s_to_n`, `rt.n_to_s` and `rt.put_bs`, allocation and ownership helpers such as `rt.new`, `rt.new_t`, `rt.free`, `rt.gc`, `rt.memreport`, and host functions including `rt.syscall` and `rt.readfile`. their signatures and descriptions are in the runtime source comments and built-in hover table. `src/lib/std.zc` declares the `std` namespace; its static groups are called as `std.Group.name(args)`: `std.Text`, `std.Num`, `std.Host`, `std.Bytes`, `std.Arena`, and `std.Buf`. the constructors `std.arena(n)` and `std.bnew(n)` and globals such as `std.argc` and `std.argv` are plain declarations in that namespace. capture uses static functions: `std.capture.start()` begins output capture and `std.capture.end(buf)` ends it and returns the buffer.
+
+`std.argc` and `std.argv` provide command-line arguments to `main`; `std.envc` and `std.envp` provide environment entries in `NAME=value` form. `std.argc` is an `int`, and `std.argv` is a `char*[?]`; the environment fields have matching types. `std.argv[0]` is the program path, or `<eval>` when source comes from stdin. `std.libinit()` fills all four before `main` when the program includes `std`. Since include dependencies do not import into the caller, runtime calls also require `include rt;`:
```
-include "src/lib/std.zc";
+include rt, std;
fn main() {
- for (int i = 0; i < argc; i++) { put_n(i); put_c(32); put_s(argv[i]); put_c(10); }
+ for (int i = 0; i < std.argc; i++) { rt.put_n(i); rt.put_c(32); rt.put_s(std.argv[i]); rt.put_c(10); }
return 0;
}
```
-`capture.start()` redirects everything the program writes into a string, and `buf.end()` stops it and hands the string back. output already redirects for the language server (`outmode`); this is the same thing turned into syntax, and `end()` is a method on a `capture` struct, which is how a plain word gets a method:
+`std.capture.start()` redirects program output into a string, and `std.capture.end(buf)` stops capture and returns the string. these are static functions in the `std.capture` group; output redirection is also used by the language server (`outmode`):
```
-let buf = capture.start();
-put_s("hello world!"); // nothing on the terminal
-buf.end();
-put_s(buf); // hello world!
+namespace;
+include rt, std;
+fn main() {
+ let buf = std.capture.start();
+ rt.put_s("hello world!"); // nothing on the terminal
+ std.capture.end(buf);
+ rt.put_s(buf); // hello world!
+}
```
## memory model: u, m and d
-`new(n)` or `new(T)` gives back a fresh object. bind it to a local and the local owns it, tagged **m**. tags live on local variables and parameters, the analyzer follows them through the code, and the compiler inserts `free` calls where owned objects die. a tag is written after the type, `*node.u p`, or alone, `.d x`. `u`, `m` and `d` are ordinary names; the tag is the dot.
+`rt.new(n)` or `rt.new(T)` gives back a fresh object. bind it to a local and the local owns it, tagged **m**. tags live on local variables and parameters, the analyzer follows them through the code, and the compiler inserts `rt.free` calls where owned objects die. a tag is written after the type, `*node.u p`, or alone, `.d x`. `u`, `m` and `d` are ordinary names; the tag is the dot.
-**m** means the object belongs to this block and has to be used in it. you can lend it to callees as often as you like, and when the block ends it's freed. an m object that's never used anywhere in its block is an error.
+**m** means the object belongs to this block and must be used in it. it may be passed to callees as a borrow; scope exit frees it. an m object that is never used in its block is an error.
-**d** means the object is going down. the first time it's handed to a lower scope its life here is over: passing it to a `.d` parameter, or touching it inside an inner block, which then owns it as m and frees it when that block ends. using it afterwards is an error, and so is ending the block with a d object that never left.
+**d** transfers the object to a lower scope. passing it to a `.d` parameter or accessing it in an inner block transfers ownership; the receiving scope owns it as m and frees it when that block ends. using it after transfer is an error, as is ending the current block before the transfer occurs.
-**u** means the object is going up. it has to leave this block for the one above, where it arrives as m. it leaves through `return`, as the value of a block expression, or by assignment to a variable of the directly enclosing block. the function's signature doesn't have to say `.u`: when its body returns a u object, the return is owned and the function is inferred to give one back. a call to a function whose body hasn't been read yet can't know that, so such a function has to be declared `.u` (like `*node.u make()` or just `.u make()`) before the call. letting it fall out of scope is an error, and so is throwing away a u result by just calling it. to discard one on purpose, recast the call with m: `m*make();` lets the unnamed object go.
+**u** transfers the object to the enclosing scope, where it becomes m. transfer occurs through `return`, a block expression value, or assignment to a variable in the directly enclosing block. returning an owned local transfers it automatically; a `.u` result declaration provides the contract to callers before the function body is available. an owned u object cannot fall out of scope or have its result discarded. bind the result to an owner and call `rt.free`, or pass it to a consuming parameter.
-recasting is `u*x`, `d*x`, `m*x`, with no space before the star. a space is ordinary multiplication by a variable of that name. it only goes away from m: m can become u or d (and u and d can swap), but nothing goes back to m in the same scope. a declaration can carry a tag too, `.u r;`, and then whatever lands in it gets that tag. `m*` also works on an unnamed u result, a call, which is how such a result is thrown away; `u*` and `d*` need a name.
+explicit recasts use `u*x`, `d*x`, or `m*x`, with no space before the star. a space is ordinary multiplication by a variable of that name. recasts remain available for low-level code, while checked returns, `.d` calls, frees, and owner assignments infer common transfers. a declaration can carry a tag too, `.u r;`, and then whatever lands in it gets that tag.
-parameters: an untagged parameter is m from the caller's side, a lend, and the callee sees a plain word. `.m` makes it a checked borrow: the callee can read and write it but can't free it, return it, recast it or store it anywhere. `.d` hands the object over, and it arrives as m.
+parameters: an untagged parameter is a lend, and the compiler infers how a checked body uses it. `.m` states a borrow contract and `.d` receives ownership. Passing an owned value to `.d` transfers it automatically, so `take(p)` can replace `take(d*p)`. A checked borrow can return an alias of its parameter, but it cannot return an alias of a local owner. A fresh result passed to a proved non-retaining function, as in `rt.put_s(message())`, is held in a compiler slot and released after the call. Unknown or retaining callees need an explicit contract or a named owner.
```
-int*.u box(int v) { int* b = new(int); *b = v; return u*b; }
-fn add(list* l, int*.d item) { l.items[l.len] = d*item; l.len++; return 0; }
+include rt;
+int*.u box(int v) { int* b = rt.new(int); *b = v; return b; }
+void add(list* l, int*.d item) { l.items[l.len] = item; l.len++; return; }
```
containers own what you store in them. `p[i] = c` or `p.f = c` needs `c` to carry the same tag as `p` (or be a fresh object), and from then on `c` follows `p`: if `p` is recast, so is `c`, if `p` is given away `c` goes with it, and when `p` is freed its children are freed too. a child can't be recast or given away on its own. storing into an object you only borrowed gives the child away: it's dead here afterwards, and the caller's object owns it. the runtime records the owner in the child's header, so freeing a container frees exactly the objects it owns.
@@ -206,7 +231,7 @@ a pointer computed from a tracked variable is tracked too. `p + k`, `p.field` an
### memory debug
-off unless a program asks. `memdebug on;` at the top level turns on header checks and allocation sites for the rest of the file, and the runtime enables them before `main`. the same line inside a function does it from there on. `memdebug scrub;` also fills a freed payload with 57005 (`0xDEAD`). `memdebug off;` stops it. `mem_set(n)` is the same switch as a call: bit 1 checks and sites, bit 2 poison, bit 4 interior pinning.
+off unless a program asks. `memdebug on;` at the top level turns on header checks and allocation sites for the rest of the file, and the runtime enables them before `main`. the same line inside a function does it from there on. `memdebug scrub;` also fills a freed payload with 57005 (`0xDEAD`). `memdebug off;` stops it. `rt.mem_set(n)` is the same switch as a call: bit 1 checks and sites, bit 2 poison, bit 4 interior pinning.
with checks on, every load and store through a heap address looks for the object header, walking back at most 64 words so `p + k` and a field still find it. a dead header stops the program:
@@ -216,7 +241,7 @@ zy: double free from line 12
zy: bad heap pointer
```
-the line is the `new` that made the block. a double free prints and returns; it does not smash the block. `memreport()` prints one line per recorded object that is still live, `leak: line 12 3 words`, and returns how many. the word count is the block's payload, which is a power-of-two chunk minus the 5 word header, not the size you asked for. the table holds 64 sites. freed blocks stay in it until the storage is reused, which is how a double free still knows the line. nothing here changes programs that never say `memdebug`.
+the line is the `new` that made the block. a double free prints and returns; it does not smash the block. `rt.memreport()` prints one line per recorded object that is still live, `leak: line 12 3 words`, and returns how many. the word count is the block's payload, which is a power-of-two chunk minus the 5 word header, not the size you asked for. the table holds 64 sites. freed blocks stay in it until the storage is reused, which is how a double free still knows the line. nothing here changes programs that never say `memdebug`.
bit 4 makes the collector treat a word that falls inside a quarantined payload as a pointer, up to 64 words back to the header. an exact pointer already did this. turn it on when you want `p + k` to keep the block. it can also pin a block that a stale integer happens to land in, so it stays off unless you set the bit.
@@ -231,20 +256,21 @@ it runs inside the compiler on every function, including code compiled at runtim
types are a compiler primitive. the compiler keeps them as records in a type area at 0x80000 (65536 words), hash-consed so the same type is always the same pointer. they stay in memory after compiling, so programs and parse words can look at them.
```
+include rt;
struct point { int x, y; };
struct list<T> { int len; *T items; };
type num = int;
T first<T>(list<T>* l) { return l.items[0]; }
-list<T>*.u mk<T>(int cap) { list<T>* l = new(list<T>); l.items = new(T, cap); return u*l; }
+list<T>*.u mk<T>(int cap) { list<T>* l = rt.new(list<T>); l.items = rt.new(T, cap); return l; }
...
list<point*>* ps = mk<point*>(8); // explicit type arguments, or inferred from the arguments
```
-the kinds are `word`, `int`, `char`, `bool`, `void`, pointers `T*`, arrays `[n]T`, `[]T` and `fixed []T`, function types `int(int, char*)`, structs, generic structs and generic functions. structs and arrays live in place (in a frame, a global or inside another struct), and their value is their address, so they're passed around by pointer. `sizeof` counts words. a generic function is compiled once for each set of type arguments it is called with, so `T` can be a number, a pointer or a struct of any size, `sizeof(T)` and `new(T)` mean the real type, and a struct can take several parameters (`struct record<K, V> { K key; V val; };`). indexing scales by the element's size, while plain pointer arithmetic stays in words, like b. `new(T)` and `new(T, n)` stamp the type into the object's header.
+the kinds are `word`, `int`, `char`, `bool`, `void`, pointers `T*`, arrays `T[n]`, `T[?]` and `fixed T[?]`, function types `int(int, char*)`, structs, generic structs and generic functions. structs and arrays live in place (in a frame, a global or inside another struct), and their value is their address, so they're passed around by pointer. `sizeof` counts words. a generic function is compiled once for each set of type arguments it is called with, so `T` can be a number, a pointer or a struct of any size, `sizeof(T)` and `rt.new(T)` mean the real type, and a struct can take several parameters (`struct record<K, V> { K key; V val; };`). indexing scales by the element's size, while plain pointer arithmetic stays in words, like b. `rt.new(T)` and `rt.new(T, n)` stamp the type into the object's header.
### pointer types
-a pointer type places its star after the type, like c: `int*`, `point*`, `vec<T>*`, `vec<point*>*`, `point**`. a tag follows the stars (`vec<T>*.u`, `int*.d x`). there is no prefix form: `*int` as a type is an error, and a star in front of an expression is only ever a dereference. a pointer to an array or to a function type is spelled with a `type` alias (`type row = [3]int;` then `row*`), because `[3]int*` reads as an array of pointers. `tprint`, and so error messages and reflection, print the same form.
+a pointer type places its star after the type, like c: `int*`, `point*`, `vec<T>*`, `vec<point*>*`, `point**`. a tag follows the stars (`vec<T>*.u`, `int*.d x`). there is no prefix form: `*int` as a type is an error, and a star in front of an expression is only ever a dereference. a pointer to an array or to a function type is spelled with a `type` alias (`type row = int[3];` then `row*`), because `int*[3]` reads as an array of pointers. `tprint`, and so error messages and reflection, print the same form.
### methods and closures
@@ -261,7 +287,9 @@ p.dist(q);
underneath, a method is an ordinary function called `point.dist` whose first parameter is `point* this`, compiled once the struct is complete, so a method can call the ones declared after it. a field with the same name wins over a method. a method of a generic struct is a generic function over the struct's parameters, compiled for each set of arguments it is called with (the receiver says which), so `this.items[i]` and `sizeof(K)` inside it are exact.
-`fn(a, b) { ... }` in an expression is a closure. the value is a heap block: word 0 is the body's entry, and the captured variables start at word 3. the vm only runs code in the code area, so the block is never executed. a call through a value checks whether the callee is at or above `HEAP_LO`. a plain function is called as it is. a closure is called with the block plus 3 as its environment and jumps to the entry in word 0. the capture is by value, taken when the closure is made. a variable from further out is captured through the closure around it. a closure's own copy is its state, so a counter inside it keeps its value between calls. the block is an owned object: bind it to a name before passing it to a parameter that isn't `.d`, and return it from a function whose result is `.u`. `tests/programs/closure.zc` and `examples/learn/25-closures.zc` cover these.
+`fn(a, b) { ... }` in an expression is a closure. the value is a heap block: word 0 is the body's entry, and its captured cell pointers start at word 3. the vm only runs code in the code area, so the block is never executed. a call through a value checks whether the callee is at or above `HEAP_LO`. a plain function is called as it is. a closure is called with the block plus 3 as its environment and jumps to the entry in word 0. captured locals and parameters share cells with the enclosing function and sibling closures, so mutations stay visible after the closure is made and after its creator returns. captured owned objects remain alive with those cells. the block is an owned object; a fresh closure can be passed directly to a proved borrower, and returning one transfers it to the caller. `examples/learn/25-closures.zc` and `examples/learn/26-closure-state.zc` cover these.
+
+`fn:type(args)` declares a closure's result and parameter types. an owned closure can be returned from a factory with `.u`; its captured cells remain alive for the returned value. [the typed closure example](examples/learn/30-typed-closure.zc) covers this form.
### hints
@@ -271,15 +299,27 @@ check mode (the language server) reports things that are legal but probably not
- a `let` with a constant initializer that is never assigned again: make it a `const`
- multiplying by a power of two: write a shift
- a number of 100 or more with no name (powers of two, masks and character literals are left alone)
-- `new` inside a loop: it allocates on every pass
-- a `new(n)` of 64 words or less that never leaves its function: a stack buffer, `let buf[n]`, avoids the heap
+- `rt.new` inside a loop: it allocates on every pass
+- a `rt.new(n)` of 64 words or less that never leaves its function: a stack buffer, `let buf[n]`, avoids the heap
- an owned object last used four or more lines above the end of its block
+by default, hints use blue informational squiggles (diagnostic severity 3). `hints on;` restores this presentation for following code, `hints small;` keeps the same hints but uses compact dots (severity 4), and `hints off;` suppresses hints for following code. each directive applies by source position to the code after it; `on` is the default.
+
+an inline block comment immediately before a numeric literal marks the literal as intentional and suppresses its magic-number hint:
+
+```zc
+return n + /* offset in memory */ 0x40ff;
+```
+
+the comment must precede the literal itself. a comment before another operand does not suppress the hint for the literal.
+
+ordinary comments immediately above a declaration provide its hover documentation; `//:` marks an implementation note instead. [the learning example](examples/learn/29-block-comments.zc) combines declaration documentation with the inline magic-number escape.
+
`examples/opt_test.zc` illustrates these cases and still runs.
### reflection
-`typeof(e)` and `typeof(T)` give a type record and `sizeof(...)` its size. both work on any expression, which is compiled dry and never runs. at runtime: `tkind(t)` (one of the `TY_` constants), `tsize`, `tname`, `tlen`, `tnfields`, `tfname(t, i)`, `tftype`, `tfoff`, `tret`, `tnparams`, `tparam`, `tparamtag` (the u/m/d tag of a parameter), `tretu`, `tnargs` and `targ` for generic instances, and `tprint(t)`. `sigof(f)` gives the type of a function by address (untyped ones get a signature built from their arity and tags), `sigofs("name")` by name, and `typeof_obj(p)` reads the type stamped on a heap object (0 when the header has no type yet: `new(n)` or `new_raw`).
+`typeof(e)` and `typeof(T)` give a type record and `sizeof(...)` its size. both work on any expression, which is compiled dry and never runs. at runtime: `tkind(t)` (one of the `TY_` constants), `tsize`, `tname`, `tlen`, `tnfields`, `tfname(t, i)`, `tftype`, `tfoff`, `tret`, `tnparams`, `tparam`, `tparamtag` (the u/m/d tag of a parameter), `tretu`, `tnargs` and `targ` for generic instances, and `tprint(t)`. `sigof(f)` gives the type of a function by address (untyped ones get a signature built from their arity and tags), `sigofs("name")` by name, and `rt.typeof_obj(p)` reads the type stamped on a heap object (0 when the header has no type yet: `rt.new(n)` or `rt.new_raw`).
### the typechecker
@@ -287,16 +327,17 @@ off by default. `typecheck on;` turns it on for what follows and `typecheck off;
### sized integers and f32
-`i8 u8 i16 u16 i32 u32` (and `byte`, which is `u8`) are integer types that keep their size. a variable, parameter, field or return value of one is still a word, but it is cut to size every time it is stored: unsigned values are masked, signed ones are masked and then sign extended, so `u8 a = 250; a = a + 10;` leaves 4, and `i8 s = 100; s = s + 100;` leaves -56. arrays and pointers of them are packed, which is where the memory goes: a `[100]u8` takes 13 words and `new(u16, n)` takes n/4 words, rounded up. indexing a packed array or pointer reads and writes the bytes with the byte ops (612 to 617), so the same code runs on every engine and o3 lifts it like any other load. `f32` is single precision: a variable holds the value already rounded to a float (arithmetic is done in double, which gives the correctly rounded float result for `+ - * /`), and an array or pointer of f32 is packed two to a word. `f64` is a second name for `float`. the rounding is the `f2s` and `s2f` micro-ops, one instruction on arm64.
+`i8 u8 i16 u16 i32 u32` (and `byte`, which is `u8`) are integer types that keep their size. a variable, parameter, field or return value of one is still a word, but it is cut to size every time it is stored: unsigned values are masked, signed ones are masked and then sign extended, so `u8 a = 250; a = a + 10;` leaves 4, and `i8 s = 100; s = s + 100;` leaves -56. arrays and pointers of them are packed, which is where the memory goes: a `u8[100]` takes 13 words and `rt.new(u16, n)` takes n/4 words, rounded up. indexing a packed array or pointer reads and writes the bytes with the byte ops (612 to 617), so the same code runs on every engine and o3 lifts it like any other load. `f32` is single precision: a variable holds the value already rounded to a float (arithmetic is done in double, which gives the correctly rounded float result for `+ - * /`), and an array or pointer of f32 is packed two to a word. `f64` is a second name for `float`. the rounding is the `f2s` and `s2f` micro-ops, one instruction on arm64.
```
+include rt;
u8 gbuf[100];
struct pix { u8 r, g, b, a; i16 dx; };
-f32* p = new(f32, 1000); // 500 words
+f32* p = rt.new(f32, 1000); // 500 words
p[3] = 0.1; // stored as the nearest float
```
-two limits. `sizeof` counts words, so `sizeof(gbuf)` is 13 and a struct of four u8s has `sizeof` 1: the fields of a non-generic struct are packed into bytes, and a field read or write goes through the same byte ops as an array element. growable `[]u8`, `[]i16`, `[]f32` and the rest work too (the handle's element size is stored as minus the byte size, and `push`, `pop`, `resize`, indexing and `for` use byte accesses for them), in o3 as well. the address of one element of a packed array cannot be taken, because it is a byte inside a word, and that is on purpose: pointer arithmetic moves in words, so such an address would not mean the same thing as `&a[i]` does elsewhere. take the array's own address and index it. `f32` is a type for `extern` and `cfunc` signatures: it goes to c as a 32-bit float and comes back rounded. a zc struct whose fields are all one word (`int`, `float`, pointers) can be passed by value to an `extern`: `extern long f(struct point p);`. it goes to c as its words, classed per eightbyte by the sysv x86-64 rules (a float word in an sse register, any other in an integer register, the whole struct on the stack when it does not fit, and a struct over 16 bytes always on the stack). arm64 (aapcs64) follows the same idea: a homogeneous float struct of up to four words goes in the v registers, a struct of two words or less in the x registers, and a bigger one is passed by reference to a copy. this is checked on x86-64 and on arm64 under qemu-user, on the interpreter, in o3 code, and on the llvm backend and jit (`tests/programs/structval.zc`). a struct can also be returned: `extern struct point mk(long x);` gives a new `point*` owned by the caller (the stub makes it with `new` and the native call fills it in). the return is at most 7 words, and its floats are classed like the arguments'. returns are checked on x86-64 and on arm64 under qemu-user, on the interpreter, and on llvm (`tests/programs/structret.zc`). apple arm64 is not checked.
+two limits. `sizeof` counts words, so `sizeof(gbuf)` is 13 and a struct of four u8s has `sizeof` 1: the fields of a non-generic struct are packed into bytes, and a field read or write goes through the same byte ops as an array element. growable `u8[?]`, `i16[?]`, `f32[?]` and the rest work too (the handle's element size is stored as minus the byte size, and `push`, `pop`, `resize`, indexing and `for` use byte accesses for them), in o3 as well. the address of one element of a packed array cannot be taken, because it is a byte inside a word, and that is on purpose: pointer arithmetic moves in words, so such an address would not mean the same thing as `&a[i]` does elsewhere. take the array's own address and index it. `f32` is a type for `extern` and `cfunc` signatures: it goes to c as a 32-bit float and comes back rounded. a zc struct whose fields are all one word (`int`, `float`, pointers) can be passed by value to an `extern`: `extern long f(struct point p);`. it goes to c as its words, classed per eightbyte by the sysv x86-64 rules (a float word in an sse register, any other in an integer register, the whole struct on the stack when it does not fit, and a struct over 16 bytes always on the stack). arm64 (aapcs64) follows the same idea: a homogeneous float struct of up to four words goes in the v registers, a struct of two words or less in the x registers, and a bigger one is passed by reference to a copy. this is checked on x86-64 and on arm64 under qemu-user, on the interpreter, in o3 code, and on the llvm backend and jit (`tests/programs/structval.zc`). a struct can also be returned: `extern struct point mk(long x);` gives a new `point*` owned by the caller (the stub makes it with `new` and the native call fills it in). the return is at most 7 words, and its floats are classed like the arguments'. returns are checked on x86-64 and on arm64 under qemu-user, on the interpreter, and on llvm (`tests/programs/structret.zc`). apple arm64 is not checked.
## virtual enhancements
@@ -311,14 +352,14 @@ enhance noisy(x) vm { 0 $ . } else soft_noisy; // can't be lifted (it does i/
a micro recipe works on registers `r0`..`r15`. inputs arrive in those registers, also under their parameter names, and outputs leave from `r0` upward. the micro-ops are `li mov add sub mul sdiv udiv srem urem and or xor shl shr sar neg not lt ltu eq ne min max minu maxu mulhs mulhu clz abs sel ld st`, followed by the bit operations `ctz popcnt bswap rotl rotr` and the single-precision conversions `f2s s2f` (rounding a double's bits to 32-bit float and back exactly). float operations and byte loads and stores are available to enhancements on the full cores. `src/runtime/rt.zc` defines callable enhancements for `clz ctz popcnt bswap rotl rotr mulhi umulhi f2s s2f`. they compile to one instruction on arm64 (popcnt uses the vector unit), an llvm intrinsic on the llvm backend, or plain c in the interpreter. the optimizer folds them when their inputs are constant. a `vm` body is straight-line vm code lifted by symbolic execution. both recipe and vm enhancements become ssa; the core optimizer folds constants, simplifies algebra, removes common subexpressions and dead code, and lowers constant division to multiply-high sequences (`ZYTRACE=1` prints the result). the arm64 jit allocates registers and inlines each use; llvm inlines the ir; the interpreter runs the optimized list. if an operation does not fit in registers, the jit calls the interpreter for that operation. when literals feed the last inputs of an enhancement (`x / 7` is `x 7 idiv`), the arm64 jit specializes that use: it substitutes the literals, optimizes again, and turns `x / 7` into multiply-high and shifts instead of division.
-an enhancement is called like a function when it has one output, or by name from asm. `enhance` installs it at compile time, so the wiring is available before `main` runs and parse words can use it immediately. an image installs its enhancements from its symbol table when it boots. that's how `/`, `%`, and `<<` work: `src/runtime/rt.zc` declares `idiv`, `imod`, and `ishl` as one-instruction recipes. `install(desc)` is the raw interface; its descriptor format is documented in `src/core/core.c`. stores performed by an enhancement do not invalidate compiled code, so use enhancements for data operations.
+an enhancement is called like a function when it has one output, or by name from asm. `enhance` installs it at compile time, so the wiring is available before `main` runs and parse words can use it immediately. an image installs its enhancements from its symbol table when it boots. that's how `/`, `%`, and `<<` work: `src/runtime/rt.zc` declares `idiv`, `imod`, and `ishl` as one-instruction recipes. `rt.install(desc)` is the raw interface; its descriptor format is documented in `src/core/core.c`. stores performed by an enhancement do not invalidate compiled code, so use enhancements for data operations.
## o3
`o3` marks code for the optimizing tier. it goes on a function, a block or a single variable:
```
-o3 int sum(fixed []int a) { int s = 0; for (int i = 0; i < len(a); i++) s += a[i]; return s; }
+o3 int sum(fixed int[?] a) { int s = 0; for (int i = 0; i < len(a); i++) s += a[i]; return s; }
fn main() {
o3 int hits = 0; // one o3 variable
o3 { for (int i = 0; i < 1000; i++) if (i % 7 == 3) hits++; }
@@ -326,9 +367,9 @@ fn main() {
}
```
-what it gives up, as asked: variables declared in o3 code must be typed (`let` and untyped parameters are errors), they have no address (`&x` and naming one from `asm` are errors), an untyped `new(n)` in o3 code is `new_oz` (no type stamp, and the words asked for are zero), and an o3 function skips the frame stack check in its prologue. typed `new(T)` and `new(T, n)` stamp the header and zero the payload, in o3 the same as outside it. what it gets: the compiler hands each o3 function body and each o3 block to the core as a region (install kind 2: the range of vm words plus its typed scalar variables' frame slots, each with a register), and the core compiles the region right away. the cpu has 7 registers for variables. a variable takes the one most recently freed by a scope that closed, or the next unused one, so loop counters in sibling loops share a register, and a variable declared when all 7 are busy stays in memory. a register shared by several slots gets written back to all of them when the region exits, and they never hold different live values, since their scopes don't overlap.
+what it gives up, as asked: variables declared in o3 code must be typed (`let` and untyped parameters are errors), they have no address (`&x` and naming one from `asm` are errors), an untyped `rt.new(n)` in o3 code is `new_oz` (no type stamp, and the words asked for are zero), and an o3 function skips the frame stack check in its prologue. typed `rt.new(T)` and `rt.new(T, n)` stamp the header and zero the payload, in o3 the same as outside it. what it gets: the compiler hands each o3 function body and each o3 block to the core as a region (install kind 2: the range of vm words plus its typed scalar variables' frame slots, each with a register), and the core compiles the region right away. the cpu has 7 registers for variables. a variable takes the one most recently freed by a scope that closed, or the next unused one, so loop counters in sibling loops share a register, and a variable declared when all 7 are busy stays in memory. a register shared by several slots gets written back to all of them when the region exits, and they never hold different live values, since their scopes don't overlap.
-inside the core, a region is cut into basic blocks at jump targets, branch fallthroughs and call return points. an empty jump is then folded into its target, a block with one predecessor is pasted onto that predecessor, and a straight loop is rotated so one iteration is a single block that branches to itself. a call, a return, a value left on the stack, or a second branch stays where it is, and an entry is never pasted over. each block is lifted to ssa the way enhancements are: the vm stack becomes values, and only what's still on it at the end of the block is written to memory. the frame pointer and the promoted slots become variables that live in registers across the whole region, so `i++` on an o3 int is one add. enhancements are spliced into the block, so `/`, `%` and `<<` become one instruction or a magic-number sequence. a call to `ld8`, `st8`, `ld16`, `st16`, `ld32` or `st32` from o3 code emits opcodes 612..617, and the region lifts those instead of calling out. every block then goes through the same optimizer (folding, algebra, cse, dce, division by constants). in o3 code `a[i]` on a `[]T` is compiled inline with its bounds check (`outside(i, len)`, one enhancement), and `len(a)` is a load. a `fixed []T` in an o3 variable does better: its items pointer and length can't change, so the compiler gives it two hidden o3 variables holding them. they're loaded when the region starts and whenever the array variable is assigned, so indexing is a compare against a register and an indexed load, with no loads through the handle.
+inside the core, a region is cut into basic blocks at jump targets, branch fallthroughs and call return points. an empty jump is then folded into its target, a block with one predecessor is pasted onto that predecessor, and a straight loop is rotated so one iteration is a single block that branches to itself. a call, a return, a value left on the stack, or a second branch stays where it is, and an entry is never pasted over. each block is lifted to ssa the way enhancements are: the vm stack becomes values, and only what's still on it at the end of the block is written to memory. the frame pointer and the promoted slots become variables that live in registers across the whole region, so `i++` on an o3 int is one add. enhancements are spliced into the block, so `/`, `%` and `<<` become one instruction or a magic-number sequence. a call to `ld8`, `st8`, `ld16`, `st16`, `ld32` or `st32` from o3 code emits opcodes 612..617, and the region lifts those instead of calling out. every block then goes through the same optimizer (folding, algebra, cse, dce, division by constants). in o3 code `a[i]` on a `T[?]` is compiled inline with its bounds check (`outside(i, len)`, one enhancement), and `len(a)` is a load. a `fixed T[?]` in an o3 variable does better: its items pointer and length can't change, so the compiler gives it two hidden o3 variables holding them. they're loaded when the region starts and whenever the array variable is assigned, so indexing is a compare against a register and an indexed load, with no loads through the handle.
the backends: on arm64 the region becomes one piece of native code. variables sit in w0..w7, block values in w8..w14 and w26..w28, constants are folded into instructions where they fit, loads from a fixed address are one `ldr`, and blocks branch to each other directly with no dispatch and no stack checks. on the llvm build the region becomes one function, with the variables in allocas that mem2reg turns into ssa across blocks, and it goes through llvm's O3 pipeline. on the plain interpreter, a small region interpreter runs the optimized blocks with the variables in locals.
@@ -356,9 +397,9 @@ fib gains nothing and can lose a little, since every call leaves the region thro
### leaving o3: fattening
-going into o3 is free: o3 code reads normal objects the same way and ignores their metadata. typed `new(T)` stamps the type into the header and zeroes the payload, so a node is born with the type it will keep. a child store in o3 code writes `p[-2] = owner` in the region when memdebug is off (a shared child keeps the owner it has, and a null store does not land) and does not call `own`. it also skips `fatten` when that value was just allocated, or returned from an o3 function whose type has a pointer. `new(n)` with no type inside o3 calls `new_oz`: the type word stays -(n + 1), and the words asked for are zero. `new_raw` still leaves those words uninitialized and zeroes only the slack past them.
+going into o3 is free: o3 code reads normal objects the same way and ignores their metadata. typed `rt.new(T)` stamps the type into the header and zeroes the payload, so a node is born with the type it will keep. a child store in o3 code writes `p[-2] = owner` in the region when memdebug is off (a shared child keeps the owner it has, and a null store does not land) and does not call `own`. it also skips `fatten` when that value was just allocated, or returned from an o3 function whose type has a pointer. `rt.new(n)` with no type inside o3 calls `rt.new_oz`: the type word stays -(n + 1), and the words asked for are zero. `rt.new_raw` still leaves those words uninitialized and zeroes only the slack past them.
-`fatten` runs where o3 data reaches code that isn't o3, using the static type known there: storing a pointer into a normal local or a global, passing one to a function that isn't o3 (`free` and `recycle` excepted), the call site where ordinary code calls an o3 function whose result has pointers, `push` into a growable array, and, at the end of an o3 block, whatever the o3 variables declared outside it hold. a call or a return from one o3 function to another does not fatten, does not call `own`, and does not run the callee's prologue: the caller enters the callee's region directly. `fatten(p, t)` (in the compiler, so it can read type records) stamps the type, as `[0]T` when more than one element was asked for, zeroes the slack, and then follows every typed pointer, field, array element and growable array element inside. an object whose header already holds a type stops the walk, and a depth limit of 32 stops cycles through growable arrays. untyped words carry no hint, so pointers kept in `word` variables stay raw until typed code sees them.
+`fatten` runs where o3 data reaches code that isn't o3, using the static type known there: storing a pointer into a normal local or a global, passing one to a function that isn't o3 (`rt.free` and `rt.recycle` excepted), the call site where ordinary code calls an o3 function whose result has pointers, `push` into a growable array, and, at the end of an o3 block, whatever the o3 variables declared outside it hold. a call or a return from one o3 function to another does not fatten, does not call `own`, and does not run the callee's prologue: the caller enters the callee's region directly. `run.fatten(p, t)` (in the compiler, so it can read type records) stamps the type, as `T[0]` when more than one element was asked for, zeroes the slack, and then follows every typed pointer, field, array element and growable array element inside. an object whose header already holds a type stops the walk, and a depth limit of 32 stops cycles through growable arrays. untyped words carry no hint, so pointers kept in `word` variables stay raw until typed code sees them.
## the llvm backend
@@ -366,15 +407,16 @@ a tiered jit for hosts the arm64 backend doesn't cover. the interpreter counts h
## the heap
-**constant-size allocations.** when a `new` has a size the compiler knows (`new(T)` of a type, or `new(6)`), it works out the size class itself and calls `new_k(n, t, k)`, which pops a block straight off the class's free list and skips the class search. a typed `new` of a struct made only of numbers (no pointer fields, no `word`, no generics) is also freed through `recycle_leaf(p, t, k)` when its owner goes out of scope: it checks the type word and the class still match what the compiler assumed (the variable may have been reassigned), and if so pushes the block back on the head of its list with no walk over children. a loop that makes such an object each pass and frees it at the end of the block therefore gets the same block back every time: the pair costs a dozen steps, where the general path searched classes, split blocks and walked the object. memdebug, an empty list, or a block inside the cut-off type area use the general path, so the checks and the messages stay what they were. in the interpreter this cut a 200000-pass loop of `point* p = new(point)` from about 1.9 s to 0.8 s.
+**constant-size allocations.** when an `rt.new` has a size the compiler knows (`rt.new(T)` of a type, or `rt.new(6)`), it works out the size class itself and calls `rt.new_k(n, t, k)`, which pops a block straight off the class's free list and skips the class search. a typed `rt.new` of a struct made only of numbers (no pointer fields, no `word`, no generics) is also freed through `recycle_leaf(p, t, k)` when its owner goes out of scope: it checks the type word and the class still match what the compiler assumed (the variable may have been reassigned), and if so pushes the block back on the head of its list with no walk over children. a loop that makes such an object each pass and frees it at the end of the block therefore gets the same block back every time: the pair costs a dozen steps, where the general path searched classes, split blocks and walked the object. memdebug, an empty list, or a block inside the cut-off type area use the general path, so the checks and the messages stay what they were. in the interpreter this cut a 200000-pass loop of `point* p = rt.new(point)` from about 1.9 s to 0.8 s.
-the heap is a buddy allocator over [HEAP_LO, top), where top starts at HEAP_HI and moves up. when no block is big enough, `new_raw` asks `heap_grow` to append a piece (a few times the failed request, at least 0x40000 words, capped by MW). the new range enters the same free lists, so it behaves like part of the heap from startup. growth stops when `heap_reset` cuts the allocator off below the type area; the language server also runs with this cut in place. with a host that answers `syscall(9, 0)`, the heap can grow until the host denies more memory. `tests/programs/growheap.zc` allocates well past the initial heap, frees it, then takes a block larger than any part of the original heap. when available, host operations 18, 19, and 20 fill, copy, and scan memory in one call, avoiding long zc loops. blocks are aligned powers of two, and a freed block merges with its free buddy so space can be reused at any size. an earlier bug let a child merge into a container while the container was still being freed; growable arrays exposed it because their handles and item blocks are often buddies. allocation finds the smallest free block that fits and splits it. each size class has a doubly linked free list, so removing a buddy during a merge takes constant time. checked code frees deterministically. `free` quarantines a block: it marks the header free but leaves the bytes intact, so a stale alias reads a dead block and a second free through it does nothing. the next collection returns a quarantined block to the free lists only when no word points to it. `free(d*x)` and compiler-generated cleanup for an owned local call `recycle` instead, returning the block to its size-class list after freeing its children. `arena_new(n)` creates an owned block for bump allocation; `arena_alloc` returns interior words and `arena_reset` rewinds the bump. the mark recognizes exact pointers by their check word. with memdebug bit 4, it also accepts interior pointers up to 64 words into the payload. `memdebug scrub` fills the payload with 57005 after releasing its children. the mark-sweep collector is conservative and non-moving; it handles globals, unchecked code, and values passed to it. `gc()` returns the number of objects freed, and collection also runs when an allocation does not fit. the sweep frees through the same merging path. a shared object (header flag bit 8) is not quarantined: `free` leaves it in place, and collection reclaims it when unmarked. the low three flag bits are the live, mark, and pin state; the share bit stays set across a mark.
+the heap is a buddy allocator over [HEAP_LO, top), where top starts at HEAP_HI and moves up. when no block is big enough, `rt.new_raw` asks `heap_grow` to append a piece (a few times the failed request, at least 0x40000 words, capped by MW). the new range enters the same free lists, so it behaves like part of the heap from startup. growth stops when `heap_reset` cuts the allocator off below the type area; the language server also runs with this cut in place. with a host that answers `core.syscall(9, 0)`, the heap can grow until the host denies more memory. `tests/programs/growheap.zc` allocates well past the initial heap, frees it, then takes a block larger than any part of the original heap. when available, host operations 18, 19, and 20 fill, copy, and scan memory in one call, avoiding long zc loops. blocks are aligned powers of two, and a freed block merges with its free buddy so space can be reused at any size. an earlier bug let a child merge into a container while the container was still being freed; growable arrays exposed it because their handles and item blocks are often buddies. allocation finds the smallest free block that fits and splits it. each size class has a doubly linked free list, so removing a buddy during a merge takes constant time. checked code frees deterministically. `rt.free` quarantines a block: it marks the header free but leaves the bytes intact, so a stale alias reads a dead block and a second free through it does nothing. the next collection returns a quarantined block to the free lists only when no word points to it. `rt.free(x)` and compiler-generated cleanup for an owned local call `rt.recycle` instead, returning the block to its size-class list after freeing its children. `rt.arena_new(n)` creates an owned block for bump allocation; `rt.arena_alloc` returns interior words and `rt.arena_reset` rewinds the bump. the mark recognizes exact pointers by their check word. with memdebug bit 4, it also accepts interior pointers up to 64 words into the payload. `memdebug scrub` fills the payload with 57005 after releasing its children. the mark-sweep collector is conservative and non-moving; it handles globals, unchecked code, and values passed to it. `rt.gc()` returns the number of objects freed, and collection also runs when an allocation does not fit. the sweep frees through the same merging path. a shared object (header flag bit 8) is not quarantined: `rt.free` leaves it in place, and collection reclaims it when unmarked. the low three flag bits are the live, mark, and pin state; the share bit stays set across a mark.
## c abi
zc can call native c functions and hand zc functions to native code. it is built on the syscall gate (ops 14 to 17), so it needs a host with `dlsym`: the interpreter, the llvm backend and the arm64 jit on linux and macos (the jit has to be a dynamic binary, since a static one has no `dlsym`). the microcore, the javascript vm, the web build and the windows scripts answer -1, and a native call there gives -1 and nothing else.
```
+include rt;
extern double pow(double x, double y);
extern long c_strlen(str s) = "len_w"; // zc name = "c name"
extern void qsort(word *base, long n, long size, void *cmp);
@@ -385,20 +427,20 @@ o3 int cmp3(int *a, int *b) { return *a - *b; }
fn main() {
let v[4];
v[0] = 3; v[1] = 1; v[2] = 2; v[3] = 0;
- qsort(v, 4, 8, cfunc(cmp3)); // native code calls back into o3
- put_n(c_strlen(b"hello"));
+ qsort(v, 4, 8, rt.cfunc(cmp3)); // native code calls back into o3
+ rt.put_n(c_strlen(b"hello"));
}
```
-`extern` reads a c prototype and defines an ordinary zc function with that name, so a call is a normal call. types are spelled the c way. the integer types (`long`, `size_t`, `int64_t` and the other 64-bit names) pass a whole word; `int`, `short`, `char` and their unsigned forms pass and come back as that width, sign or zero extended; `double` and `float` are a zc float (a `float` goes to the c side as a 32-bit float); any `T *` is a byte address, the same number `addr_b` gives, except `word *`, which is a word address (a zc object or array, what `fixed []int` or `new` hands you), and `str`, which is a zc byte string (`b"..."`) passed as the address of its bytes. a result that points inside vm memory comes back as a byte address or word address by the same rule, and any other pointer (a `malloc` result, a `FILE *`) is an opaque number you can hand back to c. up to 15 arguments in all. the first six integers and eight floats go in registers and the rest on the c stack, in order, so a function like `snprintf` with many arguments works; more than 15 is a compile error. from o3 code the call is a call instruction in the generated code: the arm64 jit emits it, and the llvm backend emits a direct call to the symbol's address, so llvm can see through it, and `sqrt`, `fabs`, `floor`, `ceil`, `trunc`, `round`, `sin`, `cos`, `exp`, `exp2`, `log`, `log2`, `log10`, `pow`, `copysign` and `fma` on doubles become llvm's own intrinsics and fold or vectorize like any other arithmetic. the darwin host adapts the known fixed prefixes of `printf`, `fprintf`, `sprintf`, and `snprintf`; other darwin variadic exports still need an explicit ABI adapter. a variadic function is declared once per shape you call it with (`extern int snprintf2(word *buf, long cap, str fmt, long a, double b) = "snprintf";`), and the call sets up the registers a variadic call needs. a missing symbol prints `zy: no native symbol name` and the call gives -1. `cload("libfoo.so")` (a byte string) opens a library so its symbols resolve, `cfind(name)` asks whether one does, and the math library is opened on demand.
+`extern` reads a c prototype and defines an ordinary zc function with that name, so a call is a normal call. types are spelled the c way. the integer types (`long`, `size_t`, `int64_t` and the other 64-bit names) pass a whole word; `int`, `short`, `char` and their unsigned forms pass and come back as that width, sign or zero extended; `double` and `float` are a zc float (a `float` goes to the c side as a 32-bit float); any `T *` is a byte address, the same number `addr_b` gives, except `word *`, which is a word address (a zc object or array, what `fixed int[?]` or `new` hands you), and `str`, which is a zc byte string (`b"..."`) passed as the address of its bytes. a result that points inside vm memory comes back as a byte address or word address by the same rule, and any other pointer (a `malloc` result, a `FILE *`) is an opaque number you can hand back to c. up to 15 arguments in all. the first six integers and eight floats go in registers and the rest on the c stack, in order, so a function like `snprintf` with many arguments works; more than 15 is a compile error. from o3 code the call is a call instruction in the generated code: the arm64 jit emits it, and the llvm backend emits a direct call to the symbol's address, so llvm can see through it, and `sqrt`, `fabs`, `floor`, `ceil`, `trunc`, `round`, `sin`, `cos`, `exp`, `exp2`, `log`, `log2`, `log10`, `pow`, `copysign` and `fma` on doubles become llvm's own intrinsics and fold or vectorize like any other arithmetic. the darwin host adapts the known fixed prefixes of `printf`, `fprintf`, `sprintf`, and `snprintf`; other darwin variadic exports still need an explicit ABI adapter. a variadic function is declared once per shape you call it with (`extern int snprintf2(word *buf, long cap, str fmt, long a, double b) = "snprintf";`), and the call sets up the registers a variadic call needs. a missing symbol prints `zy: no native symbol name` and the call gives -1. `rt.cload("libfoo.so")` (a byte string) opens a library so its symbols resolve, `rt.cfind(name)` asks whether one does, and the math library is opened on demand.
-`cfunc(f)` gives a native function pointer that runs the zc function `f`. the c signature comes from `f`'s declared types: `float` is a double, a pointer (`int *`, `[]T`) is a word address (so `int cmp(int *a, int *b)` reads the elements c points at), and everything else is a long. an `o3` function is entered at its compiled region, so a comparator or a worker in o3 runs as native code both ways. the pointer is only good while a native call from zc is running (it runs vm code on top of the stacks as they were at that call), so it belongs in an argument of the call that uses it or in a callback registration that is done again each time. there are 16 distinct callbacks per result class (long, double, float); the same function and signature reuse their slot. a callback that makes native calls of its own works, to any depth the stacks allow.
+`rt.cfunc(f)` gives a native function pointer that runs the zc function `f`. the c signature comes from `f`'s declared types: `float` is a double, a pointer (`int *`, `T[?]`) is a word address (so `int cmp(int *a, int *b)` reads the elements c points at), and everything else is a long. an `o3` function is entered at its compiled region, so a comparator or a worker in o3 runs as native code both ways. the pointer is only good while a native call from zc is running (it runs vm code on top of the stacks as they were at that call), so it belongs in an argument of the call that uses it or in a callback registration that is done again each time. there are 16 distinct callbacks per result class (long, double, float); the same function and signature reuse their slot. a callback that makes native calls of its own works, to any depth the stacks allow.
-how it works: the compiler writes the symbol's name, a signature word (4 bits per kind: the result first, then each argument) and room for the arguments as data in the code stream, and the stub it compiles stores its arguments there and does `syscall(14, block)`. the host looks the name up with `dlsym` (and remembers the answer in the block), passes the integer arguments in the first six integer registers and the floats in the first eight fp registers by calling through a prototype of six longs and eight doubles, and converts the result. `cfunc` asks gate op 16 for a trampoline: one of 16 prebuilt c functions per result class, which pushes its arguments on the vm data stack, pushes a return address that stops the nested run, runs the vm function with the same `vm_until` an o3 call uses, and pops the result. the stack pointers it needs are published by the gate store (a global in the interpreter, an alloca store in llvm blocks, and the same two words the jit saves on exit). a call from o3 code to an extern leaves the region like any call to ordinary code, and the stub's store to the gate goes through the host, so a native call costs a gate round trip. lowering it to a direct call inside a region (so llvm sees it) is not done.
+how it works: the compiler writes the symbol's name, a signature word (4 bits per kind: the result first, then each argument) and room for the arguments as data in the code stream, and the stub it compiles stores its arguments there and does `core.syscall(14, block)`. the host looks the name up with `dlsym` (and remembers the answer in the block), passes the integer arguments in the first six integer registers and the floats in the first eight fp registers by calling through a prototype of six longs and eight doubles, and converts the result. `rt.cfunc` asks gate op 16 for a trampoline: one of 16 prebuilt c functions per result class, which pushes its arguments on the vm data stack, pushes a return address that stops the nested run, runs the vm function with the same `vm_until` an o3 call uses, and pops the result. the stack pointers it needs are published by the gate store (a global in the interpreter, an alloca store in llvm blocks, and the same two words the jit saves on exit). a call from o3 code to an extern leaves the region like any call to ordinary code, and the stub's store to the gate goes through the host, so a native call costs a gate round trip. lowering it to a direct call inside a region (so llvm sees it) is not done.
## syscalls
-the host is reached by storing an op number at word `0x84002`. the three words after it are the arguments, and the store leaves the result in place. `syscall(op, a, b, c)` does that. ops 1 to 5 still pass word strings, one character per word. ops 10 to 13 pass the byte address of a packed string's payload (`addr_b(s + 1)`), which is NUL-terminated.
+the host is reached by storing an op number at word `0x84002`. the three words after it are the arguments, and the store leaves the result in place. `core.syscall(op, a, b, c)` gates core vm requests; `rt.syscall(op, a, b, c)` is the runtime wrapper used by runtime helpers. ops 1 to 5 still pass word strings, one character per word. ops 10 to 13 pass the byte address of a packed string's payload (`rt.addr_b(s + 1)`), which is NUL-terminated.
```
SYS_OPEN 1 open(path, 0 read, or 1 create/truncate for write) -> fd or -1
@@ -411,7 +453,8 @@ SYS_BOPEN 10 open a packed path
SYS_BREAD 11 read into packed bytes
SYS_BWRITE 12 write packed bytes
SYS_BGETENV 13 getenv into packed bytes
-SYS_RAW 40 any linux syscall: the block at `rawblk` holds the number and six arguments; the result is the kernel's, or -errno. `Host.raw(nr, a, b, c, d, e, f)` in src/lib/std.zc builds the block. linux only, -1 elsewhere
+SYS_RAW 40 any linux syscall: the block at `rawblk` holds the number and six arguments; the result is the kernel's, or -errno. `std.Host.raw(nr, a, b, c, d, e, f)` in src/lib/std.zc builds the block. linux only, -1 elsewhere
+SYS_POLL 42 1 when input can be read without waiting, else 0 (-1 where the host cannot tell). a streaming host (the wasm and javascript servers) suspends the run once at the call so its event loop can deliver input that is waiting, then answers; the language server uses it to drop a stale check
SYS_MEMINFO 9 memory facts: 0 total words, 1 heap start, 2 heap end, 3 total bytes
SYS_FILL 18 fill n words at an address with a value
SYS_COPY 19 copy n words (overlap safe)
@@ -424,33 +467,33 @@ SYS_ENV 24 envp(index, buf, cap) -> length, or -1 past the end
ops 21 to 24 are the command line and the environment. the string ops take an index and copy one entry across one character per word, like `SYS_GETENV`: a zero buffer asks for the length alone, so the caller sizes the string before it fetches it. `argv[0]` is the program path, and `<eval>` when the source was read from stdin, so `argc` is 1 even with nothing on the command line. the javascript and wasm vms are given `args` and `env` when they are constructed, and a bare-metal port has neither, so it answers `<eval>` on its own.
-`readfile(path)` returns a fresh string, or 0 if the file can't be opened. `include` uses it. ops 14 to 17 are the c abi. a host that doesn't know an op answers -1, and the zc side then does the work itself with a loop, so the microcore and the javascript vm run the same programs. these are not available from o3 code, whose stores don't go through the gate.
+`rt.readfile(path)` returns a fresh string, or 0 if the file can't be opened. `include` uses it. ops 14 to 17 are the c abi. a host that doesn't know an op answers -1, and the zc side then does the work itself with a loop, so the microcore and the javascript vm run the same programs. these are not available from o3 code, whose stores don't go through the gate.
## code as data
-the compiler is in memory with your program and every name in it is callable.
+the compiler resides in vm memory with the program; its functions and globals can be called by compiled code through the `run` namespace, imported with `include run;`.
`asm { }` emits raw vm words: numbers push, `-5` pushes a negative, `#n` emits a raw word, one-char tokens from the mnemonic table are opcodes, `:name` defines a label local to the block, enhancement names emit their opcode, and other names push the address of a variable or function.
-`quote { ... }` is a token list as a fresh object: `[n, kind, value, line, kind, value, line, ...]`, so errors inside a macro expansion point at the line the tokens came from, where names, strings and operators are interned pointers (`intern("while")` gets one). `tokens(s)` makes one from a string, and `qdup(q)` copies one.
+`quote { ... }` is a token list as a fresh object: `[n, kind, value, line, kind, value, line, ...]`, so errors inside a macro expansion point at the line the tokens came from, where names, strings and operators are interned pointers (`run.intern("while")` gets one). `run.tokens(s)` makes one from a string, and `run.qdup(q)` copies one.
-`eval(q)` compiles a token list as a function body and runs it. `compile(q)` returns its address instead, `load(q)` compiles top-level declarations, and `evals(s)` runs `eval` on a string.
+`run.eval(q)` compiles a token list as a function body and runs it. `run.compile(q)` returns its address instead, `run.load(q)` compiles top-level declarations, and `run.evals(s)` runs `run.eval` on a string.
-parse words: `immediate fn name() { }` runs at compile time whenever `name` shows up as a statement, inside an expression or at the top level. it starts with the current token on its own name and works the compiler directly: `next()`, `peek()` and `peekat(k)` (up to 4 tokens ahead), `tk`/`tv` (current token kind and value), `expect`, `accept`, `expr()`, `stmt()`, `substmt()`, `ptype()`, `emit`, `lit`, `op`, `hole`/`setlit` for jumps, and `here`. `grab()` reads a `{ }` group into a token list, and `feed(q)` pushes a list back into the input, which is all a macro needs. parse words can call `eval` partway through a compile: the compiler saves its state and emits the new function in place with a jump around it. the built-in keywords are entries in the same table, so a parse word called `while` replaces `while`.
+parse words: `immediate fn name() { }` runs at compile time whenever `name` shows up as a statement, inside an expression or at the top level. it starts with the current token on its own name and works the compiler directly: `run.next()`, `run.peek()` and `run.peekat(k)` (up to 4 tokens ahead), `run.tk`/`run.tv` (current token kind and value), `run.expect`, `run.accept`, `run.expr()`, `run.stmt()`, `run.substmt()`, `run.ptype()`, `run.emit`, `run.lit`, `run.op`, `run.hole`/`run.setlit` for jumps, and `run.here`. `run.grab()` reads a `{ }` group into a token list, and `run.feed(q)` pushes a list back into the input, which is all a macro needs. parse words can call `run.eval` partway through a compile: the compiler saves its state and emits the new function in place with a jump around it. the built-in keywords are entries in the same table, so a parse word called `while` replaces `while`.
-tables: `add_binop("**", 11, "ipow c")` adds or replaces a binary operator with a precedence and its code, written like asm (names resolve when it's used). `add_unop` does the same for prefix operators, and the lexer learns new spellings from both. `mnem` is the asm mnemonic table, `q@!$_~cjzr,?.+-*&|^><%`, and can be rewritten.
+tables: `run.add_binop("**", 11, "ipow c")` adds or replaces a binary operator with a precedence and its code, written like asm (names resolve when it's used). `run.add_unop` does the same for prefix operators, and the lexer learns new spellings from both. `run.mnem` is the asm mnemonic table, `q@!$_~cjzr,?.+-*&|^><%`, and can be rewritten.
## the bootstrap
-`src/tools/zc0.c` compiles the subset of zc used by `src/compiler/compiler.zc` (without the analyzer, types, quote, parse words, or block expressions, but with `enhance` declarations) into the first genome. that genome compiles `src/compiler/compiler.zc` again in the VM, where the full compiler, analyzer, and native division code meet their own source. the compiler passes its lifetime checks and compiles itself into identical words; the test suite checks this fixpoint on the jit and llvm backends when available. an image build (`build_image(org, dst)`) compiles with only the image's names, keywords, and built-in types visible, writes code for the requested address, and appends a symbol table so the booted genome knows its names and enhancements.
+`src/tools/zc0.c` compiles the subset of zc used by `src/compiler/compiler.zc` (without the analyzer, types, quote, parse words, or block expressions, but with `enhance` declarations) into the first genome. that genome compiles `src/compiler/compiler.zc` again in the VM using the full compiler, analyzer, and native division code. the compiler passes its lifetime checks and compiles itself into identical words; the test suite checks this fixpoint on the jit and llvm backends when available. an image build (`build_image(org, dst)`) compiles with only the image's names, keywords, and built-in types visible, writes code for the requested address, and appends a symbol table containing its names and enhancements.
## the instruction set
-every word is 64 bits. an odd word is a literal that pushes `w>>1` (63 bits, sign extended). an even word is an opcode, `w/2`. there's a data stack and a return stack, both stored inside memory. a literal cannot hold the minimum word (`-2^63`); `put_n` and `itoa` special-case it. shift counts wrap at 64. `mulhs` and `mulhu` are the high half of a 128-bit product, and `clz` of 0 is 64.
+every word is 64 bits. an odd word is a literal that pushes `w>>1` (63 bits, sign extended). an even word is an opcode, `w/2`. there's a data stack and a return stack, both stored inside memory. a literal cannot hold the minimum word (`-2^63`); `rt.put_n` and `std.Text.itoa` special-case it. shift counts wrap at 64. `mulhs` and `mulhu` are the high half of a 128-bit product, and `clz` of 0 is 64.
-byte addresses name the same memory as word addresses. word `w` is the eight bytes `w << 3` through `(w << 3) + 7`, and byte 0 of a word is its low 8 bits, so the layout does not depend on the host's endianness. `addr_b(w)` and `addr_w(b)` convert. a byte address is an ordinary integer: the odd/even tag is only for code words, and `u`/`m`/`d` stay on the word pointer of a heap object. opcodes 612..617 load and store 8, 16 and 32 bits, little-endian, and a 16- or 32-bit access may cross into the next word. an access that does not lie entirely in memory loads 0 and stores nothing. `ld8`, `st8`, `ld16`, `st16`, `ld32` and `st32` in the runtime are those opcodes. a byte store of a word some translation covers is dropped the same way a word store is. it does not fire the syscall gate; that is still a word store at `SYSGATE`. `bops.h` is the shared implementation.
+byte addresses name the same memory as word addresses. word `w` is the eight bytes `w << 3` through `(w << 3) + 7`, and byte 0 of a word is its low 8 bits, so the layout does not depend on the host's endianness. `rt.addr_b(w)` and `rt.addr_w(b)` convert. a byte address is an ordinary integer: the odd/even tag is only for code words, and `u`/`m`/`d` stay on the word pointer of a heap object. opcodes 612..617 load and store 8, 16 and 32 bits, little-endian, and a 16- or 32-bit access may cross into the next word. an access that does not lie entirely in memory loads 0 and stores nothing. `rt.ld8`, `rt.st8`, `rt.ld16`, `rt.st16`, `rt.ld32` and `rt.st32` in the runtime are those opcodes. a byte store of a word some translation covers is dropped the same way a word store is. it does not fire the syscall gate; that is still a word store at `SYSGATE`. `bops.h` is the shared implementation.
-a byte string is that layout as a value. `b"..."` (the `b` against the quote) packs the characters into the code stream: word 0 is the length, the bytes start at the next word, eight per word, and a 0 byte follows the last character. `"..."` stays one character per word. `len` on a byte string is `len_s`. `s[i]` loads one byte and is not a place to store. `s[i:j]` copies a slice into a fresh object, end exclusive, and clamps the bounds. `+` concatenates two byte strings. the six comparisons require a byte string on both sides and order unsigned bytes. `s_to_n` parses a signed integer and `n_to_s` builds a fresh string; both special-case the minimum word. `put_s` of one byte string writes its bytes. a result of `+`, a slice, `n_to_s` or `new_s` is a shared heap object. the compiler records that a local holds a byte string in bit 6 of its tag. `cons(a, b)` and `triple(a, b, c)` are shared cells on the same heap; `car`, `cdr` and `tget` read them, and `share(p)` publishes an existing object. bit 8 of the header flags is the share bit. it is orthogonal to live, marked and releasing. `free` of a shared object does nothing, `own` and `release` do not follow one, and the sweep collects it when no word points at it, cycles included. a unique object still quarantines on `free`. putting a heap object into a cell publishes it, so the old owner's later `free` does not clear a cell that still points at it.
+a byte string is that layout as a value. `b"..."` (the `b` against the quote) has type `bstr` and packs the characters into the code stream: word 0 is the length, the bytes start at the next word, eight per word, and a 0 byte follows the last character. `"..."` has type `char*` and stays one character per word. `len` on a byte string uses the packed representation. `s[i]` loads one byte and is not a place to store. `s[i:j]` copies a slice into a fresh object, end exclusive, and clamps the bounds. `+` concatenates two byte strings. the six comparisons require a byte string on both sides and order unsigned bytes. `rt.s_to_n` parses a signed integer and `rt.n_to_s` builds a fresh string; both special-case the minimum word. `rt.put_s` of one byte string writes its bytes. a result of `+`, a slice, `rt.n_to_s` or `rt.new_s` is a shared heap object. the compiler records that a local holds a byte string in bit 6 of its tag. `rt.cons(a, b)` and `rt.triple(a, b, c)` are shared cells on the same heap; `rt.car`, `rt.cdr` and `rt.tget` read them, and `rt.share(p)` publishes an existing object. bit 8 of the header flags is the share bit. it is orthogonal to live, marked and releasing. `rt.free` of a shared object does nothing, `rt.own` and `rt.release` do not follow one, and the sweep collects it when no word points at it, cycles included. a unique object still quarantines on `rt.free`. putting a heap object into a cell publishes it, so the old owner's later `rt.free` does not clear a cell that still points at it.
a float is an IEEE-754 binary64 value in one whole word, not an odd literal: 63 bits is not enough, and bit 0 of a double is data. opcode 600 (`flit`, the word 1200) is followed by those raw bits, and the decoder consumes that word itself. 601..604 are add, sub, mul, div. 605..607 are `==`, `<` and `<=` (the other relations are those plus a swap or a xor). 608 and 609 convert int to float and float to int, truncating toward zero. 610 prints six digits after the dot and 611 parses a word-string into bits. 618 prints with a mode: 0 is the shortest spelling that parses back to the same bits, 1 is fixed (digits after the dot) and 2 is scientific (significant digits), and the precision is capped at 17. they sit above the enhancement range, so slots 22..533 do not move. where the processor has an FPU the arithmetic is that FPU: the arm64 jit emits `fadd`, `fmul`, `fcmp`, `scvtf` and `fcvtzs`; llvm lowers the same opcodes to native `double` operations; the interpreter and the microcore call `fops.h`, which is a volatile host `double` so gcc emits SSE2, AArch64 FP or RV64 `fadd.d`. both virt boards turn the FPU on before `main` (RISC-V `mstatus.FS`, AArch64 `CPACR_EL1.FPEN`) and compile with a hard-float ABI. a port with no FPU builds with `-DZY_SOFTFLOAT`: the same `fops.h` entry points are then an integer IEEE implementation, round ties to even, and the float line is unchanged. `fops.h` is also the shared parser and the printer, and both are integer arithmetic, so the text is the same on every platform and does not call libc. opcode 610 is an optional minus, the integer digits, a dot, and six fractional digits, with a tie in the next digit rounded away from zero, for the whole finite range. opcode 618 is the shortest round-trip spelling (a plain decimal when the exponent is between -4 and 15, otherwise scientific), or a fixed or scientific form. the parser keeps the digits it is given, rounds ties to even, and accepts an exponent, `inf` and `nan`. `nan`, `inf` and `-inf` are the non-finite texts, and `-0` is distinct from `0`. `float` is a type. the compiler tracks it through locals (bit 5 of the tag), globals, fields, parameters, call results and returns, and promotes an integer operand of `+ - * /` and the comparisons, so a use does not need its own annotation. `++` is still an integer add. `o3` lifts 600..609: two constant operands fold by the IEEE operation itself, float values stay in FP registers, and `x - x` is left alone so a NaN stays a NaN. `0.0` and `-0.0` are both false in `if`, `while`, `for` and `&&` `||`: a float is compared with `+0` before the integer test. `1.field` is a field access. a dot after digits is a fraction only when a digit or an exponent follows it, and an integer address uses a field name that belongs to one struct.
@@ -470,9 +513,9 @@ stack effects: `!` is `(v addr --)`, `$` is `(n -- copy of the nth item)` so `0
## memory map and abi
-see [memory configuration](docs/design/memory.md) for build controls and the remaining runtime growth work.
+see [memory configuration](docs/design/memory.md) for build controls, runtime reservations, and host limits.
-the machine has two sizes. PW is the end of the fixed part: the code area, the compiler's tables, the genome and the stacks. every table indexed by a code address is PW words long, and the program counter never goes past it. MW is the total number of words the host gives the machine (1 GiB, 0x8000000 words, on the default build), and the heap starts at PW and grows into the rest of it while the program runs. the host's array behind MW is static and untouched until the heap reaches it, so a program that never allocates much never pays for the memory. a build can pick another total with `-DMW=...`; `src/core/memmap.h` refuses to compile if it is smaller than the initial heap.
+the initial layout reserves code, compiler tables, the genome, and stacks below `PW`. the heap starts at `PW` on hosted growing maps. tables and execution stacks grow when needed, and emitted code uses stable heap segments after its initial region fills. native code-address metadata grows with it. `MW` selects the default host memory limit (1 GiB, `0x8000000` words, on the default build); `TINYVM_MEMORY_WORDS` and `TINYVM_MAX_MEMORY_WORDS` configure startup memory and its growth limit. the native core reserves stable backing up to the startup maximum, while portable hosts replace their backing allocation when they grow. a build can select another default with `-DMW=...`; `src/core/memmap.h` rejects a value smaller than the initial heap.
```
0x000000..0x07FFFF code: the program, with globals between functions (CODE_END)
@@ -487,7 +530,7 @@ the machine has two sizes. PW is the end of the fixed part: the code area, the c
0x1C5000..0x224FFF the heap to start with (0x60000 words), then it grows up to MW
```
-the numbers are for the default build. its regions are sized so a program does not run into them (the tables and stacks are large, and words nobody writes cost nothing, because the host's array behind MW is not touched until it is used), where the small and run presets are tight and sized from measurements; `src/tools/genmap` writes the same layout for a smaller machine (see "smaller machines") and `--preset fixed` gives the older arrangement with a fixed heap and no growth, which the language server uses. the host answers `syscall(9, k)` with facts about the map (0 total words, 1 heap start, 2 heap end for now, 3 bytes), so zc code never assumes a size.
+the numbers describe the default build's initial reservations. compact compiler and run-only maps start with smaller reservations and also support growth. `bin/genmap` writes matching layouts (see "smaller machines"); `--preset fixed` retains the older bounded heap for compatibility. the host answers `core.syscall(9, k)` with facts about the map (0 total words, 1 heap start, 2 heap end for now, 3 bytes). argument 4 requests memory growth, and argument 5 changes the growth limit within the host's backing constraints.
calls push the arguments left to right and the callee's prologue pops them into its frame. locals live in the frame at `*0x90000 + slot`. every function leaves exactly one word on the data stack. heap objects have a 5 word header: size, flags with the size class in the high bits, a check word, owner, type.
@@ -495,13 +538,13 @@ calls push the arguments left to right and the callee's prologue pops them into
registers: x19 holds the memory base, x20 the data stack pointer, x21 the return stack pointer, x22 the translation table and x23 the state block (memory, s, r, table, coverage map, Q, G, &J, store hook, install hook, enhancement fallback). o3 regions also use x26..x28, which the entry stub saves.
-compilation is lazy. the first time control reaches an address, that block is translated from live memory. a block ends at call, jump, ret or halt. a `z` doesn't end it: the not-taken side keeps compiling, which makes superblocks. a literal followed by `c`, `j` or `z` is fused into a constant target. every exit goes through a dispatch stub that checks both stack pointers against their limits (kept in x24 and x25), bounds-checks the target, loads `tab[t]`, and branches straight to it when it's already compiled. otherwise it returns to c, which compiles the block and re-enters. enhancements are inlined: inputs load from the data stack into x0..x15, the optimized micro-ops run in registers with x16 and x17 as scratch, and the outputs go back. an install calls into c and then returns to the loop, which flushes every translation, since code built before the install may have inlined the old version. a flush drops o3 regions too, and they lift again on their next entry. when the loop reaches an o3 entry point, it compiles the region (or reuses it) instead of translating a block.
+compilation is lazy. the first time control reaches an address, that block is translated from live memory. a block ends at call, jump, ret or halt. a `z` doesn't end it: the not-taken side keeps compiling, which makes superblocks. a literal followed by `c`, `j` or `z` is fused into a constant target. every exit goes through a dispatch stub that checks both stack pointers against their limits (kept in x24 and x25), bounds-checks the target, loads `tab[t]`, and branches straight to it when it's already compiled. otherwise it returns to c, which compiles the block and re-enters. enhancements are inlined: inputs load from the data stack into x0..x15, the optimized micro-ops run in registers with x16 and x17 as scratch, and the outputs go back. an install calls into c and invalidates translations and o3 regions that depend on its slot. invalidated code is rebuilt on its next entry. when the loop reaches an o3 entry point, it compiles the region (or reuses it) instead of translating a block.
self-modification is handled with a counted coverage map: one byte per word, the number of live translations that cover it, sticky at 255. a store checks that byte inline: an uncovered target, the usual case, gets a plain store. a covered target calls the hook, which looks the word up in a page index (64-word pages, each with a list of the blocks that touch it) and drops only the translations containing that word, decrementing the coverage of just their words. the jit code then returns to c at the next instruction, so even patches to the very next word take effect. writing back the same value is a no-op. installing an enhancement does not flush: every translation records which enhancement slots it inlined, and an install drops only those (and the o3 regions that depend on the slot). if the 16 mb code buffer or the block index fills, everything is flushed. on llvm each block has its own resource tracker, so dropped code is freed once no compiled code is running.
-the stacks are checked. the data and return stacks are checked at every block exit on all three backends, so the cost is a couple of compares per block rather than per push. a block pushes at most about 1024 words, which the data stack limit leaves room for. overflowing either one stops the cpu with `zy: data stack overflow` or `zy: return stack overflow`. the frame stack is checked in every function prologue, which resets the frame pointer and stops with `zy: frame stack overflow`.
+the data and return stacks are checked at block boundaries and grow before their guard space is exhausted. `reserve_data(n)` and `reserve_returns(n)` request capacities explicitly. the frame stack grows in stable segments, preserving pointers into older frames; `reserve_frames(n)` controls the next segment's minimum size. when the host or heap cannot provide more storage, execution stops with a controlled overflow or allocation diagnostic.
-platform: the darwin path uses `MAP_JIT`, `pthread_jit_write_protect_np` and `sys_icache_invalidate`; linux uses rwx mmap and `__builtin___clear_cache`. this session verified the macOS arm64 native path for dispatch, frames, stacks, and code segments, along with the FFI learning example and the LLVM 23 backend. the shell runner was exercised with bash on macOS. no linux-host runtime tests were run in this session, and the Windows batch runner was not run on a Windows host. under the hardened runtime, the darwin path also needs the allow-jit entitlement. every host shares the same semantics: shift counts wrap at 64, stores outside memory are ignored, loads outside memory read 0, and jumping outside memory or to an unwired opcode halts. cells are 8 bytes on the host; the word addresses in the map above did not move.
+platform: the darwin path uses `MAP_JIT`, `pthread_jit_write_protect_np` and `sys_icache_invalidate`; linux uses rwx mmap and `__builtin___clear_cache`. this session verified the macos arm64 native path for dispatch, frames, stacks, and code segments, along with the FFI learning example and the LLVM 23 backend. the shell runner was exercised with bash on macos. no linux-host runtime tests were run in this session, and the windows batch runner was not run on a windows host. under the hardened runtime, the darwin path also needs the allow-jit entitlement. every host shares the same semantics: shift counts wrap at 64, stores outside memory are ignored, loads outside memory read 0, and jumping outside memory or to an unwired opcode halts. cells are 8 bytes on the host; the word addresses in the map above did not move.
## the microcore
@@ -517,6 +560,8 @@ what it leaves out, on purpose: native code, the optimizer, o3's register interp
`src/tools/genmap.c` computes the layout for each build. the `small` preset carries the compiler; `run` carries the runtime and a precompiled program. `src/tools/automap.sh` sizes the genome reservation to the actual image and rebuilds the bootstrap compiler for that map.
+`make run PROG=...` and `src/tools/zcc` build run-only images through stage 0, so their source must fit the bootstrap compiler's supported subset. `src/tools/build.zc` is the full-language image builder: it compiles the source after its backtick into a little-endian genome image. the run-only runtime installs enhancement descriptors before heap initialization, so enhanced operations and full-language array algorithms are available in self-hosted images.
+
```sh
bash src/tools/automap.sh small build/small
cp src/micro/micro.c src/micro/micro_port.h src/micro/port_linux.c src/core/fops.h src/core/bops.h src/core/stack.h build/small/