From 2f52ef8662b9922cfbe970f0f01a35f83bc761d4 Mon Sep 17 00:00:00 2001 From: zinc Date: Wed, 14 Oct 2026 09:56:23 +0000 Subject: docs: add SPEC.md, the first written specification of zc Reference for lexing, types, expressions (full precedence table), statements, declarations, the ownership analyzer and its messages, compile-time execution and the runtime contract, with appendices for the zc0 subset and for doc/implementation disagreements and bugs found while checking the rules against build/bin/zy. Pointers added to README.md and ONBOARD.md. --- docs/SPEC.md | 1093 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1093 insertions(+) create mode 100644 docs/SPEC.md (limited to 'docs/SPEC.md') diff --git a/docs/SPEC.md b/docs/SPEC.md new file mode 100644 index 0000000..51ed5d1 --- /dev/null +++ b/docs/SPEC.md @@ -0,0 +1,1093 @@ +# zc language specification + +this is the reference for the zc language. it is written from `src/compiler/compiler.zc` (the self-hosted compiler, the reference implementation), checked by running programs on `build/bin/zy`. where `README.md` or `docs/LEARN.md` disagree with the code, the code wins and the disagreement is listed in appendix b. `src/tools/zc0.c` (the stage-0 compiler) accepts only a subset; appendix a says which features are in it. + +## 0. conventions + +- a rule is normative. an example shows the expected result after `=>`. "error: text" means the compiler prints a line `zc: line N: text` (see 8.2) and the program does not run. a message given in this document is matched as a substring of the line after `zc: line N: `, except where the whole line is shown. +- "word" means a 64-bit machine word. all addresses are word addresses unless a rule says byte address. +- "zy" means `build/bin/zy`, which compiles the source on stdin and runs `main`. examples assume a prelude that is not shown: `include rt;` at the top, and the statements inside `fn main() { ... return 0; }` when no function is shown. `pn(x)` stands for `rt.put_n(x); rt.put_c(10);`. +- "unspecified" means a conforming program must not depend on it. "implementation note" marks something observed that the reference implementation does, which this spec does not promise. +- the compiler reports at most 20 errors, then stops (8.2). + +## 1. source text + +### 1.1 characters and white space + +1.1.1 the source is a sequence of bytes. only ascii is meaningful outside comments, string literals and character literals. a byte of 128 or more inside a string literal is one word of the string (a utf-8 character is several words). a byte of 128 or more elsewhere is a one-character operator token and is an error where an expression or declaration is expected. + +1.1.2 white space is space (32), tab (9), line feed (10) and carriage return (13). it separates tokens and is otherwise ignored. + +1.1.3 a unit (the text of one compile) is read from stdin to end of input or to the first source-ending backtick (1.5), whichever is first. + +### 1.2 comments + +1.2.1 `//` starts a comment that runs to the end of the line. + +1.2.2 `/* ... */` is a block comment. block comments do not nest. a block comment that is never closed runs to the end of the unit and is not an error. + +1.2.3 comments are not tokens, but `docs on;` (5.14) and `hints` (4.12) read them: `//` lines directly above a declaration document it, and `//:` lines are implementation notes that document nothing. an inline block comment right before a numeric literal marks the literal as intentional for the magic-number hint. neither changes the generated code. + +### 1.3 identifiers and keywords + +1.3.1 an identifier is a letter or `_` followed by letters, digits and `_`. letters are `a-z` and `A-Z`. names are case sensitive. + +1.3.2 there are no reserved words in the lexer. keywords are entries of a table in the compiler (1.3.4), and a name that is also a visible local variable or parameter is the variable, not the keyword. a program can replace a keyword with a parse word of the same name (7.2). + +1.3.3 module-qualified names. an identifier that is immediately followed by `.` is joined with the `.` and the identifier after it into one token (`rt.put_c`) when both hold: +- the first identifier is a known module: one of `rt`, `run`, `core`, `lsp`, `std`, `icon`, or a module loaded by `include name;` or declared by `namespace name;` earlier in the unit; and +- no local variable or parameter of that name is visible. +only one dot is joined: `util.box.mk` lexes as the token `util.box` then `.` then `mk`. the join happens whether or not the module was included by the current file; using such a name without including its module is the error in 5.12. +``` +include rt; +fn main() { rt.put_c(65); rt.put_c(10); return 0; } // prints A +``` +a name that looks like `x.y` where `x` is not a known module is the three tokens `x`, `.`, `y` (field access). + +1.3.4 the keyword table. a keyword is a statement keyword (S), a top-level keyword (T), an expression keyword (E), or a combination. these are the `add_kw` entries: + +| keyword | S | T | E | meaning | +|---|---|---|---|---| +| `let` | S | | | declaration without a type (4.1) | +| `if` `else` | S | | | 4.3 (`else` is consumed by `if`) | +| `while` `for` `do` | S | | | loops (4.4 to 4.6) | +| `switch` | S | | | 4.7 | +| `break` `continue` `return` | S | | | 4.8 | +| `enum` | S | T | | 5.8 | +| `include` | S | T | | 5.12 (a statement only by accident, 5.12.7) | +| `embed` | S | T | | 5.11 | +| `memdebug` | S | T | | 4.11 | +| `hints` | S | T | | 4.12 | +| `unchecked` | S | | | 4.9, 5.1 | +| `o3` | S | | | 4.10 | +| `asm` | S | | E | raw vm words (4.13) | +| `run.method` | S | T | | register a receiver method (7.7) | +| `run.syntax` | S | T | | register a keyword (7.6) | +| `run.statement_hook` | S | T | | register a statement hook (7.6) | +| `extern` | | T | | native function declaration (5.13) | +| `quote` | | | E | token list as data (7.4) | +| `run.share_value` | | | E | publish a value for sharing (7.8) | +| `sizeof` `typeof` | | | E | 3.12 | +| `cfunc` | | | E | native callback pointer (5.13) | +| `len` `cap` `push` `pop` `resize` | | | E | growable array intrinsics (3.13) | + +a keyword marked only S used inside an expression is the error `NAME: can't be used inside an expression`. a keyword that is not E and is used where an expression is expected is the same error. + +1.3.5 these words are recognised by position by the parser, are not in the keyword table, and are treated as ordinary names elsewhere: `fn`, `const`, `fixed`, `static`, `struct`, `type`, `typecheck`, `docs`, `namespace`, `enhance`, `immediate`, `as`, `case`, `default`, `on`, `off`, `small`, `micro`, `vm`, `this`, `new`, and the tag names `u`, `m`, `d`. `struct`, `type`, `typecheck`, `docs`, `namespace` and `enhance` are top-level only (5.1.2). + +1.3.6 the type names `int word char bool void float f64 f32 bstr i8 u8 byte i16 u16 i32 u32` are ordinary names bound to types (2.1). a local variable with such a name shadows the type name where a declaration or expression could start. + +### 1.4 literals + +1.4.1 integer literals. +- decimal: digits. there is no octal: `010` is ten. the value wraps modulo 2^64: `99999999999999999999` => 7766279631452241919. +- hexadecimal: `0x` or `0X` followed by hex digits, case-insensitive. the value wraps modulo 2^64: `0xFFFFFFFFFFFFFFFF` => -1. `0x` with no digits is 0 (and a letter after it starts a new token). +- there are no binary literals, digit separators or suffixes. +- the type of an integer literal is `int`. a literal is never negative; `-5` is unary minus applied to 5 (the constant is folded). + +1.4.2 character literals. `'c'` is one byte, type `int`, value 0 to 255. escapes: `\n` 10, `\t` 9, `\r` 13, `\0` 0, `\e` 27; a backslash before any other character is that character (`'\\'` 92, `'\''` 39, `'\x'` 120). a literal with more than one byte (including a non-ascii character) is error: bad character literal. + +1.4.3 float literals are ieee-754 binary64 values stored in one word. forms: `1.5`, `1.` , `.5`, `1e3`, `1.5e-3`, `1E+3`. the lexer reads the digits, optional `.digits`, and an optional exponent `e|E [+-] digits`; an exponent letter without digits after it is error: bad float. a dot after digits is a fraction when a digit, `e` or `E` follows it, is a field access when a letter or `_` (other than `e`/`E`) follows (`1.field`), and otherwise ends the literal (`1.` is a float). the type is `float`. + +1.4.4 word strings. `"..."` is a string of type `char*`: the bytes are stored one per word in the code area, followed by a 0 word. escapes are as for character literals. a `\0` (or any byte 0) inside the literal ends the string there (`"ab\0cd"` is `"ab"`). a literal may contain a raw line feed. an unterminated string runs to the end of the unit and then error: expected ')' (or the like) at the end of the file. adjacent literals are not concatenated. the literal is static storage that is not a heap object; it is read-only by convention (writing through it is unspecified). + +1.4.5 packed byte strings. `b"..."`, where `b` touches the opening quote, has type `bstr`: a pointer to a block whose word 0 is the length in bytes, followed by the bytes eight to a word, little-endian, followed by a 0 byte. escapes and the `\0` truncation rule are as for word strings. `b "x"` (with a space) is the name `b` then a string. +``` +let t = b"hello"; +rt.put_n(len(t)); // 5 +rt.put_s(t[1:3]); // el (slice, end exclusive, clamped) +``` +operations on `bstr` are in 3.10. + +### 1.5 the source-ending backtick + +1.5.1 a backtick (code 96) that is not inside a `//` comment, a `/* */` comment, a string literal or a character literal ends the unit. the rest of stdin is not read by the compiler; the running program reads it with `rt.get_c`. +``` +include rt; +fn main() { int c = rt.get_c(); rt.put_n(c); rt.put_n(rt.get_c()); return 0; } +`xy +``` +=> prints `120121` (the bytes `x` and `y` after the backtick). + +1.5.2 a backtick inside a comment or a literal is an ordinary character. the same rule is used by zc0 (appendix a). + +### 1.6 operator tokens + +1.6.1 the operator characters are `+ - * / % = < > ! & | ^ ~ ? : . @ # $`. other punctuation `( ) { } [ ] , ;` is always a one-character token. + +1.6.2 an operator token is read greedily: characters are appended while the text so far is still a prefix of some known operator. the known operators are the binary operators of 3.3, their compound assignments, the unary operators `! !! ~ -`, `=`, `->`, `++`, `--`, `.` and any operator added by a program with `run.add_binop` or `run.add_unop` (7.5). so `a<<=2` is `a`, `<<=`, `2`; `a< -b` needs the space only where the joined text would be an operator. + +1.6.3 `u*x`, `m*x` and `d*x` are recasts only when the `*` touches the next token with no white space and `u`, `m` or `d` is not a visible variable (3.9). + +## 2. types + +zc is untyped by default. an untyped name has type `word`, which is compatible with every type. types are checked only where 2.9 says. every value is one word except in-place aggregates (2.5, 2.6), whose value is their address. + +### 2.1 the builtin types + +| type | size (words) | notes | +|---|---|---| +| `word` | 1 | the untyped type, compatible with everything | +| `int` | 1 | 64-bit signed two's complement | +| `char` | 1 | a full word, treated as a number; `char[n]` takes n words | +| `bool` | 1 | a number; values are not normalised to 0 or 1 on store (appendix b) | +| `void` | 0 | function result only; a variable cannot be void | +| `float`, `f64` | 1 | ieee binary64 | +| `f32` | 1 | single precision: a variable holds the value already rounded to float; arrays and pointers of `f32` are packed two per word | +| `i8 u8 byte i16 u16 i32 u32` | 1 | sized integers; `byte` is `u8` (2.3) | +| `bstr` | 1 | pointer to a packed byte string (1.4.5) | + +there are no 64-bit sized names (`u64`, `i64`) and no `long`; `int` is the 64-bit type. (`long`, `size_t` and the other c names are accepted only in `extern` and `cfunc` signatures, 5.13.) + +### 2.2 type syntax + +``` +type := type-name { "*" | "[" "]" | "[" "?" "]" | "[" const-expr "]" } { "(" param-types ")" } +type-name := builtin | struct-name | enum-name | alias | generic-name "<" type { "," type } ">" | "fixed" type | "const" type +``` +2.2.1 `T*` is a pointer to `T`. the star goes after the type. there is no prefix form: `*int x;` is error: int: is a type, not a value (a star in front of an expression is only a dereference). array types are postfix too: `[3]int a;` is error: array types are postfix: T[N], T[] or T[?]. + +2.2.2 array suffixes apply left to right on the element type: `int[2][3]` is an array of 2 arrays of 3 ints. + +2.2.3 a function type is a return type followed by a parenthesised parameter list: `int(int, char*)`. `fn(int)` is a function type that returns `word`. a parameter may carry a callback tag, `int(char*.d)`. a function type in a declaration of a variable is spelled the same way: `int(int) f = g;`. + +2.2.4 `type name = T;` (top level only) declares an alias. a pointer to an array type or to a function type needs an alias (`type row = int[3]; row* r;`), because `int[3]*` reads as an array of pointers. + +2.2.5 a generic struct is instantiated with `name`; a `>>` closing two lists is split. + +2.2.6 a type is hash-consed: the same type written twice is the same record. + +### 2.3 integers + +2.3.1 `int` and the sized integers hold a 64-bit word in variables, parameters, fields and return values. a sized integer is cut to its size on every store: unsigned are masked, signed are masked then sign-extended. +``` +u8 a = 250; a = a + 10; // a is 4 +i8 s = 100; s = s + 100; // s is -56 +u32 z = 0; z--; // z is 4294967295 +``` +2.3.2 casts to a sized type cut the value: `(u8)300` => 44, `(i8)200` => -56, `300 as u8` => 44, `(u16)-1` => 65535. + +2.3.3 arrays and pointers of sized types are packed: `u8` one byte per element, `u16`/`i16` two, `u32`/`i32`/`f32` four, in little-endian order within each word. `sizeof(u8[10])` is 2; `sizeof(u8[8])` is 1. indexing a packed array reads and writes bytes. the address of one packed element cannot be taken (error: an element of a sized-integer array has no address; it is a byte inside a word). + +2.3.4 `int` arithmetic is 64-bit two's complement and wraps (3.4). + +### 2.4 floats + +2.4.1 `float` and `f64` are the same type. `f32` is a separate type with single-precision rounding on store. + +2.4.2 a float is truthy if it is not `+0.0` or `-0.0` (in `if`, `while`, `for`, `do`, `&&`, `||`, `?:`, `!`). `0.0 == -0.0` is 1. a nan is truthy. + +2.4.3 mixing rules are in 3.5. + +### 2.5 pointers + +2.5.1 a pointer value is a word holding a word address. `p + 1` is the next word whatever `p` points at; indexing `p[i]` scales by the element size in words (or in bytes for packed element types). see 3.7. + +2.5.2 a literal `0` is the null pointer. a null pointer is the word 0. + +2.5.3 `void*` is accepted and is compatible with every pointer type. + +### 2.6 arrays + +2.6.1 `T[n]`: `n` is a constant expression (3.14) and the array lives in place: in a frame, a global, or inside a struct. its value is its address. `sizeof(T[n])` is the words the elements take (packed for sized types). there is no run-time bound check; a constant index outside `0..n-1` is error: the index is out of range for this array. + +2.6.2 `T[]` is an in-place array whose length comes from its initializer: `int[] a = {1, 2, 3};` has length 3. without an initializer, error: T[] needs an initializer to determine its length. + +2.6.3 `T[?]` is a growable array: a pointer to a handle object of four words `[len, cap, items, element size]`, made by `rt.new(T[?])` or `rt.new(T[?], n)` (n initial length, zero filled). the handle owns its items object. indexing a `T[?]` is bounds checked at run time (3.13). element size is stored negated for packed element types. + +2.6.4 `fixed T[?]` is a growable-array handle whose length cannot change: `push`, `pop` and `resize` on it are error: a fixed array can't grow or shrink. `rt.new(fixed T[?], n)` needs the length (error: a fixed array needs its length: new(fixed T[?], n)). a `fixed T[?]` cannot be passed where `T[?]` is expected (type error under typecheck, 2.9); the reverse is allowed. `fixed T[n]` means `T[n]`. + +2.6.5 initializer lists. `T[n] a = {e0, e1, ...}`, `T[] a = {...}`, `T[?] a = {...}` (inside a function) and the compound literal `(T[]){...}`. fewer values than `n` leave the rest 0. more values: error: too many elements in array initializer. a designator `[k] = e` sets element `k` (a negative designator is error). nested braces initialise nested arrays. `char[n] s = "text"` and `u8[n] s = b"text"`-style string initialisers fill the array. a struct local or global has no initializer list; fields are assigned. +``` +int[5] b = {1, 2}; // b[4] => 0 +int[3] d = {[1] = 5}; // d[1] => 5, d[0] => 0 +char[4] s = "abc"; // s[3] => 0 +int* p = (int[]){7, 8, 9}; // p[1] => 8; the literal is a temporary (6.9) +``` + +2.6.6 an array or struct is never assigned or passed by value. an array used as a value decays to a pointer to its first element (its address). a parameter of array type is a pointer. passing a struct or array by value is error: structs and arrays are passed by pointer. assigning to an aggregate is error: this can't be assigned to (for a local aggregate) or a whole struct or array can't be assigned; assign its fields (for a field or element). + +### 2.7 structs + +2.7.1 `struct name { members };` (5.5). members are fields, methods, static functions and constants. + +2.7.2 layout. fields are laid out in declaration order. a field whose type is a sized integer or `f32` is packed: it takes its size in bytes inside the current word at its own alignment (offset a multiple of its size); every other field starts a new word and takes `sizeof(type)` words. the size of a struct is the number of words used, rounded up. an empty-looking example: +``` +struct pix { u8 r, g, b, a; i16 dx; int w; }; +sizeof(pix) // 2: r g b a dx use 6 bytes of word 0 (padded to 8), w is word 1 +sizeof(pix[3]) // 6 +``` +a struct cannot contain itself by value (error: a struct can't contain itself by value; use a pointer). a struct's size is the sum of its fields in words; there is no padding between whole-word fields. + +2.7.3 `p.x` on a struct or a pointer to a struct reads a field; `p->x` is identical (3.8). a struct value is its address. + +2.7.4 a struct field name that belongs to exactly one struct type can be used on an untyped (`word` or integer) value: `x.name` with `x` a word. when two or more structs have a field of that name, error: .name is a field of more than one struct. + +### 2.8 enums + +2.8.1 plain enum. `enum color { RED, GREEN = 5, BLUE };` declares the global constants `RED` 0, `GREEN` 5, `BLUE` 6, also reachable as `color.RED` in expressions and case labels. a member without a value is the previous value plus one, starting at 0. the name is optional (`enum { A, B };`). values are constant expressions (3.14). a member's value cannot name an earlier member of the same enum (the members are defined when the closing brace is reached; error: expected a constant). under `typecheck on;` the enum is a distinct type (2.9). + +2.8.2 tagged enum. when any member has a `{ fields }` block the enum is tagged: `enum shape { circle { int r; }, rect { int w; int h; }, dot };`. rules: +- it needs a name; payload fields are `int`, `float` or `let` (a word), at most 8 per member; at most 63 members; members take no explicit values. +- it declares a struct `shape` of `1 + (all payload fields)` words. word 0 is `tag` (the member index, from 0). the payload field of member `m` named `f` is the field `m_f` (`s.circle_r`). +- each member `m` becomes the global constant `m` (its tag number) and a static function `shape.m(fields...)` that allocates an owned `shape*`, sets the tag and fields, and returns it (`.u`). `shape.dot()` takes no arguments. +- a value of type `shape*` is switched with pattern cases (4.7.3). +``` +shape* s = shape.rect(4, 5); +pn(s.tag); // 1 +pn(s.rect_w); // 4 +rt.free(s); +``` +a constructor called with the wrong number of arguments is error: shape.circle: wrong number of arguments. + +### 2.9 conversions and `typecheck` + +2.9.1 conversions that always happen, whatever `typecheck` says: +- storing into a sized integer cuts it (2.3.1). +- at an initializer, assignment, argument or return whose destination type is `float`/`f32`/`f64`, an `int` source is converted to float (`float f = 1;` => 1.0); where the destination is an integer type and the source is a float, the float is truncated toward zero (`int i = 2.5;` => 2, `int j = -2.5;` => -2). an `f32` destination rounds. a destination with no type (`let`, an untyped parameter, a `word`) keeps the bits (`word w = 2.5;` stores the float's bits). +- `+ - * /` and the six comparisons with a float operand convert the other operand (3.5). +- `(T)e` and `e as T` (3.9). + +2.9.2 `typecheck on;` (top level) turns on checking for the code that follows; `typecheck off;` turns it off. when on, the compiler checks initializers, assignments, arguments, return values, arithmetic, comparisons, dereferences, indexing, calls and the sides of `?:`. the first error is printed as `type error: WHAT: expected X, got Y` (or `type error: WHAT, got Y`). rules (function `compat`, destination `d`, source `s`): +- `d == s` is compatible. `word` on either side is compatible, with one exception: a `char*`, `char[]` or `bstr` source does not go to a `word` destination (fixed, so a `word` cannot silently hold a string). +- numbers (`int`, `char`, `bool`, sized integers) convert to each other freely. floats convert to floats. a plain enum converts to `int`; an `int` does not convert to an enum, and an enum converts to no other enum (error: initializer: expected color, got int). +- pointers need the same element type, unless either element is `word` or `void`. an array decays to a pointer to its element type. an in-place struct supplies its address to a pointer to that struct. +- the literal `0` converts to any pointer, function or `T[?]` type. +- a `bstr` converts only to a `bstr`. +- a `T[?]` converts to a `T[?]` of the same element; a `fixed T[?]` does not convert to a `T[?]` (error: argument: expected int[?], got fixed int[?]). a read-only source does not convert to a mutable reference (2.10). +- a `+ - * / % & | ^ << >>` operand must be a number (or `word`); pointer plus or minus a number is that pointer; pointer minus pointer is `int`; `==` and `!=` need compatible operands; `<` etc. need numbers or two pointers (error: type error: arithmetic needs numbers, got int*). +- `&&` and `||` need scalars. +- dereferencing needs a pointer (error: only pointers can be dereferenced, got int); calling needs a function type or `word`; indexing needs a pointer, array or `word`, and an index that is a number. +- a generic call unifies parameter types with argument types (error: argument: expected int, got char*). +some checks apply with `typecheck off;` because code generation needs them: field names, field access on a non-struct (error: .x: that's a int, not a struct or a pointer to one), `point has no field z`, argument count. + +2.9.3 with `typecheck off;` (the default), pointer and number mixes are accepted: `char* s = 5;` compiles. +``` +typecheck on; +fn main() { char* s = 5; return 0; } +``` +error: type error: initializer: expected char*, got int. + +### 2.10 const + +2.10.1 `const T` marks storage read-only. `const T*` is a pointer to read-only data; `const int[3] a = {1,2,3};` is a read-only array. writing through read-only storage (an element, a field, a compound assignment, `++`, rebinding a `const` handle) is error: const storage cannot be modified, whatever `typecheck` says. a read-only reference cannot become a mutable reference (error: const storage cannot be converted to a mutable reference). reading is allowed, and copying an element into an ordinary variable is allowed. `fixed const T[n]` and `const fixed T[n]` are the same. + +2.10.2 a local `const k = 5;` is a constant with the type of its initializer (4.1.4). + +### 2.11 function types + +2.11.1 a function value is the word address of the function (for a plain function) or of a closure block (3.15). calling a value, `f(args)`, works for both; the compiler checks at run time whether the callee is a plain function or a closure block. + +2.11.2 `int(int, int) f = add;` stores the address. `let h = add;` infers the function type. the type of a closure is `R(P...)` built from the closure's parameter types and returns; `fn:T(params) { ... }` declares the result type `T` (3.15). + +### 2.12 ownership tags + +2.12.1 a tag follows a type after a dot: `int*.u`, `node*.m p`, `.d x` (tag alone, type omitted). tags are `.u`, `.m`, `.d`. they are ordinary names after the dot; `u`, `m` and `d` are otherwise unreserved. the meaning of each tag is in section 6. + +2.12.2 a tag may appear: on a function result (`.u` only), on a parameter (`.m` or `.d`), on a local declaration (`.u`, `.m`, `.d`), and on a callback parameter inside a function type (`.m` or `.d`). a tag on a global or a field is error: tags go on locals and parameters, not globals; a field has no tag. + +### 2.13 generics + +2.13.1 `struct list { ... }` declares a generic struct (up to 8 parameters). a generic struct cannot have static functions (error: a static function can't be in a generic struct). its methods are compiled once per instance, and `this`, `sizeof(T)` and `rt.new(T)` mean the real type. the fields of a generic struct instance are not packed: each takes `sizeof(type)` words. + +2.13.2 a generic function lists type parameters after its name: `T first(T* p) { ... }`. in the return type, an unknown name is taken as a type parameter and the list after the name must confirm it (error: T: isn't a type). a call gives the types explicitly, `first(p)`, or infers them by unifying parameter types with argument types. each distinct set of type arguments gets its own compiled copy. a generic function value without a call needs the explicit form: `f`. +``` +T first(T* p) { return p[0]; } +int[3] ia; +pn(first(ia)); // 0 +``` + +### 2.14 `sizeof` and `typeof` + +3.12. + +## 3. expressions + +### 3.1 grammar levels + +``` +expr := binary [ "?" expr ":" expr ] | binary assign-op expr (right associative) +binary := unary { binop unary } (precedence climbing) +unary := prefix-op unary | cast | recast | postfix +postfix := primary { "[" expr "]" | "[" expr ":" expr "]" | "." name | "->" name + | "(" args ")" | "++" | "--" | "as" type } +primary := number | float | char | string | bstring | name | "(" expr ")" + | "{" block-expression "}" | "fn" closure | "$" "{" tokens "}" | keyword-expression +``` + +### 3.2 operator precedence and associativity + +3.2.1 from loosest to tightest. every binary level is left associative. + +| level | operators | associativity | +|---|---|---| +| 1 | assignment `=` and compound `*= /= %= += -= <<= >>= &= ^= \|=`; `?:` | right | +| 2 | `\|\|` (logical or) | left | +| 3 | `&&` (logical and) | left | +| 4 | `\|` | left | +| 5 | `^` | left | +| 6 | `&` | left | +| 7 | `==` `!=` | left | +| 8 | `<` `>` `<=` `>=` | left | +| 9 | `<<` `>>` | left | +| 10 | `+` `-` | left | +| 11 | `*` `/` `%` | left | +| 12 (prefix) | `-` `!` `!!` `~` `*` `&` `++` `--` casts and recasts | right | +| 13 (postfix) | `[]` `.` `->` `()` `++` `--` `as` | left | + +in the compiler's table the binary levels are numbered 1 to 10 (`||` 1, `&&` 2, `|` 3, `^` 4, `&` 5, equality 6, relational 7, shift 8, additive 9, multiplicative 10), and operators added with `run.add_binop` take any number. + +3.2.2 consequences, each verified: +``` +1 || 0 && 0 => 1 // && binds tighter than || +0 && 0 || 1 => 1 +1 | 2 && 0 => 0 // | binds tighter than &&: (1|2) && 0 +0 == 0 | 2 => 3 // == binds tighter than |: (0==0) | 2 +6 & 3 == 3 => 0 // == binds tighter than &: 6 & (3==3) = 6 & 1 +1 + 2 << 1 => 6 // + binds tighter than << +1 << 2 + 1 => 8 +2 < 3 == 1 => 1 // relational binds tighter than equality +1 ^ 3 & 2 => 3 // & binds tighter than ^ +7 & 3 ^ 1 | 8 => 10 // ((7&3)^1)|8 +10 - 3 - 2 => 5 // left associative +100 / 10 / 5 => 2 +1 < 2 < 3 => 1 // (1<2) < 3 +3 > 2 > 1 => 0 // (3>2) > 1 +0 ? 1 : 0 ? 2 : 3 => 3 // ?: nests to the right +a = b = 7 // assigns both, value 7 +-2 as int + 1 => -1 // as binds tighter than any binary operator +``` +`&&` and `||` are lower than `|`. this matches c. + +3.2.3 `as` is a postfix operator: it applies to the operand to its left and binds tighter than every binary operator, so `a + b as T` is `a + (b as T)`. a type after `as` takes following stars: `x as int * y` parses `int*` then fails. parenthesise. + +3.2.4 the condition of `?:` is a `binary` expression (so `a = b ? c : d` is `a = (b ? c : d)`), the middle operand and the last operand are full `expr` (assignments allowed). `a ? b : c = 5` therefore parses as `a ? b : (c = 5)`: with `a` 1 the assignment does not run. + +### 3.3 the binary operators + +`* / % + - << >> < > <= >= == != & ^ | && ||`. each has a compound assignment `op=` except `&&`, `||`, and the comparisons: `*= /= %= += -= <<= >>= &= ^= |=`. `x op= y` loads `x`, applies `op` with `y`, stores the result and has that value. a program may add operators (7.5); those with precedence 9 or more, or exactly 3, 4 or 5, also get an `op=` form. + +### 3.4 integer semantics + +3.4.1 `+ - *` and unary `-` wrap modulo 2^64. `0x7fffffffffffffff + 1` => -9223372036854775808. + +3.4.2 `/` truncates toward zero; `%` has the sign of the dividend: `7 / -2` => -3, `-7 / 2` => -3, `-7 % 2` => -1, `7 % -2` => 1. + +3.4.3 division by zero does not trap: `n / 0` => 0 and `n % 0` => n. `INT_MIN / -1` => INT_MIN and `INT_MIN % -1` => 0. + +3.4.4 shifts: the count is taken modulo 64 (its low six bits). `1 << 64` => 1, `1 << 65` => 2, `1 << -1` => 1 << 63, `5 >> -63` => 2. `>>` is an arithmetic shift (sign preserving): `-8 >> 1` => -4, `-1 >> 63` => -1, `-1 >> 64` => -1. `<<` fills with zeros. + +3.4.5 comparisons are signed, give 0 or 1, and `!`, `!!`, `&&`, `||` give 0 or 1. `!x` is 1 for 0 and 0 otherwise; `!!x` is 0 for 0 and 1 otherwise; `~x` is bitwise not; `& | ^` are bitwise on all 64 bits. + +3.4.6 the value of an `&&` or `||` is 0 or 1: `5 && 7` => 1, `0 || 9` => 1. + +3.4.7 `++` and `--` (prefix and postfix) add or subtract 1 as an integer add on the stored word, even for a float variable (so they change the float's bit pattern). the postfix form yields the old value. they need an assignable operand (error: ++ and -- need something to assign to). + +### 3.5 float semantics + +3.5.1 if either operand of `+ - * /` or of a comparison is a float, the other operand is converted to a float (an integer operand is converted by value), the operation is ieee binary64, and comparisons yield 0 or 1. `7 / 2.0` => 3.5; `1.0 / 0.0` => inf; `-1.0 / 0.0` => -inf; `0.0 / 0.0` => nan. an expression is a float if either operand is, or if it is a float variable, field, element, call result or literal; the compiler tracks this through locals, globals, fields, parameters, call results and returns, but not through untyped `word`/`let` copies of a float's bits. + +3.5.2 integer division of two integers is integer division even if the result is stored in a float: `float r = 7 / 2;` => 3.0. `%`, `&`, `|`, `^`, `<<`, `>>`, `!`, `~` are integer operations on the bits when an operand is a float. + +3.5.3 `(int)f` and `(float)i` convert by value. `f as int` and `i as float` keep the bits (3.9). `(int)-2.9` => -2. + +3.5.4 `-x` of a float is computed as `0.0 - x`, so `-0.0` evaluates to `+0.0`. do not depend on the sign of zero. + +3.5.5 `?:` with one float branch and one integer branch is broken in the reference implementation (appendix b). write both branches as floats. + +### 3.6 evaluation order and short circuit + +3.6.1 operands are evaluated left to right, arguments left to right. `a++ + ++a` with `a` 18 gives 18 + 20. `j + (j = 5) * 2` with `j` 1 gives 11. + +3.6.2 in an assignment `target = value`, the address of `target` is computed before `value` is evaluated: with `int i = 0; int a[3];`, `a[i] = i = 2` assigns `a[0]`. + +3.6.3 `&&` evaluates its right operand only if the left is nonzero; `||` evaluates its right operand only if the left is zero. a call in the right operand may therefore be skipped. +``` +int calls; +int seen(int v) { calls++; return v; } +// calls = 0; int r = seen(1) || seen(0) && seen(1); => r 1, calls 1 +``` + +3.6.4 `c ? a : b` evaluates exactly one of `a` and `b`. + +### 3.7 pointers and indexing + +3.7.1 `p + n` and `p - n` add or subtract `n` words, whatever `p` points at. `p - q` is the distance in words. `p[i]` is the element at word offset `i * sizeof(element)`; for a packed element type it is the byte at `i * bytes`. `*(p + 1)` and `p[1]` therefore differ when `sizeof(*p)` is not 1: +``` +// struct point { int x, y; } +point* p = rt.new(point, 3); +p[1].x = 5; +int* q = (int*)p; +pn(q[2]); // 5: p[1] starts 2 words (sizeof(point)) after p +pn((int)(p + 1) - (int)p); // 1: p + 1 moves one word +``` +3.7.2 the unary `*` dereferences a pointer. on a pointer to a struct or array it yields the aggregate, whose value is its address; `*(p + 2)` with `p` a `point*` evaluates to the address `p + 2`. `*` of a pointer to a sized type yields the first packed element. + +3.7.3 `&x` is the address of a variable, field or element. it applies to a local (which then lives in memory and cannot be an o3 variable), a global, a field and an element of a non-packed array. an element of a packed array has no address (2.3.3). + +3.7.4 indexing a new object (`rt.new(int)[0]`) is error: index a new object only after binding it to a name. indexing a `T[?]` is bounds checked (3.13). indexing a `T[n]` or a pointer is not checked at run time. + +3.7.5 an index must be a number (typecheck on). + +### 3.8 member access + +3.8.1 `e.name` and `e->name` are the same operator. if `e` has type `T*` (one level of pointer) or `T` (a struct), the field `name` of `T` is accessed. two pointer levels are an error (the `.` does not go through two). + +3.8.2 a field with the same name as a method wins over the method. + +3.8.3 `e.name(args)` calls a method (5.5.4) or an array intrinsic (3.13). a missing field or method is error: T has no field name; did you mean 'other'? (a suggestion is added when a close name exists). + +3.8.4 field access on an integer or `word` value is allowed when 2.7.4 permits. + +### 3.9 casts, recasts and `as` + +3.9.1 `(T)e` converts `e` to type `T`. the parser takes `(` as a cast only when what follows is a type name (or `fn`) and then, after at most two stars, `)`; or when the type name is followed by `[`, `<` or a function-parameter list. `(T)` where `T` is a variable name is a parenthesised expression. a cast between pointers or integers keeps the bits; `(int)` of a float truncates and `(float)` of an integer converts by value (3.5.3); a cast to a sized integer cuts the value (2.3.2); a cast to `f32` rounds. a cast of an array literal, `(int[]){1,2}` or `(T[?]){...}`, is the compound literal (2.6.5). + +3.9.2 `e as T` keeps the bits and changes the type: `x as char` is `x` unchanged, `3 as float` is the integer 3's bit pattern read as a float (0.000000 when printed). `x as u8` cuts the value to 8 bits. + +3.9.3 recasts `u*x`, `m*x`, `d*x` change the ownership state of `x` (6.7). they are recasts only when `*` follows the tag with no space and the tag names `u`, `m` or `d` are not variables in scope. `u * x` (with spaces) is multiplication by a variable named `u`. + +### 3.10 byte strings + +3.10.1 operations on `bstr` values: `+` concatenates (a new shared object), `== != < > <= >=` compare as unsigned bytes then by length, `s[i]` is the byte (0 when `i` is out of range), `s[i:j]` is a copy of bytes `i` up to but not including `j` with `i` clamped to 0 and `j` clamped to `len(s)` (and to at least `i`), `len(s)` is the length. `s[i]` is not an lvalue. mixing a `bstr` with a non-`bstr` operand in these operators is error: a byte string needs a byte string on the other side. + +3.10.2 `rt.put_s(b)` with a `bstr` argument writes the bytes. results of `+`, a slice, `rt.n_to_s` and `rt.new_s` are shared heap objects: `rt.free` does nothing on them and the collector reclaims them. + +### 3.11 calls + +3.11.1 `f(args)` calls a function by name, a function value (2.11), a closure, or an enhancement (5.10). the argument count of a function with a known signature must match: error: f: wrong number of arguments. an untyped call to a function not yet declared is resolved after the whole unit is read (a unit is scanned for signatures first, so calls may precede definitions). + +3.11.2 every function leaves exactly one word on the stack. a `void` function's value is 0 (`rt.put_n(f())` for a `void f()` prints 0). + +3.11.3 arguments are converted to the parameter type if it is `float`/`f32` or an integer type (2.9.1); sized parameters cut. at most 24 parameters; only the first 12 can carry tags. + +3.11.4 static calls: `S.f(args)` calls a `static` function of struct `S`; `S.CONST` reads a struct constant; `E.MEMBER` reads an enum member. `S` that is a type name used as a value otherwise is error: S: is a type, not a value. + +3.11.5 `new(T)` without `rt.` is accepted as `rt.new(T)`; the name `new` alone is the only rt name that resolves unqualified (any other, `put_c(65)`, is error: put_c is used but never defined; did you mean to use rt.put_c?). + +### 3.12 `sizeof` and `typeof` + +3.12.1 `sizeof(T)` and `sizeof(e)` give the size in words. the expression form is compiled but never run: `int i = 0; int n = sizeof(i++);` leaves `i` 0 and `n` 1. a `u8[100]` is 13 words, a struct of four `u8` fields is 1. + +3.12.2 `typeof(T)` and `typeof(e)` give the address of the type record (a word). the readers are in `run` (`run.tprint`, `run.tkind`, `run.tsize`, ...); they need `include run;`. `run.tprint(typeof(int(int, char*)))` prints `int(int, char*)`; `fixed u8[?]` prints `fixed u8[?]`. + +### 3.13 growable-array intrinsics + +3.13.1 `len(a)`, `cap(a)`, `push(a, x)`, `pop(a)`, `resize(a, n)` also exist as receiver calls `a.len()`, `a.cap()`, `a.push(x)`, `a.pop()`, `a.resize(n)`. `len` of a `T[n]` is the constant `n` and `cap` too. + +3.13.2 `len(a)` and `cap(a)`: the length, and the number of elements that fit before the next growth. a `T[?]` created by `rt.new(T[?])` has length 0; a minimum capacity of 4 is allocated, and capacity at least doubles when a push exceeds it. + +3.13.3 `push(a, x)` appends and grows; `pop(a)` removes and returns the last element; `resize(a, n)` sets the length, zero-filling new elements and dropping the rest (freeing owned elements). `push` and `pop` on aggregate element types are errors (push takes scalar or reference elements; use resize for aggregates). `push`, `pop`, `resize` on a `fixed` array: error: a fixed array can't grow or shrink. + +3.13.4 indexing: `a[i]` with `i < 0` or `i >= len(a)` prints `zy: index I out of range (length N)` on the output and stops the program (`rt.halt`, exit status 0). `pop` on an empty array reports the same with index -1. a constant index known to be out of range also produces a compile-time warning: `zc: line N: warning: out-of-bounds access: index 2 is outside length 2 (storage declared or allocated on line N)`, and the program still runs. +``` +int[?] a = {1, 2}; +push(a, 3); a.push(4); +pn(len(a)); pn(cap(a)); pn(a[3]); // 4 4 4 +pn(pop(a)); pn(a.pop()); pn(a.len()); // 4 3 2 +a[2]; // zy: index 2 out of range (length 2) +``` + +3.13.5 the length of a non-array (`len("text")` on a `char*`) is unspecified when `typecheck` is off (it reads word 0) and error: len needs an array under `typecheck on;`. use `rt.len_w` for word strings and `len` for `bstr`. + +### 3.14 constant expressions + +3.14.1 constant expressions are used for array lengths, enum values, `const` initializers, global initializers, `case` labels, `embed` array sizes and asm operands. the grammar is: number | character | named constant | `-` operand | `~` operand | `( constant )` | and the binary operators `+ - * | & << >>` only, evaluated left to right with no precedence. +``` +const X = 1 + 2 * 3; // 9, not 7 +const Y = 10 - 2 - 3; // 5 +const V = 1 | 2 + 4; // 7 +const W = ~0 & 255; // 255 +``` +other operators (`/ % ^ < ?:` and so on) are not part of a constant expression: `const Z = 6 / 2;` is error: expected ';', found '/'. + +3.14.2 the names allowed are global constants (`const`, enum members, `Enum.MEMBER`, `Struct.CONST` inside that struct as the bare name), global variables and arrays (their value is their address), and defined functions (their address). a local `const` is not visible to a constant expression: `const n = 5; int[n] a;` inside a function is error: expected a constant. + +### 3.15 closures + +3.15.1 `fn(a, b) { ... }` in an expression is a closure. parameters may be typed (`fn(int a)`) and tagged. `fn:T(a) { ... }` declares the result type `T` and coerces and checks returns against it. a closure is an owned heap object (a fresh object, 6.1): bind it to a local, pass it to a `.d` parameter or to a borrower, or return it from a `.u` function. + +3.15.2 a closure captures the locals and parameters of the enclosing function by reference (shared cells): later changes are visible both ways, and the cells outlive the creating function. +``` +int k = 10; +let cl = fn(x) { return x + k; }; +k = 20; +pn(cl(1)); // 21 +``` +3.15.3 closures cannot be created inside an o3 region (error: closures can't be made inside an o3 region). + +### 3.16 block expressions + +3.16.1 `{ stmts; tail }` in expression position is a block expression: it opens a scope, its value is the last expression without a trailing semicolon (0 if there is none). `return`, `break` and `continue` cannot leave a block expression (error: return can't leave a block expression, break can't leave ..., continue can't leave ...). a `{ }` group that has no `;` directly inside it is an array literal, not a block expression (`{ 1, 2, 3 }` and `{ 5 }` are arrays; `{ 5; }` is a block with value 0) (2.6.5). +``` +let sq = { let t = 3 + 4; t * t }; // 49 +``` +3.16.2 ownership: an owned local may leave as the block's value only if it is `u` (`{ int* t = rt.new(int); u*t }`). a plain `m` local as the tail is error: t: has to be u to be handed up as the block's value. see 6.8. + +### 3.17 assignment + +3.17.1 `lvalue = expr` stores and yields the stored value. lvalues: a local or global variable, a field, an element, a dereference. otherwise error: this can't be assigned to. an aggregate cannot be assigned (2.6.6). a store into const storage is an error (2.10). the analyzer's rules for what an assignment means for ownership are in 6. + +3.17.2 compound assignment `x op= y` is `x = x op y` with `x` evaluated once. on a local that owns an object it is error: x: owns an object; compound assignment would lose it. + +### 3.18 unary operators + +`-e` (negation; on a float it is `0.0 - e`), `!e`, `!!e`, `~e`, `*e`, `&e`, `++e`, `--e`, `(T)e`, and a program's own unary operators (7.5). `-` of a literal is folded. + +## 4. statements + +### 4.1 declarations + +4.1.1 typed: `T a = e, b, c[10];`. `T` is any type (2.2) optionally followed by a tag (`T*.u`). a declarator may be followed by `[n]` which makes an array of `n` (an untyped array when the type is omitted). an initializer is an expression, or for an array or `T[?]` a brace list. + +4.1.2 `let a = e, b[10];` declares without a type. the type is inferred from the initializer: `char*` from a word-string literal, `bstr` from a byte-string literal, a float local from a float initializer, the pointer or function type of the initializer otherwise, and a plain word for integers. `let a[10];` is an untyped array of 10 words (type `word[10]`). + +4.1.3 a declaration with only a tag, `.u r = e;`, declares an untyped local with that ownership tag. + +4.1.4 `const k = e;` declares a local constant: no type is written, it takes the initializer's type (read-only, 2.10), needs an initializer (error: a const needs an initializer), and cannot be assigned (error: const storage cannot be modified). `const T k = e;` is a typed const. + +4.1.5 a local with no initializer is uninitialised: its value is whatever the frame held. a global with no initializer is 0. an array or struct local has no zeroing. +``` +int b; pn(b); // unspecified value +``` +4.1.6 a local may shadow a local of an outer block, a parameter, a global or a function name. redeclaring a name in the same block is accepted and the new declaration hides the old one from that point. + +4.1.7 `void` cannot be the type of a variable (error: a variable can't be void). + +4.1.8 a local declaration is scoped to its block; a `for` initializer declaration is scoped to the loop (a use of `i` after `for (int i...)` is error: i is used but never defined). + +### 4.2 expression statements and blocks + +an expression followed by `;` is a statement; its value is dropped. a fresh owned value thrown away is error: a new object (or a u result) can't be thrown away. `;` alone is empty. `{ ... }` is a block with its own scope. + +### 4.3 if + +`if (cond) stmt [else stmt]`. the condition is any scalar; a float is false when `+0.0` or `-0.0`. each branch is its own block even without braces. `else` binds to the nearest `if`. the two branches of an if are analysed from the same starting state and joined (6.10). + +### 4.4 while + +`while (cond) stmt`. tests before each pass. `while (1)` (a literal nonzero followed by `)`) is an infinite loop for the analyzer: the only exits are `break`/`return`. + +### 4.5 do + +`do stmt while (cond);` runs the body, then tests. `continue` jumps to the test. +``` +int n = 0, i = 0; +do { i++; if (i == 2) continue; n = n + i; } while (i < 5); +pn(n); // 13 +``` + +### 4.6 for + +`for (init; cond; step) stmt`. `init` is a declaration (typed or `let`) or an expression or empty; `cond` empty means true; `step` is an expression or empty. `continue` runs `step` then tests. the loop variable's scope is the loop. + +### 4.7 switch + +4.7.1 `switch (e) { case c1, c2: stmts ... default: stmts }`. the labels are constant expressions (3.14) evaluated at compile time, comma separated. cases do not fall through: when a case's statements end, control leaves the switch. `break` inside a switch leaves it early; `continue` inside a switch continues the enclosing loop. `default:` (and `case _:`, a catch-all with the same meaning) runs when no label matches, wherever it is written. a switch can have only one `default`/`_` (error: a switch has one default, a switch has one catch-all case). +``` +switch (k) { case 1, 2: ps("one-two"); case 3: ps("three"); break; default: ps("d"); } +// k = 2: prints one-two only. no fallthrough into case 3. +``` +4.7.2 duplicate labels are not an error: the first matching case in source order runs. a non-constant label is error: expected a constant. the statements of a case form their own block. + +4.7.3 tagged enums. when `e` has type pointer-to-tagged-enum, `case member(a, b):` matches the member and binds each payload field, in order, to a new local scoped to that case; the name count must equal the member's field count (error: a case binds every field of its member). `case member:` or `case member():` with no names matches by tag. a switch on a tagged enum with no `default` that omits a member prints a warning `zc: line N: warning: this switch doesn't cover M1, M2` (not an error; compile continues). +``` +shape* s = shape.rect(4, 5); +switch (s) { case circle(r): pn(r); case rect(w, h): pn(w * h); case dot: ps("dot"); } // prints 20 +``` + +### 4.8 break, continue, return + +4.8.1 `break;` leaves the innermost loop or switch; outside both: error: break outside a loop or a switch. `continue;` goes to the next iteration of the innermost loop (a `for` runs its step first; a `do`/`while` jumps to the test); outside a loop: error: continue outside a loop. + +4.8.2 `return e;` returns the value (converted to the result type if the function has a float or integer result type). `return;` returns 0. falling off the end of a function returns 0. `return` outside a function is an error. `return` also frees every owned local of the scopes it leaves (6.5) and runs the ownership return rule (6.8). + +4.8.3 a function whose declared result is `void` accepts `return e;` when `typecheck` is off (the value is ignored); under `typecheck on;` it is error: type error: return value: expected void, got int. + +### 4.9 unchecked + +`unchecked stmt` runs the statement with the analyzer off (6.12). `unchecked;` at top level switches the analyzer off for the rest of the unit. `unchecked fn f() {}` makes one function unchecked. code in an unchecked block cannot touch a local that the analyzer tracks (error: x: is tracked by the analyzer and can't be used in unchecked code). + +### 4.10 o3 + +4.10.1 `o3 { ... }` marks a block, `o3 T x = e;` one variable, and `o3 T f(...) { ... }` / `o3 fn f(...)` a function, for the optimising tier. o3 changes speed, not meaning, subject to these restrictions on code in an o3 region: +- every variable and parameter must be typed (errors: o3 variables need a type, o3 parameters need a type). +- no address-of on an o3 variable and no naming one in `asm` (errors: an o3 variable has no address, asm can't reach an o3 variable). +- no closures (3.15.3). +- an untyped `rt.new(n)` is `rt.new_oz` (zeroed, no type stamp). +4.10.2 an `o3 { }` outside a function is an ordinary block; a nested `o3` is an ordinary block. `o3` appears before the return type of a function (`o3 int sum(...)`). +``` +o3 int sum(fixed int[?] a) { int s = 0; for (int i = 0; i < len(a); i++) s += a[i]; return s; } +``` + +### 4.11 memdebug + +`memdebug on;` turns on heap header checks and allocation-site tracking from this point in the file (top level) or from this point in the function (statement); the runtime enables the checks before `main` for the top-level form. `memdebug scrub;` additionally fills freed payloads with 57005; `memdebug off;` stops. with checks on, a use after free prints `zy: use after free from line N`, a double free prints `zy: double free from line N` and returns, a stray address prints `zy: bad heap pointer`. `rt.memreport()` prints a line `leak: line N W words` per live recorded object and returns the count. (`memdebug scrub;` is rejected by the reference implementation, appendix b.) + +### 4.12 hints + +`hints on;`, `hints small;`, `hints off;` set how check mode (the language server) shows non-error findings from that point on. they have no effect on a normal run or on code generation. + +### 4.13 asm + +4.13.1 `asm { tokens }` emits raw vm words and is both a statement and an expression. in an expression it leaves one value; the compiler assumes the block's stack effect and treats the result as an untyped word. tokens: a number pushes it, `-n` pushes a negative (the `-` must touch the digits), `#n` emits the raw word `n`, a one-character token from the mnemonic table `q@!$_~cjzr,?.+-*&|^><%` emits that opcode, `:name` defines a label local to the block, a later label name pushes its address, a float literal emits `flit` and its bits, a string pushes its address, the name of a local pushes its address, the name of a global or constant pushes its value or address, a function name pushes its address, an enhancement name emits its opcode. an unknown name is error: unknown name in asm. +``` +int x = asm { 2 3 + }; // 5 +int y = 7; asm { y @ 1 + y ! } // y is 8 +``` + +## 5. declarations at the top level + +### 5.1 items + +5.1.1 a unit is a sequence of items. an item is: a function definition or prototype, a global variable, `const`, `struct`, `type`, `enum`, `include`, `embed`, `namespace`, `typecheck`, `docs`, `memdebug`, `hints`, `enhance`, `extern`, `unchecked;`, or a use of a top-level parse word (7.2). + +5.1.2 `struct`, `type`, `typecheck`, `docs`, `namespace`, `enhance` and `extern` are valid only at the top level. `enum`, `embed`, `memdebug`, `hints` and `const` also appear as statements in functions. `include` is top-level only (5.12.7). + +5.1.3 the unit is scanned once for function signatures before it is compiled, so calls may precede definitions and prototypes are optional (an untyped function called before its tags are declared that later declares `.m`/`.d` tags is error: f: was called before its tags were declared; put a prototype above the first call). + +### 5.2 functions + +5.2.1 forms: +``` +int f(int x) { ... } // typed +void log(char* s) { ... } // no result +node*.u make() { ... } // owned result (.u) +.u grab() { ... } // tag alone: untyped, owned result +fn f(a, b) { ... } // untyped; types are inferred +fn f(int a, b) { ... } // parameters may mix +T first(T* p) { ... } // generic +int proto(int n); // prototype (parameter names are required) +``` +a bare `name(params) { ... }` with no `fn` and no type is not a function declaration (error: expected a declaration). + +5.2.2 parameters: `T name`, `T*.m name`, `T*.d name`, `.m name`, `.d name`, `name`. up to 24; only the first 12 can carry tags (error: only the first 12 parameters can be tagged). a parameter of struct or array type is error: structs and arrays are passed by pointer (a parameter of array type decays to a pointer). + +5.2.3 the result: `void` or a type, optionally `.u`; the only return tag is `.u` (error: the only return tag is .u). a function with no result type returns a plain word. falling off the end returns 0; a `.u` function that falls off the end is error: a function returning .u has to end with a return. + +5.2.4 untyped functions (`fn`) infer parameter and result types by propagating constraints from calls and returns until a fixed point (README, 32-inferred-calls); the exact algorithm is not specified here. typed and untyped functions can be called interchangeably. + +5.2.5 `main`. after the unit compiles without errors, the runtime calls `main` with no arguments (declared parameters read unspecified values; `main(a, b)` prints 0 0 in the reference implementation). its return value is ignored and is not the exit status. 8.1. + +5.2.6 a function is defined once. redefining a name that already has a body is, per the rule, error: f: is defined twice; the reference implementation does not detect an ordinary redefinition (appendix b). a prototype must agree with the definition: error: f: doesn't match its earlier declaration. + +5.2.7 `immediate fn` defines a parse word (7.2). `unchecked fn` and `o3 fn` are 4.9 and 4.10. + +### 5.3 globals + +5.3.1 `T a = e, b, c[10];`, `let g = 1, buf[10];`, `int[10] buf;`, `point origin;`. storage is emitted in place, zero initialised. + +5.3.2 an initializer is: a constant expression (3.14), a word string, a byte string, or a brace list or compound literal for an array. anything else is an error: for `int a = f();` error: expected ';', found '('. a float literal initializer is rejected (error: expected a constant) in the reference implementation (appendix b); assign floats in code. a name of another global in the initializer evaluates to its address, not its value: `int a = 5; int b = a + 1;` gives `b` the value `&a + 1`. + +5.3.3 a global cannot be tagged (error: tags go on locals and parameters, not globals), cannot be `void`, and an array global needs a length or an initializer. + +5.3.4 owned objects stored in a global are collector-managed (6.9). + +### 5.4 constants + +`const A = 1, B = A + 2;` and `const T A = 1;` (the type is read and dropped). the value is a constant expression (3.14) of the earlier constants; an untyped constant has no storage. + +### 5.5 structs and methods + +5.5.1 `struct name { members };` (the trailing `;` is optional). members: +- fields: `T a, b;` or `let a, b;` (word fields). +- constants: `const MAX = 24, MIN = 1;` (or `const T MAX = 24;`), constant expressions, take no space. read as `name.MAX` from outside, and as the bare name `MAX` inside the struct's methods and static functions. a constant initializer that is not constant is error: a struct constant must be a constant expression. a duplicate is error: MAX: is defined twice in this struct. +- methods: `fn name(params) { body }` or `T name(params) { body }`. `unchecked` may precede. +- static functions: `static fn name(params) { body }` or `static T name(params) { body }`. +a field of a struct cannot have a tag. + +5.5.2 a method has a hidden first parameter `this`, of type pointer to the struct. fields are reached only through `this` (`this.x`; the bare name `x` is not in scope). declaring `this` is error: this is implicit and cannot be declared as a method parameter. methods are compiled after the struct is complete, so a method can call methods declared after it. + +5.5.3 a method is an ordinary function named `Struct.method` whose first parameter is `Struct* this`. + +5.5.4 `e.name(args)` calls method `name` with `e` as `this`, unless the struct has a field `name` (a field wins). when `e` is an integer or `word` and exactly one struct has a method of that name, that method is called (error if several: name: is a method of more than one struct). + +5.5.5 a static function has no `this` and is called `Struct.name(args)`. + +5.5.6 `Struct.MAX`, `Struct.name(...)` with a misspelled member: error: MAXX: is not a static function of this struct; did you mean 'MAX'? + +5.5.7 receiver extensions: `run.method("member", function);` registers a library function whose first parameter is the receiver, so `a.member(x)` calls `function(a, x)` for arrays and other typed receivers (7.7). `len cap push pop resize` are built-in receiver methods of arrays. + +### 5.6 type aliases + +`type name = T;` (2.2.4). + +### 5.7 generics + +2.13. + +### 5.8 enums + +2.8. + +### 5.9 `typecheck` + +`typecheck on;` / `typecheck off;` (2.9.2). top level only. + +### 5.10 enhancements + +`enhance [n] name(a, b) micro "recipe";` or `... vm { asm } [else fn];` installs a virtual opcode at compile time. `n` is the number of outputs (default 1). an enhancement is called like a function when it has one output (`name(x, y)`), or by name from `asm`. the argument count must match (error: name: wrong number of inputs). at most 8 inputs, at most 512 enhancements. `/`, `%` and `<<` compile to the runtime's `idiv`, `imod` and `ishl` enhancements. the recipe language and the vm-code form are described in README ("virtual enhancements"); they are outside this spec. + +### 5.11 embed + +`embed "path" as [type] name;` declares `name` with the bytes of a file read at compile time. the path is tried next to the including file, then in the workspace folder, then relative to the working directory. forms: +- no type or `bstr`: a byte string of the whole file. +- `char*`: a word string, one byte per word. +- a number type: the file's bytes little-endian, zero-extended (sign-extended for `i8 i16 i32`); an integer type takes at most its size; `float`/`f64` needs exactly 8 bytes and `f32` exactly 4 (error: an embedded float takes exactly as many bytes as its type). +- `T[]` or `T[n]` of a number type: elements packed for sized types and one byte per element for `char[]`; the size must be a whole number of elements (error: the embedded file's size is not a whole number of array elements) and fit `T[n]` (error: the embedded file is larger than the array it is declared as). +- anything else: error: embed declares a bstr, a char*, a number, or an array of numbers. +a missing file is error: can't read the file named by embed. at top level `name` is a global; in a function it is a local (an array is copied into the frame each time the declaration runs). + +### 5.12 include and namespace + +5.12.1 `include name;` or `include a, b, c;` loads module `name`. the modules `rt` (the runtime, always resident), `run` (the compiler's own functions, always resident), `core`, `lsp` (resident) and `std`, `icon` (embedded libraries compiled on first include) need no file. any other name `x` is read from the file `src/lib/x.zc`, found relative to the including file's directory, then the workspace folder, then the working directory. + +5.12.2 `include "path";` reads a file and compiles it at that point, signatures first. + +5.12.3 `namespace name;` makes the declarations that follow in this file belong to module `name` (their global names become `name.x`). `namespace;` returns to the global namespace. a file that is a library starts with `namespace name;`. a program that is run alone does not need `namespace;`. + +5.12.4 imports are per file. a name `m.x` is accepted only if the current file has `include m;` (the module's own file counts as importing itself). otherwise error: module access requires `include m;`. includes are not transitive: if `a` includes `b`, a file that includes `a` does not get `b`. the compiler reports each missing module once per unit. +``` +// util.zc: namespace util; int twice(int n) { return n * 2; } +// a program with include rt, mid; where mid includes util: +rt.put_n(util.twice(1)); // error: module access requires `include util;` +``` +5.12.5 a file included by path (`include "util.zc";`) contributes its declarations under its own namespace; its bare names are not imported into the includer (error: twice is used but never defined; did you mean to include util and use util.twice?). + +5.12.6 in a namespaced file, a declaration `f` is `ns.f`. an unqualified name is looked up first in the file's namespace, then globally, and `rt` is not searched for bare names (3.11.5). + +5.12.7 `include` is parsed at the top level; written inside a function body it mis-compiles (appendix b). + +5.12.8 when a library is included more than once, it is compiled once. + +### 5.13 extern and cfunc + +5.13.1 `extern RET name(PARAMS) [= "cname"];` declares a native (c abi) function callable as an ordinary zc function; `cfunc(f)` gives a native function pointer that runs the zc function `f`. the c types accepted are `void`, `char short int long` and unsigned forms, `size_t ssize_t intptr_t ptrdiff_t off_t time_t` and the `intN_t`/`uintN_t` names, `float double`, `str` (a byte string passed as the address of its bytes), `T*` (a byte address; `word *` is a word address), and `struct name` for a zc struct of one-word fields (by value, up to 7 words returned). up to 15 arguments. an unknown c type is error: isn't a c type extern knows; a struct that is not a zc struct is error: isn't a zc struct. this needs host ffi support (the microcore answers -1). the full description is in README ("c abi"); outside the rest of this spec. + +### 5.14 docs and hints + +`docs on;` keeps doc comments of what follows in the image; `docs off;` stops. no effect on a program's behaviour. + +## 6. ownership and lifetimes + +this section describes the analyzer: a pass that runs inside the compiler on every function (including code compiled at run time by `run.eval` and friends), tracks heap objects through local variables, inserts frees, and reports violations. it can be switched off (6.12). what it proves is limited to what is stated here. + +### 6.1 vocabulary + +- an object is a heap block returned by `rt.new` or a function with an owned result. +- a fresh expression produces an object nobody owns yet. the fresh expressions are: a call to `rt.new` (all forms), a call to a function whose result is owned (`.u` declared, or inferred, 6.8), a closure `fn(...) {...}`, `pop` of a reference-element array, `quote`/`$`, and a block expression that hands up a `u` local. +- the analyzer tracks state for each local variable and parameter, called its ownership state. + +### 6.2 states of a local + +| state | meaning | +|---|---| +| plain | not tracked: numbers and anything the analyzer does not follow; also a lent (untagged) parameter | +| `m` | the local owns an object that belongs to this block; the compiler frees it at block exit | +| `u` | the local owns an object that must leave this block: by return, as the block's value, or by assignment to a variable of the directly enclosing block | +| `d` | the local owns an object that must be given away (transferred to a lower scope or a consumer) before the block ends | +| dead | the object was given away, freed or moved; any later use is error | +| child | the local is owned by a container (it was stored into one), or is an alias (borrow) of another local's object | +| borrow | a `.m` parameter: the callee may use the object but not keep or free it | + +a local that owns an object (`m`, `u`, `d`) has the `own` flag. an alias is a child state with no own flag; `lroot` follows it to the owner. + +### 6.3 where a local gets a state + +6.3.1 initializing or assigning a fresh expression to a plain local makes it `m`. if the declaration carries a tag (`int*.u x = rt.new(int);`) it gets that state instead. + +6.3.2 a `.d` parameter is `m` inside the callee (it owns the object and frees it at the end). a `.m` parameter is a borrow. an untagged parameter is lent (plain); the compiler infers how a checked body uses it (6.6). + +6.3.3 `u*x` sets `x` to `u`, `d*x` sets `x` to `d` (a recast, 6.7). a function call result with an owned return is fresh. + +6.3.4 a pointer computed from a tracked local, `p + k`, `p.field` or `p[i]` (when the field or element is a pointer), `&p[i]`, or the result of a call that returns an alias of its argument (6.6.3), is stored in the local it lands in as a child alias of `p`: using the alias after `p` is dead is error. the alias is not an owner and is not freed. + +6.3.5 storing an owner into a container (`p.f = c`, `p[i] = c`, `push(a, c)`) makes `c` a child of `p`: it is freed with `p`, it follows `p` when `p` is given away, and it cannot be recast or given away on its own. + +### 6.4 what each tag means + +at a parameter: +- untagged: a lend. the caller keeps ownership. the callee may read and write through it. a checked callee that stores the pointer somewhere that outlives the call, or consumes it, is an error at the call when the argument is a named owner or a fresh temporary: this callee retains or consumes a borrowed owner; declare its parameter .d to transfer ownership. +- `.m`: an explicit borrow contract. the callee may not keep, free or return it as owned. storing a `.m` parameter in a global is error: p: is not an owner, so it can't be stored in a global or in untracked memory (the collector takes it over). +- `.d`: the callee takes ownership; the caller's variable is dead after the call, even without a recast: `take(x)` is `take(d*x)`. + +at a result: `.u` means the result is a fresh object the caller owns (and must bind, return or consume). a function without `.u` that returns an owned local or a fresh object is inferred owned (6.8). a function not `.u` returns a plain pointer. + +on a local declaration: the tag is the initial state when a fresh object lands in it. + +### 6.5 freeing + +6.5.1 every `m` local is freed (`rt.recycle`: the object and, recursively, the objects it owns) at the end of the block where it was declared, and on every early exit from that block (`return`, `break`, `continue`), in reverse order of declaration. a user does not call `rt.free` on a tracked local that goes out of scope; an explicit `rt.free(x)` frees early and makes `x` dead. + +6.5.2 an owned object that is never used in its block is error: x: owned object is never used. + +6.5.3 a free of a container frees its children. `rt.free(p)` takes its argument as `.d`. + +6.5.4 a fresh result passed to a parameter that the callee provably only reads (`rt.put_s(message())`) is held in a hidden frame slot and freed after the full expression. nested calls, short-circuit branches, `?:`, loops and early returns all release it. passing a fresh value to a retaining callee is error: a temporary argument requires a proved non-escaping borrow; this callee retains or consumes it, so transfer ownership through a .d parameter; to an unchecked, external or indirect callee: ... bind it to a name first or provide an explicit parameter contract. + +### 6.6 borrowing and inference + +6.6.1 for checked functions the compiler records, for each parameter, whether the body reads it, writes through it, retains it (stores it where it outlives the call), consumes it (frees or gives it away), or returns it or an alias of it. the summaries are propagated through calls to a fixed point, including recursion. unknown bodies (unchecked, external, indirect calls, raw asm, integer-to-pointer casts) are treated as unknown and conservative. + +6.6.2 a read or write through an untagged parameter leaves the caller's variable `m` and usable after the call. +``` +void show(int* p) { rt.put_n(*p); } +int* q = rt.new(int); *q = 3; +show(q); show(q); // prints 3 3; q freed at block end +``` +6.6.3 a function that returns its parameter, or a pointer computed from it, makes the result an alias (borrow) of the argument: the result is tied to the argument's owner, and using the result after the owner is freed or given away is error. the result must come from a function with no owned result. when several arguments have different owners, no single owner is assumed and the result is untracked. +``` +char* tail(char* s) { return s + 1; } +char* b = rt.new(4); b[0] = 'a'; b[1] = 'b'; b[2] = 0; +char* t = tail(b); +rt.free(b); +rt.put_s(t); // error: t: used after it was given away; owner b was transferred on line N (declared on line M) +``` +placing `rt.free(b)` after the use is accepted (the alias's last use precedes the transfer). + +6.6.4 a checked function cannot return an alias of a local owner (error: a borrow of a local owner cannot escape through return; return the owner). it can return an alias of a parameter (the caller owns it), but not as an owned result of `.u` (error: a borrowed parameter cannot be returned as an owned result; a borrowed projection cannot be returned as an owned result). + +### 6.7 recasts + +`u*x`, `d*x`, `m*x` (no space between tag and `*`). +- `u*x`: x becomes `u`. it then must leave the block (6.8). a block ending with `u` still held is error: x: u object must leave this block (block value, assignment to the enclosing block, or return). +- `d*x`: x becomes `d`: x must be given away before the block ends, as an argument to a `.d` parameter, by `rt.free`, or stored into a global. a `d` left over is error: x: d object was never given away. a `d` object used in an inner block is moved into that block, where it is `m` and freed at its end. +- `m*x`: sets state `m`; legal only on a local that is already `m` (error otherwise: x: can't be recast back to m, e.g. after `d*x`), or on a fresh expression (`m*rt.new(4)` is allowed and lets the result be discarded), or to lend a local to a global (6.11). +- a recast of a local that owns nothing is error: x: isn't an owned object, so there is no ownership to recast. a recast of an expression with no owner: only a named owned object can be recast; this expression has no owner. a recast of a child: x: belongs to a container; recast the container instead. of a borrow: x: is borrowed and can't be recast. of a dead object: x: used after it was given away. + +### 6.8 transfer, return, block values + +6.8.1 passing an `m`/`d`/`u` owner (that has the own flag) to a `.d` parameter transfers it: the local becomes dead and the transfer line is remembered for messages. passing an `m` to a `.d` parameter needs no recast. passing a `d` object to a plain or `.m` parameter is error: x: is d; pass it to a d parameter (lending it would lose it). passing a child or borrow to a `.d` parameter: x: belongs to a container and can't be given away alone; x: is borrowed and can't be given away; x: is u; write d*x to give it away instead of returning it (for a `u`). + +6.8.2 passing a fresh value: to a `.d` parameter it is given directly; to any other parameter it is a temporary (6.5.4) when the callee is a proved borrower; otherwise error: a new object can only go straight to a d parameter; bind it to a name first. + +6.8.3 `return x;` where `x` is `m` with the own flag, or `d`, or `u`: the object leaves with the return and `x` becomes `u` then dead. a function with a `.u` result requires it; a function without `.u` that returns an owned local or a fresh object becomes an owned-result function from that point (its signature gets the owned bit), and callers compiled after it see that. a call compiled before the body is known gets error: f: is called before its body, so its owned return can't be known there; declare the return type with .u. returning fresh from a function that is not allowed to own: error: returns a new object; give the function a .u return type. returning an owner from a function that cannot own: x: is owned and can't be returned from a function whose return type isn't .u. a `.u` function returning a non-`u` owned: x: has to be u to be returned. + +6.8.4 a block expression's value (3.16): the tail may be a `u` local of that block (it is handed up and its state becomes dead, the expression is fresh). an `m` tail is error: x: has to be u to be handed up as the block's value; a `u` of an outer block: x: only a u object of this block can be its value. + +6.8.5 assignment of an owner into another local. `a = b` where `b` is an owner: `a` must be empty (plain or dead). three cases are accepted: (1) `b` is `u` and `a` belongs to the block directly enclosing `b`'s block (hand up one level: `b` is dead, `a` is `m`); (2) `a` is dead with the own flag (it was freed or given away) and `b` is `m` with the own flag in the same block or a deeper one: `rt.free(a); a = b;` moves the object, `b` and its aliases expire; (3) `b` is a borrow or alias: `a` becomes an alias. otherwise: x: is an owned object and can't be copied into another variable; or, for a `u` to anything but the enclosing block: x: is u; it can only be handed to a variable of the enclosing block, or returned. overwriting a variable that still owns: x: still owns an object; overwriting it would leak it. + +6.8.6 an `m` local stored in a `T[?]` or container is a child (6.3.5): `push(a, x)` is a store with tag-match rules: the tag of the stored local must equal the container's (both `m`, `u` or `d`), the stored local must not live in an outer block than the container (error: ... lives in an outer block than the container it's stored into), and a borrowed container takes the child by giving it away. messages: c: its tag doesn't match its container's (both have to be m, u or d); c: already belongs to a container; c: is borrowed and can't be stored; c: is u; it has to leave through the block above, not into a borrowed object. + +### 6.9 globals, untracked memory and the collector + +6.9.1 globals, plain pointers and arrays are not tracked. storing an owner in a global needs a recast: `g = d*x;` (or a fresh object directly: `g = rt.new(int);`). storing a non-owner (a borrow, an alias, a `.m` parameter) in a global is error: p: is not an owner, so it can't be stored in a global or in untracked memory (the collector takes it over). + +6.9.2 after `g = x;` of an owner, `x` is dead (error on use: x: used after it was given away). + +6.9.3 objects reachable only through globals or unchecked code belong to the conservative mark-sweep collector (`rt.gc()`), which also reclaims quarantined and shared objects. + +6.9.4 an array literal (`(int[]){1,2}`) is a stack temporary. it cannot escape into a global, container or return (errors: an array literal cannot escape into a global or container; an array literal cannot escape its function through return) and cannot go to a `.d` parameter (an array literal cannot transfer ownership to a d parameter). + +### 6.10 control flow joins + +6.10.1 after an `if`/`else`, `?:`, `switch` or loop exit, every local must be in the same state on every path that reaches the join (a path that returned or broke does not reach it), with these exceptions: plain and dead merge to dead; a variable that is dead or `d` on some paths and dead or `m`/plain on the others becomes dead, and the compiler frees it on the paths where it was still alive, on their way in. a `u` or borrow does not get the dead treatment: it must leave. otherwise error: x: ends the branches of an if in different states (one path leaves it m, another given away); for `?:`: ends the two sides of ?: in different states; both sides of `?:` have to be fresh, or neither (error: both sides of ?: have to be new objects, or neither). + +6.10.2 loops: an object cannot be given away on a path that comes back to the top of the loop (error: x: is given away inside a loop that comes back around to it; give it away on a path that leaves the loop). any other state change over a pass: x: changes state inside a loop; each pass has to leave it as it found it (it went from A to B). giving away and then `break` is accepted. a loop's exit state is the join of its `break` states and its normal exit. + +6.10.3 `switch`: every case is analysed from the state at the top of the switch and joined. + +### 6.11 lends to globals + +`g = m*x;` where `x` is an `m` local lends `x` to the global `g`: `x` keeps ownership and stays usable; the lend ends when `g` is assigned again in the same block (the restore must be unconditional; an assignment inside a nested branch does not end it). while the lend is open, giving `x` away or freeing it is error: x: is lent to global g; assign the global again before giving the local away; leaving the block (end, `return`, `break`) with the lend open is error: x: is lent to global g; assign the global again before the block that lent it ends. +``` +word saved = rt.outbuf; +rt.outbuf = m*buffer; +render(); +rt.outbuf = saved; +``` + +### 6.12 unchecked + +`unchecked fn`, `unchecked stmt`/`unchecked { }` and `unchecked;` turn the analyzer off for that code: no state tracking, no inserted frees, no errors from the rules above. what unchecked code allocates belongs to the collector (or to an explicit `rt.free`). checked locals (tracked) cannot be used from unchecked code (4.9). calling an unchecked function with an owned or fresh argument is treated as an unknown callee (6.5.4). + +### 6.13 the error messages + +each fixture in `tests/errors/` starts with `// expect: MESSAGE`; the messages that the analyzer produces are (X is a variable name): + +- X: used after it was given away; owner Y was transferred on line N (declared on line M) +- X: owned object is never used +- X: still owns an object; overwriting it would leak it +- X: is an owned object and can't be copied into another variable +- X: u object must leave this block (block value, assignment to the enclosing block, or return) +- X: d object was never given away +- X: is u; it can only be handed to a variable of the enclosing block, or returned +- X: is d; pass it to a d parameter (lending it would lose it) +- X: is m; write d*x to give it away (an m object with no own flag) +- X: is not an owner, so it can't be stored in a global or in untracked memory (the collector takes it over) +- X: isn't an owned object, so there is no ownership to recast +- X: can't be recast back to m; belongs to a container; recast the container instead; is borrowed and can't be recast +- X: its tag doesn't match its container's (both have to be m, u or d) +- X: ends the branches of an if in different states +- X: is given away inside a loop that comes back around to it +- X: is lent to global G; assign the global again before ... +- X: is tracked by the analyzer and can't be used in unchecked code +- X: owns an object and can't be incremented; owns an object; compound assignment would lose it +- a new object can't be used as a plain value; bind it to a name first +- a new object (or a u result) can't be thrown away +- a new object can only go straight to a d parameter; bind it to a name first +- use a field of a new object only after binding it to a name; index a new object only after binding it to a name +- a borrowed projection cannot transfer ownership; transfer its owner +- a borrowed temporary cannot escape its expression (into a variable or container / through return) +- a borrow of a local owner cannot escape through return; return the owner +- a borrowed parameter cannot be returned as an owned result +- a temporary argument requires a proved non-escaping borrow ... +- this callee retains or consumes a borrowed owner; declare its parameter .d to transfer ownership +- an array literal cannot escape ... (into a global or container / its function through return) / cannot transfer ownership to a d parameter +- element borrow was invalidated by array mutation on line N; borrow created on line M (after `push`, `pop`, `resize` while an element address is held) +- f: is called before its body, so its owned return can't be known there; declare the return type with .u +- a function returning .u has to end with a return + +the `tests/errors/*.zc` files are exact reproductions; a verifier can run `bash tests/runerr.sh build/bin/zy tests/errors`. + +### 6.14 warnings + +the compiler also prints non-error findings as `zc: line N: warning: ...` (a switch that does not cover a tagged enum, a constant index out of range of a `T[?]`, a use after free provable at compile time, ...). warnings do not stop the program. + +## 7. compile-time execution + +the compiler lives in vm memory with the program. functions and globals of the compiler are callable through the module `run` (`include run;`). + +### 7.1 `run` access + +`run.next()`, `run.peek()`, `run.peekat(k)` (k from 0 to 3: up to four tokens ahead), `run.tk` and `run.tv` (current token kind and value), `run.tline`, `run.expect(s)`, `run.accept(s)`, `run.intern(text)`, `run.expr()`, `run.rval()`, `run.plainuse(c)`, `run.stmt()`, `run.substmt()`, `run.ptype()`, `run.emit(w)`, `run.lit(n)`, `run.op(k)`, `run.hole()`, `run.setlit(at, v)`, `run.here`, `run.callname(name)`, `run.emitseq`, `run.useq`/`run.ulen`, `run.grab()`, `run.feed(q)`, `run.tokens(s)`, `run.qdup(q)`, `run.serr(msg)`, and the type readers `run.tkind tsize tname tlen tnfields tfname tftype tfoff tret tnparams tparam tparamtag tretu tnargs targ tprint`, `run.sigof(f)`, `run.sigofs("name")`, `run.fatten(p, t)`. the token kinds are `run.T_EOF T_NUM T_ID T_STR T_OP T_FLT T_BSTR`; the type kinds are `run.TY_*`. these are not otherwise specified; they are the compiler's own functions. + +### 7.2 parse words + +`immediate fn name() { ... }` (or `immediate T name()`) defines a function that is also a keyword: the compiler calls it when `name` appears as a statement, inside an expression, or at the top level. it is entered with the current token on its own name. it must read tokens: a parse word that reads nothing (not even its own name with `run.next()`) is error: a parse word has to read its own name with next(). +``` +immediate fn five() { run.next(); run.lit(5); return 0; } +fn main() { int x = five + 1; rt.put_n(x); return 0; } // prints 6 +``` +a keyword defined this way replaces a built-in of the same name. a parse word cannot be defined while building an image. + +### 7.3 `run.eval`, `run.evals`, `run.compile`, `run.load` + +`run.eval(q)` compiles the token list `q` as a function body and runs it, returning its result; `run.evals(s)` does the same for a string; `run.compile(q)` returns the address instead. `run.load(q)` compiles top-level declarations. the body sees the module and imports of the caller. a `fn main()` inside an `evals` string is a closure expression, not a definition. +``` +rt.put_n(run.evals("return 6 * 7;")); // 42 +run.evals("rt.put_n(40 + 2); rt.put_c(10); return 7;"); // prints 42 +``` + +### 7.4 quote and `$` + +`quote { tokens }` and `$ { tokens }` give a fresh token list object `[n, kind, value, line, kind, value, line, ...]` as data: names, strings and operators are interned pointers. `q[0]` is the number of tokens. the list is an owned fresh object. `run.tokens(s)` makes one from a string, `run.qdup(q)` copies one. `run.feed(q)` pushes a list back into the input (it is freed once read); `run.grab()` reads a `{ }` group into a list. + +### 7.5 operator tables + +`run.add_binop("**", 11, "ipow c")` adds or replaces a binary operator with a precedence and its code, written like asm (names resolve when used; a function name pushes its address, so a call needs the `c` mnemonic). after it, `2 ** 10` is 1024 and `2 * 3 ** 2` is 18. `run.add_unop(name, code)` adds a prefix operator. the lexer learns new spellings from both (1.6.2). + +### 7.6 syntax extension + +`run.syntax("keyword", handler);` registers `handler` (a function already defined above, with no parameters) as a keyword valid at the top level and as a statement; the handler is called like a parse word and must read the keyword. `run.statement_hook(f);` registers a function called (with no arguments, its result dropped) before every statement of the functions compiled after the registration; a hook that is not guarded may re-enter itself. both can appear at the top level of a library and take effect for the files that include it. `src/lib/actor.zc` registers the `actor` syntax and a hook this way. +``` +void kw_hi() { run.next(); run.expect(run.intern(";")); + run.lit('H'); run.callname(run.intern("rt.put_c")); run.op(run.O_DROP); return; } +run.syntax("hi", kw_hi); +fn main() { hi; hi; rt.put_c(10); return 0; } // HH +``` +a hook counting statements: with `int ticks; int tick() { ticks++; return 0; } run.statement_hook(tick);` and a `main` of `int a = 1; a = a + 1; a = a + 1; rt.put_n(ticks);` the output is 4 (the hook runs before each of the four statements). + +### 7.7 receiver methods + +`run.method("member", function);` registers an extension method (5.5.7). + +### 7.8 sharing + +`run.share_value(e)` publishes a pointer value so shallow copies are safe, calling `rt.share`; sharing an aggregate is error: sharing an aggregate requires sharing its pointer fields explicitly. + +### 7.9 `run.add_*` timing + +changes to the operator and keyword tables take effect for the text read after the call; they persist for the rest of the compile. + +## 8. runtime contract + +### 8.1 the program and `main` + +8.1.1 `include rt;` is required before any `rt.` name: `rt.put_c` without it is error: module access requires `include rt;`. + +8.1.2 after a unit compiles with no errors, the runtime calls `std.libinit()` if the unit includes `std`, then `main()` if a function `main` is defined (no arguments; it works in a namespaced file too), then stops the machine. a unit with no `main` runs nothing. a unit with errors does not run. + +8.1.3 the value `main` returns is discarded. the process exit status of `zy` is 0 in all cases, including compile errors and `rt.halt()`. + +8.1.4 compile errors are printed on standard output (not standard error), the same stream as the program's output. + +### 8.2 compile diagnostics + +each error is a line `zc: line N: MESSAGE`. after the unit, a summary line `zc: 1 error` or `zc: K errors`. a syntax error skips to the next `;` or `}` (or the next top-level item) and later errors in the skipped text are not reported; the compiler reports at most 20 errors, then prints `zc: too many errors, stopping`. warnings are `zc: line N: warning: MESSAGE`. an error in a nested include carries the line of that file. + +### 8.3 the `rt` functions a program relies on + +| function | contract | +|---|---| +| `rt.put_c(ch)` | write the low 8 bits of `ch` to the output; returns 0 | +| `rt.put_s(s)` | write each word of a zero-terminated `char*` as a byte (low 8 bits); a `bstr` argument writes its bytes | +| `rt.put_n(n)` | write the signed decimal form of `n` (also the minimum integer) | +| `rt.get_c()` | the next input byte, 0 to 255; -1 at the end of input | +| `rt.eof()` | nonzero when the input has no more bytes | +| `rt.new(n)` | a fresh zeroed object of `n` words, owned (`.u`); `rt.new(T)`, `rt.new(T, n)`, `rt.new(T[?])`, `rt.new(T[?], n)`, `rt.new(fixed T[?], n)` allocate a typed object; failure to allocate halts with a diagnostic | +| `rt.free(p)` | releases `p` and every object it owns; takes `p` as `.d`; a shared object (a result of `+` on byte strings, a slice, `rt.n_to_s`, `rt.cons`) is left to the collector; in checked code a second free of a tracked local is a compile error, and in unchecked code it is ignored (with `memdebug on;` it prints `zy: double free from line N`) | +| `rt.halt()` | stop the machine at once (exit status 0) | +| `rt.len_w(s)`, `rt.eq_w(a, b)`, `rt.copy_w(dst, src, n)`, `rt.fill_w(dst, v, n)` | word-string length, equality, word copy, word fill | +| `rt.len_s rt.at_s rt.slice_s rt.cat_s rt.eq_s rt.s_to_n rt.n_to_s rt.put_bs` | packed byte strings | +| `rt.gc() rt.memreport() rt.syscall rt.readfile` | collector, leak report, host gate, file read | + +heap objects have a 5 word header before the pointer (size, flags, check word, owner, type); `rt.new` words are zeroed. output is the process's standard output, one byte per `put_c`. + +### 8.4 run-time errors + +the program prints `zy: index I out of range (length N)` and stops on an out-of-range `T[?]` index; `zy: use after free from line N`, `zy: double free from line N`, `zy: bad heap pointer` under `memdebug`. the machine does not trap on integer division by zero (3.4.3) or on a stray write inside memory. + +## appendix a: what zc0 accepts + +zc0 (`src/tools/zc0.c`) is the stage-0 bootstrap compiler. it reads source on stdin and writes a genome image; to run a program through it use `make run PROG=file.zc` (the source starts with `namespace;` and defines `main`). it has no types (they are read and dropped), no analyzer, no checks. in the zc0 subset: + +- lexing: comments, decimal/hex integers (but see appendix b for octal and saturation), floats, character literals, word strings, `b"..."`, the operator set `== != <= >= << >> && || ++ -- += -= *= /= %= &= |= ^= -> <<= >>=`, the source-ending backtick, `rt.`/`run.`/`core.`/`lsp.`/`std.`/`icon.` joined names. +- top level: `namespace`, `include` (the names are read and ignored; resident modules only), `embed`, `unchecked`, `fn`, `.u name(...)`, typed functions and globals, `const`, `enhance`, `let`/`var` globals, prototypes. +- statements: blocks, declarations (typed or `let`/`var`, arrays `x[n]` and `T[n]`), `unchecked stmt`, `if`/`else`, `while`, `for`, `break`, `continue`, `return`, `asm`, expression statements. +- expressions: all binary operators with the c precedences, assignment and compound assignment, unary `- ! ~ * & ++ --`, recasts `u*x m*x d*x` (no code), postfix `[]`, `[i:j]` and `[i]` on byte strings, calls (including through a value), `++ --`, `x as name`, `asm { }`, float arithmetic and comparisons, byte strings (`+`, comparisons). + +not in the zc0 subset: `struct`, methods, `enum`, `type`, generics, `do`/`while`, `switch`, `?:`, `(T)e` casts, `!!`, `.`/`->` field access, `sizeof`, `typeof`, `len`/`cap`/`push`/`pop`/`resize` on growable arrays, `T[?]`, `rt.new(T)` typed forms (`new` of a size is a plain call), `fixed`, `const` types, the ownership analyzer and its errors, `typecheck`, `memdebug`, `hints`, `docs`, `o3`, `extern`, `cfunc`, `immediate`, closures, block expressions, array initializer lists, compound literals, `quote`/`$`, `run.*` access, operator tables, `include` of anything not resident, `namespace`-scoped imports checks, `case _`. + +## appendix b: disagreements and bugs + +each item gives a minimal program, the expected behavior (per this spec) and the actual behavior of the reference implementation on `build/bin/zy` (unless it names zc0). none are fixed here. + +### b.1 disagreements between the prose docs and the code + +1. README ("memory debug"): "`memdebug scrub;` also fills a freed payload with 57005". the compiler compares the word after `memdebug` with the interned name `rt.scrub` instead of `scrub`, so the statement is rejected: `memdebug scrub;` => `zc: line N: memdebug on, memdebug scrub or memdebug off`. (also b.2.1) +2. README ("zc, the language") says a cast type "has to be a simple type ... `int` or `*node`". the prefix pointer spelling `*node` is not a type (README's own "pointer types" section says so, and `(*int)y` => `int: is a type, not a value`). the postfix form `(node*)x` is the one that works. +3. README says "operators follow C precedence". this holds for expressions. constant expressions (array lengths, enum values, `const`, `case` labels) have no precedence: `+ - * | & << >>` only, left to right (3.14). `const X = 1 + 2 * 3;` is 9, and `/ % ^` are rejected (`const Z = 6 / 2;` => `expected ';', found '/'`). README does not mention either fact. +4. README lists `switch` forms `case 1, 2:` and `default:`. the catch-all `case _:` exists in the compiler and is not documented. +5. README ("zc, the language"): "a unit is read whole and scanned for function signatures before it compiles, so calls can come before definitions, tags included, and prototypes are optional". prototypes that are written must name their parameters (`int f(int);` => `expected a name, found ')'`). +6. README/LEARN do not say that an embedded `\0` ends a string literal: `b"hi\n\0x"` has `len` 3 and `"ab\0cd"` is `ab`. +7. README ("zc, the language") says `const k = 5;` in a function is a local constant. it is not a constant expression: `const n = 5; int[n] a;` in a function => `expected a constant`. +8. README (reflection) describes `run.tparamtag` as "the u/m/d tag of a parameter"; the code returns 0 untagged, 1 `.m`, 2 `.d` (there is no u at a parameter). +9. README/LEARN do not say that local variables and arrays are not zeroed (4.1.5). +10. zc0 vs compiler, the case that needed a referee: with `||` and `&&` now at c precedence, zc0 and compiler.zc agree on `1 || 0 && 0` => 1, `0 && 0 || 1` => 1, `1 | 2 && 0` => 0, `1 + 2 << 1` => 6, `7 & 3 ^ 1 | 8` => 10, `6 & 3 == 3` => 0 (checked on both). +11. zc0 vs compiler, integer literals. zc0 reads them with `strtoll` base 0: `rt.put_n(010)` prints 8 in zc0 and 10 in the compiler; `99999999999999999999` and `0xFFFFFFFFFFFFFFFF` saturate to 9223372036854775807 in zc0 and wrap to 7766279631452241919 and -1 in the compiler. this spec follows the compiler. + +### b.2 implementation bugs and limitations + +1. `memdebug scrub;` is rejected (b.1.1). expected: poison freed payloads with 57005. +2. enum members cannot name earlier members of the same enum: `enum { HI = 16, LO = HI + 1 };` => `expected a constant` (the members are defined after the closing brace). members of an earlier enum and constants work. +3. a function can be silently redefined. `int f() { return 1; } int g() { return f() + 10; } int f() { return 2; }`, then `g()` gives 11 and a later call of `f()` gives 2. expected: error `f: is defined twice` (the message exists in the compiler and is unreachable for an ordinary second definition). +4. a prototype after the definition hides it. `int f() { return 1; } int f();` then `f()` => `zc: line 0: f is used but never defined`. expected: the definition stays. +5. `?:` with one float branch and one integer branch converts wrongly. `pf(1 ? 1 : 2.5)` prints `0.000000` (the integer 1's bits read as a float); `pf(1 ? 2.5 : 1)` prints `4612811918334230528.000000` (the taken float converted again as if an integer); `pf(0 ? 2.5 : 1)` prints 1.000000 and `pf(0 ? 1 : 2.5)` 2.500000 (these two are right). expected 1.0, 2.5, 1.0, 2.5. +6. `bool` does not normalise: `bool b = 5; rt.put_n(b); rt.put_n(b == 1); rt.put_n(!b);` prints 5, 0, 0. expected 1, 1, 0 if `bool` is a 0/1 type; it behaves as a plain integer. +7. a float literal as a global initializer: `float g = 1.5;` at the top level => `expected a constant`. expected: allowed (a float local initializer works). +8. a global initializer that names another global uses the other global's address, not its value: `int a = 5; int b = a + 1;` gives `b` the value `&a + 1` (1 when `a` is at address 0). expected: the value of `a`, or an error. +9. `include std;` written inside a function body: `zc: line N: return outside a function`. expected: an error saying include is top level only. +10. a syntax error hides errors in the statements skipped by recovery: `int a = ; int b = 3 +;` reports only the first error (8.2). by design, but easy to miss. +11. an unterminated `/* ...` comment is accepted silently (1.2.2); an unterminated string literal is only reported as a later syntax error. +12. `x as int * y` parses `int*` as the type (3.2.3). +13. prototypes need parameter names: `int f(int);` => `expected a name, found ')'`. +14. `len(s)` on a `char*` returns word 0 of the string when `typecheck` is off. +15. duplicate `case` labels are not diagnosed (4.7.2). +16. a local `const` cannot be used in a constant expression (b.1.7). + +## appendix c: how this spec was checked + +- `build/bin/zy` ran every example that shows an expected result in sections 1 to 7: the precedence table (3.2.2), integer semantics (3.4), float rules (3.5), literals (1.4), string forms, switch (4.7), struct layout (2.7), enums (2.8), growable arrays (3.13), closures (3.15), modules (5.12), ownership (6), parse words (7). +- the ownership messages in 6.13 are the `// expect:` lines of `tests/errors/*.zc` plus the messages found in the compiler's `cerr` and `cerrn` calls. +- zc0 was run on the precedence and literal cases by building a run-only image with `src/tools/automap.sh run` and the microcore. -- cgit v1.2.3