a

6 files changed, 194 insertions, 0 deletions

diff --git a/docs/design/adt.md b/docs/design/adt.md
new file mode 100644
index 0000000..2011337
--- /dev/null
+++ b/docs/design/adt.md
@@ -0,0 +1,19 @@
+# tagged enums
+
+tagged enums let one pointer refer to a value with one of several payload shapes. a member with fields is declared using a block of field declarations; a member without fields has no block:
+
+```zc
+enum shape { circle { int r; }, rect { int w; int h; }, dot };
+int area(shape* s) {
+    switch (s) {
+    case circle(r): return r * r * 3;
+    case rect(w, h): return w * h;
+    case dot: return 0;
+    }
+    return -1;
+}
+```
+
+each constructor allocates an owned enum object and returns a pointer. word zero stores the tag; payload fields follow it. the object is sized for the largest member, and unused payload words are zeroed. a `case member(names)` arm tests the tag and binds that member's fields as locals scoped to the arm. a member omitted from a switch without `default` produces a warning, in typed and untyped code.
+
+see [the tagged enum example](../../examples/learn/16-tagged-enums.zc). the syntax and behavior described here are implemented by the parser, typechecker, code generator and language server.
diff --git a/docs/design/handoff.md b/docs/design/handoff.md
new file mode 100644
index 0000000..0b08048
--- /dev/null
+++ b/docs/design/handoff.md
@@ -0,0 +1,20 @@
+# implementation notes
+
+this note summarizes implemented language and tooling features that were previously tracked across design documents. feature details live in the project readme and examples.
+
+## language and library
+
+- struct methods and static functions support typed parameters and return types. the existing untyped form remains available.
+- `src/lib/std.zc` groups helpers in structs of static functions, called with `Group.name(args)`. `Text`, `Bytes`, `Host`, `Arena` and `Buf` provide string, byte, host, arena and buffer helpers. linux `syscall` is host-only and bypasses vm checks.
+- tagged enums support payload members, constructors that return owned enum pointers, and `case member(fields)` bindings in ordinary `switch` statements. omitted members produce a warning when there is no `default`.
+- struct values can cross native calls on x86-64 sysv and linux arm64 aapcs64 for the interpreter, o3, llvm and jit paths described in the readme. apple arm64 remains unchecked.
+
+## language server
+
+the language server runs the compiler in check mode, so diagnostics and semantic highlighting use the same parser and symbol tables as compilation. hover supports declarations, parameters, locals, built-ins and documented keywords. it does not follow declarations reached through `include` yet.
+
+## optimization
+
+o3 remains the explicit way to request native regions; automatic tiering was dropped. llvm inlining and global copy propagation are implemented. see [tiering](tiering.md) for the rationale and [struct-by-value](struct-by-value.md) for native aggregate limits.
+
+the runtime memory map is generated from a preset. fixed code and compiler tables have finite capacity; the heap grows within the configured machine memory. see the project readme's memory map section for the current build controls.
diff --git a/docs/design/memory.md b/docs/design/memory.md
new file mode 100644
index 0000000..bcbdf35
--- /dev/null
+++ b/docs/design/memory.md
@@ -0,0 +1,106 @@
+# memory configuration
+
+## one image and one layout
+
+the compiler, runtime, and host must agree on their addresses. `genmap` writes a matching `memmap.h`, runtime source, and compiler source; `src/tools/automap.sh` builds an image against those files. rebuild the host against the generated headers. changing a header alone does not relocate an existing genome. compact compiler and run-only maps also put their heap above the fixed regions, so compiler storage can grow after startup.
+
+```sh
+bin/genmap --preset default --check
+bash src/tools/automap.sh small build/small
+bin/genmap --preset default --code 65536 --symbols 256 --locals 64 --fwd 512 --pool 8192 --hash 1024 -o build/custom
+```
+
+run `bin/genmap --help` for the table, heap, genome, and stack options. sizes are in vm words unless an option says otherwise; one word is eight bytes. `--snap-cap` selects the initial snapshot reservation and `--quote-cap` selects an initial token count, with three words per token. both regions grow after startup.
+
+## layout inventory
+
+the generated addresses reserve initial storage. native code-address tables start at `PW` and grow together when code needs new addresses; the microcore and javascript execute within their mapped memory. every host must use the same layout as its image.
+
+| region | build control | runtime behavior |
+| --- | --- | --- |
+| initial emitted program, through `CODE_END` | `--code` | links stable heap segments when full |
+| type arena below `ZFP` | `--types` | stable type chunks and a growing index |
+| compiler names and columns | `--hash`, `--pool`, `--symbols`, `--locals`, `--fwd` | grow independently |
+| compiler scratch regions | `--snap`, `--quote`, `--src`, `--seq`, `--ctx`, `--loop` | grow independently |
+| enum members, tagged-enum index, struct methods, generated source | `--enum-members`, `--tagged-enums`, `--methods`, `--generated-text` | grow independently |
+| operator columns | `--binary`, `--unary`, `--assignments`, `--keywords` | grow independently |
+| assembly labels and patches at `R_ASM` | `--asm-labels`, `--asm-refs` | columns grow |
+| exit storage at `R_EXIT` | `--exit` | grows |
+| genome reservation, from `GORG` to `FBOT` | `--genome` | existing image addresses stay fixed |
+| frame region, from `FBOT` to `FTOP` | `--frame` | adds stable heap segments |
+| return region, from `RS0` to `MARK0` | `--return` | copies live return addresses to larger storage |
+| collector work at `MARK0` | `--mark` | grows during collection |
+| data region, from `DSLO` to `DSHI` | `--data` | copies live operands to larger storage |
+| initial heap | `--heap`, `--heap-lo` on compact maps | grows when the host supports it |
+| total host memory | `--total`, host startup options | growth limit can change during execution |
+
+`--words` selects the fixed-map size; unused words extend the initial data region. `--total` selects the host reservation for a growing map. inside `genmap`, the fixed-map size is called `MW` and the host reservation `TW`; generated headers name them `PW` and `MW`. `HEAP_HI` is the initial heap end, not its eventual limit. `SYSGATE` stays at `ZFP + 2`, and the linked image supplies `GORG`.
+
+parameter tags, native calling conventions, packed token positions, and opcode formats also have limits. these describe the language or machine interface; they are separate from the storage capacities above. their parser buffers grow independently; for example, an enum index can grow while one tagged enum keeps its 63-member exhaustiveness mask. function parameter tags and generic arity retain their current encoding. packed token positions retain a 24-bit line, 16-bit token length, and 21-bit byte offset; this is an encoding constraint, not a growing buffer.
+
+## compiler tables during execution
+
+the generated layout supplies initial storage. the following tables grow when they fill:
+
+| storage | growth behavior | explicit reservation |
+| --- | --- | --- |
+| symbols and their types | copy columns; symbol indices stay valid | `reserve_symbols(n)` |
+| locals and analyzer facts | copy every local column | `reserve_locals(n)` |
+| forward references | copy patch columns; links retain indices | `reserve_fixes(n)` |
+| intern hash | rehash pointers; interned spellings stay at their old addresses | `reserve_names(n)` |
+| string pool | add larger chunks; old handles and symbol-chain heads stay valid | `poolchunk` sets the next chunk's minimum size |
+| type records and type index | add stable chunks and grow the index | automatic |
+| loop-range facts, joined paths, queued o3 regions | copy records before appending; build controls are `--facts`, `--paths`, `--regions` | automatic |
+| analyzer snapshots and exits | copy storage; saved offsets stay valid | `reserve_R_SNAP(n)`, `reserve_R_EXIT(n)` |
+| source, loop, and compiler-context stacks | copy complete frames | `reserve_R_SRC(n)`, `reserve_R_LOOP(n)`, `reserve_R_CTX(n)` |
+| quote tokens and operator sequences | copy storage before appending | `reserve_R_QBUF(n)`, `reserve_R_SEQ(n)` |
+| emitted code | link stable heap segments; code addresses remain valid | `reserve_code(n)`, `code_chunk` |
+| operator columns | copy columns before adding an operator | `reserve_binary(n)`, `reserve_unary(n)`, `reserve_assignment(n)`, `reserve_operators(n)` |
+| execution frames | add segments without moving older local addresses | `reserve_frames(n)` |
+
+reservation functions take minimum capacities and round upward. they can be called by running programs or immediate functions. scratch reservations count words; source frames use 32 words, loop frames use eight, and compiler contexts use 34. local and symbol capacities count entries. the intern hash stays a power of two and grows before it is half full.
+
+new chunks and columns are collector roots. garbage collection can reclaim replaced heap columns, while chunk chains retain interned text and type records for the current compiler state. fixed map storage remains reserved; growing a table does not reuse another map region.
+
+## language server
+
+document slots, uri text, hover text, declaration types, parsed type spans, resolved names, comment ranges, and diagnostics grow independently. hint deduplication uses a growing set of source positions. local name lookup uses growing name heads and shadow links; leaving a scope restores its previous bindings. a large source file must not lose its later diagnostics because earlier hints filled a buffer. the editor bundle must be rebuilt when the server changes; packaging always rebuilds its image.
+
+inference function records retain stable addresses while their index grows. inference edits, replacement text, and annotation verification buffers also grow; later declarations are not dropped at the original function-count or edit-count limits. protocol method names, request identifiers, json keys, nesting paths, and header lines also grow; request identifiers are echoed in full.
+
+## freestanding backing
+
+the stub and board ports reserve `MICRO_PORT_WORDS` cells at build time; it defaults to the initial image and heap size. the vm's active size and growth limit still change at runtime, within that physical backing. set the macro when linking a board with more reserved ram. a rejected physical allocation preserves the current memory. hosted maps keep their heap above the fixed regions; the historical `fixed` preset retains its bounded layout for compatibility.
+
+## host memory and image storage
+
+the default native map places a growing heap above the fixed regions. hosted runners accept `TINYVM_MEMORY_WORDS` for initial memory and `TINYVM_MAX_MEMORY_WORDS` for its growth limit. javascript also accepts `memoryWords` and `maxMemoryWords` when constructing a vm. invalid configurations are rejected before execution.
+
+```sh
+TINYVM_MAX_MEMORY_WORDS=16777216 bin/zy < examples/hello.zc
+TINYVM_MEMORY_WORDS=4194304 bin/zym < examples/hello.zc
+```
+
+the native core reserves a stable backing allocation up to its startup maximum; increasing the available size keeps addresses and native code pointers valid. the microcore and javascript vm replace backing allocations when they grow, preserving existing contents. native bounds use the reservation; portable bounds use the current allocation. memory syscall `9` with argument `4` requests growth, and argument `5` changes the growth limit during execution. a limit cannot fall below the current size. the native limit cannot exceed its startup reservation.
+
+data and return stacks grow automatically before their guard space is exhausted. `reserve_data(n)` and `reserve_returns(n)` request minimum capacities during execution; failure returns `-1` and preserves the old stack. live operands and return addresses are copied before old heap storage is returned to its free list. locals remain in frame storage, so pointers into locals keep their addresses. raw indices returned by `dsp()` describe the current operand stack and must be read again after a reservation.
+
+the host takes stack blocks directly from the runtime's buddy free lists. it does not call vm code or collect while servicing an overflow. when it needs another heap extent, the host's configured memory limit still applies. newly allocated stack blocks are excluded from user-object counts and are protected if collection is already running. operand-stack roots use the current stack's live range.
+
+frame segments grow on the heap, preserve pointers into older frames, and return empty segments to the allocator. collection scans only the live ranges of each segment. `frame_chunk` controls the minimum size of the next segment; `reserve_frames(n)` raises it. the growth helper uses a guard region to allocate without recursively growing its own frame. program emission links new segments through `reserve_code(n)`, with `code_chunk` controlling their minimum size. earlier functions and globals retain their addresses. strings, quotes, global arrays, and enhancement recipes reserve contiguous storage.
+
+optimizer regions that cross a segment boundary run as ordinary bytecode. image output grows independently through `build_image_grow(org, initial)`; its length is returned, and `image_words` and `image_size` expose its output. `build_image(org, dst)` retains the explicit-destination interface. a finished artifact still needs a host layout whose genome reservation fits it; `src/tools/automap.sh` sizes that reservation before linking.
+
+assembly columns, enum members, tagged-enum indices, struct methods, and generated source grow. native program-address metadata grows through memory syscall `9`, argument `9`; a denied request preserves the previous tables. annotation records and edit buffers also grow, within the host’s configured memory limit. generated initial addresses remain part of the image format; language and machine-interface limits do not change when storage grows.
+
+mark work grows through `reserve_mark(n)`, including during a collection. generic templates and instances grow through `reserve_generics(n)` and `reserve_instances(n)`. closure frames and capture rows grow through `reserve_closures(n)` and `reserve_captures(n)`. lexer text, documentation text, and stored signatures also grow. documentation objects keep stable addresses, and compiler storage is excluded from `live_objects()`.
+
+## verification
+
+`tests/drivers/compiler_growth_test.py` checks locals, distinct array types, long names, generic records, captures, and collector work, then reserves larger tables and scratch stacks and runs collection. `tests/drivers/frame_growth_test.py` checks deep recursion, oversized frames, and collection with a pointer into an older segment. `tests/drivers/inference_growth_test.py` checks 600 functions and 5000 local declarations. `tests/drivers/core_lsp_test.js` opens every maintained core zc source through the editor protocol and checks diagnostics and highlighting. `tests/drivers/memory_limits_test.py` checks startup validation and runtime growth on the native, microcore, and javascript hosts. a bootstrap fixpoint alone does not verify runtime growth. collection is conservative: code literals can resemble heap addresses, particularly on compact maps, so reclamation counts can differ between layouts. checks must verify collection progress and retained roots without assuming identical counts.
+
+## bootstrap storage
+
+stage 0 reads source and grows its temporary image buffer on the host. `MAXW`, when supplied at build time, selects an initial image reservation; it is not a final size limit. the resident compiler builds images with `build_image_grow(org, initial)` and returns their current buffer through `image_words`. image output is temporary storage, so it can move while the logical addresses recorded in the image stay unchanged. a matching generated layout must still fit the finished artifact before it is loaded.
+
+vm literals carry 63 signed bits in a single odd word. the compiler emits a short sequence for other 64-bit values; float payloads use their own full word. `setlit` patches one encoded word and requires a value that fits that single-word format.
diff --git a/docs/design/storage-analysis.md b/docs/design/storage-analysis.md
new file mode 100644
index 0000000..59fbcde
--- /dev/null
+++ b/docs/design/storage-analysis.md
@@ -0,0 +1,33 @@
+# bounds and lifetime findings
+
+## what the compiler knows
+
+the compiler records constant element counts for `new(n)`, `new(T, n)`, fixed arrays, and newly created array handles. a direct local alias shares the allocation's facts. a definite `free` or `recycle` ends that allocation's lifetime; closing a scope ends its stack storage and automatically released owned objects.
+
+an index outside a known length produces a warning. dereferencing, indexing, or accessing a field after a recorded lifetime end also produces a warning. these findings apply to unchecked code too. existing ownership errors and fixed-array index errors keep their current behavior. warnings do not prevent compilation.
+
+```zc
+unchecked fn example() {
+    let p = new(int, 3);
+    let q = p;
+    q[3] = 1;       // warns: index 3 is outside length 3
+    free(p);
+    return q[0];    // warns: this allocation was freed
+}
+```
+
+## what stays unknown
+
+this is basic analysis of local facts, not a proof of the whole program. branches, loops, short-circuit operators, raw assembly, and nested compilation discard facts instead of merging uncertain paths. unknown callees invalidate facts about allocations passed to them. array mutations invalidate recorded lengths. pointer arithmetic, explicit casts, pointer-valued fields, stored aliases, and function return contracts are not inferred by this pass.
+
+an unknown fact does not produce a definite warning. runtime array checks still run. the pass may miss an error across one of these boundaries; silence is not a safety guarantee. the broader analysis in [BORROW.md](../BORROW.md) remains a design plan.
+
+## editor previews
+
+array and pointer hovers show the known element count and lifetime at that reference. a live stack allocation names its scope exit as its lifetime end. a live heap allocation ends when freed or released by its owner. a reference after a recorded end names that line. unavailable sizes and lifetimes say `unknown`.
+
+facts are copied into each reference's editor record while the compiler parses it. a later `free` therefore does not make an earlier hover claim the object was already dead. block comments and inferred type text remain part of the same markdown preview.
+
+## focused checks
+
+`node tests/drivers/storage_analysis_test.js bin/zclsp` checks known bounds, packed elements, direct aliases, free and scope lifetimes, hover metadata, and safe or uncertain neighboring cases.
diff --git a/docs/design/struct-by-value.md b/docs/design/struct-by-value.md
new file mode 100644
index 0000000..bbc8f26
--- /dev/null
+++ b/docs/design/struct-by-value.md
@@ -0,0 +1,9 @@
+# structs in native calls
+
+zc structs whose fields are one word each (`int`, `float` or pointers) can be passed by value to `extern` functions, and native functions can return structs. the interpreter, o3, llvm and jit paths support sysv x86-64 and linux arm64 aapcs64. apple arm64 is not verified.
+
+arguments are classified according to the host ABI. on sysv x86-64, structs up to 16 bytes use integer or sse registers when the whole value fits; otherwise they go on the stack. larger structs go on the stack. on aapcs64, homogeneous float structs of up to four words use v registers, integer structs of up to two words use x registers, and other arguments follow the platform ABI.
+
+returns are limited to seven words. the call stub allocates an owned zc object and passes its address through the call block; the native ABI determines how the result is filled. the llvm descriptor path does not handle grouped struct signatures, so these calls use the generic native-call path.
+
+see `tests/programs/structval.zc` and `tests/programs/structret.zc` for executable examples. platform-specific behavior beyond x86-64 sysv and linux arm64 aapcs64 remains unverified.
diff --git a/docs/design/tiering.md b/docs/design/tiering.md
new file mode 100644
index 0000000..21a74c9
--- /dev/null
+++ b/docs/design/tiering.md
@@ -0,0 +1,7 @@
+# automatic tiering
+
+automatic tiering was dropped. the `o3` keyword remains the explicit way to request an optimized native region.
+
+tiering would require runtime information the compiler currently records only for `o3` functions, including frame layout, address-taken slots, arrays and argument placement. `o3` also uses a different calling convention for dynamic arrays and enforces compile-time restrictions such as closure and gate-store limits. promoting an ordinary function would therefore need either a uniform calling convention, duplicate compiled versions with call-site switching, or equivalent runtime checks.
+
+keeping `o3` explicit avoids those changes. the existing optimization work—including direct `o3` calls, llvm inlining, global constant and copy propagation, loop-invariant code motion and cross-block common-subexpression elimination—remains available.