diff options
| author | 2026-10-14 08:29:46 +0330 | |
|---|---|---|
| committer | 2026-10-14 08:29:46 +0330 | |
| commit | 1c53304d03414890af0da7432d1aea87e754a14a (patch) | |
| tree | 97190d86cf548c9bc63f68c86a4d2a4f09809713 /docs/ICON.md | |
| parent | a (diff) | |
feat: grow memory limits, add icon/actor libraries, overhaul LSP diagnostics
Compiler and VM
- make every memory map limit dynamic and adjustable at both build time and
runtime, including compiler tables, stacks, and every execution region
- reserve and grow tables, operand and return stacks, frame segments, and
image-building output without relocating live code or globals
- rename i/o and string helpers consistently (puts -> put_s, putn -> put_n,
scat -> cat_s) across compiler, runtime, examples, tests, and docs
- recast and consume values assigned to globals from inside functions
- allow fixed and const together, add typed closure returns (fn:t(...)),
multi-include syntax (include rt, run, core;), and recurse type inference
- never infer string literals as word, anywhere, as argument, variable, or
return type
- draft docs/BORROW.md: a borrow checker less intrusive than Rust's but more
intelligent
LSP and editor integration
- require explicit includes and give every toolchain area its own namespace
(rt, lsp, run, core)
- hover for built-in types, inferred types, values known at hover time, array
and pointer lifetime/size, library names in include statements, and inferred
lifetimes with explanations
- honor /* */ comments as documentation for variables and arguments, and stop
misattributing section headers or trailing comments as variable hints
- stream diagnostics and cancel stale requests on change
- jump to definition and references, including cross-file
- quick fixes with automatic library import, and namespace completion
Hints and diagnostics
- render hints as blue squigglies by default, with hints on/small/off per file
- suppress magic-number hints via a preceding /* */ comment, and say so
- warn on unused arguments; allow multiple _ discards in parameters, pattern
matching, and tagged enums, warning if a _ is actually used
- rewrite hint and comment prose in professional compiler-doc register
Libraries
- add icon.zc: run-time self-modification, find/replace syntax, type and block
edits, and step-through inspection of locals and globals
- add actor.zc: async actors with Actor<I,O>, implicit this, spawn/recv/send,
injected into the compiler machinery at include time rather than built in
Tooling and docs
- reorganize sources, tests, and designs into subdirectories
- unify windows entry points in tinyvm.bat and add a matching tinyvm.sh
- add --help across the toolchain
- polish the learning guide, stdlib docs, and design notes
Tests
- add accept, reject, and regression coverage across the compiler, LSP,
- runtime, icon, actors, threads, formatting, and the toolchain contracts,
plus fuzzers and property tests
Diffstat (limited to 'docs/ICON.md')
| -rw-r--r-- | docs/ICON.md | 112 |
1 files changed, 112 insertions, 0 deletions
diff --git a/docs/ICON.md b/docs/ICON.md new file mode 100644 index 0000000..9c222c8 --- /dev/null +++ b/docs/ICON.md @@ -0,0 +1,112 @@ +# runtime syntax transformations + +`src/lib/icon.zc` is a prototype for `icon` operations that transform compiler token lists and recompile retained functions. its current tests and examples do not constitute the full integration planned in TODO.md. include `run` when calling compiler operations directly, and `rt` for runtime output: + +```zc +include rt; +include run; +include icon; + +int addition() { return 7 + 3; } + +fn main() { + icon.find_replace(addition, ${ $int + $int }, ${ $int - $int }); + rt.put_n(addition()); // 4 + return 0; +} +``` + +the templates are compiler token lists. identifiers, strings, operators and numeric values remain distinct token kinds; a pattern cannot accidentally match text inside a string or comment. edits preserve token source lines for compiler diagnostics. + +## pattern captures + +a pattern contains ordinary tokens and capture markers. ordinary tokens match both token kind and value. capture markers accept the following ranges: + +| marker | accepted range | +| --- | --- | +| `$int` | one integer literal token | +| `$name` | one identifier token | +| `$expr` | one complete expression in the supported expression grammar | +| `$stmt` | one semicolon-terminated token range, brace block or braced `if`/`while` statement | +| `$block` | one balanced brace block | + +name a capture with `:name`: `$expr:left`. use `$left` in the replacement to insert its captured tokens. capture-class names are reserved and cannot also name a capture. repeated named captures require identical token sequences. unnamed captures are substituted by class and occurrence order: the first replacement `$int` uses the first captured integer, and the next uses the second. + +```zc +let source = run.tokens("return (9 + 2) * 3;"); +let pattern = run.tokens("$expr:left * $int:right"); +let replacement = run.tokens("$left + $right"); +let edited = icon.rewrite(source, pattern, replacement); +rt.put_n(run.eval(edited)); // 14 +``` + +expression recognition covers literals, names, grouping, calls, indexing, member access, unary `!`, `~`, `+`, `-`, `*` and `&`, postfix increment/decrement, ordinary binary operators and conditional expressions. capture classes describe token structure, rather than performing type inference: `$int` matches integer literals, while `$expr` also accepts variable expressions. `$stmt` checks a supported statement boundary rather than validating its entire grammar; blocks and braced `if`/`while` captures stop at their closing boundary. casts, custom operators, closure literals and compound initializers require exact token patterns or block captures. + +matching operates on balanced token ranges. it is not a type-directed tree rewrite. preserve parentheses when a replacement changes operator precedence. matches are nonoverlapping, replacement tokens are not searched again, and patterns cannot match an empty range. originals and templates remain unchanged. + +## editing token lists + +the edit operations return independent lists. a null result indicates a rejected edit; `icon.error()` supplies its status. + +| operation | behavior | +| --- | --- | +| `rewrite(tokens, pattern, replacement)` | replace every matching range | +| `splice(tokens, start, end, insert)` | replace the token range `[start, end)` | +| `undeclare(tokens, name)` | remove a complete, single-binding declaration statement | +| `change_type(tokens, name, type_tokens)` | replace the type prefix of the first simple declaration | +| `insert_block(tokens, position, body_tokens)` | insert a new scoped block | +| `calculate(tokens, start, end)` | compile and evaluate the selected expression | +| `validate(tokens)` | check delimiter structure | + +`undeclare` rejects declarations containing multiple bindings and statements that merely reference the name. simple declarations inside nested blocks are included in the search, and `change_type` preserves a preceding `const` qualifier. edits select the first declaration with the requested spelling; they do not perform name-resolution-based rewriting of every declaration or use. removing a declaration leaves uses elsewhere for the compiler to diagnose. `validate` checks delimiters, rather than all language rules; the compiler remains responsible for type, ownership and syntax checking before execution. + +`calculate` executes an expression through the compiler. calls in that expression retain their ordinary effects. it is an evaluator, rather than a purity or constant-folding guarantee. + +## publishing a function replacement + +`find_replace(function, pattern, replacement)` obtains the function's retained body tokens, transforms them, and asks the compiler to publish the replacement. it returns the number of matched ranges after successful publication, or zero when there is no match or publication is rejected. existing references to the original function entry use the published version. + +publication must preserve the original parameter and return contract. it does not retroactively alter frames that are already executing the original body. closure environment mutation and inspection of native execution frames require compiler or vm support beyond these token-list operations. + +## resumable execution and inspection + +`begin(tokens)` creates a resumable frame with its own token list, local binding map and global binding map. `begin_function(function)` starts from retained body tokens; provide each argument with `set_local` before execution. `step(frame)` executes one statement or one control-flow decision. `resume(frame, budget)` executes at most the requested number of steps, leaving an unfinished frame available for inspection or later execution. + +```zc +let code = run.tokens("int n = 4; n = n * 3; return n + bias;"); +let frame = icon.begin(code); +icon.set_global(frame, "bias", 2); +icon.step(frame); // n is 4 +icon.step(frame); // n is 12 +icon.step(frame); // completed result is 14 +rt.put_n(icon.result(frame)); +icon.close(d*frame); +``` + +the frame evaluator supports single-binding `int`, `word` and `let` declarations containing integer values, assignments and `+=`, `-=`, `*=`, `/=`, standalone increment and decrement, return, blocks, and braced `if`, `else` and `while`. expression evaluation uses the compiler after substituting the frame's integer bindings. increment or decrement inside an expression is rejected because substitution does not preserve an assignable location. block-local declarations are removed at block exit, and shadowed bindings become visible again. + +use `local(frame, name)` and `global(frame, name)` to read bindings; `set_local` and `set_global` update them. `local_count` and `local_name` enumerate the currently visible storage records, including shadowed records within an active block. `position` returns the next token position and `steps` reports executed steps. `result` is available after a return or normal completion. + +frame states are `0` for resumable execution, `1` for completion and `2` for an unsupported statement. unsupported syntax sets status `5` and does not execute the statement. `for`, switch and pattern matching, pointer or aggregate mutation, exceptions, and arbitrary native-frame suspension are not supported by this evaluator. direct reads of program globals require an explicit frame binding. expressions may call ordinary program functions and use constants; those calls retain their ordinary effects and run to completion within the step. + +`bind_global(frame, name)` explicitly connects one global binding to a program global of type `int` or `word`. reads observe its current value and frame assignments update its actual storage. binding a function, array, pointer, packed integer, floating value or missing symbol is rejected. this operation changes the isolation boundary for that one binding, so use it only when updating the program global is intended. + +`close(d*frame)` releases the frame's tokens, bindings and control-flow storage. the caller must release independently created token lists according to their normal ownership rules. + +## status values + +| status | meaning | +| --- | --- | +| `0` | operation completed | +| `1` | invalid delimiters or malformed or empty pattern | +| `2` | invalid range, declaration or missing binding | +| `3` | replacement references an unavailable capture | +| `4` | function tokens are unavailable | +| `5` | unsupported frame statement | +| `6` | function publication rejected | + +the status applies to the most recent transformation or binding operation. successful binding reads and writes clear it; inspection of frame state does not clear it. function publication is staged: a parse, type or signature failure rejects the replacement and retains the previous function entry. compiler errors during ordinary expression evaluation follow the compiler's normal diagnostic and termination behavior; these status codes do not intercept them. + +## focused tests + +the acceptance, rejection and regression fixtures live under `tests/icon`. run them with `python3 tests/drivers/icon_test.py bin/zy`; use `--js` for the javascript vm. select one fixture or category with `--case`, for example `python3 tests/drivers/icon_test.py bin/zym --case loop_step`. add `--source` to test the working library source before rebuilding the embedded compiler module; omit it when checking the built image. |
