diff options
| author |  Moon Flower <moonflower@elhenney.local>
| 2026-10-14 08:29:46 +0330 |
|---|---|---|
| committer | Moon Flower <moonflower@elhenney.local>
| 2026-10-14 08:29:46 +0330 |
| commit | 1c53304d03414890af0da7432d1aea87e754a14a (patch) | |
| tree | 97190d86cf548c9bc63f68c86a4d2a4f09809713 /src/compiler/compiler.zc | |
| parent | a (diff) | |
feat: grow memory limits, add icon/actor libraries, overhaul LSP diagnostics
Compiler and VM
- make every memory map limit dynamic and adjustable at both build time and
runtime, including compiler tables, stacks, and every execution region
- reserve and grow tables, operand and return stacks, frame segments, and
image-building output without relocating live code or globals
- rename i/o and string helpers consistently (puts -> put_s, putn -> put_n,
scat -> cat_s) across compiler, runtime, examples, tests, and docs
- recast and consume values assigned to globals from inside functions
- allow fixed and const together, add typed closure returns (fn:t(...)),
multi-include syntax (include rt, run, core;), and recurse type inference
- never infer string literals as word, anywhere, as argument, variable, or
return type
- draft docs/BORROW.md: a borrow checker less intrusive than Rust's but more
intelligent
LSP and editor integration
- require explicit includes and give every toolchain area its own namespace
(rt, lsp, run, core)
- hover for built-in types, inferred types, values known at hover time, array
and pointer lifetime/size, library names in include statements, and inferred
lifetimes with explanations
- honor /* */ comments as documentation for variables and arguments, and stop
misattributing section headers or trailing comments as variable hints
- stream diagnostics and cancel stale requests on change
- jump to definition and references, including cross-file
- quick fixes with automatic library import, and namespace completion
Hints and diagnostics
- render hints as blue squigglies by default, with hints on/small/off per file
- suppress magic-number hints via a preceding /* */ comment, and say so
- warn on unused arguments; allow multiple _ discards in parameters, pattern
matching, and tagged enums, warning if a _ is actually used
- rewrite hint and comment prose in professional compiler-doc register
Libraries
- add icon.zc: run-time self-modification, find/replace syntax, type and block
edits, and step-through inspection of locals and globals
- add actor.zc: async actors with Actor<I,O>, implicit this, spawn/recv/send,
injected into the compiler machinery at include time rather than built in
Tooling and docs
- reorganize sources, tests, and designs into subdirectories
- unify windows entry points in tinyvm.bat and add a matching tinyvm.sh
- add --help across the toolchain
- polish the learning guide, stdlib docs, and design notes
Tests
- add accept, reject, and regression coverage across the compiler, LSP,
- runtime, icon, actors, threads, formatting, and the toolchain contracts,
plus fuzzers and property tests
Diffstat (limited to 'src/compiler/compiler.zc')
| -rw-r--r-- | src/compiler/compiler.zc | 3954 |
1 files changed, 2896 insertions, 1058 deletions
diff --git a/src/compiler/compiler.zc b/src/compiler/compiler.zc index db7355b..d274eff 100644 --- a/src/compiler/compiler.zc +++ b/src/compiler/compiler.zc @@ -1,177 +1,202 @@ -// ============================================================================ -// compiler.zc: the resident zc compiler. it lives in the genome next to the -// runtime, compiles source straight to vm words in memory, and is visible to -// the programs it compiles: every function and global here can be called or -// patched by name, and parse words (immediate fns) run inside it. -// ============================================================================ +namespace run; +include rt; + +//: ============================================================================ +//: compiler.zc: the resident zc compiler. it lives in the genome next to the +//: runtime, compiles source straight to vm words in memory, and is visible to +//: the programs it compiles: every function and global here can be called or +//: patched by name, and parse words (immediate fns) run inside it. +//: ============================================================================ + +//: how this file is organized, in the order a source file travels through it: +//: +//: 1 lexer bytes -> tokens (nextch, lex1, next/peek). a token is a +//: kind (tk), a value (tv: number or interned name) and a +//: place (line, offset, length). sources nest: stdin, a +//: token list for macros, a string for eval. +//: 2 symbols every name is interned once in a string pool; symbols +//: (functions, globals, constants, keywords, types) hang +//: off the interned name and shadow one another. +//: 3 types records in the type area, hash-consed, so comparing two +//: types is comparing two words. typing is optional: an +//: untyped program is all "word". +//: 4 analyzer the ownership/lifetime pass. it runs while the parser +//: emits code, tracking for every local whether it owns an +//: object in this block (m), moves one up and out (u), or +//: moves one down into a lower scope (d). borrowing is +//: tracked separately. the pass inserts frees, and joins +//: merge that knowledge at +//: if/else, loops and switches. +//: 5 expressions precedence climbing over the operator tables, which a +//: program can extend at compile time. +//: 6 statements blocks, control flow, switch, asm blocks, quote/grab. +//: 7 functions signatures, prologues, closures, methods, generics, +//: fattening (typed new) and the top level. +//: 8 setup the keyword table, built-in names, entry point. +//: 9 check mode the same compiler run over an in-memory document for the +//: language server: errors and hints become records. +//: +//: the compiler emits vm words directly into memory at "here" (there is no +//: intermediate tree). a forward reference emits a placeholder literal and +//: remembers the site on a patch chain; defining the name patches the chain. +//: the compiler is itself written in zc and lives in the genome next to rt.zc, +//: so a program being compiled can call any function in this file, and +//: immediate fns (parse words) run in the middle of compilation and can read +//: tokens, emit code, or add operators. a few conventions that recur: +//: - "plain word": most values are untyped 64-bit words. a pointer is just a +//: word that happens to be a heap address. +//: - a code word is lit(v) = 2v+1 (odd) or op(k) = 2k (even). see lit and op. +//: - names ending in an s_ prefix are interned spellings the parser compares +//: against by pointer, which is why intern exists. +//: - serr reports and carries on; die reports and skips input to the next +//: ; or }. nothing in here raises or unwinds. +//: - a backtick outside a comment or a literal ends the source (srcend). zc0, +//: the c bootstrap, reads the same way, so a backtick in a comment is fine. +//: +//: ---- compiler data region (@map-cc) ---- -// how this file is organized, in the order a source file travels through it: -// -// 1 lexer bytes -> tokens (nextch, lex1, next/peek). a token is a -// kind (tk), a value (tv: number or interned name) and a -// place (line, offset, length). sources nest: stdin, a -// token list for macros, a string for eval. -// 2 symbols every name is interned once in a string pool; symbols -// (functions, globals, constants, keywords, types) hang -// off the interned name and shadow one another. -// 3 types records in the type area, hash-consed, so comparing two -// types is comparing two words. typing is optional: an -// untyped program is all "word". -// 4 analyzer the ownership/lifetime pass. it runs while the parser -// emits code, tracking for every local whether it owns an -// object in this block (m), moves one up and out (u), or -// moves one down into a lower scope (d). borrowing is -// tracked separately. the pass inserts frees, and joins -// merge that knowledge at -// if/else, loops and switches. -// 5 expressions precedence climbing over the operator tables, which a -// program can extend at compile time. -// 6 statements blocks, control flow, switch, asm blocks, quote/grab. -// 7 functions signatures, prologues, closures, methods, generics, -// fattening (typed new) and the top level. -// 8 setup the keyword table, built-in names, entry point. -// 9 check mode the same compiler run over an in-memory document for the -// language server: errors and hints become records. -// -// the compiler emits vm words directly into memory at "here" (there is no -// intermediate tree). a forward reference emits a placeholder literal and -// remembers the site on a patch chain; defining the name patches the chain. -// the compiler is itself written in zc and lives in the genome next to rt.zc, -// so a program being compiled can call any function in this file, and -// immediate fns (parse words) run in the middle of compilation and can read -// tokens, emit code, or add operators. a few conventions that recur: -// - "plain word": most values are untyped 64-bit words. a pointer is just a -// word that happens to be a heap address. -// - a code word is lit(v) = 2v+1 (odd) or op(k) = 2k (even). see lit and op. -// - names ending in an s_ prefix are interned spellings the parser compares -// against by pointer, which is why intern exists. -// - serr reports and carries on; die reports and skips input to the next -// ; or }. nothing in here raises or unwinds. -// - a backtick outside a comment or a literal ends the source (srcend). zc0, -// the c bootstrap, reads the same way, so a backtick in a comment is fine. -// -// ---- compiler data region (@map-cc) ---- // src/tools/genmap computes these addresses from the compiler table sizes for each // build. later regions (the type area, local types, exit records) are computed // from these names. // @map-cc - -const int IHN = 0x8000; -const word R_IH = 0x90100; -const word R_POOL = 0x98100; -const word R_POOLEND = 0xD8100; -const int NS = 0x4000; -const word R_SYM = 0xD8200; -const word sname_BASE = 0xD8200; -const word skind_BASE = 0xDC200; -const word sval_BASE = 0xE0200; -const word ssig_BASE = 0xE4200; -const word sflag_BASE = 0xE8200; -const word sfix_BASE = 0xEC200; -const word snext_BASE = 0xF0200; -const int NL = 0x200; -const word lname_BASE = 0xF4200; -const word lslot_BASE = 0xF4400; -const word lhome_BASE = 0xF4600; -const word lstate_BASE = 0xF4800; -const word lowner_BASE = 0xF4A00; -const word lused_BASE = 0xF4C00; -const word larr_BASE = 0xF4E00; -const word lown_BASE = 0xF5000; -const int NF = 0x8000; -const word fsite_BASE = 0xF5200; -const word fnext_BASE = 0xFD200; -const word R_SNAP_BASE = 0x105200; -const word R_QBUF_BASE = 0x109200; -const word R_SRC_BASE = 0x10D200; -const word R_SEQ_BASE = 0x10D600; -const word bname_BASE = 0x10DE00; -const word bprec_BASE = 0x10DE40; -const word bseq_BASE = 0x10DE80; -const word blen_BASE = 0x10DEC0; -const word uname_BASE = 0x10DF00; -const word useq_BASE = 0x10DF20; -const word ulen_BASE = 0x10DF40; -const word aname_BASE = 0x10DF60; -const word abin_BASE = 0x10DFA0; -const word kops_BASE = 0x10E000; -const word R_CTX_BASE = 0x10E200; -const word R_LOOP_BASE = 0x10EA00; -const word R_ASM = 0x10EE00; -const int ASM_LABELS = 0x40; -const int ASM_REFS = 0x100; -const int ASM_WORDS = 0x400; -const int EXIT_WORDS = 0xA00; -const int ENUM_MEMBERS = 0x80; -const int TAGGED_ENUMS = 0x10; -const int STRUCT_METHODS = 0x20; -const int GENERATED_WORDS = 0x2000; -const int RANGE_FACTS = 0x20; -const int JOIN_PATHS = 0x40; -const int O3_REGIONS = 0xA; +const IHN = 0x8000; +const R_IH = 0x90100; +const R_POOL = 0x98100; +const R_POOLEND = 0xD8100; +const NS = 0x4000; +const R_SYM = 0xD8200; +const sname_BASE = 0xD8200; +const skind_BASE = 0xDC200; +const sval_BASE = 0xE0200; +const ssig_BASE = 0xE4200; +const sflag_BASE = 0xE8200; +const sfix_BASE = 0xEC200; +const snext_BASE = 0xF0200; +const NL = 0x200; +const lname_BASE = 0xF4200; +const lslot_BASE = 0xF4400; +const lhome_BASE = 0xF4600; +const lstate_BASE = 0xF4800; +const lowner_BASE = 0xF4A00; +const lused_BASE = 0xF4C00; +const larr_BASE = 0xF4E00; +const lown_BASE = 0xF5000; +const NF = 0x8000; +const fsite_BASE = 0xF5200; +const fnext_BASE = 0xFD200; +const R_SNAP_BASE = 0x105200; +const R_QBUF_BASE = 0x109200; +const R_SRC_BASE = 0x10D200; +const R_SEQ_BASE = 0x10D600; +const bname_BASE = 0x10DE00; +const bprec_BASE = 0x10DE40; +const bseq_BASE = 0x10DE80; +const blen_BASE = 0x10DEC0; +const uname_BASE = 0x10DF00; +const useq_BASE = 0x10DF20; +const ulen_BASE = 0x10DF40; +const aname_BASE = 0x10DF60; +const abin_BASE = 0x10DFA0; +const kops_BASE = 0x10E000; +const R_CTX_BASE = 0x10E200; +const R_LOOP_BASE = 0x10EA00; +const R_ASM = 0x10EE00; +const ASM_LABELS = 0x40; +const ASM_REFS = 0x100; +const ASM_WORDS = 0x400; +const EXIT_WORDS = 0xA00; +const ENUM_MEMBERS = 0x80; +const TAGGED_ENUMS = 0x10; +const STRUCT_METHODS = 0x20; +const GENERATED_WORDS = 0x2000; +const RANGE_FACTS = 0x20; +const JOIN_PATHS = 0x40; +const O3_REGIONS = 0xA; // @end-map -// table bases start in the generated map; growth copies columns without changing indices. -word sname = sname_BASE; +//: table bases start in the generated map; growth copies columns without changing indices. + +/* interned spelling of each symbol, indexed by symbol id; use lookup(name) to find an id. */ +char** sname = sname_BASE; +/* kind of each symbol: K_FN, K_GVAR, K_GARR, K_CONST, K_KW, K_ENH, K_TYPE or K_DOC. */ word skind = skind_BASE; +/* value of each symbol, indexed by symbol id: a function code address, variable storage address, constant value, enhancement slot or type record, according to skind. */ word sval = sval_BASE; +/* packed function signature for each symbol; includes the owned-return bit, parameter count and parameter lifetime tags. */ word ssig = ssig_BASE; +/* definition and compilation flags for each symbol, such as F_DEF, F_DECL, F_GEN and F_STATIC. */ word sflag = sflag_BASE; +/* head of unresolved code references for each symbol; references are patched when the symbol is defined. */ word sfix = sfix_BASE; +/* previous symbol with the same interned name, stored as id plus one; zero ends its shadow chain. */ word snext = snext_BASE; -word lname = lname_BASE; +// retained source metadata for functions that can be transformed at runtime. +word fnmeta; +int syntax_host_floor, syntax_host_from; word syntax_host_symbols; +/* interned name of each currently active local variable or parameter. */ +char** lname = lname_BASE; +/* frame slot of each local; laddr(lslot[i]) emits its address relative to the current frame. */ word lslot = lslot_BASE; +/* scope depth where each local was declared; the lifetime analyzer uses it when values move between blocks. */ word lhome = lhome_BASE; +/* ownership state of each local: plain, m, u, d, dead, child or borrowed. */ word lstate = lstate_BASE; +/* owner local of a borrowed or container-owned local; lroot(i) follows this chain. */ word lowner = lowner_BASE; +/* source line of each local's last tracked use; zero means it has not been used. */ word lused = lused_BASE; +/* nonzero for locals stored in place as fixed arrays or structs; their expression value is their address. */ word larr = larr_BASE; +/* nonzero when a local is responsible for releasing its owned object. */ word lown = lown_BASE; +/* code location to patch for each unresolved reference record. */ word fsite = fsite_BASE; +/* next unresolved reference record, stored as index plus one; zero ends a patch chain. */ word fnext = fnext_BASE; int symcap = NS, loccap = NL, fixcap = NF; -const word R_TY = ZFP - 16384, R_TYEND = ZFP; -// interned text and type records never move: only their index vectors grow. -word namehash = R_IH, poolend = R_POOLEND; +const word R_TY = rt.ZFP - 16384, R_TYEND = rt.ZFP; +//: interned text and type records never move: only their index vectors grow. +char** namehash = R_IH; word poolend = R_POOLEND; word poolblocks, typeblocks; int hashcap = IHN, namecount, poolchunk = R_POOLEND - R_POOL; word typelist = R_TY, tyend = R_TYEND; int typecap = 1024; -// growing scratch regions retain offsets used by nested parser frames. +//: growing scratch regions retain offsets used by nested parser frames. word R_SNAP = R_SNAP_BASE; int R_SNAP_cap = R_QBUF_BASE - R_SNAP_BASE; word R_QBUF = R_QBUF_BASE; int R_QBUF_cap = R_SRC_BASE - R_QBUF_BASE; word R_SRC = R_SRC_BASE; int R_SRC_cap = R_SEQ_BASE - R_SRC_BASE; word R_SEQ = R_SEQ_BASE; int R_SEQ_cap = bname_BASE - R_SEQ_BASE; word R_CTX = R_CTX_BASE; int R_CTX_cap = R_LOOP_BASE - R_CTX_BASE; word R_LOOP = R_LOOP_BASE; int R_LOOP_cap = R_ASM - R_LOOP_BASE; -[60]word lpfact_initial; word lpfact = lpfact_initial; int loopfacts_cap = 60; +word[60] lpfact_initial; word lpfact = lpfact_initial; int loopfacts_cap = 60; -// operator columns grow independently of their initial map reservations. -word bname = bname_BASE; +//: operator columns grow independently of their initial map reservations. +char** bname = bname_BASE; word bprec = bprec_BASE; word bseq = bseq_BASE; word blen = blen_BASE; -word uname = uname_BASE; +char** uname = uname_BASE; word useq = useq_BASE; word ulen = ulen_BASE; -word aname = aname_BASE; +char** aname = aname_BASE; word abin = abin_BASE; word kops = kops_BASE; int bincap = bprec_BASE - bname_BASE, uncap = useq_BASE - uname_BASE, asgcap = abin_BASE - aname_BASE, kopcap = R_CTX_BASE - kops_BASE; -// ---- token kinds, symbol kinds and flags ---- +//: ---- token kinds, symbol kinds and flags ---- + const int T_EOF = 0, T_NUM = 1, T_ID = 2, T_STR = 3, T_OP = 4, T_FLT = 5, T_BSTR = 6; const int K_FN = 1, K_GVAR = 2, K_GARR = 3, K_CONST = 4, K_KW = 5, K_ENH = 6, K_TYPE = 7, K_DOC = 8; const int F_DEF = 1, F_SIG = 2, F_UNTYPED = 4, F_STMT = 8, F_EXPR = 16, F_TOP = 32, F_FALLBACK = 64, F_GEN = 128, F_PRE = 256, F_O3 = 512, F_EXT = 1024, F_TPL = 2048, F_STATIC = 4096, F_DECL = 8192, F_RETOWN = 16384, F_EARLY = 32768; -// variable states for the lifetime analyzer +//: variable states for the lifetime analyzer const int S_PLAIN = 0, S_M = 1, S_U = 2, S_D = 3, S_DEAD = 4, S_CHILD = 5, S_BORROW = 6; -// expression classes: plain word, a new object, or a bare variable (C_VAR + index) +//: expression classes: plain word, a new object, or a bare variable (C_VAR + index) const int C_PLAIN = 0, C_FRESH = 1, C_VAR = 2; -// lvalue kinds +//: lvalue kinds const int LV_NONE = 0, LV_LOCAL = 1, LV_GLOBAL = 2, LV_ELEM = 3, LV_OTHER = 4; -// opcodes +//: opcodes const int O_Q = 0, O_LD = 1, O_ST = 2, O_PICK = 3, O_DROP = 4, O_SWAP = 5, O_CALL = 6, O_JMP = 7; const int O_JZ = 8, O_RET = 9, O_ADD = 13, O_SUB = 14, O_LT = 20; const int XOP0 = 22; // first enhancement opcode @@ -180,37 +205,48 @@ const int OP_FEQ = 605, OP_FLT = 606, OP_FLE = 607, OP_I2F = 608, OP_F2I = 609; const int SEQWORD = -4; // in an operator sequence: the next word is a full-width literal const int SEQREF = -2; // in an operator sequence: the next word names a symbol -// ---- state ---- +//: ---- state ---- + int chk_heap; int nenh; word pooltop; int nsym, ubase, vislo, nfixu, fixfree; +char* module_name; word module_imports; int nbin, nun, nasg, nkop, seqtop; char* mnem = "q@!$_~cjzr,?.+-*&|^><%"; int check_on = 1; // set to 0 to switch the analyzer off for what follows int tk; word tv; int tglue, tline, line = 1, ch, chkind; word chptr, chbase; int qdone, isp = -1, pendot; -// where the current token sits in its source (chars from the start, -1 for a token list), how long it is, and where the one before it ended +//: where the current token sits in its source (chars from the start, -1 for a token list), how long it is, and where the one before it ended int toff = -1, toklen, tpend = -1, tpline; int ntok; // tokens taken so far, so a parse word that reads nothing can be caught int qst, qesc, qp; // srcend's reading state: normal, line comment, block comment, string, char literal -[1100]word tbuf_initial; word tbuf = tbuf_initial; int tbufcap = 1100; -// doc comments: the comment lines directly above a declaration (no blank line -// between them or before the declaration) are its documentation. the lexer -// collects them in dbuf, dlast is the line of the last one, and dline the -// line of the token that follows them. only when docs are on. +char[1100] tbuf_initial; char* tbuf = tbuf_initial; int tbufcap = 1100; +//: doc comments: the comment lines directly above a declaration (no blank line +//: between them or before the declaration) are its documentation. the lexer +//: collects them in dbuf, dlast is the line of the last one, and dline the +//: line of the token that follows them. only when docs are on. int docs_on; // docs on; keeps them: a symbol "doc name" holds a byte string -[2100]word dbuf_initial; word dbuf = dbuf_initial; int docbufcap = 2100; int dn, dlast, dline, docpending; -[2100]word dsnap_initial; word dsnap = dsnap_initial; int docsnapcap = 2100; int dsn, dnm; [300]word dnmb_initial; word dnmb = dnmb_initial; int docnamecap = 300; -int pk_tk, pk_tv, pq_n; [4]word pq_tk; [4]word pq_tv; [4]word pq_gl; [4]word pq_ln; // up to 4 tokens of lookahead +char[2100] dbuf_initial; char* dbuf = dbuf_initial; int docbufcap = 2100; int dn, dlast, dline, docpending, dsect; +char[2100] dsnap_initial; char* dsnap = dsnap_initial; int docsnapcap = 2100; int dsn, dnm; char[300] dnmb_initial; char* dnmb = dnmb_initial; int docnamecap = 300; +int pk_tk, pk_tv, pq_n; word[4] pq_tk; word[4] pq_tv; word[4] pq_gl; word[4] pq_ln; // up to 4 tokens of lookahead int here, hdelta, hlimit, ptop; -int code_chunk = CODE_END; +int code_chunk = rt.CODE_END; word image_buffer, image_words; int image_active, image_org, image_capacity, image_size; int nloc, lbase, fdepth, fslots, fchk, fret, live, bxdepth, epichain, fframe, infn; int finsym, finfer; // the function being compiled, and whether its owned return is still to be inferred (no .u) +int fn_return_type; // a consistent return type inferred from the function body +//: effects come from the real compiler's uses, stores, returns and call edges; unknown bodies stay conservative. +const int EF_READ = 1, EF_WRITE = 2, EF_RETAIN = 4, EF_CONSUME = 8, EF_UNKNOWN = 16, EF_ALIAS = 32, EF_IDENTITY = 64; +word effect_symbols, effect_edges, effect_requirements, effect_current; int effect_floor, effect_argument_base; +word local_origin, local_origin_mask, local_cell, local_captured, local_ended, local_end_info; word effect_value, argorigin_mask; int capture_enabled; word[1] capture_dummy; +int borrow_pending, borrow_depth, borrow_jump = -1, borrow_entry; word borrow_slots; +int argorigin = -1, argtemporary, argowner = -1; word argowner_fact; int stubgen; // 1 while an extern's generated stub is compiled: the stub gets no hints +word xprojection; int af_mutation_tick; int xbor = -1; // local a pointer value is borrowed from, or -1 int mdbg, mdbg_flags; // memdebug: compile checks in, and the flags start() turns on int lsp, loopbase, ctxsp, snaptop; int lv, lvk, lvi, kw_expr, xcls, fty; -// interned spellings the parser compares against +//: interned spellings the parser compares against + int s_lp, s_rp, s_lb, s_rb, s_lk, s_rk, s_comma, s_semi, s_eq, s_arrow, s_inc, s_dec; int s_star, s_slash, s_amp, s_colon, s_hash, s_minus, s_andand, s_oror; int s_dot, s_fn, s_var, s_const, s_else, s_as, s_unchecked, s_immediate, s_m, s_d, s_u; @@ -218,116 +254,143 @@ int s_q, s_case, s_default, s_fovf, s_start, s_main, s_free, s_recycle, s_own, s int s_frame_floor, s_frame_top, s_frame_node, s_frame_busy, s_frame_guard, s_frame_grow, s_frame_leave; int s_do, s_while, s_enum, s_include, s_memdebug, s_scrub, s_alloc, s_check, s_memset, s_static; -// ---- types: kinds, tables and parser state (see the types section) ---- +//: ---- types: kinds, tables and parser state (see the types section) ---- + const int TY_WORD = 0, TY_INT = 1, TY_CHAR = 2, TY_BOOL = 3, TY_VOID = 4, TY_PTR = 5, TY_ARR = 6; -// dyn: []T, a = element, n = 1 if fixed -const int TY_FN = 7, TY_STRUCT = 8, TY_VAR = 9, TY_GEN = 10, TY_DYN = 11, TY_FLOAT = 12; +//: dyn: T[?], a = element, n = 1 if fixed +const int TY_FN = 7, TY_STRUCT = 8, TY_VAR = 9, TY_GEN = 10, TY_DYN = 11, TY_FLOAT = 12, TY_BSTR = 13; const int NTL = 1024; // the first NTL words list every record -// type per local, declared tag per local, type per symbol +//: type per local, declared tag per local, type per symbol + const word LTYPE_BASE = R_ASM + ASM_WORDS, LTAG_BASE = LTYPE_BASE + NL, STYPE_BASE = LTAG_BASE + NL; word ltype = LTYPE_BASE, ltag = LTAG_BASE, stype = STYPE_BASE; -int tytop, ntl; word tword, tint, tchar, tbool, tvoid, tfloat, tf32; -int ckst, ckn; // extern: the zc struct a by-value parameter names (ckind), and its name +int tytop, ntl; word tword, tint, tchar, tbool, tvoid, tfloat, tf32, tbstr; +int ckst; char* ckn; // extern: the zc struct a by-value parameter names (ckind), and its name int tc_on; // the optional typechecker, off unless a program says typecheck on; -// type, literal 0, float, byte string, dry runs (typeof), unique giveaway +//: type, literal 0, float, byte string, dry runs (typeof), unique giveaway + word xty; int xzero, xfloat, xstr, dry, xgive; -// the type facts of annotate mode (see type facts below): the tables, and where the declaration being read starts +//: the type facts of annotate mode (see type facts below): the tables, and where the declaration being read starts + const int NIF = 431, IFP = 24, IFW = 80, IFN = 4096, IFT = 32768; int ifon; word ifs, ifr; int nif; word ifx; int nifx; word ift; int nift; word ifcur; int ifkp, ifkk, ifcall, ifgk, ifstrict; +word if_edges, if_value, if_local; int ifrcap = NIF, ifxcap = IFN, iftcap = IFT; int fldst, fldi; // the one struct a bare address's field name resolved to -[8]word tpn; int ntp; // type parameters in scope -// reading a return type: unknown names are type parameters declared after the function's name +char*[8] tpn; int ntp; // type parameters in scope +//: reading a return type: unknown names are type parameters declared after the function's name int tvauto; +word readonly_types; int dty, dtag; // result of dtype() -int s_lt, s_gt, s_shr, s_le, s_ge, s_eqeq, s_ne, s_plus, s_struct, s_type, s_typecheck, s_docs, s_on, s_off, s_new, s_new_t; +int s_lt, s_gt, s_shr, s_le, s_ge, s_eqeq, s_ne, s_plus, s_struct, s_type, s_typecheck, s_docs, s_small, s_std, s_icon, s_on, s_off, s_new, s_new_t; int s_o3, s_lfa, s_outside, s_new_raw, s_new_oz, s_oob; int s_ld8, s_st8, s_ld16, s_st16, s_ld32, s_st32; -// ---- o3 region state ---- +//: ---- o3 region state ---- -// each promoted slot maps to a register; pending regions use 67-word records and a descriptor buffer. -int o3on, o3nxt, o3one; [32]word o3sl; [32]word o3rg; int no3sl; [O3_REGIONS * 67]word o3pend_initial; word o3pend = o3pend_initial; int o3pendcap = O3_REGIONS, no3pend; [80]word o3dsc; int o3fs, o3na, o3fn0; -// o3free holds registers released by closed scopes; o3cache holds the items pointer and length slots for fixed arrays. -[8]word o3free; int no3free, o3nreg; [NL]word o3cache_initial; word o3cache = o3cache_initial; int argloc, s_hitems, s_hlen, s_fatten; -// operator names used by numeric type checks. -[12]word numops; int nnumops, s_fixed, s_dlen, s_dcap, s_dnew, s_dat, s_dpush, s_dpop, s_dresize, xconst, xcval; +//: each promoted slot maps to a register; pending regions use 67-word records and a descriptor buffer. +int o3on, o3nxt, o3one; word[32] o3sl; word[32] o3rg; int no3sl; word[O3_REGIONS * 67] o3pend_initial; word o3pend = o3pend_initial; int o3pendcap = O3_REGIONS, no3pend; word[80] o3dsc; int o3fs, o3na, o3fn0; +//: o3free holds registers released by closed scopes; o3cache holds the items pointer and length slots for fixed arrays. +word[8] o3free; int no3free, o3nreg; word[NL] o3cache_initial; word o3cache = o3cache_initial; int argloc, s_hitems, s_hlen, s_fatten; +//: operator names used by numeric type checks. +word[12] numops; int nnumops, s_fixed, s_dlen, s_dcap, s_dnew, s_dat, s_dpush, s_dpop, s_dresize, xconst, xcval; int xlenloc, arrloc, fortest, forstep, ft_iv, ft_bd, ft_ir, ft_nops, fs_ok, xstamp; -// local lower bound / len(array); one proof record per live loop -int nfact, factcap = RANGE_FACTS; [RANGE_FACTS]word floc_initial, flo_initial, flen_initial; word floc = floc_initial, flo = flo_initial, flen = flen_initial; +//: local lower bound / len(array); one proof record per live loop +int nfact, factcap = RANGE_FACTS; word[RANGE_FACTS] floc_initial, flo_initial, flen_initial; word floc = floc_initial, flo = flo_initial, flen = flen_initial; int s_len_s, s_at_s, s_slice_s, s_cat_s, s_eq_s, s_lt_s, s_le_s, s_put_bs, s_put_s, s_n_to_s, s_new_s; -int mdg; [8]word mdn; // the type parameters of the generic struct whose methods are being compiled -[8]word gbind; int ngbind; // explicit type arguments for the next call: f<int>(...) -// generic functions. the template (type parameters stay variables) is compiled once, as before, -// and its tokens are kept; a call whose type arguments are all known gets an instance, the same -// tokens compiled again with the parameters bound to those types, so new(T) and sizeof(T) -// inside it mean the real type. instances are compiled after the item that asked for them. +int mdg; word[8] mdn; // the type parameters of the generic struct whose methods are being compiled +word[8] gbind; int ngbind; // explicit type arguments for the next call: f<int>(...) +//: generic functions. the template (type parameters stay variables) is compiled once, as before, +//: and its tokens are kept; a call whose type arguments are all known gets an instance, the same +//: tokens compiled again with the parameters bound to those types, so new(T) and sizeof(T) +//: inside it mean the real type. instances are compiled after the item that asked for them. int grec; word gbuf; int gn, gcap; // the tokens of the top-level item being read -int gnew, gnew_np; [8]word gnew_nm; // a generic function was just defined -word gtab; int gt_n, gtcap = 64, gicap = 128; word gt_sym, gt_tok, gt_np; int gt_nm; // the columns are allocated at the first generic function +int gnew, gnew_np; word[8] gnew_nm; // a generic function was just defined +int fn_method_compile, fnpatchseq; +word gtab; int gt_n, gtcap = 64, gicap = 128; word gt_sym, gt_tok, gt_np, gt_module, gt_imports, gt_source; int gt_nm; // the columns are allocated at the first generic function int gi_n; word gi_t, gi_sym, gi_ar, gi_done; -int ginst; [8]word gnm; [8]word gib; int gng, ginsn; // compiling an instance: the names and what they stand for +int ginst; word[8] gnm; word[8] gib; int gng, ginsn; // compiling an instance: the names and what they stand for int s_this; -// closures being compiled, innermost last: the enclosing function's locals -// (lo .. hi), how many variables the body has captured so far (names and the -// enclosing local each came from), and the body's own hidden env parameter -const int NCLO = 8, NCAP = 32; +const int NCLO = 8; /* initial closure nesting capacity; grows when more closures are compiled */ +const int NCAP = 32; /* initial capture capacity per closure; grows when more variables are captured */ + +//: ---- closure compilation state ---- +//: closures being compiled, innermost last: the enclosing function's locals +//: (lo .. hi), how many variables the body has captured so far (names and the +//: enclosing local each came from), and the body's own hidden env parameter + int nclo, closure_cap = NCLO, capture_width = NCAP; -[NCLO]word clo_lo_initial; word clo_lo = clo_lo_initial; -[NCLO]word clo_hi_initial; word clo_hi = clo_hi_initial; -[NCLO]word clo_n_initial; word clo_n = clo_n_initial; -[NCLO]word clo_env_initial; word clo_env = clo_env_initial; -[NCLO]word clo_ret_initial; word clo_ret = clo_ret_initial; -[256]word clo_name_initial; word clo_name = clo_name_initial; -[256]word clo_loc_initial; word clo_loc = clo_loc_initial; -[256]word clo_via_initial; word clo_via = clo_via_initial; +word[NCLO] clo_lo_initial; word clo_lo = clo_lo_initial; +word[NCLO] clo_hi_initial; word clo_hi = clo_hi_initial; +word[NCLO] clo_n_initial; word clo_n = clo_n_initial; +word[NCLO] clo_env_initial; word clo_env = clo_env_initial; +word[NCLO] clo_ret_initial; word clo_ret = clo_ret_initial; +char*[256] clo_name_initial; char** clo_name = clo_name_initial; +word[256] clo_loc_initial; word clo_loc = clo_loc_initial; +word[256] clo_via_initial; word clo_via = clo_via_initial; -// ---- errors ---- -// the compiler reports up to 20 errors before it stops. a syntax error -// (die) skips the token it happened on and sets panic, which skips ahead to -// the next ; or } (or the next top-level item) and keeps quiet until then. -// other errors (serr) just report: the parse is still in step. -// -// check mode (cmode) is for the language server, which runs the real compiler -// over a document it holds in memory. nothing is printed and nothing halts: -// each error is written into dgbuf as a record [severity, line, offset, -// length, chars...] and giveup raises chkabort, which makes the lexer return -// end of file so every parsing loop winds down by itself. +//: ---- errors ---- + +//: the compiler reports up to 20 errors before it stops. a syntax error +//: (die) skips the token it happened on and sets panic, which skips ahead to +//: the next ; or } (or the next top-level item) and keeps quiet until then. +//: other errors (serr) just report: the parse is still in step. +//: +//: check mode (cmode) is for the language server, which runs the real compiler +//: over a document it holds in memory. nothing is printed and nothing halts: +//: each error is written into dgbuf as a record [severity, line, offset, +//: length, chars...] and giveup raises chkabort, which makes the lexer return +//: end of file so every parsing loop winds down by itself. int nerr, panic; -// bits kept in a local's ltag for the hints: assigned after its declaration, -// read, declared with let, initialized with a literal; the size of the object -// a new gave it (bits 16 to 39); and how many times it was used (from bit 56) +//: bits kept in a local's ltag for the hints: assigned after its declaration, +//: read, declared with let, initialized with a literal; the size of the object +//: a new gave it (bits 16 to 39); and how many times it was used (from bit 56) const int LASG = 0x1000, LREAD = 0x2000, LDECL = 0x4000, LCONST = 0x8000, LCNT1 = 0x100000000000000; const int NEWTYPED = 0x800000; -[NL]word lpos_initial; word lpos = lpos_initial; [NL]word lnsz_initial; word lnsz = lnsz_initial; [NL]word lleaf_initial; word lleaf = lleaf_initial; [NL]word lrsoff_initial; word lrsoff = lrsoff_initial; [NL]word lrskind_initial; word lrskind = lrskind_initial; int newsz, newleaf, nomagic, callpos; -[NL]word lptr_initial; word lptr = lptr_initial; // annotate mode: the local was indexed or dereferenced, so it holds an address -// storage facts are shared by aliases. they describe one allocation, not a -// variable slot: [length, lifetime, kind, allocation place, end place, owner]. -// lifetime is 0 unknown, 1 live, 2 ended; kind is 1 stack, 2 heap, 3 array handle. -[NL]word lafact_initial; word lafact = lafact_initial; -[NL]word laaddr_initial; word laaddr = laaddr_initial; word xaf, argfact; -// local name lookup keeps a head for each spelling and a shadow link per slot. -[NL]word lprev_initial; word lprev = lprev_initial; -[256]word lkeys_initial; word lkeys = lkeys_initial; -[256]word lheads_initial; word lheads = lheads_initial; int lkeycap = 256, lkeyn; +word[NL] lpos_initial; word lpos = lpos_initial; word[NL] lreadpos_initial; word lreadpos = lreadpos_initial; word[NL] lnsz_initial; word lnsz = lnsz_initial; word[NL] lleaf_initial; word lleaf = lleaf_initial; word[NL] lrsoff_initial; word lrsoff = lrsoff_initial; word[NL] lrskind_initial; word lrskind = lrskind_initial; int newsz, newleaf, nomagic, callpos; +word[NL] lptr_initial; word lptr = lptr_initial; // annotate mode: the local was indexed or dereferenced, so it holds an address +//: storage facts are shared by aliases. they describe one allocation, not a +//: variable slot: [length, lifetime, kind, allocation place, end place, owner]. +//: lifetime is 0 unknown, 1 live, 2 ended; kind is 1 stack, 2 heap, 3 array handle. +word[NL] lafact_initial; word lafact = lafact_initial; +word[NL] laaddr_initial; word laaddr = laaddr_initial; word xaf, argfact; + +//: value previews are separate from optimization facts. each reference keeps +//: the value proved at that position: 1 integer, 2 float bits, 3 string, 4 bytes. +word lvkind, lvvalue, lvescaped, svkind, svvalue; int xvk; word xvv; +//: local name lookup keeps a head for each spelling and a shadow link per slot. +word[NL] lprev_initial; word lprev = lprev_initial; +word[256] lkeys_initial; word lkeys = lkeys_initial; +word[256] lheads_initial; word lheads = lheads_initial; int lkeycap = 256, lkeyn; int s_new_k, s_recycle_leaf; -// dgprev: the next record sits at the end of the previous token +//: dgprev: the next record sits at the end of the previous token +//: chk_hook, when set, is called after each top-level item of the document in check mode: the language +//: server streams diagnostics and drops a check whose source has been replaced +word chk_hook; int cmode, chkabort, chkfuel; word dgbuf, dggrow; int dgcap, dgpos, dgn, dgopen, dgprev, dgmode; // expand a diagnostic in progress, keeping its output offset within the buffer. int dg_expand() { - dgbuf = chk_resize(dgbuf, dgpos + 5 + outn, dgcap * 2); - dgcap = dgcap * 2; outbuf = dgbuf + dgpos + 5; outcap = dgcap - dgpos - 5; - return 0; -} -// diagnostic records for check mode. dg_open starts a record [severity, line, -// offset, length, text length, text...] at the current token (or just past the -// previous one when dgprev is set) and redirects the error text into it with -// outmode 2; dg_close seals it with the length of the text and restores output. -// severity 1 is an error, 2 a warning, and 4 a hint. + dgbuf = chk_resize(dgbuf, dgpos + 5 + rt.outn, dgcap * 2); + dgcap = dgcap * 2; rt.outbuf = dgbuf + dgpos + 5; rt.outcap = dgcap - dgpos - 5; + return 0; +} +//: diagnostic records for check mode. dg_open starts a record [severity, line, +//: offset, length, text length, text...] at the current token (or just past the +//: previous one when dgprev is set) and redirects the error text into it with +//: outmode 2; dg_close seals it with the length of the text and restores output. +//: severity 1 is an error, 2 a warning, and 4 a hint. +//: completion: the language server asks what is in scope at a place in the document. probe_off is that place +//: (a byte offset; -1 when nobody asks). the check keeps a copy of the locals at the last statement that starts at or +//: before it, with whether the lifetime analysis has used each one up, and, when the place is a member name after +//: a dot, what is before the dot. the check stops as soon as it has what was asked for +int probe_off = -1, probe_has, probe_n, probe_cap, probe_static; +char** probe_name; word probe_kind, probe_state, probe_type, probe_decl, probe_mem; +char* curinc; int incdepth, dgdrop; int incplace; // how many includes deep the compile is; a record to throw away; the place of the outermost include int dg_close() { if (!dgopen) return 0; - dgbuf[dgpos + 4] = outn; dgpos = dgpos + 5 + outn; dgn++; - dgopen = 0; outmode = dgmode; outgrow = dggrow; + if (!dgdrop) { dgbuf[dgpos + 4] = rt.outn; dgpos = dgpos + 5 + rt.outn; dgn++; } + dgdrop = 0; + dgopen = 0; rt.outmode = dgmode; rt.outgrow = dggrow; return 0; } // open a check-mode diagnostic record of the given severity. @@ -344,24 +407,31 @@ int dg_open(int sev) { } else { dgbuf[dgpos + 1] = tline; dgbuf[dgpos + 2] = toff; dgbuf[dgpos + 3] = toklen; } dgbuf[dgpos] = sev; - outbuf = dgbuf + dgpos + 5; outn = 0; outcap = dgcap - dgpos - 5; - dggrow = outgrow; outgrow = dg_expand; - outmode = 2; dgopen = 1; + // inside an included file the document has no place for the finding: hints and warnings about a library are + // left out, and an error is shown on the include that brought the file in, with where it really is + if (incdepth > 0) { + dgbuf[dgpos + 1] = incplace & 0xFFFFFF; dgbuf[dgpos + 2] = ((incplace >> 40) & 0x1FFFFF) - 1; dgbuf[dgpos + 3] = (incplace >> 24) & 0xFFFF; + if (sev != 1) dgdrop = 1; + } + rt.outbuf = dgbuf + dgpos + 5; rt.outn = 0; rt.outcap = dgcap - dgpos - 5; + dggrow = rt.outgrow; rt.outgrow = dg_expand; + rt.outmode = 2; dgopen = 1; + if (incdepth > 0 && sev == 1) { rt.put_s("in "); rt.put_s(curinc); rt.put_s(", line "); rt.put_n(tline); rt.put_s(": "); } return 0; } -[256]word dgseen_initial; word dgseen = dgseen_initial; int dgseen_cap = 256, dgseen_n; +word[256] dgseen_initial; word dgseen = dgseen_initial; int dgseen_cap = 256, dgseen_n; // start and stop a check run. chk_begin points the diagnostics at a buffer // the caller owns; chk_end flushes any open record and restores output. int chk_begin(int buf, int cap) { cmode = 1; chkabort = 0; chkfuel = 0; dgbuf = buf; dgcap = cap; dgpos = 0; dgn = 0; dgopen = 0; - nerr = 0; panic = 0; dgmode = outmode; fill_w(dgseen, 0, dgseen_cap); dgseen_n = 0; + nerr = 0; panic = 0; dgmode = rt.outmode; rt.fill_w(dgseen, 0, dgseen_cap); dgseen_n = 0; return 0; } // end check mode and put the output back the way it was. int chk_end() { dg_close(); cmode = 0; - outmode = dgmode; + rt.outmode = dgmode; return 0; } // a hint is a finding that isn't an error: the program still compiles, and @@ -385,13 +455,36 @@ int dg_dup() { while (dgseen[i]) { if (dgseen[i] == key) return 1; i = (i + 1) & (dgseen_cap - 1); } dgseen[i] = key; dgseen_n++; return 0; } -// start a check-mode hint at token position p, or say no when hints are off. -int hint_begin(int p) { +//: hints on; hints small; hints off; set how many hints the code after it gets: +//: on uses informational ranges, small uses compact hint markers, and off +//: suppresses hints. each directive is kept with its byte +//: offset, so a hint is judged by where its code is, not where it was reported. +const int HINT_SMALL = 1, HINT_ALL = 2; +word hl_off, hl_lev; int hl_n, hl_cap = 16; +int hint_set(int level, int at) { + if (!cmode || dry || isp != 0 || chkind != 3) return 0; + if (hl_n && hl_off[hl_n - 1] >= at) return 0; + if (!hl_off) { hl_cap = 16; hl_off = chk_resize(0, 0, hl_cap); hl_lev = chk_resize(0, 0, hl_cap); } + if (hl_n >= hl_cap) { hl_off = chk_resize(hl_off, hl_n, hl_cap * 2); hl_lev = chk_resize(hl_lev, hl_n, hl_cap * 2); hl_cap = hl_cap * 2; } + hl_off[hl_n] = at; hl_lev[hl_n] = level; hl_n++; + return 0; +} +// the hint level at byte offset o: 2 all, 1 small, 0 off +int hint_level(int o) { + int k = hl_n; + while (k) { k--; if (hl_off[k] <= o) return hl_lev[k]; } + return 2; +} +// start a check-mode hint at token position p, or say no when hints are off or +// this kind (HINT_SMALL or HINT_ALL) is not wanted there. +int hint_begin(int p, int kind) { int sv, ok; if (!cmode || dry || panic || chkabort || stubgen) return 0; + int level = hint_level(((p >> 40) & 0x1FFFFF) - 1); + if (!level) return 0; sv = tpack(); tunpack(p); ok = !dg_dup(); - if (ok) { dg_open(4); ok = dgopen; } + if (ok) { if (level == 1) dg_open(4); else dg_open(3); ok = dgopen; } tunpack(sv); return ok; } @@ -405,9 +498,9 @@ int errat() { if (cmode) { dg_open(1); } else { - put_s("zc: line "); - put_n(tline); - put_s(": "); + rt.put_s("zc: line "); + rt.put_n(tline); + rt.put_s(": "); } nerr++; return 0; @@ -415,45 +508,45 @@ int errat() { // finish an error line; the compiler stops after 20 errors. int errend() { if (cmode) { dg_close(); if (nerr >= 20) chkabort = 1; return 0; } - put_c(10); - if (nerr >= 20) { put_s("zc: too many errors, stopping\n"); halt(); } + rt.put_c(10); + if (nerr >= 20) { rt.put_s("zc: too many errors, stopping\n"); rt.halt(); } return 0; } // stop compiling after errors and report how many there were. int giveup() { if (cmode) { dg_close(); chkabort = 1; return 0; } - put_s("zc: "); put_n(nerr); if (nerr == 1) put_s(" error\n"); else put_s(" errors\n"); halt(); + rt.put_s("zc: "); rt.put_n(nerr); if (nerr == 1) rt.put_s(" error\n"); else rt.put_s(" errors\n"); rt.halt(); return 0; } // report a fatal error and stop, or record it in check mode. -int fatal(int msg) { - errat(); put_s(msg); +int fatal(char* msg) { + errat(); rt.put_s(msg); if (cmode) { dg_close(); chkabort = 1; return 0; } - put_c(10); halt(); + rt.put_c(10); rt.halt(); return 0; } // report a syntax error at the current token, once per recovery. -int serr(int msg) { +int serr(char* msg) { if (!panic && !dry) { errat(); - put_s(msg); + rt.put_s(msg); errend(); } return 0; } // report an error that names something, then the message. -int serrn(int name, int msg) { +int serrn(char* name, char* msg) { if (!panic && !dry) { errat(); - put_s(name); - put_s(": "); - put_s(msg); + rt.put_s(name); + rt.put_s(": "); + rt.put_s(msg); errend(); } return 0; } // report an error and skip to the next top-level item. -int die(int msg) { +int die(char* msg) { serr(msg); panic = 1; if (tk == T_EOF) giveup(); @@ -462,19 +555,19 @@ int die(int msg) { } // what the parser is looking at, for the "found" part of a message int tokshow() { - if (tk == T_EOF) { put_s("the end of the file"); return 0; } - if (tk == T_NUM) { put_s("the number "); put_n(tv); return 0; } - if (tk == T_STR) { put_s("a string"); return 0; } - if (tk == T_BSTR) { put_s("a byte string"); return 0; } - if (tk == T_FLT) { put_s("a float"); return 0; } - put_s("'"); put_s(tv); put_s("'"); + if (tk == T_EOF) { rt.put_s("the end of the file"); return 0; } + if (tk == T_NUM) { rt.put_s("the number "); rt.put_n(tv); return 0; } + if (tk == T_STR) { rt.put_s("a string"); return 0; } + if (tk == T_BSTR) { rt.put_s("a byte string"); return 0; } + if (tk == T_FLT) { rt.put_s("a float"); return 0; } + rt.put_s("'"); rt.put_s(tv); rt.put_s("'"); return 0; } // like die, and says what was found instead -int diefound(int msg) { +int diefound(char* msg) { if (!panic && !dry) { errat(); - put_s(msg); put_s(", found "); tokshow(); + rt.put_s(msg); rt.put_s(", found "); tokshow(); errend(); } panic = 1; @@ -483,15 +576,15 @@ int diefound(int msg) { return 0; } // die with a name shown before the message. -int dien(int name, int msg) { +int dien(char* name, char* msg) { serrn(name, msg); panic = 1; if (tk == T_EOF) giveup(); next(); return 0; } -// four classification bits live next to each interned spelling. this works -// for names in later pool chunks too, without indexing by a fixed pool offset. +//: four classification bits live next to each interned spelling. this works +//: for names in later pool chunks too, without indexing by a fixed pool offset. const int LSW = 1; word lsets; int lsmark(int set, int n) { @@ -502,19 +595,19 @@ word lsget(int set, int n) { if (!lsets) return 0; return (n[-2] >> set) & 1; } -// name resolution for the language server. while a check runs over the document -// at rsbase, every name a use or a declaration resolves to is written into rsrec -// as a record: the byte offset, kind (RS_LOCAL, RS_PARAM, RS_SYM or RS_FIELD), -// declaration or symbol, and storage facts at that position. a local names its declaration by offset -// (-1 when it has none), a symbol names its kind and flags (see rs_sym). the -// compiler's own scopes decide, so the highlighter and hover see locals and -// parameters first, then globals and built-ins, the way the compiler does. -const int RS_LOCAL = 1, RS_PARAM = 2, RS_SYM = 3, RS_FIELD = 4, RS_CAP = 32768, RS_W = 9; +//: name resolution for the language server. while a check runs over the document +//: at rsbase, every name a use or a declaration resolves to is written into rsrec +//: as a record: the byte offset, kind (RS_LOCAL, RS_PARAM, RS_SYM or RS_FIELD), +//: declaration or symbol, and storage facts at that position. a local names its declaration by offset +//: (-1 when it has none), a symbol names its kind and flags (see rs_sym). the +//: compiler's own scopes decide, so the highlighter and hover see locals and +//: parameters first, then globals and built-ins, the way the compiler does. +const int RS_LOCAL = 1, RS_PARAM = 2, RS_SYM = 3, RS_FIELD = 4, RS_CAP = 32768, RS_W = 13; int lt_on, lt_n, ltcap = 256, ltxtcap = 2048; word lt_off, lt_at, lt_len, lt_type, ltxt; int ltxn; int ht_n, htcap = 256; word ht_start, ht_end, ht_type; -int fieldn, fieldcap = 128; word fieldtypes, fieldnames, fieldoffs; -int field_note(word t, word n, int off) { +int fieldn, fieldcap = 128; word fieldtypes, fieldoffs; char** fieldnames; +int field_note(word t, char* n, int off) { if (!lt_on || isp != 0 || dry) return 0; if (!fieldtypes || fieldn == fieldcap) { if (fieldtypes) fieldcap = fieldcap * 2; @@ -525,18 +618,32 @@ int field_note(word t, word n, int off) { fieldtypes[fieldn] = t; fieldnames[fieldn] = n; fieldoffs[fieldn] = off; fieldn++; return 0; } -int field_decl(word t, word n) { +int field_decl(word t, char* n) { int k = 0; while (k < fieldn) { if (fieldtypes[k] == t && fieldnames[k] == n) return fieldoffs[k]; k++; } return 0 - 1; } -word rsrec, rstmp; [32]word pnoffs; int rsn, rscap, rsbase, nameoff; -// resize check-mode tables; their global pointers keep them visible to gc. -unchecked word chk_resize(word old, int count, int capacity) { - word p = new(capacity); - p[-4] = p[-4] | F_INTERNAL; - fill_w(p, 0, capacity); - if (old && count) copy_w(p, old, count); +//: where each symbol was defined, for go to definition: sdoff is the byte offset of its name + 1 +//: (0 when unknown, as for built-ins) and sdsrc the include path whose text it is in (0 for the document) +word sdoff, sdsrc; +//: includes: incdir is the directory of the file being compiled (0 for the program on stdin), increoot the +//: workspace folder an editor passes in, inc_hook a function that gives the text of a file the editor +//: sent (a host with no files of its own), incres the path an include was found at +char* incdir, increoot, incres, std_buf, icon_buf; char*(char*.m) inc_hook; int std_done, icon_done; +word rsrec, rstmp; word[32] pnoffs; word[32] pnplaces; int rsn, rscap, rsbase, nameoff; +//: ownership at a reference: record word 11 holds the local's state (S_* + 1, plus 16 when +//: it owns its object) and word 12 the declaration offset (+ 1) of the owner it borrows from. +//: transfers are logged as events, EV_W words each: the local's declaration offset, what +//: happened (EV_*), its line, and the target's declaration offset (+ 1) when there is one. +const int EV_RETURN = 1, EV_VALUE = 2, EV_UP = 3, EV_DOWN = 4, EV_STORE = 5, EV_GLOBAL = 6, EV_FREE = 7, EV_CAPTURE = 8, EV_MOVE = 9, EV_W = 4; +word evrec; int evn, evcap = 64; +// resize compiler tables. these buffers are collector-managed: m* relinquishes +// scope cleanup, and their local or global pointers keep them visible to gc. +word chk_resize(word old, int count, int capacity) { + word p = m*rt.new(capacity); + p[-4] = p[-4] | rt.F_INTERNAL; + rt.fill_w(p, 0, capacity); + if (old && count) rt.copy_w(p, old, count); return p; } // lexer and documentation buffers grow before storing the next character. @@ -566,7 +673,11 @@ int reserve_symbols(int need) { sfix = chk_resize(sfix, nsym, cap); snext = chk_resize(snext, nsym, cap); stype = chk_resize(stype, nsym, cap); + if (effect_symbols) effect_symbols = chk_resize(effect_symbols, nsym, cap); + if (fnmeta) fnmeta = chk_resize(fnmeta, nsym, cap); if (ifs) ifs = chk_resize(ifs, symcap, cap); + if (svkind) { svkind = chk_resize(svkind, nsym, cap); svvalue = chk_resize(svvalue, nsym, cap); } + if (sdoff) { sdoff = chk_resize(sdoff, nsym, cap); sdsrc = chk_resize(sdsrc, nsym, cap); } symcap = cap; return cap; } int reserve_locals(int need) { @@ -584,12 +695,21 @@ int reserve_locals(int need) { ltype = chk_resize(ltype, nloc, cap); ltag = chk_resize(ltag, nloc, cap); lpos = chk_resize(lpos, nloc, cap); + lreadpos = chk_resize(lreadpos, nloc, cap); lnsz = chk_resize(lnsz, nloc, cap); lleaf = chk_resize(lleaf, nloc, cap); lrsoff = chk_resize(lrsoff, nloc, cap); lrskind = chk_resize(lrskind, nloc, cap); lptr = chk_resize(lptr, nloc, cap); + if (if_local) if_local = chk_resize(if_local, nloc, cap); + if (local_origin) local_origin = chk_resize(local_origin, nloc, cap); + if (local_origin_mask) local_origin_mask = chk_resize(local_origin_mask, nloc, cap); + if (local_cell) local_cell = chk_resize(local_cell, nloc, cap); + if (local_captured) local_captured = chk_resize(local_captured, nloc, cap); + if (local_ended) local_ended = chk_resize(local_ended, nloc, cap); + if (local_end_info) local_end_info = chk_resize(local_end_info, nloc, cap); lafact = chk_resize(lafact, nloc, cap); laaddr = chk_resize(laaddr, nloc, cap); lprev = chk_resize(lprev, nloc, cap); + if (lvkind) { lvkind = chk_resize(lvkind, nloc, cap); lvvalue = chk_resize(lvvalue, nloc, cap); lvescaped = chk_resize(lvescaped, nloc, cap); } o3cache = chk_resize(o3cache, nloc, cap); loccap = cap; return cap; } @@ -699,10 +819,36 @@ int rs_put(int off, int kind, int ref, int sym) { } at = rsn * RS_W; rsrec[at] = off; rsrec[at + 1] = kind; rsrec[at + 2] = ref; rsrec[at + 3] = sym; - fill_w(rsrec + at + 4, 0, RS_W - 4); + rt.fill_w(rsrec + at + 4, 0, RS_W - 4); rsn++; return 0; } +// the ownership of local i at the record for the name at off +int os_at(int off, int i) { + int at, r, k = rsn; + if (!rsn || !cmode || dry || i < 0 || chkind != 3 || chbase != rsbase) return 0; + while (k && rsrec[(k - 1) * RS_W] != off) k--; + if (!k) return 0; + at = (k - 1) * RS_W; + rsrec[at + 11] = lstate[i] + 1; + if (lown[i]) rsrec[at + 11] = rsrec[at + 11] + 16; + if (lstate[i] == S_CHILD) { r = lroot(i); if (lrsoff[r] >= 0) rsrec[at + 12] = lrsoff[r] + 1; } + return 0; +} +// log a transfer of local i; the last few events are searched so a loop body is not logged twice +int ev_put(int i, int kind, int target) { + int k = evn, n = 0, t = 0; + if (!rsrec || !cmode || dry || chkind != 3 || chbase != rsbase || i < 0 || lrsoff[i] < 0) return 0; + if (target >= 0 && lrsoff[target] >= 0) t = lrsoff[target] + 1; + while (k && n < 8) { + k--; n++; + if (evrec[k * EV_W] == lrsoff[i] && evrec[k * EV_W + 1] == kind && evrec[k * EV_W + 2] == tline) return 0; + } + if (!evrec) { evcap = 64; evrec = chk_resize(0, 0, evcap * EV_W); } + if (evn >= evcap) { evrec = chk_resize(evrec, evn * EV_W, evcap * 2 * EV_W); evcap = evcap * 2; } + evrec[evn * EV_W] = lrsoff[i]; evrec[evn * EV_W + 1] = kind; evrec[evn * EV_W + 2] = tline; evrec[evn * EV_W + 3] = t; + evn++; return 0; +} // write a record for the name token that is current. int rs_note(int kind, int ref) { return rs_put(toff, kind, ref, 0 - 1); } // the flags of every symbol record, read again once the check has defined the @@ -710,7 +856,10 @@ int rs_note(int kind, int ref) { return rs_put(toff, kind, ref, 0 - 1); } int rs_fix() { int i = 0; while (i < rsn) { - if (rsrec[i * RS_W + 1] == RS_SYM && rsrec[i * RS_W + 3] >= 0) rsrec[i * RS_W + 2] = rs_sym(rsrec[i * RS_W + 3]); + if (rsrec[i * RS_W + 1] == RS_SYM && rsrec[i * RS_W + 3] >= 0) { + int s = rsrec[i * RS_W + 3]; rsrec[i * RS_W + 2] = rs_sym(s); + if (skind[s] == K_CONST) vf_at(i, 1, sval[s]); + } i++; } return 0; @@ -725,7 +874,7 @@ int rs_sym(int s) { } // copies record a of from to record b of to (the merge sort moves records this way). int rs_move(word from, int a, word to, int b) { - copy_w(to + b * RS_W, from + a * RS_W, RS_W); + rt.copy_w(to + b * RS_W, from + a * RS_W, RS_W); return 0; } // sort the records by offset: a bottom-up merge sort through rstmp, so a long @@ -755,29 +904,44 @@ int rs_sort() { return 0; } // analyzer errors only count in checked, reachable code -int cerr(int msg) { +int cerr(char* msg) { if (fchk && live) serr(msg); return 0; } // report an analyzer error about local i, named in the message. -int cerrn(int i, int msg) { - if (fchk && live) serrn(lname[i], msg); - return 0; +int cerrn(int i, char* msg) { + int root = i, cause = 0; word info = 0; + if (fchk && live) { + if (rt.eq_w(msg, "used after it was given away")) { + root = lroot(i); if (local_end_info) info = local_end_info[i]; + if (!info && local_end_info) info = local_end_info[root]; + if (info) cause = info[2]; else if (local_ended) cause = local_ended[root]; + } + if (cause && !panic && !dry) { + errat(); rt.put_s(lname[i]); rt.put_s(": "); rt.put_s(msg); + rt.put_s("; owner "); if (info) rt.put_s(info[0]); else rt.put_s(lname[root]); + rt.put_s(" was transferred on line "); rt.put_n(cause & 0xFFFFFF); + if (info && info[1]) { rt.put_s(" (declared on line "); rt.put_n(info[1] & 0xFFFFFF); rt.put_c(')'); } + errend(); + } else serrn(lname[i], msg); + } + return 0; } -// ---- interning ---- +//: ---- interning ---- + // the pool: an open addressing hash table (R_IH) of pointers into a string // pool (R_POOL). each spelling has two spare words: its symbol-chain head at // p[-1] and editor classification bits at p[-2]. equal text returns the same // stable handle, which lookup can use directly. -int hashs(word s) { +int hashs(char* s) { int h = 0; while (*s) { h = h * 31 + *s; s++; } return h & (hashcap - 1); } // rehash the index without moving strings or their symbol-chain heads. int reserve_names(int need) { - int cap = hashcap, previous = hashcap, i = 0, h; word p, old; + int cap = hashcap, previous = hashcap, i = 0, h; char* p; char** old; if (need <= cap) return cap; while (cap < need) cap = cap * 2; old = namehash; @@ -791,17 +955,17 @@ int reserve_names(int need) { return cap; } // return the interned copy of a spelling. new chunks preserve every old handle. -word intern(word s) { +char* intern(char* s) { int h; word p; int n; if ((namecount + 1) * 2 >= hashcap) reserve_names(hashcap * 2); h = hashs(s); while (1) { p = namehash[h]; if (!p) break; - if (p[0] == s[0] && eq_w(p, s)) return p; + if (p[0] == s[0] && rt.eq_w(p, s)) return p; h = (h + 1) & (hashcap - 1); } - n = len_w(s); + n = rt.len_w(s); if (pooltop + n + 3 > poolend) { if (poolchunk < n + 3) poolchunk = n + 3; else if (poolchunk < 0x20000000) poolchunk = poolchunk * 2; @@ -810,18 +974,26 @@ word intern(word s) { } pooltop[0] = 0; pooltop[1] = 0; p = pooltop + 2; - copy_w(p, s, n + 1); + rt.copy_w(p, s, n + 1); pooltop = pooltop + n + 3; namehash[h] = p; namecount++; return p; } -// ---- symbols ---- +//: ---- symbols ---- + // each interned name keeps the index (+1) of its newest symbol in p[-1], and // each symbol links to the one it shadows, so lookup is one step. -int addsym(word name, int kind, int val) { +// remember where symbol s is defined, when the prescan made it before its definition was read +int def_note(int s, int off) { + if (cmode && !dry && !ginst && sdoff && off >= 0 && !sdoff[s]) { sdoff[s] = off + 1; sdsrc[s] = curinc; } + return 0; +} +int addsym(char* name, int kind, int val) { int i = nsym; + if (module_name && kind != K_KW) name = module_symbol(name); if (nsym >= symcap) reserve_symbols(nsym + 1); + if (!fnmeta) fnmeta = chk_resize(0, 0, symcap); nsym++; sname[i] = name; skind[i] = kind; @@ -829,27 +1001,88 @@ int addsym(word name, int kind, int val) { ssig[i] = 0; sflag[i] = F_DEF; sfix[i] = 0; + fnmeta[i] = 0; stype[i] = 0; snext[i] = name[-1]; name[-1] = i + 1; + if (cmode && !dry && !sdoff) { sdoff = chk_resize(0, 0, symcap); sdsrc = chk_resize(0, 0, symcap); } + if (sdoff) { + sdoff[i] = 0; sdsrc[i] = 0; + if (cmode && !dry && !ginst && nameoff >= 0) { sdoff[i] = nameoff + 1; sdsrc[i] = curinc; } + } return i; } // while an image builds, names below vislo are hidden, except keywords -int lookup(word name) { - int i = name[-1] - 1; +int lookup_raw(char* name) { + int i = name[-1] - 1, resident; + if (syntax_host_floor && i >= syntax_host_floor && i < syntax_host_from) { + resident = i; + while (resident >= syntax_host_floor) resident = snext[resident] - 1; + if (resident >= 0) return resident; + if (skind[i] == K_VAR) serr("a syntax handler cannot access emitted program globals"); + } while (i >= 0) { if (i >= vislo || skind[i] == K_KW || skind[i] == K_TYPE) return i; i = snext[i] - 1; } return -1; } +// qualify a declaration within its file namespace; generated method names retain their prefix. +char* module_symbol(char* name) { + int i = 0, j; char* b; char* q; + while (name[i]) { if (name[i] == '.') return name; i++; } + if (!module_name) return name; + if (((name[0] == 'd' && name[1] == 'o' && name[2] == 'c') || (name[0] == 's' && name[1] == 'i' && name[2] == 'g')) && name[3] == 32) { + q = methname(module_name, name + 4); b = chk_resize(0, 0, rt.len_w(q) + 5); + rt.copy_w(b, name, 4); + j = 0; while (q[j]) { b[j + 4] = q[j]; j++; } b[j + 4] = 0; + return intern(b); + } + return methname(module_name, name); +} +int lookup(char* name) { + int s, i = 0; + if (module_name) { s = lookup_raw(module_symbol(name)); if (s >= 0) return s; } + s = lookup_raw(name); if (s >= 0) return s; + while (name[i]) { if (name[i] == '.') return -1; i++; } + return lookup_raw(methname("rt", name)); +} +// modules loaded from source are tracked independently of each file's imports. +word included_modules; +int module_loaded(char* name) { + word p = included_modules; + while (p) { if (rt.eq_w(p[1], name)) return 1; p = p[0]; } + return 0; +} +int module_known(char* name) { + return module_loaded(name) || rt.eq_w(name, "rt") || rt.eq_w(name, "run") || rt.eq_w(name, "core") || rt.eq_w(name, "lsp") || rt.eq_w(name, "std") || rt.eq_w(name, "icon"); +} +int module_imported(char* name) { + word p = module_imports; + if (module_name && rt.eq_w(name, module_name)) return 1; + while (p) { if (rt.eq_w(p[1], name)) return 1; p = p[0]; } + return 0; +} +// source references require their module's explicit import; compiler-generated calls use lookup directly. +int source_lookup(char* name) { + int i = 0, s; char* owner; + while (name[i] && name[i] != '.') i++; + if (name[i]) { + owner = chk_resize(0, 0, i + 1); rt.copy_w(owner, name, i); owner[i] = 0; + if ((module_known(owner) || module_loaded(owner)) && !module_imported(owner)) { + errat(); rt.put_s("module access requires `include "); rt.put_s(owner); rt.put_s(";`"); errend(); return -1; + } + } + if (module_name) { s = lookup_raw(module_symbol(name)); if (s >= 0) return s; } + return lookup_raw(name); +} // drop the symbols above index to, which ends a scope. int popsyms(int to) { while (nsym > to) { nsym--; sname[nsym][-1] = snext[nsym]; } return 0; } // look a name up from a plain string. -word lookups(int s) { return lookup(intern(s)); } +int lookups(char* s) { return lookup(intern(s)); } // patch nodes int fixnew(int site, int link) { @@ -872,7 +1105,8 @@ int fixall(int head, int v) { return 0; } -// ---- code emission ---- +//: ---- code emission ---- + // code goes straight into memory at "here". hdelta shifts where it is written // (an image build emits at one address and stores at another, see nested // compiles), and dry turns emission off entirely, which typeof and similar @@ -898,9 +1132,9 @@ int reserve_code(int need) { if (here + 1 >= hlimit) fatal("no room for a code link"); cap = code_chunk; if (cap < need + 5) cap = need + 5; - p = chk_resize(0, 0, cap); top = p + p[-5] - HDR; - if (syscall(9, 9, top, 0) < top) fatal("code address reservation failed"); - p[0] = code_blocks; p[1] = p + 3; p[2] = top; code_blocks = p; + p = chk_resize(0, 0, cap); top = p + p[-5] - rt.HDR; + if (rt.syscall(9, 9, top, 0) < top) fatal("code address reservation failed"); + p[0] = rt.code_blocks; p[1] = p + 3; p[2] = top; rt.code_blocks = p; *here = (p + 3) * 2 + 1; here[1] = O_JMP * 2; here = p + 3; hlimit = top; if (cap < 0x20000000) code_chunk = cap * 2; @@ -908,8 +1142,8 @@ int reserve_code(int need) { } // a region handed to an optimizer must fit in one contiguous segment. int code_contiguous(int lo, int hi) { - word p = code_blocks; - if (image_active || (lo >= 0 && hi <= CODE_END)) return 1; + word p = rt.code_blocks; + if (image_active || (lo >= 0 && hi <= rt.CODE_END)) return 1; while (p) { if (lo >= p[1] && hi <= p[2] - 2) return 1; p = p[0]; } return 0; } @@ -949,7 +1183,7 @@ int jmp(int t) { int hole() { int a; reserve_code(1); a = here; - lit(MW - 1); + lit(rt.MW - 1); return a; } // forward references. calling a function that is not defined yet creates a @@ -958,26 +1192,31 @@ int hole() { // itself to the symbol's chain (sfix); defining the function runs fixall to // patch every site. MW-1 is a jump target that halts, so an unpatched call // stops the machine instead of running garbage. -int fwdfn_at(int name, int p) { +int fwdfn_at(char* name, int p) { int s = addsym(name, K_FN, p); sflag[s] = 0; return s; } // the place of the first use (line, and offset above bit 24), until it's defined -int fwdfn(int name) { +int fwdfn(char* name) { return fwdfn_at(name, tpack()); } // emit a reference to function s, patched later if it is not defined yet. int fnref(int s) { + if (syntax_host_floor && s >= syntax_host_floor && s < syntax_host_from) { + s = syntax_host_function(s); + if (s < 0) { lit(rt.MW - 1); return 0; } + } if (sflag[s] & F_DEF) { lit(sval[s]); return 0; } if (dry) return 0; reserve_code(1); sfix[s] = fixnew(here, sfix[s]); - lit(MW - 1); // a jump to MW-1 halts if it ever runs unpatched + lit(rt.MW - 1); // a jump to MW-1 halts if it ever runs unpatched return 0; } // emit a call to the function with this name, making a forward slot if it is not defined. -int callname(int name) { +int callname(char* name) { int s = lookup(name); + if (s < 0) { name = methname("rt", name); s = lookup(name); } if (s < 0) s = fwdfn(name); fnref(s); op(O_CALL); @@ -986,7 +1225,7 @@ int callname(int name) { // the address of local slot k: the frame pointer lives in the word ZFP, and a // local is fp + k. (o3 locals live in cpu registers instead; see vaddr.) int laddr(int k) { - lit(ZFP); + lit(rt.ZFP); op(O_LD); if (k) { lit(k); op(O_ADD); } return 0; @@ -1003,11 +1242,12 @@ word vaddr(int i) { return 0; } } - return laddr(lslot[i]); + laddr(lslot[i]); if (local_cell && local_cell[i]) op(O_LD); return 0; } -// registers for an o3 region's variables: the cpu has 7. a variable takes the -// one its region freed last (when a scope closed), or the next unused one -// ---- o3: the optimizing region compiler's bookkeeping ---- +//: registers for an o3 region's variables: the cpu has 7. a variable takes the +//: one its region freed last (when a scope closed), or the next unused one +//: ---- o3: the optimizing region compiler's bookkeeping ---- + // code inside an o3 region (or a function marked o3) keeps hot locals in the // cpu's registers. the compiler does not change what it emits for them; it // records which frame slot maps to which register and hands the finished @@ -1063,7 +1303,7 @@ int o3promote(int i) { o3rg[no3sl] = r; no3sl++; return 0; } -// a fixed []T can't change its items or its length, so o3 code keeps both in +// a fixed T[?] can't change its items or its length, so o3 code keeps both in // two hidden variables, loaded when the array variable is set (or when the // region starts), and indexes through them int o3mkcache(int i) { @@ -1090,7 +1330,7 @@ int o3region(int lo, int hi, word sl, word rg, int n) { o3dsc[0] = 0; o3dsc[1] = 2; o3dsc[2] = lo; o3dsc[3] = hi; o3dsc[4] = n; while (i < n) { o3dsc[5 + i + i] = sl[i]; o3dsc[6 + i + i] = rg[i]; i++; } o3dsc[5 + n + n] = o3fs; o3dsc[6 + n + n] = o3na; o3dsc[7 + n + n] = o3fn0; - install(o3dsc); + rt.install(o3dsc); return 0; } // o3 blocks go to the cpu once their function is done, so every jump in them is final @@ -1103,10 +1343,10 @@ int o3flush() { } // a string literal: the characters (one per word, then a 0) are emitted into // the code stream behind a jump, and the literal's value is their address -int strlit(word s) { +int strlit(char* s) { int j = hole(), a; op(O_JMP); - reserve_code(len_w(s) + 1); a = here; + reserve_code(rt.len_w(s) + 1); a = here; while (*s) { emit(*s); s++; } emit(0); setlit(j, here); @@ -1114,7 +1354,8 @@ int strlit(word s) { return 0; } -// ---- input sources ---- +//: ---- input sources ---- + // frame: 0 kind (0 stdin, 1 token list, 2 string) 1 object 2 index 3 end // 4 transparent 5..7 saved token 8..10 saved char state 11 owned 12 saved line // 13 saved token line 14 saved lookahead count 15..30 saved lookahead 31 saved char buffer @@ -1152,14 +1393,16 @@ int srcend(int c) { } // the character level of the lexer: ch is the current char (-1 at the end), // and chkind says where it comes from (0 stdin, 3 a byte string, anything -// else a word string). a stdin source ends at a backtick (see srcend). +// else a zero-terminated char* string). a stdin source ends at a backtick (see srcend). int nextch() { if (chkind == 3) { // the usual case: the unit readsrc packed ch = (chbase[chptr >> 2] >> ((chptr & 3) * 8)) & 255; if (ch) chptr++; else ch = -1; } else if (chkind == 0) { - if (qdone || eof()) { ch = -1; return 0; } - ch = get_c() & 255; + if (qdone || rt.eof()) { ch = -1; return 0; } + ch = rt.get_c() & 255; + // a read that waited for a slow pipe and found its end returns -1: that is the end, not a 0xFF byte + if (ch == 255 && rt.eof()) { qdone = 1; ch = -1; return 0; } if (srcend(ch)) { qdone = 1; ch = -1; return 0; } // a backtick ends the source } else { ch = *chptr & 255; @@ -1191,7 +1434,7 @@ int cpos() { // saved into a 32 word frame so src_pop can put them back exactly. kind 0 is // stdin, 1 a token list (obj[0] is the count, then triples of kind, value and // packed place: this is how macros, quote/grab and feed replay code), 2 a -// word string and 3 a byte string. trans means "transparent": when a token +// char* string and 3 a byte string. trans means "transparent": when a token // list runs out the reader pops it and carries on in the source below, which // is what lets a fed list splice into the text around it. owned means the // frame frees obj when it pops. @@ -1231,7 +1474,7 @@ int src_pop() { i++; } isp--; - if (f[11]) free(f[1]); + if (f[11]) rt.free(f[1]); f[1] = 0; // a dead source pointer would pin a quarantined block return 0; } @@ -1317,9 +1560,8 @@ int setstr(int i, int f) { } // b"...": length word, then bytes packed 8 per word, little-endian, and a 0 // after the last byte. a jump skips the data. the address pushed is the length. -int bstrlit(word s) { - int j = hole(), a, n = 0, i = 0, w, k; - op(O_JMP); +int bstrdata(char* s) { + int a, n = 0, i = 0, w, k; while (s[n]) n++; reserve_code((n >> 3) + 2); a = here; emit(n); @@ -1329,11 +1571,14 @@ int bstrlit(word s) { emit(w); } if (!(n & 7)) emit(0); - setlit(j, here); - lit(a); - return 0; + return a; } -// ---- float literals and float code ---- +int bstrlit(char* s) { + int j = hole(), a; + op(O_JMP); a = bstrdata(s); setlit(j, here); lit(a); return 0; +} +//: ---- float literals and float code ---- + // a float is one word holding the ieee bits. the lexer collects the literal's // text in tbuf and fbits asks the core (opcode 611) to convert it, so the // compiler needs no float parser of its own. float operations are single vm @@ -1399,6 +1644,8 @@ int ftest() { // a float stored into an int becomes an int, and an int stored into a float // becomes a float. word keeps the bits and drops the float mark. int fcoerce(word dst) { + if (dst && treadonly(xty) && isref(xty) && !treadonly(dst) && dst[0] != TY_VAR) serr("const storage cannot be converted to a mutable reference"); + vf_coerce(dst, 1); if (!dst || dst[0] == TY_VAR) return 0; if (isfl(dst)) { if (!xfloat) { op(OP_I2F); xfloat = 1; xty = tfloat; } @@ -1476,13 +1723,18 @@ int fieldscan(int n) { // doc comment being collected, if the line above was part of it, or starts a // new one; a comment after code on the same line is not documentation. int doccomment(int nl) { - int cl = line; + int cl = line, start; nextch(); + if (ch == ':') { while (ch >= 0 && ch != 10) nextch(); dn = 0; dsect = 1; dlast = cl; docpending = 1; return 0; } if (!nl) { while (ch >= 0 && ch != 10) nextch(); return 0; } - if (dn && dlast != cl - 1) dn = 0; + if (dlast != cl - 1) { dn = 0; dsect = 0; } if (ch == 32) nextch(); + start = dn; while (ch >= 0 && ch != 10) { doc_char(ch); nextch(); } - while (dn && (dbuf[dn - 1] == 32 || dbuf[dn - 1] == 9 || dbuf[dn - 1] == 13)) dn--; + while (dn > start && (dbuf[dn - 1] == 32 || dbuf[dn - 1] == 9 || dbuf[dn - 1] == 13)) dn--; + // a comment block with a section heading is section text: it documents nothing + if (dn - start >= 4 && dbuf[start] == '-' && dbuf[start + 1] == '-' && dbuf[dn - 1] == '-' && dbuf[dn - 2] == '-') dsect = 1; + if (dsect) { dn = 0; dlast = cl; docpending = 1; return 0; } doc_char(10); dlast = cl; docpending = 1; return 0; @@ -1490,8 +1742,9 @@ int doccomment(int nl) { // block documentation can end on the declaration's line. remove the optional // leading star on each line, while leaving markdown inside the comment intact. int docblock(int nl) { - int cl = line, bol = 1, c; + int cl = line, bol = 1, c, adj = dsect && dlast == cl - 1; if (!nl || (dn && dlast != cl - 1)) dn = 0; + if (!adj) dsect = 0; nextch(); if (ch == '*') { nextch(); if (ch == '/') { nextch(); return 0; } } while (ch >= 0) { @@ -1507,6 +1760,7 @@ int docblock(int nl) { } while (dn && (dbuf[dn - 1] == 32 || dbuf[dn - 1] == 9 || dbuf[dn - 1] == 10 || dbuf[dn - 1] == 13)) dn--; doc_char(10); dlast = cl; docpending = 1; + if (adj) dn = 0; return 0; } // the lexer proper: skip white space and comments, then read one token into @@ -1516,8 +1770,31 @@ int docblock(int nl) { // white space, 0 when it does not, and 3 for a char literal (so the magic // number hint can leave 'a' alone). asm blocks use it to read -5 as one // literal. +// commented numeric literals are recorded by source offset, including lookahead. +word literal_docs; int literal_doc_n, literal_doc_cap = 32, token_block_doc; +int literal_documented(int off) { + int key = off + 1, i; + if (!literal_doc_n || key <= 0) return 0; + i = dg_hash(key, literal_doc_cap); + while (literal_docs[i]) { if (literal_docs[i] == key) return 1; i = (i + 1) & (literal_doc_cap - 1); } + return 0; +} +int literal_doc_note(int off) { + int key = off + 1, i, cap, k; word old; + if (literal_documented(off)) return 0; + if (!literal_docs) literal_docs = chk_resize(0, 0, literal_doc_cap); + if ((literal_doc_n + 1) * 2 >= literal_doc_cap) { + old = literal_docs; cap = literal_doc_cap * 2; literal_docs = chk_resize(0, 0, cap); + i = 0; + while (i < literal_doc_cap) { if (old[i]) { k = dg_hash(old[i], cap); while (literal_docs[k]) k = (k + 1) & (cap - 1); literal_docs[k] = old[i]; } i++; } + literal_doc_cap = cap; + } + i = dg_hash(key, literal_doc_cap); while (literal_docs[i]) i = (i + 1) & (literal_doc_cap - 1); + literal_docs[i] = key; literal_doc_n++; return 0; +} int lex1() { int n = 0, v, dg, nl; + token_block_doc = 0; if (pendot) { pendot = 0; tglue = 1; tline = line; toff = cpos(); tbuf[0] = '.'; tbuf[1] = 0; tk = T_OP; tv = intern(tbuf); @@ -1534,6 +1811,7 @@ int lex1() { tglue = 0; continue; } if (ch == '*') { + token_block_doc = 1; if (docs_on) { docblock(nl); tglue = 0; continue; } nextch(); while (ch >= 0) { if (ch == '*') { nextch(); if (ch == '/') break; } else nextch(); } @@ -1584,6 +1862,11 @@ int lex1() { tbuf[n] = 0; tk = T_BSTR; tv = intern(tbuf); return 0; } + if (ch == '.' && module_known(tbuf) && findloc(intern(tbuf)) < 0) { + n = token_char(n, '.'); nextch(); + while (isidc(ch)) { n = token_char(n, ch); nextch(); } + tbuf[n] = 0; + } tk = T_ID; tv = intern(tbuf); return 0; } if (ch == 39) { @@ -1625,6 +1908,9 @@ int lex1() { // lex1 plus the token's length, so the language server knows its extent int lex() { lex1(); + if (cmode && tk == T_NUM && token_block_doc && chkind == 3 && chbase == rsbase && toff >= 0) { + literal_doc_note(toff); + } if (toff < 0) toklen = 0; else toklen = cpos() - toff; return 0; } @@ -1670,7 +1956,7 @@ int peek() { return peekat(0); } // case in them. int next0() { word f, q; int i; - if (chkabort) { tk = T_EOF; tv = 0; chkfuel++; if (chkfuel > 3000000) halt(); return 0; } + if (chkabort) { tk = T_EOF; tv = 0; chkfuel++; if (chkfuel > 3000000) rt.halt(); return 0; } f = R_SRC + isp * 32; if (f[0] != 1) { lex(); return 0; } i = f[2]; @@ -1703,7 +1989,7 @@ int expect(int s) { dgprev = 1; errat(); dgprev = 0; - put_s("expected '"); put_s(s); put_s("', found "); tokshow(); + rt.put_s("expected '"); rt.put_s(s); rt.put_s("', found "); tokshow(); errend(); } panic = 1; @@ -1720,10 +2006,11 @@ int name() { return n; } -// ---- operator tables ---- +//: ---- operator tables ---- + // a sequence is written like asm: one-char mnemonics, numbers, and names // (resolved when the operator is used, so image builds stay self-contained) -int seqasm(word s) { +int seqasm(char* s) { int start = seqtop, n, neg, v, k, c; while (1) { while (*s == 32) s++; @@ -1814,7 +2101,7 @@ int asgfind(int p) { } // add or replace a binary operator. code is a sequence string, or 0 for the // built-in && (prec 2) and || (prec 1) behaviour, chosen by -1 / -2 in prec. -int add_binop(int nm, int prec, int code) { +int add_binop(char* nm, int prec, char* code) { int p = intern(nm), i = binfind(p), a, n; if (i < 0) { reserve_binary(nbin + 1); i = nbin; nbin++; } bname[i] = p; bprec[i] = prec; @@ -1822,9 +2109,9 @@ int add_binop(int nm, int prec, int code) { bseq[i] = seqasm(code); blen[i] = seqtop - bseq[i]; addkop(p); // and its compound assignment, x op= y - n = len_w(nm); + n = rt.len_w(nm); while (1002 + n > tbufcap) { tbuf = chk_resize(tbuf, 0, tbufcap * 2); tbufcap = tbufcap * 2; } - copy_w(tbuf + 1000, nm, n); + rt.copy_w(tbuf + 1000, nm, n); tbuf[1000 + n] = '='; tbuf[1001 + n] = 0; a = intern(tbuf + 1000); if (prec >= 8 || prec == 5 || prec == 4 || prec == 3) { @@ -1835,7 +2122,7 @@ int add_binop(int nm, int prec, int code) { return i; } // register a unary operator and the sequence it expands to. -int add_unop(int nm, int code) { +int add_unop(char* nm, char* code) { int p = intern(nm), i = unfind(p); if (i < 0) { reserve_unary(nun + 1); i = nun; nun++; } uname[i] = p; useq[i] = seqasm(code); ulen[i] = seqtop - useq[i]; @@ -1843,48 +2130,376 @@ int add_unop(int nm, int code) { return i; } // register a keyword with its handler and flags. -int add_kw(int nm, int handler, int flags) { +int add_kw(char* nm, int handler, int flags) { int s = addsym(intern(nm), K_KW, handler); sflag[s] = F_DEF | flags; return s; } +// register a library function as a receiver method. its first parameter +// determines the receiver type; remaining parameters follow the member name. +word receiver_methods, closure_expected; +int add_method(char* member, char* function_name) { + int s = lookup_raw(intern(function_name)); word link; + if (s < 0 || skind[s] != K_FN || !stype[s] || !stype[s][3]) { + serr("a receiver method requires a function with at least one typed parameter"); return -1; + } + link = chk_resize(0, 0, 3); + link[0] = receiver_methods; link[1] = intern(member); link[2] = s; receiver_methods = link; + return s; +} +int receiver_method(word receiver_type, int member) { + word link = receiver_methods, ft, actual; word[8] bs; int found = -1, s; + while (link) { + s = link[2]; + if (s < nsym && link[1] == member) { + ft = stype[s]; actual = receiver_type; + if (actual[0] == TY_ARR && ft[7][0] == TY_DYN && ft[7][3]) actual = tdyn(actual[4], 1); + rt.fill_w(bs, 0, 8); + if (unify(ft[7], actual, bs)) { if (found >= 0 && found != s) return -2; found = s; } + } + link = link[0]; + } + return found; +} +// run.method("member", function) registers a receiver extension while parsing +// the importing source, including when its code is emitted into an image. +int kw_method() { + char* member; int s; + stmtonly(); next(); expect(s_lp); + if (tk != T_STR) { die("a receiver method name must be a character string"); return 0; } + member = tv; next(); expect(s_comma); + s = -1; if (tk == T_ID) s = source_lookup(tv); + if (s < 0 || skind[s] != K_FN) serr("a receiver extension requires a function name"); + next(); expect(s_rp); expect(s_semi); + if (s >= 0) add_method(member, sname[s]); + return 0; +} +// publish a reference for shallow copies while preserving its declared type. +// scalar values are unchanged; in-place aggregates require explicit copying. +int kw_share_value() { + int c; word t; + next(); expect(s_lp); c = expr(); rval(); plainuse(c); t = xty; expect(s_rp); + if (isagg(t)) serr("sharing an aggregate requires sharing its pointer fields explicitly"); + else if (isptr(t) || t[0] == TY_DYN || t[0] == TY_FN || t == tword) callname(intern("rt.share")); + xbor = -1; xaf = 0; effect_value = 0; xty = t; xconst = 0; xvk = 0; + return C_PLAIN; +} + +word statement_hooks; + +// source libraries can register syntax without emitting a runtime registration call. +int kw_syntax() { + char* keyword; int handler, entry = -1; + stmtonly(); next(); expect(s_lp); + if (tk != T_STR) { die("a syntax keyword must be a character string"); return 0; } + keyword = tv; next(); expect(s_comma); handler = -1; + if (tk == T_ID) handler = source_lookup(tv); + if (handler < 0 || skind[handler] != K_FN) serr("a syntax extension requires a function name"); + next(); expect(s_rp); expect(s_semi); + if (handler >= 0) { + if (hdelta && handler >= ubase) entry = syntax_resident(handler); else entry = sval[handler]; + if (entry >= 0) add_kw(keyword, entry, F_TOP | F_STMT); + } + return 0; +} +int kw_statement_hook() { + int handler = -1; word link; + stmtonly(); next(); expect(s_lp); if (tk == T_ID) handler = source_lookup(tv); + if (handler < 0 || skind[handler] != K_FN) serr("a statement hook requires a function name"); + next(); expect(s_rp); expect(s_semi); + if (handler >= 0) { link = chk_resize(0, 0, 3); link[0] = statement_hooks; link[1] = handler; link[2] = 1; statement_hooks = link; } + return 0; +} +// optional statement callbacks installed by source extensions at include time. +// callbacks execute at ordinary vm call boundaries and must guard reentrancy. +int add_statement_hook(int callback) { + word link = chk_resize(0, 0, 3); link[0] = statement_hooks; link[1] = callback; link[2] = 0; statement_hooks = link; + return 0; +} +int emit_statement_hooks() { + word link = statement_hooks; + if (dry || o3on || cmode || incdepth || !infn) return 0; + while (link) { if (link[2]) fnref(link[1]); else lit(link[1]); op(O_CALL); op(O_DROP); link = link[0]; } + return 0; +} + +//: ---- function effects and temporary borrows ---- +word effect_record(int symbol) { + if (symbol < 0) return 0; + if (!effect_symbols) effect_symbols = chk_resize(0, 0, symcap); + if (!effect_symbols[symbol]) effect_symbols[symbol] = chk_resize(0, 0, 50); + return effect_symbols[symbol]; +} +// trusted runtime contracts apply only to built-ins, never a user function with the same name. +int effect_contract(int symbol, int parameter) { + word n; + if (symbol < 0 || symbol >= ubase) return EF_UNKNOWN; + n = sname[symbol]; + if (n == s_put_s || rt.eq_w(n, "rt.put_bs") || rt.eq_w(n, "rt.len_w") || rt.eq_w(n, "rt.len_s") || rt.eq_w(n, "rt.eq_w") || rt.eq_w(n, "rt.eq_s") || rt.eq_w(n, "rt.at_s") || rt.eq_w(n, "rt.put_n") || rt.eq_w(n, "rt.put_c")) return EF_READ; + if (n == s_free || n == s_recycle) return EF_CONSUME; + if (rt.eq_w(n, "rt.copy_w")) { if (!parameter) return EF_WRITE; return EF_READ; } + if (rt.eq_w(n, "rt.fill_w")) { if (!parameter) return EF_WRITE; return EF_READ; } + return EF_UNKNOWN; +} +int effect_get(int symbol, int parameter) { + word r; + if (parameter < 0 || parameter >= 24) return EF_UNKNOWN; + if (symbol < 0 || symbol >= nsym) return EF_UNKNOWN; + if (symbol < ubase) return effect_contract(symbol, parameter); + r = effect_record(symbol); + if (r[0] != 1 || (sflag[symbol] & F_EXT)) { + if (parameter < 12 && (((ssig[symbol] >> 6) >> (parameter + parameter)) & 3) == 1) return EF_READ | EF_WRITE | EF_ALIAS; + return EF_UNKNOWN; + } + return r[2 + parameter]; +} +int effect_origin(int c) { + int i = xbor; + if (c >= C_VAR) i = c - C_VAR; + if (i < lbase || i >= nloc) return -1; + if (local_origin && local_origin[i] >= 0) return local_origin[i]; + if (lrskind[i] == RS_PARAM && i - lbase < 24) return i - lbase; + return -1; +} +word effect_mask(int c) { + int i; + if (c >= C_VAR) { + i = c - C_VAR; + if (local_origin_mask && local_origin_mask[i]) return local_origin_mask[i]; + if (i >= lbase && lrskind[i] == RS_PARAM && i - lbase < 24) return 1 << (i - lbase); + } + return effect_value; +} +int effect_note_mask(word mask, int flags) { + int i = 0; + while (mask && i < 24) { if (mask & 1) effect_note(i, flags); mask = mask >> 1; i++; } + return 0; +} +int effect_args_mask(int callee, int parameter, word mask) { + int i = 0; + while (mask && i < 24) { if (mask & 1) effect_arg(callee, parameter, i); mask = mask >> 1; i++; } + return 0; +} +int effect_note(int origin, int flags) { + if (effect_current && origin >= 0 && origin < 24) { effect_current[2 + origin] = effect_current[2 + origin] | flags; effect_current[26 + origin] = effect_current[26 + origin] | flags; } + return 0; +} +int effect_arg(int callee, int parameter, int origin) { + word e; + if (!effect_current || origin < 0) return 0; + e = talloc(6); e[0] = effect_edges; e[1] = effect_current; e[2] = origin; e[3] = callee; e[4] = parameter; e[5] = 0; + effect_edges = e; return 0; +} +int effect_require_mode(int callee, int parameter, int place, int mode) { + word e; + if (!fchk || !live || dry) return 0; + e = talloc(7); e[6] = mode; + e[5] = effect_floor; + e[0] = effect_requirements; e[1] = callee; e[2] = parameter; e[3] = place; e[4] = curinc; + effect_requirements = e; return 0; +} +int effect_require(int callee, int parameter, int place) { return effect_require_mode(callee, parameter, place, 0); } +// argument checks for a generic call use the instantiated body's effects. +int effect_specialize(int original, int instance, word requirements, word edges) { + word e = effect_requirements; + while (e && e != requirements) { if (e[1] == original) e[1] = instance; e = e[0]; } + e = effect_edges; + while (e && e != edges) { if (e[3] == original) e[3] = instance; e = e[0]; } + return 0; +} +int effect_finish(int from) { + word e, r, next; int changed = 1, flags, prior, save = tpack(), inc = curinc, i = 0; + if (chkabort) return 0; + // direct body effects are immutable inputs; recompute callee effects so an earlier nested unit cannot leave stale unknowns. + while (effect_symbols && i < nsym) { r = effect_symbols[i]; if (r) rt.copy_w(r + 2, r + 26, 24); i++; } + while (changed) { + changed = 0; e = effect_edges; + while (e) { + r = e[1]; flags = effect_get(e[3], e[4]) & ~(EF_ALIAS | EF_IDENTITY); prior = r[2 + e[2]]; + r[2 + e[2]] = prior | flags; if (prior != r[2 + e[2]]) changed = 1; + e = e[0]; + } + } + e = effect_requirements; effect_requirements = 0; + while (e) { + next = e[0]; + if (e[5] < from) { e[0] = effect_requirements; effect_requirements = e; e = next; continue; } + flags = effect_get(e[1], e[2]); + if ((!e[6] && (flags & (EF_RETAIN | EF_CONSUME | EF_UNKNOWN))) || (e[6] && !(flags & EF_UNKNOWN) && (flags & (EF_RETAIN | EF_CONSUME)))) { + tunpack(e[3]); curinc = e[4]; + if (e[6]) serr("this callee retains or consumes a borrowed owner; declare its parameter .d to transfer ownership"); + else if (flags & EF_UNKNOWN) serr("a temporary argument requires a proved non-escaping borrow; bind it to a name first or provide an explicit parameter contract"); + else serr("a temporary argument requires a proved non-escaping borrow; this callee retains or consumes it, so transfer ownership through a .d parameter"); + panic = 0; + } + e = next; + } + tunpack(save); curinc = inc; return 0; +} +int borrow_begin() { return 0; } +int borrow_prologue() { borrow_jump = hole(); op(O_JMP); borrow_entry = here; return 0; } +int borrow_initializer() { + word p = borrow_slots; + if (borrow_jump < 0) return 0; + if (!p) { setlit(borrow_jump, borrow_entry); return 0; } + setlit(borrow_jump, here); + while (p) { lit(0); laddr(p[1]); op(O_ST); p = p[0]; } + jmp(borrow_entry); return 0; +} +int emit_borrow_hold() { + word p; + if (!infn) { cerr("temporary borrow outside a function"); return 0; } + p = talloc(2); p[0] = borrow_slots; p[1] = fslots; fslots++; borrow_slots = p; + lit(0); op(O_PICK); laddr(p[1]); op(O_ST); borrow_pending++; return 0; +} +// release only temporaries introduced by this full expression. zero slots cover untaken branches. +int borrow_cleanup_since(word stop) { + word p = borrow_slots; int skip; + while (p && p != stop) { + laddr(p[1]); op(O_LD); skip = hole(); op(O_JZ); + laddr(p[1]); op(O_LD); callname(s_recycle); op(O_DROP); + lit(0); laddr(p[1]); op(O_ST); setlit(skip, here); p = p[0]; + } + return 0; +} +int borrow_cleanup() { return borrow_cleanup_since(0); } +int borrow_condition() { + word before = borrow_slots; int c = expr(); rval(); plainuse(c); ftest(); borrow_cleanup_since(before); return c; +} + +//: ---- locals and the analyzer's view of them ---- + +// value tracking is for check mode only; it never changes emitted code. +int vf_symbols() { + if (!svkind) { svkind = chk_resize(0, 0, symcap); svvalue = chk_resize(0, 0, symcap); } + return 0; +} +int vf_at(int r, int kind, word value) { + if (!cmode || dry || chkind != 3 || chbase != rsbase || r < 0 || r >= rsn) return 0; + rsrec[r * RS_W + 9] = kind; rsrec[r * RS_W + 10] = value; return 0; +} +char* vf_string(char* text) { + word p; int n; + if (!cmode || dry) return 0; + n = rt.len_w(text) + 1; p = talloc(n); rt.copy_w(p, text, n); return p; +} +int vf_forget() { + if (!cmode || dry) return 0; + if (lvkind) rt.fill_w(lvkind + lbase, 0, nloc - lbase); + if (svkind) rt.fill_w(svkind, 0, nsym); + xvk = 0; return 0; +} +// a call or indirect write can change globals, escaped locals, and string data. +int vf_effect() { + int i = lbase; + if (!cmode || dry) return 0; + while (i < nloc) { if (lvescaped[i] || lvkind[i] >= 3) lvkind[i] = 0; i++; } + if (svkind) rt.fill_w(svkind, 0, nsym); + xvk = 0; return 0; +} +word vf_i2f(word v) { return asm { v @ #1216 }; } +word vf_f2i(word v) { return asm { v @ #1218 }; } +word vf_negf(word v) { return asm { 0.0 v @ #1204 }; } +int vf_coerce(word t, int convert) { + if (!xvk || !t) return 0; + if (xvk >= 3) { if (t != tword && t != tbstr && t[0] != TY_PTR) xvk = 0; return 0; } + if (isfl(t)) { + if (xvk == 1) { if (convert) xvv = vf_i2f(xvv); xvk = 2; } + if (t == tf32 && convert) xvv = rt.zs2f(rt.zf2s(xvv)); + } else { + if (xvk == 2 && convert && isnum(t)) xvv = vf_f2i(xvv); + xvk = 1; xvv = cnorm(t, xvv); + } + return 0; +} +int vf_store(int i) { + if (!cmode || dry || !live) return 0; + lvkind[i] = xvk; lvvalue[i] = xvv; return 0; +} -// ---- locals and the analyzer's view of them ---- // these warnings add no ownership rules. facts are discarded at control-flow // joins and after unknown effects, so a possible failure is never called definite. word af_new(int length, int kind, int place) { word f; if (dry || stubgen) return 0; - f = talloc(6); f[0] = length; f[1] = 1; f[2] = kind; + f = talloc(15); rt.fill_w(f, 0, 15); f[0] = length; f[1] = 1; f[2] = kind; f[3] = place; f[4] = 0; f[5] = -1; return f; } +// owner-relative projections retain provenance and the array generation at the time of borrowing. +word af_base(word f) { while (f && f[6]) f = f[6]; return f; } +word af_project(word f, int offset, int extent, int place, int elements) { + word p, base = af_base(f); int start = offset; + if (!f || !base) return 0; + if (f[6] && f[7] >= 0 && offset >= 0) start = f[7] + offset; + p = af_new(extent, base[2], base[3]); + if (!p) return 0; + p[1] = base[1]; p[4] = base[4]; p[5] = base[5]; p[6] = base; p[7] = start; + p[8] = elements; p[9] = base[10]; p[12] = place; + if (base[2] == 3) p[2] = 2; + if (elements) { + p[0] = -1; + if (offset >= 0 && f[0] >= 0) { p[0] = f[0] - offset; p[13] = f[13] - offset; p[14] = 1; } + } + return p; +} +int af_mutate(word f, int place) { + word base = af_base(f); + if (!base || dry || !live) return 0; + af_mutation_tick++; base[10] = af_mutation_tick; base[11] = place; base[0] = -1; return 0; +} +int af_projection_access(word f, int place) { + word base; int saved; + if (!f || !f[6] || !f[8]) return 0; + base = af_base(f); + if (base[10] < 0 || base[10] == f[9] || dry || !live) return 0; + saved = tpack(); tunpack(place); + if (fchk) { if (panic) { tunpack(saved); return 0; } errat(); } + else if (!af_warning(place)) { tunpack(saved); return 0; } + rt.put_s("element borrow was invalidated by array mutation on line "); rt.put_n(base[11] & 0xFFFFFF); + rt.put_s("; borrow created on line "); rt.put_n(f[12] & 0xFFFFFF); + if (fchk) errend(); else af_warning_end(); tunpack(saved); return 0; +} + int af_forget() { + vf_forget(); int i = lbase; while (i < nloc) { lafact[i] = 0; i++; } xaf = 0; return 0; } // an unknown callee may change a passed allocation's length or lifetime. int af_escape(word f) { - if (f) { f[0] = -1; f[1] = 0; } + if (f) { word base = af_base(f); base[0] = -1; base[1] = 0; base[10] = -1; f[0] = -1; f[1] = 0; } return 0; } int af_store(int i) { if (!dry) lafact[i] = xaf; return 0; } +int af_expired(word f) { + word base; + if (!f) return 0; + if (f[1] == 2) return 1; + if (!f[6] || !f[8]) return 0; + base = af_base(f); return base[10] >= 0 && base[10] != f[9]; +} +int af_snapshot(word f) { + if (f && f[6] && af_expired(f)) { f[1] = 2; f[4] = af_base(f)[11]; } + return 0; +} int af_record(int off, word f) { int at; if (!f || !cmode || dry || chkind != 3 || chbase != rsbase || !rsn) return 0; + af_snapshot(f); at = (rsn - 1) * RS_W; if (rsrec[at] != off) return 0; - copy_w(rsrec + at + 4, f, 5); return 0; + rt.copy_w(rsrec + at + 4, f, 5); return 0; } // keep declaration previews as well as the facts at each use site. int af_declaration(int off, word f) { int i = rsn; if (!f || !cmode || dry || chkind != 3 || chbase != rsbase) return 0; - while (i) { i--; if (rsrec[i * RS_W] == off) { copy_w(rsrec + i * RS_W + 4, f, 5); return 0; } } + while (i) { i--; if (rsrec[i * RS_W] == off) { rt.copy_w(rsrec + i * RS_W + 4, f, 5); return 0; } } return 0; } int af_warning(int place) { @@ -1892,20 +2507,27 @@ int af_warning(int place) { if (dry || panic || chkabort || stubgen || !live) return 0; sv = tpack(); tunpack(place); if (cmode) dg_open(2); - else { put_s("zc: line "); put_n(tline); put_s(": warning: "); } + else { rt.put_s("zc: line "); rt.put_n(tline); rt.put_s(": warning: "); } tunpack(sv); return 1; } -int af_warning_end() { if (cmode) dg_close(); else put_c(10); return 0; } +int af_warning_end() { if (cmode) dg_close(); else rt.put_c(10); return 0; } int af_access(word f, int place) { + af_projection_access(f, place); if (f && f[6]) { word base = af_base(f); f[1] = base[1]; f[4] = base[4]; } if (!f || f[1] != 2 || !af_warning(place)) return 0; - put_s("use after free: this storage's lifetime ended on line "); - put_n(f[4] & 0xFFFFFF); af_warning_end(); return 0; + rt.put_s("use after free: this storage's lifetime ended on line "); + rt.put_n(f[4] & 0xFFFFFF); af_warning_end(); return 0; +} +int af_use(word f, int place, int c) { + if (fchk && c >= C_VAR && lstate[lroot(c - C_VAR)] == S_DEAD) return 0; + return af_access(f, place); } int af_index(word f, int place) { - if (!f || f[1] == 2 || f[0] < 0 || !xconst || (xcval >= 0 && xcval < f[0])) return 0; + int lower = 0; + if (f && f[6]) { word base = af_base(f); if (base[1] == 0 || base[10] < 0) return 0; if (f[14]) lower = f[13]; } + if (!f || f[1] == 2 || f[0] < 0 || !xconst || (xcval >= lower && xcval < f[0])) return 0; if (!af_warning(place)) return 0; - put_s("out-of-bounds access: index "); put_n(xcval); - put_s(" is outside length "); put_n(f[0]); af_warning_end(); return 0; + rt.put_s("out-of-bounds access: index "); rt.put_n(xcval); + rt.put_s(" is outside length "); rt.put_n(f[0]); rt.put_s(" (storage declared or allocated on line "); rt.put_n(f[3] & 0xFFFFFF); rt.put_c(')'); af_warning_end(); return 0; } // freeing an alias ends the same allocation as freeing its original name. int af_end(word f, int place) { @@ -1916,9 +2538,9 @@ int af_close(int save) { int i = save; word f; if (dry) return 0; while (i < nloc) { - f = laaddr[i]; if (f) af_end(f, tpack()); + f = laaddr[i]; if (f && f[2] == 1) af_end(f, tpack()); f = lafact[i]; - if (f && fchk && lstate[i] == S_M && lown[i]) af_end(f, tpack()); + if (f && fchk && lstate[i] == S_M && lown[i] && !local_captured[i]) af_end(f, tpack()); i++; } return 0; @@ -1987,6 +2609,15 @@ int newloc(int n) { if (nloc >= loccap) reserve_locals(nloc + 1); nloc++; lname[i] = n; local_bind(n, i); lslot[i] = fslots; lhome[i] = fdepth; lstate[i] = S_PLAIN; lsmark(0, n); + if (!local_origin) local_origin = chk_resize(0, 0, loccap); local_origin[i] = -1; + if (if_local) if_local[i] = 0; + if (!local_origin_mask) local_origin_mask = chk_resize(0, 0, loccap); local_origin_mask[i] = 0; + if (!local_cell) local_cell = chk_resize(0, 0, loccap); local_cell[i] = 0; + if (!local_captured) local_captured = chk_resize(0, 0, loccap); local_captured[i] = 0; + if (!local_ended) local_ended = chk_resize(0, 0, loccap); local_ended[i] = 0; + if (!local_end_info) local_end_info = chk_resize(0, 0, loccap); local_end_info[i] = 0; + lpos[i] = 0; + lreadpos[i] = 0; lowner[i] = 0; lrsoff[i] = 0 - 1; lrskind[i] = RS_LOCAL; lused[i] = 0; @@ -1996,6 +2627,10 @@ int newloc(int n) { ltag[i] = 0; o3cache[i] = 0; lafact[i] = 0; laaddr[i] = 0; + if (cmode) { + if (!lvkind) { lvkind = chk_resize(0, 0, loccap); lvvalue = chk_resize(0, 0, loccap); lvescaped = chk_resize(0, 0, loccap); } + lvkind[i] = 0; lvvalue[i] = 0; lvescaped[i] = 0; + } return i; } // a local's known lower bound, and the array whose length it holds. -1 is unknown. @@ -2065,6 +2700,12 @@ int bpoison(int i) { } // note a store to a local: a constant or a length it now holds. int note_store(int i) { + if (if_local && !dry) { + if (!if_local[i]) if_local[i] = chk_resize(0, 0, 1); + if (if_value) if_link(if_local[i], if_value); + else if (xty == tword) *if_local[i] = -1; + if_join(if_local[i], xty); + } int lo = -1, ln = -1; if (xconst && xcval >= 0) lo = xcval; if (xlenloc >= 0) ln = xlenloc; @@ -2088,12 +2729,22 @@ int lroot(int i) { while (lstate[i] == S_CHILD) i = lowner[i]; return i; } +// retire existing aliases together with their owner. dead aliases do not keep later loop snapshots live. +int end_owner(int i) { + int j = lbase; word info = talloc(3); + info[0] = lname[i]; info[1] = lpos[i]; info[2] = tpack(); + local_ended[i] = info[2]; local_end_info[i] = info; lstate[i] = S_DEAD; + while (j < nloc) { + if (j != i && isalias(j) && lroot(j) == i) { local_ended[j] = info[2]; local_end_info[j] = info; lstate[j] = S_DEAD; } + j++; + } + return 0; +} // true when a local has an ownership state the analyzer follows. word tracked(int i) { return lstate[i] != S_PLAIN; } // a child link with nothing owned: a pointer computed from some other variable word isalias(int i) { return lstate[i] == S_CHILD && !lown[i]; } -// record the last-use line and saturating count read by the hints ("only -// handed off once"). +// record the last-use line and saturating reference count for diagnostics. int luse(int i) { lused[i] = tline; if ((ltag[i] >> 56) < 100) ltag[i] = ltag[i] + LCNT1; @@ -2123,13 +2774,14 @@ int plainuse(int c) { // the size class of an n word object, for a constant-size new int szclass(int n) { int k = 3; - while (pow2[k] < n + HDR) k++; + while (rt.pow2[k] < n + rt.HDR) k++; return k; } // freeing an owned local. one that holds a typed object with no pointer // fields goes back through recycle_leaf, which knows its class int freevar(int i) { - laddr(lslot[i]); op(O_LD); + if (local_captured && local_captured[i]) { lit(0); laddr(lslot[i]); op(O_ST); return 0; } + vaddr(i); op(O_LD); if (lleaf[i] && !o3on && !dry && lookup(s_recycle_leaf) >= 0) { lit(lleaf[i]); lit(szclass(lnsz[i])); @@ -2148,7 +2800,7 @@ int exit_vars(int dep, int final) { if (lhome[i] < dep) continue; st = lstate[i]; if (st == S_M) { - freevar(i); + if (!local_captured || !local_captured[i]) ev_put(i, EV_FREE, -1); freevar(i); } else if (st == S_U) { cerrn(i, "u object must leave this block (block value, assignment to the enclosing block, or return)"); } @@ -2160,15 +2812,15 @@ int exit_vars(int dep, int final) { // block ends. int unused_check(int from) { while (from < nloc) { - if (fchk && lown[from] && !lused[from]) serrn(lname[from], "m object is never used"); + if (fchk && lown[from] && !lused[from] && !(lrskind[from] == RS_PARAM && rt.eq_w(lname[from], "_"))) serrn(lname[from], "owned object is never used"); from++; } return 0; } // what a block's own variables show once it ends (check mode only) -int hint_name(int i, int msg) { - if (!hint_begin(lpos[i])) return 0; - put_s(lname[i]); put_s(msg); +int hint_name(int i, char* msg, int kind) { + if (!hint_begin(lpos[i], kind)) return 0; + rt.put_s(lname[i]); rt.put_s(msg); dg_close(); return 0; } @@ -2178,11 +2830,15 @@ int local_hints(int from) { if (!cmode || dry || panic || chkabort || stubgen) return 0; while (i < nloc) { f = ltag[i]; - if ((f & LDECL) && !larr[i]) { - if (!(f & LREAD)) hint_name(i, " is never read"); - else if ((f & LCONST) && !(f & LASG)) hint_name(i, " is never reassigned; make it a const"); - else if (fchk && lstate[i] == S_M && lown[i] && lnsz[i] > 0 && lnsz[i] <= 64) hint_name(i, " never leaves this function; a stack buffer (let name[n]) avoids the heap"); - else if (fchk && lstate[i] == S_M && lown[i] && lused[i] && tline - lused[i] >= 4) hint_name(i, " is last used several lines above the end of its block; close the block sooner so it is freed sooner"); + if (lrskind[i] == RS_PARAM && lname[i] != s_this && !rt.eq_w(lname[i], "_") && !(f & LREAD)) { + if (hint_begin(lpos[i], HINT_SMALL)) { + rt.put_s("unused parameter `"); rt.put_s(lname[i]); rt.put_s("`"); dg_close(); + } + } else if ((f & LDECL) && !larr[i]) { + if (!(f & LREAD)) hint_name(i, " is never read", HINT_SMALL); + else if ((f & LCONST) && !(f & LASG)) hint_name(i, " is never reassigned; make it a const", HINT_ALL); + else if (fchk && lstate[i] == S_M && lown[i] && lnsz[i] > 0 && lnsz[i] <= 64) hint_name(i, " never leaves this function; a stack buffer (let name[n]) avoids the heap", HINT_SMALL); + else if (fchk && lstate[i] == S_M && lown[i] && lused[i] && tline - lused[i] >= 4) hint_name(i, " is last used several lines above the end of its block; close the block sooner so it is freed sooner", HINT_ALL); } i++; } @@ -2234,7 +2890,7 @@ int alias_end(int dep, int save) { return 0; } // snapshots: [lbase] [nloc] then state, owner, home per local -// save the states of all current locals; returns the snapshot's offset +// save the states of all current locals; returns the snapshot offset int snap_push() { int b = snaptop, i = lbase; word p; af_forget(); @@ -2286,30 +2942,32 @@ int statename(int s) { // report that local i ends two paths in different states int merge_err(int i, int sa, int what) { if (!fchk || panic || dry) return 0; - errat(); put_s(lname[i]); put_s(": "); put_s(what); - put_s(" (one path leaves it "); - put_s(statename(sa)); - put_s(", another "); - put_s(statename(lstate[i])); - put_s(")"); + errat(); rt.put_s(lname[i]); rt.put_s(": "); rt.put_s(what); + rt.put_s(" (one path leaves it "); + rt.put_s(statename(sa)); + rt.put_s(", another "); + rt.put_s(statename(lstate[i])); + rt.put_s(")"); errend(); return 0; } -// ---- snapshots and comparing paths ---- -// to analyze an if, the compiler snapshots the locals' states before the -// branch, compiles the then path, restores the snapshot for the else path, and -// then compares or merges the two. a snapshot is [lbase, nloc] followed by -// state, owner and home for each local; the stack of them is at R_SNAP. -// ---- joins ---- -// where control flow comes back together (after an if, a loop, a switch or -// ?:) every variable has to agree across the paths that reach the join. one -// exception: a variable that became d, or was given away, on some path is dead -// after the join, and the paths where it's still alive free it on the way in -// (fixups). u objects and borrows don't get this treatment: a u object still -// has to leave, and a borrow never changes. +//: ---- snapshots and comparing paths ---- -int extop; [JOIN_PATHS]word jps_initial; word jps = jps_initial; int jpscap = JOIN_PATHS, njps; // the paths being joined: [lbase, nloc, triples] +//: to analyze an if, the compiler snapshots the locals' states before the +//: branch, compiles the then path, restores the snapshot for the else path, and +//: then compares or merges the two. a snapshot is [lbase, nloc] followed by +//: state, owner and home for each local; the stack of them is at R_SNAP. +//: ---- joins ---- + +//: where control flow comes back together (after an if, a loop, a switch or +//: ?:) every variable has to agree across the paths that reach the join. one +//: exception: a variable that became d, or was given away, on some path is dead +//: after the join, and the paths where it's still alive free it on the way in +//: (fixups). u objects and borrows don't get this treatment: a u object still +//: has to leave, and a borrow never changes. + +int extop; word[JOIN_PATHS] jps_initial; word jps = jps_initial; int jpscap = JOIN_PATHS, njps; // the paths being joined: [lbase, nloc, triples] // accessors on a path record p (a snapshot or an exit record): the state, // owner and home of local i on that path. locals declared after the snapshot // read as plain. @@ -2373,12 +3031,12 @@ int join(int n0, int what) { } } else if (dead && okdead) lstate[i] = S_DEAD; else if (fchk && !panic && !dry) { - errat(); put_s(lname[i]); put_s(": "); put_s(what); - put_s(" (one path leaves it "); - put_s(statename(s0)); - put_s(", another "); - put_s(statename(jst(jps[bad], i))); - put_s(")"); + errat(); rt.put_s(lname[i]); rt.put_s(": "); rt.put_s(what); + rt.put_s(" (one path leaves it "); + rt.put_s(statename(s0)); + rt.put_s(", another "); + rt.put_s(statename(jst(jps[bad], i))); + rt.put_s(")"); errend(); } i++; @@ -2410,9 +3068,9 @@ int backedge(int s0) { } else if (!panic && !dry) { errat(); - put_s(lname[i]); - put_s(": changes state inside a loop; each pass has to leave it as it found it (it went from "); - put_s(statename(jst(p, i))); put_s(" to "); put_s(statename(s)); put_s(")"); errend(); + rt.put_s(lname[i]); + rt.put_s(": changes state inside a loop; each pass has to leave it as it found it (it went from "); + rt.put_s(statename(jst(p, i))); rt.put_s(" to "); rt.put_s(statename(s)); rt.put_s(")"); errend(); } } i++; @@ -2436,13 +3094,14 @@ int join2(int sa, int livea, int ja, int livebv, int n0, int what) { return 0; } -// the rules below are the heart of the analyzer. each is a function the parser -// calls at one kind of place where a value moves: recast (u*x), argument -// passing, assignment, return, block values, and discarding. the argument -// c they take is an expression class: C_PLAIN (an ordinary word), C_FRESH (a -// new object nobody owns yet) or C_VAR + n (a bare local, number n), which is -// how the analyzer knows the thing being moved is a variable it tracks. -// ---- analyzer rules for the places values flow ---- +//: the rules below are the heart of the analyzer. each is a function the parser +//: calls at one kind of place where a value moves: recast (u*x), argument +//: passing, assignment, return, block values, and discarding. the argument +//: c they take is an expression class: C_PLAIN (an ordinary word), C_FRESH (a +//: new object nobody owns yet) or C_VAR + n (a bare local, number n), which is +//: how the analyzer knows the thing being moved is a variable it tracks. +//: ---- analyzer rules for the places values flow ---- + // recast: x as u / x as d / x as m int recast(int c, int t) { int i, st; @@ -2463,7 +3122,10 @@ int recast(int c, int t) { // an argument for a parameter tagged t (0 plain, 1 m, 2 d) int argcheck(int c, int t) { int i, st; + if (fchk && t == 2 && xaf && xaf[2] == 5) { cerr("a borrowed temporary cannot transfer ownership"); return 0; } + if (fchk && t == 2 && xaf && xaf[2] == 4 && (isptr(xty) || isagg(xty))) { cerr("an array literal cannot transfer ownership to a d parameter"); return 0; } if (!fchk) return 0; + if (t == 2 && c < C_VAR && c != C_FRESH && xbor >= 0 && tracked(lroot(xbor))) { cerr("a borrowed projection cannot transfer ownership; transfer its owner"); return 0; } if (c == C_FRESH) { if (t != 2) { cerr("a new object can only go straight to a d parameter; bind it to a name first"); @@ -2474,6 +3136,7 @@ int argcheck(int c, int t) { } if (c < C_VAR || !tracked(c - C_VAR)) return 0; i = c - C_VAR; + if (t == 2 && isalias(i)) { cerr("a borrowed projection cannot transfer ownership; transfer its owner"); return 0; } st = lstate[i]; if (lstate[lroot(i)] == S_DEAD) { cerrn(i, "used after it was given away"); return 0; } if (t <= 1) { // untagged parameters are m: a lend @@ -2484,7 +3147,7 @@ int argcheck(int c, int t) { touch(i); return 0; } - if (st == S_D) { lstate[i] = S_DEAD; luse(i); xgive = 1; return 0; } + if (st == S_D || ((st == S_M || st == S_U) && lown[i])) { end_owner(i); luse(i); xgive = 1; ev_put(i, EV_DOWN, -1); return 0; } if (st == S_M) cerrn(i, "is m; write d*x to give it away"); else if (st == S_U) cerrn(i, "is u; write d*x to give it away instead of returning it"); else if (st == S_CHILD) cerrn(i, "belongs to a container and can't be given away alone"); @@ -2512,8 +3175,8 @@ int o3own() { // owner in the child's header. a growable array's elements live in its items // object, so that is the owner there. int emitown(int p) { - laddr(lslot[p]); op(O_LD); - // a []T's elements live in its items object + vaddr(p); op(O_LD); + // a T[?]'s elements live in its items object if (ltype[p][0] == TY_DYN) { lit(2); op(O_ADD); @@ -2526,9 +3189,15 @@ int emitown(int p) { } // a store of class rc into an lvalue of kind k (local or container index i) int assigncheck(int k, int i, int rc) { - int st, j, r, js, bor; + int st, j, r, js, bor, origin = effect_origin(rc); + if (k == LV_LOCAL && local_origin) { local_origin[i] = origin; local_origin_mask[i] = local_origin_mask[i] | effect_mask(rc); } + if (k == LV_ELEM) { effect_note_mask(effect_mask(C_VAR + i), EF_WRITE); } + if (k != LV_LOCAL && isptr(xty)) effect_note_mask(effect_mask(rc), EF_RETAIN); + else if (k != LV_LOCAL && xty == tword) effect_note_mask(effect_mask(rc), EF_UNKNOWN); + if (fchk && xaf && xaf[2] == 5 && (isptr(xty) || xty == tword)) cerr("a borrowed temporary cannot escape its expression into a variable or container"); bor = xbor; xbor = -1; if (!fchk) return C_PLAIN; + if (k != LV_LOCAL && xaf && xaf[2] == 4 && (isptr(xty) || isagg(xty))) { cerr("an array literal cannot escape into a global or container"); return C_PLAIN; } j = -1; if (rc >= C_VAR && tracked(rc - C_VAR)) j = rc - C_VAR; if (j >= 0 && lstate[lroot(j)] == S_DEAD) { @@ -2540,7 +3209,7 @@ int assigncheck(int k, int i, int rc) { if (isalias(i)) st = S_PLAIN; // a borrow isn't an owner; overwriting it drops the alias if (j < 0 && rc != C_FRESH) { if (!empty(st)) cerrn(i, "still owns an object; overwriting it would leak it"); - else if (bor >= 0) { lstate[i] = S_CHILD; lowner[i] = bor; lown[i] = 0; } + else if (bor >= 0 && bor != i && (tracked(bor) || larr[bor])) { lstate[i] = S_CHILD; lowner[i] = bor; lown[i] = 0; } else lstate[i] = S_PLAIN; return C_PLAIN; } @@ -2552,16 +3221,29 @@ int assigncheck(int k, int i, int rc) { lstate[i] = S_M; lused[i] = 0; lown[i] = 1; + local_ended[i] = 0; local_end_info[i] = 0; if (ltag[i] & 15) lstate[i] = ltag[i] & 15; return C_PLAIN; } js = lstate[j]; + if (js == S_BORROW || isalias(j)) { lstate[i] = S_CHILD; lowner[i] = lroot(j); lown[i] = 0; touch(j); return C_PLAIN; } + // replace a released owner by moving a same-scope owner into its empty slot. + // fresh declarations still cannot copy owners; the source and its aliases expire. + if (st == S_DEAD && lown[i] && js == S_M && lown[j] && lhome[i] == lhome[j] + && !local_captured[i] && !local_captured[j]) { + lstate[i] = S_M; lused[i] = 0; lown[i] = 1; + lnsz[i] = lnsz[j]; lleaf[i] = lleaf[j]; + local_ended[i] = 0; local_end_info[i] = 0; + end_owner(j); luse(j); ev_put(j, EV_MOVE, i); + if (ltag[i] & 15) lstate[i] = ltag[i] & 15; + return C_PLAIN; + } if (js == S_U && lhome[i] == lhome[j] - 1) { // handing a u object up one block lstate[i] = S_M; lused[i] = 0; lown[i] = 1; - lstate[j] = S_DEAD; - luse(j); + end_owner(j); + luse(j); ev_put(j, EV_UP, i); if (ltag[i] & 15) lstate[i] = ltag[i] & 15; return C_PLAIN; } @@ -2584,16 +3266,17 @@ int assigncheck(int k, int i, int rc) { if (js == S_U) { cerrn(j, "is u; it has to leave through the block above, not into a borrowed object"); } - else { lstate[j] = S_DEAD; luse(j); emitown(i); } // storing it gives it away + else { end_owner(j); luse(j); ev_put(j, EV_STORE, i); emitown(i); } // storing it gives it away } else if (js != st) cerrn(j, "its tag doesn't match its container's (both have to be m, u or d)"); else if (lhome[j] < lhome[r]) cerrn(j, "lives in an outer block than the container it's stored into"); - else { lstate[j] = S_CHILD; lowner[j] = i; luse(j); emitown(i); } + else { lstate[j] = S_CHILD; lowner[j] = i; luse(j); ev_put(j, EV_STORE, i); emitown(i); } return C_PLAIN; } // a global, or memory the analyzer doesn't track: the collector takes it + // an owner (d, u, or an m local that owns its object) is transferred as if by u*x if (j >= 0) { - if (lstate[j] == S_D) { lstate[j] = S_DEAD; luse(j); } - else cerrn(j, "must be d to be stored in a global or in untracked memory (the collector takes it over)"); + if (lstate[j] == S_D || lstate[j] == S_U || (lstate[j] == S_M && lown[j])) { ev_put(j, EV_GLOBAL, -1); end_owner(j); luse(j); } + else cerrn(j, "is not an owner, so it can't be stored in a global or in untracked memory (the collector takes it over)"); } return C_PLAIN; } @@ -2603,8 +3286,29 @@ int assigncheck(int k, int i, int rc) { // object is lent, not given, so returning it is an error, and so is a plain word // from an owned function that was called before its body. int retcheck(int c) { - int i; + int i; + if (isptr(xty) || xty == tword) { + effect_note_mask(effect_mask(c), EF_ALIAS); + if (c >= C_VAR && lrskind[c - C_VAR] == RS_PARAM) effect_note(effect_origin(c), EF_IDENTITY); + } + if (fchk && xaf && xaf[2] == 5 && (isptr(xty) || xty == tword)) cerr("a borrowed temporary cannot escape its expression through return"); if (!fchk) return 0; + if (c < C_VAR && c != C_FRESH && xbor >= 0 && (isptr(xty) || xty == tword)) { + int root = lroot(xbor); + if (fret) { cerr("a borrowed projection cannot be returned as an owned result"); return 0; } + if (lown[root] && (lstate[root] == S_M || lstate[root] == S_U || lstate[root] == S_D)) { cerr("a borrow of a local owner cannot escape through return; return the owner"); return 0; } + } + if (c >= C_VAR && isalias(c - C_VAR)) { + int root = lroot(c - C_VAR); + if (fret) cerr("a borrowed projection cannot be returned as an owned result"); + else if (lown[root]) cerr("a borrow of a local owner cannot escape through return; return the owner"); + return 0; + } + if (c >= C_VAR && lstate[c - C_VAR] == S_BORROW) { + if (fret) cerr("a borrowed parameter cannot be returned as an owned result"); + return 0; + } + if ((isptr(xty) || isagg(xty)) && ((xaf && xaf[2] == 4) || (xbor >= 0 && lafact[xbor] && lafact[xbor][2] == 4))) { cerr("an array literal cannot escape its function through return"); return 0; } i = -1; if (c >= C_VAR && tracked(c - C_VAR)) i = c - C_VAR; if (i >= 0 && lstate[lroot(i)] == S_DEAD) { cerrn(i, "used after it was given away"); return 0; } @@ -2613,11 +3317,12 @@ int retcheck(int c) { if (sflag[finsym] & F_EARLY) serrn(sname[finsym], "is called before its body, so its owned return can't be known there; declare the return type with .u"); ssig[finsym] = ssig[finsym] | 1; // the return is owned: callers read it from the signature sflag[finsym] = sflag[finsym] | F_RETOWN; + if (stype[finsym]) { word ft = stype[finsym]; stype[finsym] = tfn(ft[4], ft[3], ft + 7, ssig[finsym]); } fret = 1; finfer = 0; } if (fret) { if (i >= 0) { - if (lstate[i] == S_U) { lstate[i] = S_DEAD; luse(i); } + if (lstate[i] == S_U) { end_owner(i); luse(i); ev_put(i, EV_RETURN, -1); } else cerrn(i, "has to be u to be returned"); } return 0; @@ -2633,7 +3338,7 @@ int handup(int c) { if (c == C_FRESH) return C_FRESH; if (c < C_VAR || !tracked(c - C_VAR)) return C_PLAIN; i = c - C_VAR; - if (lstate[i] == S_U && lhome[i] == fdepth) { lstate[i] = S_DEAD; luse(i); return C_FRESH; } + if (lstate[i] == S_U && lhome[i] == fdepth) { lstate[i] = S_DEAD; luse(i); ev_put(i, EV_VALUE, -1); return C_FRESH; } if (lstate[i] == S_U) cerrn(i, "only a u object of this block can be its value"); else cerrn(i, "has to be u to be handed up as the block's value"); return C_PLAIN; @@ -2646,7 +3351,55 @@ int discard(int c) { return 0; } -// ---- types ---- +//: ---- completion ---- +//: copy local i (kind: 1 local, 2 parameter, 3 captured from the function around a closure) into the probe, +//: unless a nearer local has its name +int probe_add(int i, int kind) { + int k = 0; + while (k < probe_n) { if (probe_name[k] == lname[i]) return 0; k++; } + if (!probe_name || probe_n == probe_cap) { + if (!probe_cap) probe_cap = 32; else probe_cap = probe_cap * 2; + probe_name = chk_resize(probe_name, probe_n, probe_cap); probe_kind = chk_resize(probe_kind, probe_n, probe_cap); + probe_state = chk_resize(probe_state, probe_n, probe_cap); probe_type = chk_resize(probe_type, probe_n, probe_cap); probe_decl = chk_resize(probe_decl, probe_n, probe_cap); + } + probe_name[probe_n] = lname[i]; probe_kind[probe_n] = kind; + probe_state[probe_n] = lstate[lroot(i)] == S_DEAD || af_expired(lafact[i]); + if (lafact[i] && lafact[i][1] == 2) probe_state[probe_n] = 1; + probe_type[probe_n] = ltype[i]; probe_decl[probe_n] = lrsoff[i]; + probe_n++; + return 0; +} +// what is in scope right now: this function's locals, then those of the functions around a closure +int probe_snap() { + int i = nloc, k = nclo; + probe_n = 0; probe_has = 1; + while (i > lbase) { + i--; + if (lname[i][0] != '<') { if (lrskind[i] == RS_PARAM) probe_add(i, 2); else probe_add(i, 1); } + } + while (k) { + k--; + i = clo_hi[k]; + while (i > clo_lo[k]) { i--; if (lname[i][0] != '<') probe_add(i, 3); } + } + return 0; +} +// a statement of the document starts here: remember what is in scope, or stop when the place has been passed. +// statements of a method or generic compiled for a call belong to another function and are left alone +int probe_stmt() { + if (!cmode || dry || ginst || chkabort || incdepth != 0 || ctxsp > nclo) return 0; + probe_snap(); + if (toff > probe_off) { chkabort = 1; return 0; } + return 0; +} +// the place asked about is a member name, and t is the type before the dot +int probe_member(word t) { + probe_mem = t; probe_has = 2; chkabort = 1; + return 0; +} + +//: ---- types ---- + // types are a compiler primitive: records in the type area, hash-consed so // the same type is always the same pointer and comparing types is comparing // words. untyped code is typed word, which goes with anything. the records @@ -2666,28 +3419,49 @@ int discard(int c) { // the hash-consing: asking for the same pointer type twice returns the same // record. the list is searched linearly, which is fine for the few hundred // types a program has. -unchecked int talloc(int n) { +int talloc(int n) { int p = tytop; if (tytop + n > tyend) { - p = new(n + 16385); p[-4] = p[-4] | F_INTERNAL; p[0] = typeblocks; typeblocks = p; + p = chk_resize(0, 0, n + 16385); p[0] = typeblocks; typeblocks = p; p++; tytop = p; tyend = p + n + 16384; } tytop = tytop + n; - fill_w(p, 0, n); + rt.fill_w(p, 0, n); return p; } // make a type record of the given kind, size and name in the type area. -word tnew(int kind, int size, int nm, int extra) { +word tnew(int kind, int size, char* nm, int extra) { word t = talloc(7 + extra); t[0] = kind; t[1] = size; t[2] = nm; if (ntl >= typecap) { typelist = chk_resize(typelist, ntl, typecap * 2); typecap = typecap * 2; } typelist[ntl] = t; ntl++; return t; } +// qualified types retain their layout and reference their unqualified record. +word tunconst(word t) { + word entry = readonly_types; + while (entry) { if (entry[1] == t) return entry[2]; entry = entry[0]; } + return t; +} +int treadonly(word t) { return t && tunconst(t) != t; } +word tconst(word t) { + word entry = readonly_types, qualified; int words = 7; + if (treadonly(t)) return t; + while (entry) { if (entry[2] == t) return entry[1]; entry = entry[0]; } + if (t[0] == TY_FN) words = words + t[3]; + qualified = tnew(t[0], t[1], t[2], words - 7); rt.copy_w(qualified, t, words); + entry = talloc(3); entry[0] = readonly_types; entry[1] = qualified; entry[2] = t; readonly_types = entry; + return qualified; +} +// writes require mutable storage, independently of optional type checking. +int require_mutable(word t) { + if (treadonly(t)) serr("const storage cannot be modified"); + return 0; +} // the pointer type to type e, reused when it already exists. word tptr(int e) { int i = 0; word t; - while (i < ntl) { t = typelist[i]; if (t[0] == TY_PTR && t[4] == e) return t; i++; } + while (i < ntl) { t = typelist[i]; if (t[0] == TY_PTR && t[4] == e && !treadonly(t)) return t; i++; } t = tnew(TY_PTR, 1, 0, 0); t[4] = e; return t; } @@ -2701,9 +3475,9 @@ word tbytes(word t) { if (t[0] == TY_INT || t[0] == TY_FLOAT) return t[5]; retur // true for a packed integer or float type. word ispk(word t) { return (t[0] == TY_INT || t[0] == TY_FLOAT) && (t[6] & 2); } // the type a packed type stores its elements as, or the type itself. -word tpk(word t) { if ((t[0] == TY_INT || t[0] == TY_FLOAT) && t[4]) return t[4]; return t; } +word tpk(word t) { if (treadonly(t)) return tconst(tpk(tunconst(t))); if ((t[0] == TY_INT || t[0] == TY_FLOAT) && t[4]) return t[4]; return t; } // an enhancement used by the compiler itself, inline when the runtime has it -int xenh(int name) { +int xenh(char* name) { int s = lookup(intern(name)); if (s >= 0 && skind[s] == K_ENH) { op(XOP0 + sval[s]); return 0; } callname(intern(name)); @@ -2711,18 +3485,19 @@ int xenh(int name) { } // words an array of n elements of e takes: packed for a sized integer word tasz(word e, int n) { + if (n < 0) return 0; if (tbytes(e)) return (n * e[5] + 7) >> 3; return n * tsz(e); } // the fixed array type of n elements of type e, reused when it already exists. word tarr(int e, int n) { int i = 0; word t; - while (i < ntl) { t = typelist[i]; if (t[0] == TY_ARR && t[4] == e && t[3] == n) return t; i++; } + while (i < ntl) { t = typelist[i]; if (t[0] == TY_ARR && t[4] == e && t[3] == n && !treadonly(t)) return t; i++; } t = tnew(TY_ARR, tasz(e, n), 0, 0); t[4] = e; t[3] = n; return t; } // a sized integer type and its packed twin -word tsized(int name, int bytes, int sgn) { +word tsized(char* name, int bytes, int sgn) { word t = tnew(TY_INT, 1, intern(name), 0), twin = tnew(TY_INT, 1, t[2], 0); t[5] = bytes; t[6] = sgn; t[4] = twin; twin[5] = bytes; twin[6] = sgn | 2; @@ -2741,18 +3516,18 @@ word tfn(int ret, int np, word ps, int sig) { i++; } t = tnew(TY_FN, 1, 0, np); t[4] = ret; t[3] = np; t[5] = sig; - copy_w(t + 7, ps, np); + rt.copy_w(t + 7, ps, np); return t; } -// []T, or fixed []T +// T[?], or fixed T[?] word tdyn(int e, int f) { int i = 0; word t; - while (i < ntl) { t = typelist[i]; if (t[0] == TY_DYN && t[4] == e && t[3] == f) return t; i++; } + while (i < ntl) { t = typelist[i]; if (t[0] == TY_DYN && t[4] == e && t[3] == f && !treadonly(t)) return t; i++; } t = tnew(TY_DYN, 1, 0, 0); t[4] = e; t[3] = f; return t; } // the type variable with index i, reused when it already exists. -word tvar(int i, int nm) { +word tvar(int i, char* nm) { int k = 0; word t; while (k < ntl) { t = typelist[k]; if (t[0] == TY_VAR && t[3] == i) return t; k++; } t = tnew(TY_VAR, 1, nm, 0); t[3] = i; @@ -2771,18 +3546,18 @@ word tinst(word g, word args, int n) { } i++; } - v = talloc(n + 1); v[0] = n; copy_w(v + 1, args, n); + v = talloc(n + 1); v[0] = n; rt.copy_w(v + 1, args, n); t = tnew(TY_STRUCT, 0, g[2], 0); t[3] = -1; t[5] = g; t[6] = v; return t; } // lay out a generic instance on first use word tresolve(word t) { - word g, f; int n, i; word fs; int off = 0, ft; [8]word bs; + word g, f; int n, i; word fs; int off = 0, ft; word[8] bs; if (t[0] != TY_STRUCT || t[3] >= 0) return t; if (t[3] == -2) { serr("a struct can't contain itself by value"); t[3] = 0; return t; } t[3] = -2; g = t[5]; n = g[5]; f = g[4]; - fill_w(bs, 0, 8); copy_w(bs, t[6] + 1, t[6][0]); + rt.fill_w(bs, 0, 8); rt.copy_w(bs, t[6] + 1, t[6][0]); fs = talloc(n * 3 + 1); i = 0; while (i < n) { @@ -2801,10 +3576,12 @@ word tsz(word t) { } // replace type parameters with what they were bound to (unbound ones become word) word subst(word t, word bs) { - int k = t[0], i, n; [24]word v; word vec; + if (treadonly(t)) return tconst(subst(tunconst(t), bs)); + int k = t[0], i, n; word[24] v; word vec; if (k == TY_VAR) { if (bs[t[3]]) return bs[t[3]]; return tword; } if (k == TY_PTR) return tptr(subst(t[4], bs)); if (k == TY_ARR) return tarr(subst(t[4], bs), t[3]); + if (k == TY_DYN) return tdyn(subst(t[4], bs), t[3]); if (k == TY_FN) { n = t[3]; i = 0; @@ -2826,14 +3603,18 @@ word isnum(word t) { return t[0] == TY_INT || t[0] == TY_CHAR || t[0] == TY_BOOL // true for an aggregate type: an array or a struct. word isagg(word t) { return t[0] == TY_ARR || t[0] == TY_STRUCT; } // true for a pointer or an array type. -word isptr(word t) { return t[0] == TY_PTR || t[0] == TY_ARR; } +word isptr(word t) { return t[0] == TY_PTR || t[0] == TY_ARR || t[0] == TY_BSTR; } +// true for a reference value, including growable-array handles and functions. +word isref(word t) { return isptr(t) || t[0] == TY_DYN || t[0] == TY_FN; } // the pointer an array type turns into when used as a value. word decay(word t) { + if (treadonly(t)) return tconst(decay(tunconst(t))); if (t[0] == TY_ARR) return tptr(t[4]); return t; } // the element type of a pointer or array, or word for anything else. word telem(word t) { + if (treadonly(t)) return tconst(telem(tunconst(t))); if (t[0] == TY_PTR || t[0] == TY_ARR) return t[4]; return tword; } @@ -2843,7 +3624,12 @@ word telem(word t) { // the same element unless one side is untyped or void, and a literal 0 is a // null pointer. word compat(word dst, word src) { - if (dst == src || dst == tword || src == tword) return 1; + if (treadonly(src) && isref(src) && !treadonly(dst)) return 0; + dst = tunconst(dst); src = tunconst(src); + if (dst == src) return 1; + if (dst == tword && src && (src == tbstr || (src[0] == TY_PTR && src[4] == tchar) || (src[0] == TY_ARR && src[4] == tchar))) return 0; + if (dst == tword || src == tword) return 1; + if (dst == tbstr || src == tbstr) return dst == tbstr && xzero; if (dst[0] == TY_PTR && src[0] == TY_STRUCT && dst[4] == src) return 1; // an in-place struct supplies its address if (isen(dst) || isen(src)) { // an enum takes only its own enum; an enum still becomes an int if (isen(dst) && isen(src)) return dst == src; @@ -2865,9 +3651,25 @@ word compat(word dst, word src) { // T from its arguments. word unify(word p, word a, word bs) { int i, n; word pv, av; + if (p[0] != TY_VAR) { + if (treadonly(a) && isref(a) && !treadonly(p)) return 0; + p = tunconst(p); a = tunconst(a); + } if (p[0] == TY_VAR) { i = p[3]; if (!bs[i]) { bs[i] = a; return 1; } return compat(bs[i], a); } a = decay(a); if (p[0] == TY_PTR && a[0] == TY_PTR) { if (a[4] == tword) return 1; return unify(p[4], a[4], bs); } + if (p[0] == TY_DYN && a[0] == TY_DYN) return (p[3] || !a[3]) && unify(p[4], a[4], bs); + if (p[0] == TY_FN && a[0] == TY_FN) { + if (p[3] != a[3] || ((p[5] >> 6) != (a[5] >> 6))) return 0; + i = 0; + while (i < p[3]) { + pv = subst(p[7 + i], bs); av = a[7 + i]; + if (pv != tword && av != tword && pv != av) return 0; + if (!unify(p[7 + i], av, bs)) return 0; + i++; + } + return unify(p[4], a[4], bs); + } if (p[0] == TY_STRUCT && p[5] && a[0] == TY_STRUCT && a[5] == p[5]) { pv = p[6]; av = a[6]; n = pv[0]; i = 0; while (i < n) { if (!unify(pv[1 + i], av[1 + i], bs)) return 0; i++; } @@ -2875,39 +3677,42 @@ word unify(word p, word a, word bs) { } return compat(p, a); } -// print a type the way it is written in source. a pointer to an array or a -// function has no source spelling (T* would read as an array of pointers; give -// it a type alias), so it prints with a leading star for the reader's benefit. +// print a type the way it is written in source. function pointers use an alias +// for source spelling; array pointers use T[N]*. +int tprint_dimensions(word t) { + while (t[0] == TY_ARR || t[0] == TY_DYN) { rt.put_c('['); if (t[0] == TY_DYN) rt.put_c('?'); else if (t[3] >= 0) rt.put_n(t[3]); rt.put_c(']'); t = t[4]; } + return 0; +} word tprint(word t) { + if (treadonly(t)) { rt.put_s("const "); return tprint(tunconst(t)); } int k = t[0], i; word v; - if (k == TY_PTR) { // T*; a pointer to an array or a function keeps the prefix form, since T* would read as an array of pointers + if (k == TY_PTR) { // postfix pointers; only function pointers keep the display-only prefix form v = t[4][0]; - if (v == TY_ARR || v == TY_DYN || v == TY_FN) { put_c('*'); return tprint(t[4]); } - tprint(t[4]); put_c('*'); + if (v == TY_FN) { rt.put_c('*'); return tprint(t[4]); } + tprint(t[4]); rt.put_c('*'); return 0; } - if (k == TY_DYN) { if (t[3]) put_s("fixed "); put_s("[]"); return tprint(t[4]); } - if (k == TY_ARR) { put_c('['); put_n(t[3]); put_c(']'); return tprint(t[4]); } + if (k == TY_ARR || k == TY_DYN) { if (k == TY_DYN && t[3]) rt.put_s("fixed "); v = t; while (v[0] == TY_ARR || v[0] == TY_DYN) v = v[4]; tprint(v); return tprint_dimensions(t); } if (k == TY_FN) { - if (t[4] == tword) put_s("fn"); else tprint(t[4]); - if (t[5] & 1) put_s(".u"); - put_c('('); + if (t[4] == tword) rt.put_s("fn"); else tprint(t[4]); + if (t[5] & 1) rt.put_s(".u"); + rt.put_c('('); i = 0; while (i < t[3]) { - if (i) put_s(", "); + if (i) rt.put_s(", "); tprint(t[7 + i]); - if (i < 12 && tparamtag(t, i) == 1) put_s(".m"); - if (i < 12 && tparamtag(t, i) == 2) put_s(".d"); + if (i < 12 && tparamtag(t, i) == 1) rt.put_s(".m"); + if (i < 12 && tparamtag(t, i) == 2) rt.put_s(".d"); i++; } - put_c(')'); + rt.put_c(')'); return 0; } - if (t[2]) put_s(t[2]); else put_s("?"); + if (t[2]) rt.put_s(t[2]); else rt.put_s("?"); if (k == TY_STRUCT && t[5]) { - v = t[6]; put_c('<'); - i = 0; while (i < v[0]) { if (i) put_s(", "); tprint(v[1 + i]); i++; } - put_c('>'); + v = t[6]; rt.put_c('<'); + i = 0; while (i < v[0]) { if (i) rt.put_s(", "); tprint(v[1 + i]); i++; } + rt.put_c('>'); } return 0; } @@ -2915,15 +3720,15 @@ word tprint(word t) { // code, and never during a dry run. int tcerr(int what, int want, int got) { if (!tc_on || !live || dry || panic) return 0; - errat(); put_s("type error: "); put_s(what); - put_s(": expected "); tprint(want); put_s(", got "); tprint(got); + errat(); rt.put_s("type error: "); rt.put_s(what); + rt.put_s(": expected "); tprint(want); rt.put_s(", got "); tprint(got); errend(); return 0; } // report a type error when ok is false, showing the type it got. int tcneed(int ok, int what, int got) { if (ok || !tc_on || !live || dry || panic) return 0; - errat(); put_s("type error: "); put_s(what); put_s(", got "); tprint(got); + errat(); rt.put_s("type error: "); rt.put_s(what); rt.put_s(", got "); tprint(got); errend(); return 0; } @@ -2939,9 +3744,9 @@ int tpfind(int n) { return -1; } // true when the name n is a type, a type parameter or a generic type. -word istypename(int n) { +word istypename(char* n) { int s; - if (n == s_fixed) return 1; + if (n == s_fixed || n == s_const) return 1; if (ginst && gfind(n) >= 0) return 1; if (tpfind(n) >= 0) return 1; if (findloc(n) >= 0) return 0; @@ -2954,56 +3759,68 @@ int closeangle() { expect(s_gt); return 0; } -// a type, then any number of (params) after it: int(int, int) is a function -// returning int, fn(int) one returning a plain word -// ---- parsing types ---- +//: a type, then any number of (params) after it: int(int, int) is a function +//: returning int, fn(int) one returning a plain word +//: ---- parsing types ---- + // grammar, outermost first: // ptype a type, then any number of (params): int(int, int) is a function type -// ptype0 a type with its postfix stars: int*, vec<T>*, point** -// ptype1 a base: [n]T, []T, fixed []T, a type name, or a generic instance +// ptype0 postfix pointers and array groups: T*, T[N], T[], T[?] +// ptype1 a type name, fixed qualifier, function base, or generic instance // the first thing in ptype's comment applies here too: fn(int) is a function // returning a plain word. int ptype() { - int start = toff, t = ptype0(), np; [24]word ps; + int start = toff, t = ptype0(), np, tags, tag; word[24] ps; while (is_op(s_lp)) { - next(); np = 0; + next(); np = 0; tags = 0; if (!is_op(s_rp)) { while (1) { if (np >= 24) fatal("too many parameters"); - ps[np] = ptype(); + ps[np] = ptype(); if (ps[np][0] == TY_ARR) ps[np] = decay(ps[np]); + tag = 0; + if (accept(s_dot)) { + if (tv == s_m) tag = 1; else if (tv == s_d) tag = 2; else die("callback parameter lifetime is .m or .d"); + next(); + } + tags = tags | (tag << (np + np)); np++; if (!accept(s_comma)) break; } } expect(s_rp); - t = tfn(t, np, ps, np + np); + t = tfn(t, np, ps, np + np + tags * 64); } ht_note(start, toff, t); return t; } // a type with any stars after it: int*, vec<T>*, point** int ptype0() { - int t = ptype1(); - while (is_op(s_star)) { next(); t = tptr(t); } + int t = ptype1(), cap = 8, n, count, j; word dims = 0; + while (is_op(s_star) || is_op(s_lk)) { + if (accept(s_star)) t = tptr(t); + else { + count = 0; + while (accept(s_lk)) { + if (accept(s_q)) { expect(s_rk); n = -2; } + else if (accept(s_rk)) n = -1; + else { n = cexpr(); expect(s_rk); if (n < 0) serr("a negative array length"); } + if (!dims || count == cap) { if (dims) cap = cap * 2; dims = chk_resize(dims, count, cap); } + dims[count] = n; count++; + } + j = count; while (j) { j--; if (dims[j] == -2) t = tdyn(t, 0); else t = tarr(t, dims[j]); } + } + } + if (dims) rt.free(dims); return t; } // parse one type with no trailing function parts. word ptype1() { - int n, s; word t; [8]word args; int na = 0, i; - if (accept(s_lk)) { - if (accept(s_rk)) { // []T: grows - t = ptype0(); - return tdyn(t, 0); - } - n = cexpr(); - expect(s_rk); - t = ptype0(); - if (n < 0) serr("a negative array length"); - return tarr(t, n); - } - if (is_id(s_fixed)) { // fixed []T can't grow; fixed [n]T says what [n]T already means + int n, s; word t; word[8] args; int na = 0, i; + if (is_op(s_lk)) { die("array types are postfix: T[N], T[] or T[?]"); return tword; } + if (is_id(s_const)) { next(); return tconst(ptype0()); } + if (is_id(s_fixed)) { // fixed T[?] can't grow; fixed T[n] says what T[n] already means next(); t = ptype0(); - if (t[0] == TY_DYN) return tdyn(t[4], 1); + if (t[0] == TY_DYN) { if (treadonly(t)) return tconst(tdyn(t[4], 1)); return tdyn(t[4], 1); } if (t[0] != TY_ARR) serr("only arrays can be fixed"); return t; } @@ -3020,7 +3837,7 @@ word ptype1() { } i = tpfind(n); if (i >= 0) { next(); return tvar(i, n); } - s = lookup(n); + s = source_lookup(n); // T in T first<T>(...) if (tvauto && (s < 0 || skind[s] != K_TYPE) && ntp < 8) { tpn[ntp] = n; @@ -3030,6 +3847,7 @@ word ptype1() { return tvar(ntp - 1, n); } if (s < 0 || skind[s] != K_TYPE) { dien(n, "isn't a type"); return tword; } + rs_put(toff, RS_SYM, rs_sym(s), s); next(); t = sval[s]; if (t[0] != TY_GEN) return t; @@ -3087,9 +3905,9 @@ int dtype() { // and call its siblings. a generic struct keeps its fields unresolved and // can't have methods; each instance is laid out later by tresolve. int kw_struct() { - int n; word t; int s, nf = 0; word buf = chk_resize(0, 0, 64); int bufcap = 64, i; word fs; int off = 0, gen = 0, ft, nm = 0, mcap = STRUCT_METHODS; word mn = chk_resize(0, 0, mcap), mq = chk_resize(0, 0, mcap), mst = chk_resize(0, 0, mcap), mty = chk_resize(0, 0, mcap), mrt = chk_resize(0, 0, mcap), mrg = chk_resize(0, 0, mcap); int bo, k, stat = 0, fname = 0; + char* n; word t; int s, nf = 0; word buf = chk_resize(0, 0, 64); int bufcap = 64, i; word fs; int off = 0, gen = 0, ft, nm = 0, mcap = STRUCT_METHODS; char** mn = chk_resize(0, 0, mcap); word mq = chk_resize(0, 0, mcap), mst = chk_resize(0, 0, mcap), mty = chk_resize(0, 0, mcap), mrt = chk_resize(0, 0, mcap), mrg = chk_resize(0, 0, mcap); int bo, k, stat = 0; char* fname = 0; next(); - n = name(); + n = module_symbol(name()); if (is_op(s_lt)) { // generic next(); ntp = 0; while (1) { @@ -3102,16 +3920,17 @@ int kw_struct() { closeangle(); gen = 1; t = tnew(TY_GEN, 0, n, 0); t[3] = ntp; - fs = talloc(ntp); copy_w(fs, tpn, ntp); t[6] = fs; + fs = talloc(ntp); rt.copy_w(fs, tpn, ntp); t[6] = fs; } else { t = tnew(TY_STRUCT, 0, n, 0); t[3] = -2; } // -2 while its fields are read s = addsym(n, K_TYPE, t); expect(s_lb); while (!is_op(s_rb)) { if (tk == T_EOF) { die("missing }"); break; } stat = 0; + if (is_id(s_unchecked)) { next(); stat = 2; } if (is_id(s_static)) { // static fn name(params) { body }, or static T name(params) { body }: a function of the struct, called as Struct.name(args), with no this next(); - stat = 1; + stat = stat | 1; } if (is_id(s_fn)) { // fn name(params) { body }: a method, compiled once the struct is complete next(); @@ -3125,14 +3944,14 @@ int kw_struct() { mq[nm] = grabfn(); mty[nm] = 0; k = methdecl(n, mn[nm]); - if (stat) sflag[k] = sflag[k] | F_STATIC; + if (stat & 1) sflag[k] = sflag[k] | F_STATIC; nm++; continue; } // let x, y; fields are plain words fname = 0; if (is_id(s_var)) { - if (stat) die("static must be followed by fn or a typed method"); + if (stat & 1) die("static must be followed by fn or a typed method"); next(); ft = tword; } else { @@ -3151,11 +3970,11 @@ int kw_struct() { mty[nm] = 1; mrt[nm] = dty; mrg[nm] = dtag == S_U; if (dtag && dtag != S_U) serr("the only return tag is .u"); k = methdecl(n, fname); - if (stat) sflag[k] = sflag[k] | F_STATIC; + if (stat & 1) sflag[k] = sflag[k] | F_STATIC; nm++; continue; } - if (stat) die("static must be followed by fn or a typed method"); + if (stat & 1) die("static must be followed by fn or a typed method"); if (dtag) serr("a field has no tag"); ft = dty; } @@ -3170,9 +3989,9 @@ int kw_struct() { } next(); if (gen) { - fs = talloc(nf + nf); copy_w(fs, buf, nf + nf); + fs = talloc(nf + nf); rt.copy_w(fs, buf, nf + nf); t[4] = fs; t[5] = nf; - mdg = ntp; copy_w(mdn, tpn, ntp); + mdg = ntp; rt.copy_w(mdn, tpn, ntp); ntp = 0; } else { fs = talloc(nf * 3 + 1); @@ -3194,18 +4013,19 @@ int kw_struct() { } accept(s_semi); i = 0; - while (i < nm) { methdef(n, mn[i], mq[i], mst[i], mty[i], mrt[i], mrg[i]); i++; } + while (i < nm) { fn_method_compile = 1; methdef(n, mn[i], mq[i], mst[i], mty[i], mrt[i], mrg[i]); fn_method_compile = 0; i++; } mdg = 0; return s; } -// ---- methods ---- +//: ---- methods ---- + // a method is an ordinary function named Struct.name whose first parameter is // this, a pointer to the struct. e.method(a) calls Struct.name(e, a). the // receiver's struct comes from its type, or, for an untyped word, from the // one struct that has a method of that name. // the symbol for a method is the interned string "Struct.name" -word methname(word sn, word mn) { - word b = chk_resize(0, 0, len_w(sn) + len_w(mn) + 2); int i = 0, j = 0; +char* methname(char* sn, char* mn) { + char* b = chk_resize(0, 0, rt.len_w(sn) + rt.len_w(mn) + 2); int i = 0, j = 0; while (sn[j]) { b[i] = sn[j]; i++; j++; } b[i] = '.'; i++; j = 0; @@ -3216,13 +4036,13 @@ word methname(word sn, word mn) { // the method called n on struct type t, as a symbol index, or -1. methwhich // (below) does the untyped version: search every struct for one with that // method, which is unique or ambiguous (-2). -int methof(word t, int n) { +int methof(word t, char* n) { int s = lookup(methname(t[2], n)); if (s >= 0 && skind[s] == K_FN) return s; return 0 - 1; } // the one method called n, -1 for none, -2 for several structs having one -int methscan(int n) { +int methscan(char* n) { int i = 0; word t; int s, hit = 0 - 1; while (i < ntl) { t = typelist[i]; @@ -3237,7 +4057,7 @@ int methscan(int n) { // declare Struct.name now, so methods can call each other in any order // (the declaration is what makes mutual recursion between methods work: the // symbol exists with no body yet, and calls to it go on the patch chain) -int methdecl(int sn, int mn) { +int methdecl(char* sn, char* mn) { int nm = methname(sn, mn), s = lookup(nm); if (s >= ubase && s >= 0 && skind[s] == K_FN) { serrn(mn, "is defined twice in this struct"); @@ -3252,15 +4072,15 @@ int methdecl(int sn, int mn) { // into a token list (kind, value, place triples) up to the matching brace, and // methdef replays them later. the place travels with each token, so errors // inside a method still point at the right line. -unchecked word grabfn() { - int cap = 768; word q = new(cap); int n = 0, depth = 0; word nq; +word .u grabfn() { + int cap = 768; word q = rt.new(cap); int n = 0, depth = 0; word nq; q[0] = 0; if (!is_op(s_lp)) { die("expected ( after the method name"); return q; } while (tk != T_EOF) { if (n * 3 + 4 > cap) { - nq = new(cap + cap); - copy_w(nq, q, cap); - free(q); + nq = rt.new(cap + cap); + rt.copy_w(nq, q, cap); + rt.free(q); q = nq; cap = cap + cap; } @@ -3278,17 +4098,26 @@ unchecked word grabfn() { // method's saved tokens, push it as an input source and parse it as an // ordinary top-level function. a method is therefore just a function whose // first parameter is this; nothing else about it is special. -unchecked int tkput(word r, int k, int kind, int val, int pl) { +int tkput(word r, int k, int kind, int val, int pl) { r[1 + k * 3] = kind; r[2 + k * 3] = val; r[3 + k * 3] = pl; return k + 1; } // compile a struct method once the struct is complete. -unchecked int methdef(int sn, int mn, word q, int stat, int typed, int rty, int rtag) { - int n = q[0]; word r; int i, k = 0, lnp; - if (!n) { free(q); return 0; } - r = new((n + 8 + mdg * 4) * 3 + 1); +int methdef(char* sn, char* mn, word q, int stat, int typed, int rty, int rtag) { + int n = q[0]; word r; int i, k = 0, lnp, unchecked_method = stat & 2; + stat = stat & 1; + if (!n) { rt.free(q); return 0; } + if (!stat) { + i = 1; + while (i < n && q[2 + i * 3] != s_comma && q[2 + i * 3] != s_rp) { + if (q[1 + i * 3] == T_ID && q[2 + i * 3] == s_this) serr("this is implicit and cannot be declared as a method parameter"); + i++; + } + } + r = rt.new((n + 8 + mdg * 4) * 3 + 1); lnp = q[3]; - r[1] = T_ID; r[2] = s_fn; r[3] = lnp; k = 1; + if (unchecked_method) k = tkput(r, k, T_ID, s_unchecked, lnp); + k = tkput(r, k, T_ID, s_fn, lnp); r[1 + k * 3] = T_ID; r[2 + k * 3] = methname(sn, mn); r[3 + k * 3] = lnp; k++; if (mdg) { // a method of a generic struct is a generic function over the struct's parameters: fn S.m<K, V>(S<K, V>* this, ...) k = tkput(r, k, T_OP, s_lt, lnp); @@ -3332,12 +4161,36 @@ unchecked int methdef(int sn, int mn, word q, int stat, int typed, int rty, int i++; } r[0] = k; - free(q); + rt.free(q); + if (typed && mdg) { + // retain a typed generic method as ordinary typed function tokens. + int mode = rt.outmode, length = rt.outn, capacity = rt.outcap, prefix = unchecked_method != 0; + word output = rt.outbuf, grow = rt.outgrow, buffer = rt.new(4096), return_tokens, joined; + rt.outmode = 2; rt.outbuf = m*buffer; rt.outn = 0; rt.outcap = 4095; rt.outgrow = 0; + tprint(rty); if (rtag) rt.put_s(".u"); buffer[rt.outn] = 0; + rt.outmode = mode; rt.outbuf = output; rt.outn = length; rt.outcap = capacity; rt.outgrow = grow; + return_tokens = tokens(buffer); + int token = 0, variable; + while (token < return_tokens[0]) { + variable = 0; + while (variable < ntl) { + word descriptor = typelist[variable]; + if (descriptor[0] == TY_VAR && descriptor[3] < mdg && return_tokens[1 + token * 3] == T_ID && return_tokens[2 + token * 3] == descriptor[2]) return_tokens[2 + token * 3] = mdn[descriptor[3]]; + variable++; + } + token++; + } + joined = rt.new((r[0] + return_tokens[0]) * 3 + 1); + joined[0] = r[0] - 1 + return_tokens[0]; + if (prefix) rt.copy_w(joined + 1, r + 1, 3); + rt.copy_w(joined + 1 + prefix * 3, return_tokens + 1, return_tokens[0] * 3); + rt.copy_w(joined + 1 + (prefix + return_tokens[0]) * 3, r + 1 + (prefix + 1) * 3, (r[0] - prefix - 1) * 3); + word previous = m*r; r = d*joined; rt.free(return_tokens); rt.free(buffer); rt.free(previous); + } src_push(1, r, 0, 1); next(); - if (typed) { + if (typed && !mdg) { // a typed method: skip the fn and the name, then compile from the ( with the declared return type - if (mdg) die("a typed method of a generic struct is not supported yet"); next(); next(); ifkp = tpack(); ifkk = 0 - 1; if (isp != 0) ifkk = 0; @@ -3352,17 +4205,31 @@ unchecked int methdef(int sn, int mn, word q, int stat, int typed, int rty, int // e.m(args): by now the receiver is on the stack, so it simply becomes the // first argument. the call is checked against the method's signature if it // has been compiled, and its result class is fresh if the method returns .u. -int methcall(int s, word rst) { +int methcall(int s, word rst, word receiver_fact, int receiver, word receiver_mask) { af_forget(); - int np, tags = 0, ret = 0; word ft = stype[s]; [8]word bs; + int np, tags = 0, ret = 0, temporary, owner, aliases, i; word ft = stype[s]; word[8] bs; word owner_fact, origins; + word requirements = effect_requirements, edges = effect_edges; int original = s; + effect_args_mask(s, 0, receiver_mask); + if (receiver_fact && (receiver_fact[2] == 4 || receiver_fact[2] == 5)) effect_require(s, 0, callpos); + if (receiver >= 0 && tracked(lroot(receiver))) effect_require_mode(s, 0, callpos, 1); if (sflag[s] & F_SIG) { tags = ssig[s] >> 6; ret = ssig[s] & 1; } else sflag[s] = sflag[s] | F_UNTYPED; if (sflag[s] & F_RETOWN) ret = 1; - fill_w(bs, 0, 8); - if ((sflag[s] & F_GEN) && ft && rst && rst[0] == TY_STRUCT) unify(ft[7], tptr(rst), bs); // a method of a generic struct: the receiver says what its parameters are - np = args(tags >> 2, 0, bs, 0) + 1; + rt.fill_w(bs, 0, 8); + if (ft && rst) { + word actual = rst; if (rst[0] == TY_STRUCT) actual = tptr(rst); + if (!unify(ft[7], actual, bs)) tcerr("receiver", subst(ft[7], bs), actual); + } + ifcall = s; effect_argument_base = 1; + np = args(tags >> 2, ft, bs, 0) + 1; + temporary = argtemporary || (receiver_fact && receiver_fact[2] == 5); owner = argowner; owner_fact = argowner_fact; origins = argorigin_mask; + if (effect_get(s, 0) & (EF_ALIAS | EF_IDENTITY | EF_UNKNOWN)) { + origins = origins | receiver_mask; + if (owner < 0) { owner = receiver; owner_fact = receiver_fact; } + else if (owner != receiver) { owner = -1; owner_fact = 0; } + } if ((sflag[s] & F_SIG) && np != ((ssig[s] >> 1) & 31)) serrn(sname[s], "wrong number of arguments"); - if (sflag[s] & F_GEN) s = ginstance(s, bs); + if (sflag[s] & F_GEN) { s = ginstance(s, bs); effect_specialize(original, s, requirements, edges); } fnref(s); op(O_CALL); lv = 0; lvk = LV_NONE; @@ -3376,6 +4243,12 @@ int methcall(int s, word rst) { xty = tword; if (ft) xty = subst(ft[4], bs); if (isfl(xty)) xfloat = 1; + effect_value = 0; + if (!ret && (isptr(xty) || xty == tword)) { + effect_value = origins; + if (isptr(xty)) xbor = owner; + if (temporary) { aliases = 0; i = 0; while (i < np) { aliases = aliases | effect_get(s, i); i++; } if (aliases & (EF_ALIAS | EF_UNKNOWN)) xaf = af_new(-1, 5, callpos); } + } if (ret) return C_FRESH; return C_PLAIN; } @@ -3399,14 +4272,16 @@ int caststart() { peek(); if (pk_tk == T_ID && (istypename(pk_tv) || pk_tv == s_fn)) { k = 1; peekat(k); + if (pk_tk == T_OP && (pk_tv == s_lk || pk_tv == s_lt || (pk_tv == s_lp && pq_tv[0] == s_fn))) return 1; while (pk_tk == T_OP && pk_tv == s_star && k < 3) { k++; peekat(k); } // (int*)x, (node**)x return pk_tk == T_OP && pk_tv == s_rp; } return 0; } -// memdebug on; checks and interior pinning. memdebug scrub; also poisons frees. -// memdebug off; stops both. at the top level, start() applies it before main. -// ---- statements that only configure the compiler ---- +//: memdebug on; checks and interior pinning. memdebug scrub; also poisons frees. +//: memdebug off; stops both. at the top level, start() applies it before main. +//: ---- statements that only configure the compiler ---- + int kw_memdebug() { int f = 0; stmtonly(); @@ -3424,20 +4299,20 @@ int kw_memdebug() { // enum name { A, B = 5, C }; the names are global constants, the name is an int type // the type of an enum: an int record that the typechecker keeps apart from the // plain ints. t[6] bit 4 marks it; the members carry it in stype -word tenum(int n) { word t = tnew(TY_INT, 1, n, 0); t[6] = 16; return t; } +word tenum(char* n) { word t = tnew(TY_INT, 1, n, 0); t[6] = 16; return t; } // true for a type made by an enum declaration. word isen(word t) { return t[0] == TY_INT && (t[6] & 16); } -// enum name { A, B = 5 }: each member is a constant, plain (A) and scoped (name.A) -// member records for tagged enums and for plain enums (pm_en is -1 for a plain -// member). a member's payload fields are at most 8. -// the text a tagged enum or an extern stub is generated into -[GENERATED_WORDS]word ebuf_initial; word ebuf = ebuf_initial; int ebcap = GENERATED_WORDS, ebn; -[ENUM_MEMBERS]word pm_name_initial, pm_en_initial, pm_tag_initial, pm_nf_initial, pm_val_initial, pm_exp_initial; -word pm_name = pm_name_initial, pm_en = pm_en_initial, pm_tag = pm_tag_initial, pm_nf = pm_nf_initial, pm_val = pm_val_initial, pm_exp = pm_exp_initial; int pmcap = ENUM_MEMBERS, npm; -[ENUM_MEMBERS * 8]word pm_fnm_initial, pm_ft_initial, pm_off_initial; -word pm_fnm = pm_fnm_initial, pm_ft = pm_ft_initial, pm_off = pm_off_initial; -[TAGGED_ENUMS]word pe_st_initial, pe_first_initial, pe_n_initial, pe_name_initial; -word pe_st = pe_st_initial, pe_first = pe_first_initial, pe_n = pe_n_initial, pe_name = pe_name_initial; int pecap = TAGGED_ENUMS, npe; +//: enum name { A, B = 5 }: each member is a constant, plain (A) and scoped (name.A) +//: member records for tagged enums and for plain enums (pm_en is -1 for a plain +//: member). a member's payload fields are at most 8. +//: the text a tagged enum or an extern stub is generated into +char[GENERATED_WORDS] ebuf_initial; char* ebuf = ebuf_initial; int ebcap = GENERATED_WORDS, ebn; +char*[ENUM_MEMBERS] pm_name_initial, pm_en_initial; word[ENUM_MEMBERS] pm_tag_initial, pm_nf_initial, pm_val_initial, pm_exp_initial; +char** pm_name = pm_name_initial, pm_en = pm_en_initial; word pm_tag = pm_tag_initial, pm_nf = pm_nf_initial, pm_val = pm_val_initial, pm_exp = pm_exp_initial; int pmcap = ENUM_MEMBERS, npm; +char*[ENUM_MEMBERS * 8] pm_fnm_initial; word[ENUM_MEMBERS * 8] pm_ft_initial, pm_off_initial; +char** pm_fnm = pm_fnm_initial; word pm_ft = pm_ft_initial, pm_off = pm_off_initial; +word[TAGGED_ENUMS] pe_st_initial, pe_first_initial, pe_n_initial; char*[TAGGED_ENUMS] pe_name_initial; +word pe_st = pe_st_initial, pe_first = pe_first_initial, pe_n = pe_n_initial; char** pe_name = pe_name_initial; int pecap = TAGGED_ENUMS, npe; // grow whole member rows, keeping their payload-field indices stable. int reserve_enum_members(int need) { int cap = pmcap; @@ -3459,9 +4334,9 @@ int reserve_tagged_enums(int need) { pecap = cap; return cap; } // a field's c-like spelling in a generated struct -int ftext(int t) { if (t == tint) return "int"; if (t == tfloat) return "float"; return "let"; } +char* ftext(int t) { if (t == tint) return "int"; if (t == tfloat) return "float"; return "let"; } // the member named n of a tagged enum, or -1 -int pm_find(int n) { +int pm_find(char* n) { int gi = npm - 1; while (gi >= 0) { if (pm_name[gi] == n && pm_en[gi] >= 0) return gi; gi--; } return -1; @@ -3477,7 +4352,7 @@ int ptagged(word t) { // and scoped (name.A). a member with a block of fields makes it a tagged enum // (see enum_tagged): enum name { circle { int r; }, dot }. int kw_enum() { - int n, en = 0, first = npm, cnt = 0, tagged = 0, gi, nf, v = 0, et = 0, sk, ft, fname; + char* n; char* en = 0; int first = npm, cnt = 0, tagged = 0, gi, nf, v = 0, et = 0, sk, ft; char* fname; stmtonly(); next(); if (tk == T_ID) { en = tv; next(); } @@ -3525,8 +4400,8 @@ int kw_enum() { // function on the enum, en.member(fields...), that allocates the struct, sets // the tag and the member's fields, and returns it owned. the struct is generated // as source and compiled here, so its methods are ordinary zc. -int enum_tagged(int en, int first, int cnt) { - int gi, j, nf, w = 1; word buf; int sk; +int enum_tagged(char* en, int first, int cnt) { + int gi, j, nf, w = 1; word buf, oldimports = module_imports; int sk; if (!en) { serr("a tagged enum needs a name"); return 0; } reserve_tagged_enums(npe + 1); if (cnt > 63) { fatal("a tagged enum has at most 63 members"); return 0; } @@ -3548,7 +4423,7 @@ int enum_tagged(int en, int first, int cnt) { if (pm_ft[nf] == tint) ebs("int "); else if (pm_ft[nf] == tfloat) ebs("float "); ebs(pm_fnm[nf]); } - ebs(") { "); ebs(en); ebs("* o = new("); ebs(en); ebs("); o.tag = "); ebn10(pm_tag[gi]); ebs("; "); + ebs(") { "); ebs(en); ebs("* o = rt.new("); ebs(en); ebs("); o.tag = "); ebn10(pm_tag[gi]); ebs("; "); for (j = 0; j < pm_nf[gi]; j++) { nf = gi * 8 + j; ebs("o."); ebs(pm_name[gi]); ebs("_"); ebs(pm_fnm[nf]); ebs(" = "); ebs(pm_fnm[nf]); ebs("; "); @@ -3556,15 +4431,17 @@ int enum_tagged(int en, int first, int cnt) { ebs("return u*o; } "); } ebs("}"); - buf = new(ebn + 2); - copy_w(buf, ebuf, ebn + 1); + buf = rt.new(ebn + 14); + rt.copy_w(buf, "include rt; ", 12); + rt.copy_w(buf + 12, ebuf, ebn + 1); src_push(2, buf, 0, 0); next(); stubgen = 1; while (tk != T_EOF) topitem(); stubgen = 0; src_pop(); - free(d*buf); + module_imports = oldimports; + rt.free(d*buf); sk = lookup(en); if (sk < 0 || skind[sk] != K_TYPE) { serr("the tagged enum didn't make its struct"); return 0; } pe_st[npe] = sval[sk]; pe_first[npe] = first; pe_n[npe] = cnt; pe_name[npe] = en; @@ -3584,32 +4461,207 @@ int cover_warn(int e, int cov, int sl) { if (dry || panic || cmode) return 0; for (k = 0; k < pe_n[e]; k++) { if (k < 63 && ((cov >> k) & 1)) continue; - if (first) { put_s("zc: line "); put_n(sl); put_s(": warning: this switch doesn't cover "); first = 0; } - else put_s(", "); + if (first) { rt.put_s("zc: line "); rt.put_n(sl); rt.put_s(": warning: this switch doesn't cover "); first = 0; } + else rt.put_s(", "); gi = pe_first[e] + k; - put_s(pm_name[gi]); + rt.put_s(pm_name[gi]); } - if (!first) put_c(10); + if (!first) rt.put_c(10); + return 0; +} +//: ---- include ---- +//: the text of the embedded std library, one character to a word and ending in 0, as readfile gives a file. +//: it is built from the pieces src/tools/embed.sh put at the end of this file, the first time it is asked for +char* std_text() { + int n = 0, i = 0, k = 0, j; bstr p; char* b; + if (std_buf) return std_buf; + while (1) { p = std_part(i); if (!p) break; n = n + rt.len_s(p); i++; } + b = rt.new(n + 2); + i = 0; + while (1) { + p = std_part(i); if (!p) break; + j = 0; while (j < rt.len_s(p)) { b[k] = rt.at_s(p, j); k++; j++; } + i++; + } + b[k] = 0; + std_buf = b; + return std_buf; +} +// the embedded icon module, assembled from its source pieces on first use. +char* icon_text() { + int n = 0, i = 0, k = 0, j; bstr p; char* b; + if (icon_buf) return icon_buf; + while (1) { p = icon_part(i); if (!p) break; n = n + rt.len_s(p); i++; } + b = rt.new(n + 2); + i = 0; + while (1) { + p = icon_part(i); if (!p) break; + j = 0; while (j < rt.len_s(p)) { b[k] = rt.at_s(p, j); k++; j++; } + i++; + } + b[k] = 0; + icon_buf = b; + return icon_buf; +} +// a path with its . and .. segments and doubled slashes taken out: /a/b/../c/./d.zc is /a/c/d.zc +char*.u inc_norm(char*.m p) { + int n = rt.len_w(p), i = 0, k = 0, s, e, j, isabs = 0; char* o; + o = rt.new(n + 4); + if (p[0] == '/') isabs = 1; + while (i < n) { + while (i < n && p[i] == '/') i++; + s = i; + while (i < n && p[i] != '/') i++; + e = i; + if (e == s) continue; + if (e - s == 1 && p[s] == '.') continue; + if (e - s == 2 && p[s] == '.' && p[s + 1] == '.') { + j = k; + while (j > 0 && o[j - 1] != '/') j--; + // the last segment is o[j..k); a leading .. of a relative path stays + if (k > 0 && !(k - j == 2 && o[j] == '.' && o[j + 1] == '.')) { + if (j > 0) k = j - 1; else k = 0; + continue; + } + if (k == 0 && isabs) continue; + } + if (k > 0 || isabs) { o[k] = '/'; k++; } + j = s; while (j < e) { o[k] = p[j]; k++; j++; } + } + if (k == 0 && isabs) { o[0] = '/'; k = 1; } + o[k] = 0; + return o; +} +// dir, a slash and path +char*.u inc_cat(char*.m dir, char*.m path) { + int a = rt.len_w(dir), b = rt.len_w(path); char* o; + o = rt.new(a + b + 3); + rt.copy_w(o, dir, a); + o[a] = '/'; + rt.copy_w(o + a + 1, path, b); + o[a + b + 1] = 0; + return o; +} +// the directory part of a path, without the last slash, or 0 when there is none +char*.u inc_dir(char*.m p) { + int n = rt.len_w(p), k; char* o; + k = n; + while (k > 0 && p[k - 1] != '/') k--; + if (k == 0) return 0; + if (k > 1) k--; + o = rt.new(k + 2); + rt.copy_w(o, p, k); + o[k] = 0; + return o; +} +// the text of the file at path: the editor's copy if it sent one, else the file +char*.u inc_try(char*.m path) { + if (inc_hook) { + char* text = inc_hook(path); + if (text) { + int n = rt.len_w(text); char* copy = rt.new(n + 1); + rt.copy_w(copy, text, n + 1); return copy; + } + } + char* copy = rt.readfile(path); + return copy; +} +// find an include: next to the file that has it, then in the workspace folder, then as written (relative to the +// working directory). incres is where it was found +char*.u inc_open(char*.m path) { + incres = inc_norm(path); + if (path[0] == '/' || path[1] == ':') { + char* c = inc_norm(path), b = inc_try(c); + if (b) incres = c; + return b; + } + if (incdir) { + char* combined = inc_cat(incdir, path); char* c = inc_norm(combined), b = inc_try(c); + if (b) { incres = c; return b; } + } + if (increoot) { + char* combined = inc_cat(increoot, path); char* c = inc_norm(combined), b = inc_try(c); + if (b) { incres = c; return b; } + } + return inc_try(path); +} +// compile the text buf as if it were written here, signatures first. name is what hovers and definitions call it +// and dir the directory its own includes are found from +int inc_compile(char* buf, char* name, char*.d dir) { + char* oldname = curinc; word olddir = incdir; + char* oldmodule = module_name; word oldimports = module_imports; + module_imports = 0; module_name = 0; + prescan(2, buf); + src_push(2, buf, 0, 0); + curinc = name; incdir = dir; incdepth++; + next(); + while (tk != T_EOF) topitem(); + curinc = oldname; incdir = olddir; incdepth--; + src_pop(); + module_name = oldmodule; module_imports = oldimports; return 0; } // include "path"; reads the file and compiles it here, signatures first +// include std; does the same with the standard library, which is part of the compiler int kw_include() { - int path, buf; + int pp, sv; char* path; char* buf; stmtonly(); next(); - if (tk != T_STR) { die("include needs a string"); return 0; } + if (tk == T_ID) { + while (1) { + pp = tpack(); path = tv; + if (tk != T_ID) { die("expected a module name in include"); return 0; } + next(); + if (!module_imported(path)) { + word link = chk_resize(0, 0, 2); link[0] = module_imports; link[1] = path; module_imports = link; + } + if (path == s_std && !std_done) { + std_done = 1; if (incdepth == 0) incplace = pp; + inc_compile(std_text(), "tinyvm:std.zc", 0); + } + if (path == s_icon && !icon_done) { + icon_done = 1; if (incdepth == 0) incplace = pp; + inc_compile(icon_text(), "tinyvm:icon.zc", 0); + } + if (!module_known(path) && !module_loaded(path)) { + char* filename = inc_cat("src/lib", path); int length = rt.len_w(filename); + char* full = chk_resize(0, 0, length + 4); rt.copy_w(full, filename, length); rt.copy_w(full + length, ".zc", 4); + buf = m*inc_open(full); + if (!buf) { sv = tpack(); tunpack(pp); serr("can't read that module include"); tunpack(sv); return 0; } + word link = chk_resize(0, 0, 2); link[0] = included_modules; link[1] = path; included_modules = link; + if (incdepth == 0) incplace = pp; + inc_compile(buf, incres, inc_dir(incres)); + } + if (!accept(s_comma)) break; + } + expect(s_semi); return 0; + } + if (tk != T_STR) { die("include takes a path in quotes, or std"); return 0; } + pp = tpack(); path = tv; next(); expect(s_semi); - buf = readfile(path); - if (!buf) { serr("can't read that include"); return 0; } - prescan(2, buf); - src_push(2, buf, 0, 0); - next(); - while (tk != T_EOF) topitem(); - src_pop(); + buf = m*inc_open(path); + if (!buf) { + sv = tpack(); tunpack(pp); + serr("can't read that include"); + tunpack(sv); + return 0; + } + if (incdepth == 0) incplace = pp; + inc_compile(buf, incres, inc_dir(incres)); return 0; } +// file namespace and imports are restored when an included source ends. +int kw_namespace() { + next(); + if (is_op(s_semi)) module_name = 0; + else { + module_name = tv; name(); + if (!module_known(module_name)) { word link = chk_resize(0, 0, 2); link[0] = included_modules; link[1] = module_name; included_modules = link; } + } + module_imports = 0; expect(s_semi); return 0; +} // typecheck on; / typecheck off; int kw_typecheck() { next(); @@ -3628,11 +4680,21 @@ int kw_docs() { next(); expect(s_semi); return 0; } +// hints on; / hints small; / hints off; from here to the end of the document (or the next such line) +int kw_hints() { + int level = 2, at = toff; + next(); + if (is_id(s_on)) level = 2; else if (is_id(s_small)) level = 1; else if (is_id(s_off)) level = 0; + else { die("hints on, hints small or hints off"); return 0; } + hint_set(level, at); + next(); expect(s_semi); + return 0; +} // the doc symbol of a name: the symbol "doc name" that holds its byte string, // or -1. the chain is walked by hand because lookup hides what is below vislo -int docsym(word nm) { - int i, k = 0; word p = "doc "; - doc_name_room(len_w(nm) + 5); +int docsym(char* nm) { + int i, k = 0; char* p = "doc "; + doc_name_room(rt.len_w(nm) + 5); while (p[k]) { dnmb[k] = p[k]; k++; } i = 0; while (nm[i]) { dnmb[k] = nm[i]; k++; i++; } dnmb[k] = 0; @@ -3645,13 +4707,13 @@ int docsym(word nm) { return -1; } // the packed documentation of a name (a byte string) or 0 -word docofs(int nm) { +bstr docofs(char* nm) { int s = docsym(nm); if (s < 0) return 0; return sval[s]; } // the documentation of a function, by address -word docof(int f) { +bstr docof(int f) { int i = nsym; while (i) { i--; @@ -3661,9 +4723,9 @@ word docof(int f) { } // documentation uses stable packed objects. symbols keep them alive until an // image copies their bytes after its code; ordinary docofs calls work too. -word doc_pack(word text, int length) { +bstr doc_pack(char* text, int length) { int i = 0, k, c, w, nw = (length + 7) >> 3; word p = chk_resize(0, 0, nw + 2); - p[-4] = p[-4] | F_SHARE | F_INTERNAL; p[0] = length; + p[-4] = p[-4] | rt.F_SHARE | rt.F_INTERNAL; p[0] = length; while (i < length) { w = 0; k = 0; while (k < 8 && i < length) { c = text[i]; if (c > 255) c = 63; w = w | (c << (k * 8)); k++; i++; } @@ -3671,12 +4733,26 @@ word doc_pack(word text, int length) { } return p; } +// section headings are organizational text, never documentation for a symbol. +int doc_section(char* text, int length) { + int start = 0, end, a, b; + while (start < length) { + end = start; while (end < length && text[end] != 10) end++; + a = start; b = end; + while (a < b && (text[a] == 32 || text[a] == 9 || text[a] == 13)) a++; + while (b > a && (text[b - 1] == 32 || text[b - 1] == 9 || text[b - 1] == 13)) b--; + if (b - a >= 4 && text[a] == '-' && text[a + 1] == '-' && text[b - 1] == '-' && text[b - 2] == '-') return 1; + start = end + 1; + } + return 0; +} + // attach the doc comment above a declaration at name n. -int doc_attach(word n) { +int doc_attach(char* n) { int i = 0, w, k, c, b, nw; - if (!dsn || !n || dry || ginst) return 0; + if (!dsn || !n || dry || ginst || doc_section(dsnap, dsn)) return 0; b = doc_pack(dsnap, dsn); - doc_name_room(len_w(n) + 5); + doc_name_room(rt.len_w(n) + 5); i = 0; while ("doc "[i]) { dnmb[i] = "doc "[i]; i++; } k = 0; while (n[k]) { dnmb[i] = n[k]; i++; k++; } dnmb[i] = 0; @@ -3684,50 +4760,51 @@ int doc_attach(word n) { dsn = 0; return 0; } -// the signature of a function as source text, "int put_c(int ch)", kept in the -// image as the symbol "sig name" when docs are on, so the language server can -// show it for any genome function, with or without a doc comment. the text is -// printed through put_c with the output sent to sgb (the same redirect the -// check uses for a parse word that prints) -[400]word sgb_initial; word sgb = sgb_initial; int sigcap = 400; -int sig_print(word n, int rty, int np, word pn, word pty, int typed) { +//: the signature of a function as source text, "int put_c(int ch)", kept in the +//: image as the symbol "sig name" when docs are on, so the language server can +//: show it for any genome function, with or without a doc comment. the text is +//: printed through put_c with the output sent to sgb (the same redirect the +//: check uses for a parse word that prints) +char[400] sgb_initial; char* sgb = sgb_initial; int sigcap = 400; +int sig_print(char* n, int rty, int np, char** pn, word pty, int typed) { int i = 0; - if (typed) { tprint(rty); put_c(' '); } else put_s("fn "); - put_s(n); put_c('('); + if (typed) { tprint(rty); rt.put_c(' '); } else rt.put_s("fn "); + rt.put_s(n); rt.put_c('('); while (i < np) { - if (i) put_s(", "); - if (typed) { tprint(pty[i]); put_c(' '); } - put_s(pn[i]); i++; + if (i) rt.put_s(", "); + if (typed) { tprint(pty[i]); rt.put_c(' '); } + rt.put_s(pn[i]); i++; } - put_c(')'); return 0; + rt.put_c(')'); return 0; } // attach a complete signature, measured before capture. -int sig_attach(word n, int rty, int np, word pn, word pty, int typed) { +int sig_attach(char* n, int rty, int np, char** pn, word pty, int typed) { int i = 0, k, b, nw, sm, sb, sn, sc, sl, w, c; if (!docs_on || dry || ginst || ntp || panic) return 0; - sm = outmode; sb = outbuf; sn = outn; sc = outcap; - outmode = 3; outn = 0; sig_print(n, rty, np, pn, pty, typed); - if (outn > sigcap) { while (outn > sigcap) sigcap = sigcap * 2; sgb = chk_resize(sgb, 0, sigcap); } - outmode = 2; outbuf = sgb; outn = 0; outcap = sigcap; + sm = rt.outmode; sb = rt.outbuf; sn = rt.outn; sc = rt.outcap; + rt.outmode = 3; rt.outn = 0; sig_print(n, rty, np, pn, pty, typed); + if (rt.outn > sigcap) { while (rt.outn > sigcap) sigcap = sigcap * 2; sgb = chk_resize(sgb, 0, sigcap); } + rt.outmode = 2; rt.outbuf = sgb; rt.outn = 0; rt.outcap = sigcap; sig_print(n, rty, np, pn, pty, typed); - sl = outn; - outmode = sm; outbuf = sb; outn = sn; outcap = sc; + sl = rt.outn; + rt.outmode = sm; rt.outbuf = sb; rt.outn = sn; rt.outcap = sc; b = doc_pack(sgb, sl); - doc_name_room(len_w(n) + 5); + doc_name_room(rt.len_w(n) + 5); i = 0; while ("sig "[i]) { dnmb[i] = "sig "[i]; i++; } k = 0; while (n[k]) { dnmb[i] = n[k]; i++; k++; } dnmb[i] = 0; addsym(intern(dnmb), K_DOC, b); return 0; } -// the field of a struct type, or -1 -// ---- built-ins that look like functions ---- +//: the field of a struct type, or -1 +//: ---- built-ins that look like functions ---- + // sizeof and typeof take a type or an expression; the expression form is // compiled with dry set so it contributes only its type. len, cap, push, pop -// and resize work on growable arrays ([]T handles in rt.zc); arrarg parses +// and resize work on growable arrays (T[?] handles in rt.zc); arrarg parses // their first argument and remembers which local it was, so the o3 and // bounds-proof machinery can see it. -int tfindfield(word t, int n) { +int tfindfield(word t, char* n) { int i = 0; tresolve(t); while (i < t[3]) { @@ -3751,27 +4828,33 @@ int typearg() { return t; } // len(a), cap(a), push(a, x), pop(a), resize(a, n): growable arrays. -// len of a [n]T is just n. -word arrarg() { - int c; - expect(s_lp); c = expr(); +// len of a T[n] is just n. +word arrvalue(int c) { argloc = -1; arrloc = -1; if (lv && lvk == LV_LOCAL && xty[0] == TY_DYN) { arrloc = lvi; if (o3on && o3cache[lvi]) argloc = lvi; } - rval(); plainuse(c); af_access(xaf, tpack()); + effect_value = effect_mask(c); rval(); plainuse(c); af_access(xaf, tpack()); return xty; } +// parse an array argument and retain the receiver's mutation and bounds facts. +word arrarg() { + int c; + expect(s_lp); c = expr(); + return arrvalue(c); +} // report an error when a fixed array is asked to grow or shrink. int nogrow(word t) { - if (t[0] == TY_DYN && t[3]) serr("a fixed array can't grow or shrink"); + require_mutable(t); + if (t[0] == TY_ARR || (t[0] == TY_DYN && t[3])) serr("a fixed array can't grow or shrink"); return 0; } // compile len(a): the length of an array. -int kw_len() { - word t; int a; - next(); t = arrarg(); a = arrloc; expect(s_rp); +int array_len(word t) { + int a = arrloc; + xcls = C_PLAIN; xaf = 0; xbor = -1; effect_value = 0; + expect(s_rp); if (xstr) { callname(s_len_s); xty = tint; @@ -3788,35 +4871,38 @@ int kw_len() { if (a >= 0 && t[0] == TY_DYN && t[3]) xlenloc = a; // the whole expression is len of a fixed array return 0; } +int kw_len() { next(); return array_len(arrarg()); } // compile cap(a): the capacity of a growable array. -int kw_cap() { - word t; - next(); - t = arrarg(); +int array_cap(word t) { + xcls = C_PLAIN; xaf = 0; xbor = -1; effect_value = 0; expect(s_rp); - tcneed(t[0] == TY_DYN || t == tword, "cap needs a []T", t); - callname(s_dcap); + if (t[0] == TY_ARR) { op(O_DROP); lit(t[3]); } + else { tcneed(t[0] == TY_DYN || t == tword, "cap needs an array", t); callname(s_dcap); } xty = tint; return 0; } +int kw_cap() { next(); return array_cap(arrarg()); } // compile push(a, v): add an element to the end of a growable array. -int kw_push() { - word t; int c; - next(); t = arrarg(); af_escape(xaf); nogrow(t); - if (t[0] == TY_DYN && tsz(t[4]) != 1) serr("push takes one-word elements; use resize for structs"); - expect(s_comma); c = expr(); rval(); argcheck(c, 2); +int array_push(word t) { + int c; + tcneed(t[0] == TY_DYN || t == tword, "push needs a T[?]", t); + effect_note_mask(effect_value, EF_WRITE); af_mutate(xaf, tpack()); nogrow(t); + if (t[0] == TY_DYN && isagg(t[4])) serr("push takes scalar or reference elements; use resize for aggregates"); + c = expr(); rval(); if (t[0] == TY_DYN) fcoerce(t[4]); + if (isptr(xty) || xty == tword) effect_note_mask(effect_mask(c), EF_RETAIN); argcheck(c, 2); if (t[0] == TY_DYN && !compat(t[4], xty)) tcerr("push", t[4], xty); if (o3on && t[0] == TY_DYN && hasptr(t[4])) { lit(t[4]); callname(s_fatten); } expect(s_rp); if (t[0] == TY_DYN && tbytes(t[4])) { // packed: an f32 goes in as its 32 bits, the rest are cut by the store if (isfl(t[4])) { if (!xfloat) op(OP_I2F); xenh("zf2s"); } - callname(intern("dpushb")); xty = tint; xfloat = 0; + callname(intern("dpushb")); xty = tint; xfloat = 0; xcls = C_PLAIN; return 0; } - if (o3on && t[0] == TY_DYN && !hasptr(t[4]) && tsz(t[4]) == 1) { pushfast(); xty = tint; xfloat = 0; return 0; } - callname(s_dpush); xty = tint; + if (o3on && t[0] == TY_DYN && !hasptr(t[4]) && tsz(t[4]) == 1) { pushfast(); xty = tint; xfloat = 0; xcls = C_PLAIN; return 0; } + callname(s_dpush); xty = tint; xfloat = 0; xcls = C_PLAIN; return 0; } +int kw_push() { word t; next(); t = arrarg(); expect(s_comma); return array_push(t); } // pop in o3 code, inline: a non-empty array gives up its last word, an empty one is dpop's report int popfast() { int k, e; @@ -3825,7 +4911,7 @@ int popfast() { lit(0); op(O_PICK); op(O_LD); lit(1); op(O_SUB); lit(1); op(O_PICK); op(O_ST); lit(0); op(O_PICK); op(O_LD); lit(1); op(O_PICK); lit(2); op(O_ADD); op(O_LD); - op(O_ADD); op(O_LD); + op(O_ADD); lit(0); op(O_PICK); op(O_LD); op(O_SWAP); lit(0); op(O_SWAP); op(O_ST); op(O_SWAP); op(O_DROP); e = hole(); op(O_JMP); setlit(k, here); @@ -3853,41 +4939,63 @@ int pushfast() { return 0; } // compile pop(a): remove and return the last element of a growable array. -int kw_pop() { - word t; - next(); - t = arrarg(); af_escape(xaf); +int array_pop(word t) { + xcls = C_PLAIN; + tcneed(t[0] == TY_DYN || t == tword, "pop needs a T[?]", t); + effect_note_mask(effect_value, EF_WRITE); af_mutate(xaf, tpack()); nogrow(t); + if (t[0] == TY_DYN && isagg(t[4])) serr("pop takes scalar or reference elements; access the last aggregate before resizing"); expect(s_rp); if (t[0] == TY_DYN && tbytes(t[4])) { // packed: read zero extended, then sign extend or widen callname(intern("dpopb")); xty = t[4]; xfloat = 0; + xaf = 0; xbor = -1; effect_value = 0; pkfix(xty); if (isfl(xty)) xfloat = 1; return 0; } - if (o3on && t[0] == TY_DYN && tsz(t[4]) == 1) popfast(); else callname(s_dpop); + if (o3on && t[0] == TY_DYN && !hasptr(t[4]) && tsz(t[4]) == 1) popfast(); else callname(s_dpop); xty = telem(t); if (t[0] == TY_DYN) xty = t[4]; xfloat = 0; if (isfl(xty)) xfloat = 1; + xaf = 0; xbor = -1; effect_value = 0; + if (isref(xty)) { + xcls = C_FRESH; xaf = af_new(-1, 2, tpack()); newsz = -1; newleaf = 0; return C_FRESH; + } return 0; } +int kw_pop() { next(); return array_pop(arrarg()); } // compile resize(a, n): set the length of a growable array. -int kw_resize() { - word t; int c; - next(); - t = arrarg(); af_escape(xaf); +int array_resize(word t) { + int c; + tcneed(t[0] == TY_DYN || t == tword, "resize needs a T[?]", t); + effect_note_mask(effect_value, EF_WRITE); af_mutate(xaf, tpack()); nogrow(t); - expect(s_comma); c = expr(); rval(); + tcneed(isnum(xty) || isfl(xty) || xty == tword, "resize needs a numeric length", xty); + fcoerce(tint); plainuse(c); expect(s_rp); if (t[0] == TY_DYN && tbytes(t[4])) callname(intern("dresizeb")); else callname(s_dresize); - xty = tint; + xty = tint; xcls = C_PLAIN; xfloat = 0; return 0; } +int kw_resize() { word t; next(); t = arrarg(); expect(s_comma); return array_resize(t); } +// array receiver calls share the free-call lowering and ownership checks. +int array_method(int n) { + return n == intern("len") || n == intern("cap") || n == intern("push") || n == intern("pop") || n == intern("resize"); +} +int array_member(int c, int n) { + word t = arrvalue(c); + expect(s_lp); + if (n == intern("len")) return array_len(t); + if (n == intern("cap")) return array_cap(t); + if (n == intern("push")) return array_push(t); + if (n == intern("pop")) return array_pop(t); + return array_resize(t); +} // compile sizeof(T) or sizeof(e): the size of a type or expression in words. int kw_sizeof() { int t; @@ -3906,7 +5014,8 @@ int kw_typeof() { xty = tword; return 0; } -// ---- c abi: extern and cfunc ---- +//: ---- c abi: extern and cfunc ---- + // extern double pow(double, double); or with parameter names, and an // optional "= "cname"" to give the zc function another name than the c symbol. // the declaration is read with c's spelling of types and turned into an @@ -3926,7 +5035,7 @@ int eb_char(int c) { if (ebn + 2 > ebcap) { ebcap = ebcap * 2; ebuf = chk_resize(ebuf, ebn, ebcap); } ebuf[ebn] = c; ebn++; ebuf[ebn] = 0; return 0; } -int ebs(word s) { +int ebs(char* s) { while (*s) { eb_char(*s); s++; } ebuf[ebn] = 0; return 0; @@ -3940,14 +5049,14 @@ int ebn10(int n) { } // true for a c type keyword, like long, char or double. int ctypeword(int w) { - return eq_w(w, "unsigned") || eq_w(w, "signed") || eq_w(w, "long") || eq_w(w, "short") || eq_w(w, "int") || eq_w(w, "char") || eq_w(w, "double") || eq_w(w, "float") || eq_w(w, "void") || eq_w(w, "const") || eq_w(w, "volatile"); + return rt.eq_w(w, "unsigned") || rt.eq_w(w, "signed") || rt.eq_w(w, "long") || rt.eq_w(w, "short") || rt.eq_w(w, "int") || rt.eq_w(w, "char") || rt.eq_w(w, "double") || rt.eq_w(w, "float") || rt.eq_w(w, "void") || rt.eq_w(w, "const") || rt.eq_w(w, "volatile"); } // a c type at the current token: its signature kind, or -1 after an error. // skipname also eats a parameter name that follows. // a plain zc struct a native call takes by value: every field is one word (a long, // a double or a pointer), so word g of the struct is eightbyte g. returns its // word count, or -1 after an error. -int cstruct_ok(word st, int nm) { +int cstruct_ok(word st, char* nm) { int i = 0, n; word ft; tresolve(st); n = st[1]; @@ -3961,27 +5070,27 @@ int cstruct_ok(word st, int nm) { } // the native signature kind of the c type at the current token, or -1 after an error. int ckind(int skipname) { - int w, uns = 0, lng = 0, shrt = 0, chr = 0, ints = 0, dbl = 0, flt = 0, vd = 0, other = 0, ptr = 0, seen = 0, k = -1, sw = 0, sk, nw; + char* w; int uns = 0, lng = 0, shrt = 0, chr = 0, ints = 0, dbl = 0, flt = 0, vd = 0, ptr = 0, seen = 0, k = -1, sk, nw; word other = 0; char* sw = 0; ckst = 0; ckn = 0; while (tk == T_ID) { w = tv; - if (eq_w(w, "const") || eq_w(w, "volatile") || eq_w(w, "restrict")) { next(); continue; } - if (eq_w(w, "struct") || eq_w(w, "enum") || eq_w(w, "union")) { + if (rt.eq_w(w, "const") || rt.eq_w(w, "volatile") || rt.eq_w(w, "restrict")) { next(); continue; } + if (rt.eq_w(w, "struct") || rt.eq_w(w, "enum") || rt.eq_w(w, "union")) { next(); - if (tk == T_ID) { if (eq_w(w, "struct")) sw = tv; next(); } + if (tk == T_ID) { if (rt.eq_w(w, "struct")) sw = tv; next(); } other = 1; seen = 1; continue; } if (seen && !ctypeword(w)) break; - if (eq_w(w, "unsigned")) uns = 1; - else if (eq_w(w, "signed")) ints = 1; - else if (eq_w(w, "long")) lng++; - else if (eq_w(w, "short")) shrt = 1; - else if (eq_w(w, "char")) chr = 1; - else if (eq_w(w, "int")) ints = 1; - else if (eq_w(w, "double") || eq_w(w, "f64")) dbl = 1; - else if (eq_w(w, "float") || eq_w(w, "f32")) flt = 1; - else if (eq_w(w, "void")) vd = 1; + if (rt.eq_w(w, "unsigned")) uns = 1; + else if (rt.eq_w(w, "signed")) ints = 1; + else if (rt.eq_w(w, "long")) lng++; + else if (rt.eq_w(w, "short")) shrt = 1; + else if (rt.eq_w(w, "char")) chr = 1; + else if (rt.eq_w(w, "int")) ints = 1; + else if (rt.eq_w(w, "double") || rt.eq_w(w, "f64")) dbl = 1; + else if (rt.eq_w(w, "float") || rt.eq_w(w, "f32")) flt = 1; + else if (rt.eq_w(w, "void")) vd = 1; else other = w; seen = 1; next(); @@ -3993,7 +5102,7 @@ int ckind(int skipname) { } if (skipname && tk == T_ID) next(); if (ptr) { - if (other && other != 1 && eq_w(other, "word")) return 6; + if (other && other != 1 && rt.eq_w(other, "word")) return 6; return 3; } if (dbl) return 2; @@ -4013,22 +5122,22 @@ int ckind(int skipname) { if (nw <= 2) return 13; // in registers when they fit (sysv: 16 bytes or less) return 14; // in memory: on the stack, word after word } - if (eq_w(other, "str")) return 12; - if (eq_w(other, "size_t") || eq_w(other, "ssize_t") || eq_w(other, "intptr_t") || eq_w(other, "uintptr_t") || eq_w(other, "ptrdiff_t") || eq_w(other, "int64_t") || eq_w(other, "uint64_t") || eq_w(other, "off_t") || eq_w(other, "time_t") || eq_w(other, "word")) return 1; - if (eq_w(other, "int32_t") || eq_w(other, "i32")) return 4; - if (eq_w(other, "uint32_t") || eq_w(other, "u32")) return 7; - if (eq_w(other, "int16_t") || eq_w(other, "i16")) return 8; - if (eq_w(other, "uint16_t") || eq_w(other, "u16")) return 9; - if (eq_w(other, "int8_t") || eq_w(other, "i8")) return 10; - if (eq_w(other, "uint8_t") || eq_w(other, "u8") || eq_w(other, "byte") || eq_w(other, "bool") || eq_w(other, "_Bool")) return 11; + if (rt.eq_w(other, "str")) return 12; + if (rt.eq_w(other, "size_t") || rt.eq_w(other, "ssize_t") || rt.eq_w(other, "intptr_t") || rt.eq_w(other, "uintptr_t") || rt.eq_w(other, "ptrdiff_t") || rt.eq_w(other, "int64_t") || rt.eq_w(other, "uint64_t") || rt.eq_w(other, "off_t") || rt.eq_w(other, "time_t") || rt.eq_w(other, "word")) return 1; + if (rt.eq_w(other, "int32_t") || rt.eq_w(other, "i32")) return 4; + if (rt.eq_w(other, "uint32_t") || rt.eq_w(other, "u32")) return 7; + if (rt.eq_w(other, "int16_t") || rt.eq_w(other, "i16")) return 8; + if (rt.eq_w(other, "uint16_t") || rt.eq_w(other, "u16")) return 9; + if (rt.eq_w(other, "int8_t") || rt.eq_w(other, "i8")) return 10; + if (rt.eq_w(other, "uint8_t") || rt.eq_w(other, "u8") || rt.eq_w(other, "byte") || rt.eq_w(other, "bool") || rt.eq_w(other, "_Bool")) return 11; serrn(other, "isn't a c type extern knows (use long, int, double, a pointer, ...)"); return 0 - 1; } // true for a native kind that travels in a float register. int ckindfloat(int k) { return k == 2 || k == 5; } // parse an extern declaration: a c function with its signature, and a zc stub for it. -unchecked int kw_extern() { - int rk; [16]word ks; [16]word kst; [16]word ksn; int np = 0, k, fname; word cname; int i, a, j, n, w, sig, bad = 0; word buf; int pos, len, g; word st, rst; int rn, rsd, sret = 0; +int kw_extern() { + int rk; word[16] ks; word[16] kst; char*[16] ksn; int np = 0, k; char* fname; char* cname; int i, a, j, n, w, sig, bad = 0; char* buf; word oldimports = module_imports; int pos, len, g; word st, rst; int rn, rsd, sret = 0; stmtonly(); next(); rk = ckind(0); @@ -4097,7 +5206,7 @@ unchecked int kw_extern() { } j = hole(); op(O_JMP); - reserve_code(20 + (len_w(cname) >> 3)); a = here; + reserve_code(20 + (rt.len_w(cname) >> 3)); a = here; emit((a + 18) << 3); emit(sig); emit(rsd << 32); // the cache's half is the host's; this half is the result's descriptor @@ -4118,7 +5227,7 @@ unchecked int kw_extern() { // for the places that take the function's address; a direct call to it // compiles to the native-call opcode instead (see call) ebn = 0; ebuf[0] = 0; - if (sret) { ebs("unchecked "); ebs(rn); ebs("*.u "); } + if (sret) { ebs(rn); ebs("*.u "); } else if (rk == 5) ebs("f32 "); else if (ckindfloat(rk)) ebs("float "); else ebs("int "); ebs(fname); ebs("("); i = 0; @@ -4129,24 +5238,28 @@ unchecked int kw_extern() { i++; } ebs(") { let b = "); ebn10(a); ebs(";"); - if (sret) { ebs(" "); ebs(rn); ebs("* r = new("); ebs(rn); ebs("); b[3] = r;"); } + // the native descriptor borrows an integer address, not ownership of r. + // clear that scratch address after the call so it cannot keep the result rooted. + if (sret) { ebs(" "); ebs(rn); ebs("* r = rt.new("); ebs(rn); ebs("); b[3] = core.addr_b(r) >> 3;"); } i = 0; while (i < np) { ebs(" b["); ebn10(i + 3 + sret); ebs("] = p"); ebn10(i); ebs(";"); i++; } - if (sret) ebs(" syscall(14, b, 0, 0); return r; }"); + if (sret) ebs(" core.syscall(14, b, 0, 0); b[3] = 0; return r; }"); else { - ebs(" return syscall(14, b, 0, 0)"); + ebs(" return core.syscall(14, b, 0, 0)"); if (rk == 5) ebs(" as f32"); else if (ckindfloat(rk)) ebs(" as float"); ebs("; }"); } - buf = new(ebn + 2); - copy_w(buf, ebuf, ebn + 1); + buf = rt.new(ebn + 19); + rt.copy_w(buf, "include rt, core; ", 18); + rt.copy_w(buf + 18, ebuf, ebn + 1); src_push(2, buf, 0, 0); next(); stubgen = 1; // the stub is ours, not the document's: its locals get no hints while (tk != T_EOF) topitem(); stubgen = 0; src_pop(); - free(buf); + module_imports = oldimports; + rt.free(buf); k = lookup(fname); if (k >= 0 && skind[k] == K_FN && (sflag[k] & F_DEF) && !sret) sflag[k] = sflag[k] | F_EXT; // a struct result is called as its stub return 0; @@ -4174,7 +5287,7 @@ int kw_cfunc() { next(); expect(s_lp); if (tk != T_ID) { die("cfunc needs the name of a function"); return 0; } - s = lookup(tv); + s = source_lookup(tv); if (s < 0 || skind[s] != K_FN) { serrn(tv, "isn't a function"); return 0; } t = sigof(sval[s]); if (!t) t = stype[s]; @@ -4199,6 +5312,7 @@ int kw_cfunc() { // is_id(s_fn) is one pointer compare. int types_init() { word f32p; + readonly_types = 0; tytop = R_TY + NTL; tyend = R_TYEND; typeblocks = 0; ntl = 0; fieldn = 0; if (R_TY + typecap > R_TYEND && typelist == R_TY) typelist = chk_resize(0, 0, typecap); tword = tnew(TY_WORD, 1, intern("word"), 0); @@ -4207,6 +5321,7 @@ int types_init() { tbool = tnew(TY_BOOL, 1, intern("bool"), 0); tvoid = tnew(TY_VOID, 0, intern("void"), 0); tfloat = tnew(TY_FLOAT, 1, intern("float"), 0); + tbstr = tnew(TY_BSTR, 1, intern("bstr"), 0); addsym(tbstr[2], K_TYPE, tbstr); addsym(tword[2], K_TYPE, tword); addsym(tint[2], K_TYPE, tint); addsym(tchar[2], K_TYPE, tchar); addsym(tbool[2], K_TYPE, tbool); addsym(tvoid[2], K_TYPE, tvoid); addsym(tfloat[2], K_TYPE, tfloat); addsym(intern("f64"), K_TYPE, tfloat); @@ -4224,39 +5339,39 @@ int types_init() { s_le = intern("<="); s_ge = intern(">="); s_eqeq = intern("=="); s_ne = intern("!="); s_plus = intern("+"); - s_struct = intern("struct"); s_type = intern("type"); s_typecheck = intern("typecheck"); s_docs = intern("docs"); + s_struct = intern("struct"); s_type = intern("type"); s_typecheck = intern("typecheck"); s_docs = intern("docs"); s_std = intern("std"); s_icon = intern("icon"); s_small = intern("small"); s_on = intern("on"); s_off = intern("off"); - s_new = intern("new"); - s_new_t = intern("new_t"); - s_new_k = intern("new_k"); - s_recycle_leaf = intern("recycle_leaf"); - s_hitems = intern("#items"); s_hlen = intern("#len"); s_fatten = intern("fatten"); + s_new = intern("rt.new"); + s_new_t = intern("rt.new_t"); + s_new_k = intern("rt.new_k"); + s_recycle_leaf = intern("rt.recycle_leaf"); + s_hitems = intern("#items"); s_hlen = intern("#len"); s_fatten = intern("run.fatten"); s_this = intern("this"); s_o3 = intern("o3"); - s_lfa = intern("lfa"); - s_outside = intern("outside"); - s_new_raw = intern("new_raw"); - s_oob = intern("oob"); - s_new_oz = intern("new_oz"); - s_ld8 = intern("ld8"); s_st8 = intern("st8"); s_ld16 = intern("ld16"); s_st16 = intern("st16"); - s_ld32 = intern("ld32"); s_st32 = intern("st32"); + s_lfa = intern("rt.lfa"); + s_outside = intern("rt.outside"); + s_new_raw = intern("rt.new_raw"); + s_oob = intern("rt.oob"); + s_new_oz = intern("rt.new_oz"); + s_ld8 = intern("rt.ld8"); s_st8 = intern("rt.st8"); s_ld16 = intern("rt.ld16"); s_st16 = intern("rt.st16"); + s_ld32 = intern("rt.ld32"); s_st32 = intern("rt.st32"); s_fixed = intern("fixed"); - s_dlen = intern("dlen"); - s_dcap = intern("dcap"); - s_dnew = intern("dnew"); - s_dat = intern("dat"); - s_len_s = intern("len_s"); - s_at_s = intern("at_s"); - s_slice_s = intern("slice_s"); - s_cat_s = intern("cat_s"); - s_eq_s = intern("eq_s"); - s_lt_s = intern("lt_s"); - s_le_s = intern("le_s"); - s_put_bs = intern("put_bs"); - s_put_s = intern("put_s"); - s_n_to_s = intern("n_to_s"); s_new_s = intern("new_s"); - s_dpush = intern("dpush"); s_dpop = intern("dpop"); s_dresize = intern("dresize"); + s_dlen = intern("rt.dlen"); + s_dcap = intern("rt.dcap"); + s_dnew = intern("rt.dnew"); + s_dat = intern("rt.dat"); + s_len_s = intern("rt.len_s"); + s_at_s = intern("rt.at_s"); + s_slice_s = intern("rt.slice_s"); + s_cat_s = intern("rt.cat_s"); + s_eq_s = intern("rt.eq_s"); + s_lt_s = intern("rt.lt_s"); + s_le_s = intern("rt.le_s"); + s_put_bs = intern("rt.put_bs"); + s_put_s = intern("rt.put_s"); + s_n_to_s = intern("rt.n_to_s"); s_new_s = intern("rt.new_s"); + s_dpush = intern("rt.dpush"); s_dpop = intern("rt.dpop"); s_dresize = intern("rt.dresize"); numops[0] = intern("*"); numops[1] = intern("/"); numops[2] = intern("%"); numops[3] = intern("&"); numops[4] = intern("|"); numops[5] = intern("^"); @@ -4278,62 +5393,64 @@ int isnumop(int o) { // the result type of binary operator b, which is also where the typechecker // complains: pointer plus int is that pointer, pointer minus pointer is an int, // word on either side wins, and a float on either side makes arithmetic float. -word bintype(int b, word lt, word rt) { +word bintype(int b, word lt, word rhs_type) { int o = bname[b]; - lt = decay(lt); rt = decay(rt); - if (blen[b] < 0) { tcneed(!isagg(lt) && !isagg(rt), "&& and || need scalars", lt); return tbool; } - if (isfl(lt) || isfl(rt)) { + lt = decay(lt); rhs_type = decay(rhs_type); + if (blen[b] < 0) { tcneed(!isagg(lt) && !isagg(rhs_type), "&& and || need scalars", lt); return tbool; } + if (isfl(lt) || isfl(rhs_type)) { if (o == s_plus || o == s_minus || o == s_star || o == s_slash) { - if (lt[0] == TY_PTR || rt[0] == TY_PTR) { tcerr("+", lt, rt); return tword; } + if (lt[0] == TY_PTR || rhs_type[0] == TY_PTR) { tcerr("+", lt, rhs_type); return tword; } return tfloat; } if (o == s_eqeq || o == s_ne || o == s_lt || o == s_gt || o == s_le || o == s_ge) return tbool; } if (o == s_eqeq || o == s_ne) { - if (!compat(lt, rt) && !compat(rt, lt)) tcerr("comparison", lt, rt); + if (!compat(lt, rhs_type) && !compat(rhs_type, lt)) tcerr("comparison", lt, rhs_type); return tbool; } if (o == s_lt || o == s_gt || o == s_le || o == s_ge) { - if (!(lt == tword || rt == tword || (isnum(lt) && isnum(rt)) || (lt[0] == TY_PTR && rt[0] == TY_PTR))) { - tcerr("comparison", lt, rt); + if (!(lt == tword || rhs_type == tword || (isnum(lt) && isnum(rhs_type)) || (lt[0] == TY_PTR && rhs_type[0] == TY_PTR))) { + tcerr("comparison", lt, rhs_type); } return tbool; } if (o == s_plus) { - if (lt[0] == TY_PTR && (isnum(rt) || rt == tword)) return lt; - if (rt[0] == TY_PTR && isnum(lt)) return rt; - if (lt == tword || rt == tword) return tword; - if (isnum(lt) && isnum(rt)) return tint; - tcerr("+", lt, rt); return tword; + if (lt[0] == TY_PTR && (isnum(rhs_type) || rhs_type == tword)) return lt; + if (rhs_type[0] == TY_PTR && isnum(lt)) return rhs_type; + if (lt == tword || rhs_type == tword) return tword; + if (isnum(lt) && isnum(rhs_type)) return tint; + tcerr("+", lt, rhs_type); return tword; } if (o == s_minus) { - if (lt[0] == TY_PTR && rt[0] == TY_PTR) return tint; - if (lt[0] == TY_PTR && (isnum(rt) || rt == tword)) return lt; - if (lt == tword || rt == tword) return tword; - if (isnum(lt) && isnum(rt)) return tint; - tcerr("-", lt, rt); return tword; + if (lt[0] == TY_PTR && rhs_type[0] == TY_PTR) return tint; + if (lt[0] == TY_PTR && (isnum(rhs_type) || rhs_type == tword)) return lt; + if (lt == tword || rhs_type == tword) return tword; + if (isnum(lt) && isnum(rhs_type)) return tint; + tcerr("-", lt, rhs_type); return tword; } if (isnumop(o)) { - if (lt == tword || rt == tword) return tword; - if (isnum(lt) && isnum(rt)) return tint; - if (isnum(lt)) lt = rt; + if (lt == tword || rhs_type == tword) return tword; + if (isnum(lt) && isnum(rhs_type)) return tint; + if (isnum(lt)) lt = rhs_type; tcneed(0, "arithmetic needs numbers", lt); return tword; } return tword; // operators a program added } -// ---- reflection ---- +//: ---- reflection ---- + // for programs (and parse words) looking at types at runtime word tkind(word t) { return t[0]; } // the size of a type in words. word tsize(int t) { return tsz(t); } // the name of a type. -word tname(word t) { return t[2]; } +char* tname(word t) { return t[2]; } // the length of an array type. word tlen(word t) { return t[3]; } // the number of fields of a struct type. word tnfields(word t) { tresolve(t); return t[3]; } -// ---- fattening ---- +//: ---- fattening ---- + // typed new stamps the header, in o3 the same as outside it. fatten is for a // value that still has new_raw's blank type word when it reaches code that // isn't o3: a store into a normal local or a global, an argument to a @@ -4369,7 +5486,7 @@ word isleaf(word t) { // true when a value of type t holds a pointer that fattening must follow. word hasptr(word t) { int k = t[0], i, n; - if (k == TY_PTR || k == TY_DYN) return 1; + if (k == TY_PTR || k == TY_DYN || k == TY_BSTR) return 1; if (k == TY_ARR) return hasptr(t[4]); if (k == TY_STRUCT && !t[5]) { n = tnfields(t); @@ -4388,7 +5505,7 @@ int fatten(int p, int t) { fat1(p, t, 32); return p; } // -(n + 1) for an object of n words, which tells how many elements to type. int fat1(word p, word t, int dl) { word e; int n, k, i, es; - if (dl <= 0 || !t || !isobj(p)) return 0; + if (dl <= 0 || !t || !rt.isobj(p)) return 0; if (t[0] == TY_DYN) { // a handle is always fat; what it holds may not be if (hasptr(t[4])) { es = tsz(t[4]); @@ -4404,7 +5521,7 @@ int fat1(word p, word t, int dl) { if (e == tword || e[0] == TY_VAR) p[-1] = 0; else if (n > es) p[-1] = tarr(e, 0); else p[-1] = e; - if (osize(p) > n) fill_w(p + n, 0, osize(p) - n); + if (rt.osize(p) > n) rt.fill_w(p + n, 0, rt.osize(p) - n); if (hasptr(e)) { k = n / es; i = 0; while (i < k) { fatin(p + i * es, e, dl - 1); i++; } } return 0; } @@ -4435,7 +5552,7 @@ word fatin(word a, word t, int dl) { } // reflection accessors over type records (see the layout under types). they // resolve a generic instance first, so asking about vec<int> works. -word tfname(word t, int i) { tresolve(t); return t[4][i * 3]; } +char* tfname(word t, int i) { tresolve(t); return t[4][i * 3]; } // the type of field i of a struct. word tftype(word t, int i) { tresolve(t); return t[4][i * 3 + 1]; } // the offset of field i of a struct. @@ -4457,10 +5574,11 @@ word tnargs(word t) { } // type argument i of a generic struct instance. word targ(word t, int i) { return t[6][1 + i]; } -// the type of a function, by address. untyped functions get one made from their signature -// ---- function types by address ---- +//: the type of a function, by address. untyped functions get one made from their signature +//: ---- function types by address ---- + word sigof(int f) { - int i = nsym, s = -1; [24]word ws; int n, k; + int i = nsym, s = -1; word[24] ws; int n, k; while (i) { i--; if (skind[i] == K_FN && sval[i] == f && (sflag[i] & F_DEF)) { s = i; break; } } if (s < 0) return 0; if (stype[s]) return stype[s]; @@ -4469,13 +5587,14 @@ word sigof(int f) { return tfn(tword, n, ws, ssig[s]); } // the function type of the function named nm, or 0 when there is none. -word sigofs(int nm) { +word sigofs(char* nm) { int s = lookups(nm); if (s < 0 || skind[s] != K_FN) return 0; return sigof(sval[s]); } -// ---- expressions ---- +//: ---- expressions ---- + // every expression leaves its value on the stack, returns its analyzer class, // and sets xty to its type. lv means the stack holds an address that hasn't // been loaded yet; structs and arrays are never loaded, their address is @@ -4497,7 +5616,13 @@ int dbg_addr() { // the thing is an array or struct, whose address is its value. every place // that needs a value calls this after parsing an expression. int rval() { - if (lv && lvk == LV_LOCAL) ltag[lvi] = ltag[lvi] | LREAD; + if (lv && lvk == LV_LOCAL) { + ltag[lvi] = ltag[lvi] | LREAD; + if (rt.eq_w(lname[lvi], "_") && af_warning(lreadpos[lvi])) { + rt.put_s("discard placeholder `_` is read\nhelp: give the parameter or pattern binding a name when its value is needed"); + af_warning_end(); + } + } if (lv) { if (ispk(xty)) pkld(xty); else if (!isagg(xty)) { dbg_addr(); op(O_LD); } @@ -4512,7 +5637,7 @@ int rval() { int kwcall(int s, int e) { int f = sval[s], old = kw_expr, c, n0 = ntok; kw_expr = e; xcls = C_PLAIN; xty = tword; xzero = 0; - f(); xaf = 0; + f(); xaf = 0; xvk = 0; // a handler that never reads a token (not even its own name) would be called // again on the same token forever, so it is an error, and the token is skipped if (ntok == n0) die("a parse word has to read its own name with next()"); @@ -4533,20 +5658,47 @@ int stmtonly() { // fat asks for every pointer argument to be fattened first, for o3 code // calling code that is not o3.) int args(int tags, word ft, int bs, int fat) { - int np = 0, c, pt, t, cs = ifcall, freeing = 0, harmless = 0; word first = 0; + int np = 0, c, pt, t, cs = ifcall, freeing = 0, harmless = 0; word first = 0; int origin = -1, anytemp = 0, current, tag, place, parameter_base = effect_argument_base, owner = -1, current_owner, owner_seen = 0, aliases; word origins = 0, current_mask, owner_fact = 0; if (cs >= 0 && cs < ubase) { freeing = sname[cs] == s_free || sname[cs] == s_recycle; - harmless = sname[cs] == s_put_s || eq_w(sname[cs], "put_n") || eq_w(sname[cs], "put_c") || eq_w(sname[cs], "len") || eq_w(sname[cs], "cap"); + harmless = sname[cs] == s_put_s || rt.eq_w(sname[cs], "rt.put_n") || rt.eq_w(sname[cs], "rt.put_c") || rt.eq_w(sname[cs], "len") || rt.eq_w(sname[cs], "cap"); } - xgive = 0; ifcall = 0 - 1; + xgive = 0; ifcall = 0 - 1; effect_argument_base = 0; expect(s_lp); if (!is_op(s_rp)) { while (1) { - c = expr(); rval(); + word expected = closure_expected; + closure_expected = 0; + if (ft && np + parameter_base < ft[3] && is_id(s_fn)) closure_expected = subst(ft[7 + np + parameter_base], bs); + place = tpack(); c = expr(); rval(); closure_expected = expected; + current = effect_origin(c); current_mask = effect_mask(c); origins = origins | current_mask; effect_args_mask(cs, np + parameter_base, current_mask); + if (!np) origin = current; else if (origin != current) origin = -1; + tag = (tags >> (np + np)) & 3; + current_owner = xbor; + if (c >= C_VAR && tracked(c - C_VAR)) current_owner = lroot(c - C_VAR); + if (tag <= 1 && current_owner >= 0 && tracked(current_owner)) effect_require_mode(cs, np + parameter_base, place, 1); + aliases = effect_get(cs, np + parameter_base); + if (aliases & (EF_ALIAS | EF_IDENTITY | EF_UNKNOWN)) { + if (!owner_seen) { owner = current_owner; owner_fact = xaf; owner_seen = 1; } + else if (owner != current_owner) { owner = -1; owner_fact = 0; } + } + if (fchk && c == C_FRESH && tag != 2) { + if (!(cs < 0 && tag == 1)) effect_require(cs, np + parameter_base, place); emit_borrow_hold(); c = C_PLAIN; anytemp = 1; + } + if (xaf && xaf[2] == 5) { anytemp = 1; if (!(cs < 0 && tag == 1)) effect_require(cs, np + parameter_base, place); } + if (fchk && xaf && xaf[2] == 4 && (isptr(xty) || isagg(xty)) && !(cs < 0 && tag == 1)) effect_require(cs, np + parameter_base, place); if (!np) first = xaf; - if (!freeing && !harmless) af_escape(xaf); - if (cs >= 0) if_arg(cs, np, xty, xzero); - if (ft && np < ft[3]) fcoerce(subst(ft[7 + np], bs)); + if (!freeing && !harmless && (effect_get(cs, np + parameter_base) & (EF_WRITE | EF_RETAIN | EF_CONSUME | EF_UNKNOWN))) af_escape(xaf); + if (ifs && if_value && cs >= 0 && sname[cs] == s_new && !np) if_join(if_value, tint); + if (cs >= 0) if_arg(cs, np + parameter_base, xty, xzero); + if (if_value && cs >= 0 && np + parameter_base < IFP) { + word infer_callee = if_get(cs); + if (infer_callee) { + if (ft && np + parameter_base < ft[3]) if_join(infer_callee + 8 + 3 * (np + parameter_base), subst(ft[7 + np + parameter_base], bs)); + if_link(if_value, infer_callee + 8 + 3 * (np + parameter_base)); + } + } + if (ft && np + parameter_base < ft[3]) fcoerce(subst(ft[7 + np + parameter_base], bs)); t = xty; argcheck(c, (tags >> (np + np)) & 3); // o3 data handed to normal code @@ -4555,8 +5707,9 @@ int args(int tags, word ft, int bs, int fat) { callname(s_fatten); xty = t; } - if (ft && np < ft[3]) { - pt = ft[7 + np]; + if (ft && np + parameter_base < ft[3]) { + pt = ft[7 + np + parameter_base]; + if (cs >= 0 && sname[cs] == s_put_s && np == 0 && xstr) pt = tbstr; if (!unify(pt, xty, bs)) tcerr("argument", subst(pt, bs), xty); } np++; @@ -4564,7 +5717,7 @@ int args(int tags, word ft, int bs, int fat) { } } expect(s_rp); - argfact = first; xaf = 0; + argfact = first; argorigin = origin; argorigin_mask = origins; argtemporary = anytemp; argowner = owner; argowner_fact = owner_fact; xaf = 0; vf_effect(); return np; } // memdebug records the source line on the next new, via the runtime global @@ -4579,7 +5732,7 @@ int stamp_site() { return 0; } // new(T) and new(T, count): a typed object, its type stamped into the header -// new(T), new(T, n), new([]T), new(fixed []T, n). a growable array goes to +// new(T), new(T, n), new(T[?]), new(fixed T[?], n). a growable array goes to // dnew. otherwise the call is new_t(size, type), unless the size is known at // compile time and we are not in o3 or a dry run: then the size class is // computed here and the call is new_k(size, type, class), which pops a block @@ -4591,11 +5744,11 @@ int newtyped() { if (t[0] == TY_DYN) { // a growable array: element size, length, type if (tbytes(t[4])) lit(0 - tbytes(t[4])); else lit(tsz(t[4])); // packed elements say so with a negative size: minus their bytes if (accept(s_comma)) { c = expr(); rval(); plainuse(c); length = -1; if (xconst && xcval >= 0) length = xcval; } - else { length = 0; lit(0); if (t[3]) serr("a fixed array needs its length: new(fixed []T, n)"); } + else { length = 0; lit(0); if (t[3]) serr("a fixed array needs its length: new(fixed T[?], n)"); } lit(t); expect(s_rp); stamp_site(); - callname(s_dnew); + callname(s_dnew); vf_effect(); xaf = af_new(length, 3, place); lv = 0; lvk = LV_NONE; xty = t; xzero = 0; xstamp = 1; return C_FRESH; @@ -4617,7 +5770,7 @@ int newtyped() { } expect(s_rp); stamp_site(); - if (fast) callname(s_new_k); else callname(s_new_t); + if (fast) callname(s_new_k); else callname(s_new_t); vf_effect(); xaf = af_new(length, 2, place); lv = 0; lvk = LV_NONE; xty = tptr(t); xzero = 0; xstamp = 1; return C_FRESH; @@ -4635,7 +5788,8 @@ int byteop(int s) { if (sname[s] == s_st32) return 617; return 0; } -// ---- type facts ---- +//: ---- type facts ---- + // annotate mode (the language server asks for it, ifon) collects the types the // untyped declarations of a document could be given, as edits: a place, how // many chars it replaces (0 inserts) and the text. a local gets the type of @@ -4651,7 +5805,7 @@ word ifpure(word t) { if (!t || t == tword || t == tvoid) return 0; k = t[0]; if (k == TY_INT) return t[5] == 0; - if (k == TY_CHAR || k == TY_BOOL || k == TY_STRUCT) return t[2] != 0; + if (k == TY_CHAR || k == TY_BOOL || k == TY_STRUCT || k == TY_BSTR) return t[2] != 0; if (k == TY_PTR) { k = t[4][0]; if (k == TY_ARR || k == TY_DYN || k == TY_FN) return 0; @@ -4663,19 +5817,19 @@ word ifpure(word t) { // one edit: the token at place p is replaced by the type t and then the text // after (len < 0: the token's own length; 0: nothing is replaced) int fact_add(int p, int len, int t, int after) { - int sv, o, at, n; word e; int osm = outmode, osb = outbuf, osn = outn, osc = outcap; + int sv, o, at, n; word e; int osm = rt.outmode, osb = rt.outbuf, osn = rt.outn, osc = rt.outcap; // the text is printed with tprint into the pool: output mode 2 writes into outbuf if (!ifx || dry || panic || chkabort) return 0; sv = tpack(); tunpack(p); o = toff; if (len < 0) len = toklen; tunpack(sv); if (o < 0) return 0; at = nift; - outn = 0; outmode = 3; tprint(t); put_s(after); n = outn; + rt.outn = 0; rt.outmode = 3; tprint(t); rt.put_s(after); n = rt.outn; if (nifx >= ifxcap) { ifxcap = ifxcap * 2; ifx = chk_resize(ifx, nifx * 4, ifxcap * 4); } if (at + n > iftcap) { while (at + n > iftcap) iftcap = iftcap * 2; ift = chk_resize(ift, at, iftcap); } - outbuf = ift + at; outn = 0; outcap = n; outmode = 2; - tprint(t); put_s(after); - n = outn; - outmode = osm; outbuf = osb; outn = osn; outcap = osc; + rt.outbuf = ift + at; rt.outn = 0; rt.outcap = n; rt.outmode = 2; + tprint(t); rt.put_s(after); + n = rt.outn; + rt.outmode = osm; rt.outbuf = osb; rt.outn = osn; rt.outcap = osc; e = ifx + nifx * 4; e[0] = o; e[1] = len; e[2] = at; e[3] = n; nifx++; nift = at + n; @@ -4690,18 +5844,46 @@ word if_get(int s) { if (nif >= ifrcap) { ifrcap = ifrcap * 2; ifr = chk_resize(ifr, nif, ifrcap); } nif++; ifs[s] = nif; r = chk_resize(0, 0, IFW); ifr[nif - 1] = r; - fill_w(r, 0, IFW); + rt.fill_w(r, 0, IFW); r[0] = s; return r; } // a state is 0 (nothing seen), a type, or -1 (no single type): p points at it int if_join(word p, int t) { + if (!t || t == tword) return 0; if (*p == 0 - 1) return 0; if (!ifpure(t)) *p = 0 - 1; else if (!*p) *p = t; else if (*p != t) *p = 0 - 1; return 0; } +// equal type flows connect caller parameters, callee parameters and results. +// nodes are addresses of inference states, retained by the function records. +int if_link(word a, word b) { + word edge; + if (!ifs || !a || !b || a == b || dry) return 0; + edge = chk_resize(0, 0, 3); edge[0] = if_edges; edge[1] = a; edge[2] = b; if_edges = edge; + return 0; +} +int if_solve() { + word edge, a, b, record; int changed = 1, left, right, index = 0; + while (index < nif) { + record = ifr[index]; + if (!record[3] && (!record[2] || !record[1])) record[3] = -1; + index++; + } + while (changed) { + changed = 0; edge = if_edges; + while (edge) { + a = edge[1]; b = edge[2]; left = *a; right = *b; + if (left == -1 || right == -1 || (left && right && left != right)) { *a = -1; *b = -1; } + else if (left) *b = left; else if (right) *a = right; + if (*a != left || *b != right) changed = 1; + edge = edge[0]; + } + } + return 0; +} // argument j of a call of s has the type t (a literal 0 says nothing: it is a // null pointer as often as a number) int if_arg(int s, int j, int t, int zero) { @@ -4723,10 +5905,10 @@ int if_begin(int s, int k) { } // parameter j: p is where its type would go, kind 0 for a typed one, 1 to // insert the type and a space, 2 to insert it right in front of a tag -int if_param(int j, int p, int kind) { +int if_param(int j, int p, int kind, word type) { word r = ifcur; if (!r || j >= IFP) return 0; - r[8 + 3 * j] = 0; r[9 + 3 * j] = p; r[10 + 3 * j] = kind; + if_join(r + 8 + 3 * j, type); r[9 + 3 * j] = p; r[10 + 3 * j] = kind; if (kind) r[7] = j + 1; return 0; } @@ -4735,6 +5917,7 @@ int if_ret(int t, int zero) { word r = ifcur; if (!r || nclo || dry) return 0; r[1] = 1; + if (if_value && !zero) if_link(r + 3, if_value); if (zero) r[4] = 1; else if_join(r + 3, t); return 0; } @@ -4750,6 +5933,7 @@ int if_fin() { // the edits of every function, once all the calls have been seen int if_emit() { int i = 0, j; word r; int t, k; + if_solve(); while (i < nif) { r = ifr[i]; i++; @@ -4771,28 +5955,30 @@ int if_emit() { } return 0; } -// ---- hints ---- +//: ---- hints ---- + // findings that are not errors. each is written only in check mode, through // hint_begin, and none of them changes the code the compiler emits. int magic(int v) { - if (!hint_begin(tpack())) return 0; - put_s("the number "); put_n(v); put_s(" has no name; a const says what it means"); + if (literal_documented(toff) || !hint_begin(tpack(), HINT_ALL)) return 0; + rt.put_s("unannotated numeric literal "); rt.put_n(v); + rt.put_s("\nhelp: use a named constant or add an inline block comment before the literal to suppress this hint"); dg_close(); return 0; } // warn that a new inside a loop allocates on every pass. int newloop(int p) { - if (!hint_begin(p)) return 0; - put_s("new inside a loop allocates on every pass; allocate once above the loop if the object can be reused"); + if (!hint_begin(p, HINT_SMALL)) return 0; + rt.put_s("new inside a loop allocates on every pass; allocate once above the loop if the object can be reused"); dg_close(); return 0; } // hint when a shift by v is past the width of the value. int shifthint(int p, int v) { int k = 0; - if (!hint_begin(p)) return 0; + if (!hint_begin(p, HINT_ALL)) return 0; while ((1 << k) < v) k++; - put_s("multiplying by "); put_n(v); put_s(" is a shift; write << "); put_n(k); + rt.put_s("multiplying by "); rt.put_n(v); rt.put_s(" is a shift; write << "); rt.put_n(k); dg_close(); return 0; } @@ -4811,11 +5997,13 @@ int shifthint(int p, int v) { // 6. the result's type and class are set: .u functions return C_FRESH, which // is how the analyzer knows the caller now owns a new object int call(int s) { - int np, tags = 0, ret = 0; word ft = stype[s]; [8]word bs; int rs, rt = 0, kop, cp = callpos, original = s; word storage; + int np, tags = 0, ret = 0; word ft = stype[s]; word[8] bs; int rs, rhs_type = 0, kop, cp = callpos, original = s; word storage; int origin, temporary, owner; word origins, owner_fact; + word requirements = effect_requirements, edges = effect_edges; + borrow_depth++; if (sname[s] == s_new && skind[s] == K_FN) { peek(); - if (pk_tk == T_OP && pk_tv == s_lk) { next(); return newtyped(); } - if (pk_tk == T_ID && (istypename(pk_tv) || pk_tv == s_fn)) { next(); return newtyped(); } + if (pk_tk == T_OP && pk_tv == s_lk) { next(); borrow_depth--; return newtyped(); } + if (pk_tk == T_ID && (istypename(pk_tv) || pk_tv == s_fn)) { next(); borrow_depth--; return newtyped(); } } if (o3on && sname[s] == s_new) { tags = lookup(s_new_oz); if (tags >= 0) s = tags; tags = 0; } if (o3on) { @@ -4831,21 +6019,21 @@ int call(int s) { } lv = 0; lvk = LV_NONE; xty = tword; xzero = 0; xbor = -1; xfloat = 0; xstr = 0; xlenloc = -1; xstamp = 0; - return C_PLAIN; + borrow_depth--; return C_PLAIN; } } if (sflag[s] & F_SIG) { tags = ssig[s] >> 6; ret = ssig[s] & 1; } else sflag[s] = sflag[s] | F_UNTYPED; if (sflag[s] & F_RETOWN) ret = 1; - fill_w(bs, 0, 8); - if (ngbind) { copy_w(bs, gbind, ngbind); ngbind = 0; } + rt.fill_w(bs, 0, 8); + if (ngbind) { rt.copy_w(bs, gbind, ngbind); ngbind = 0; } np = 0; if (o3on && !(sflag[s] & F_O3) && sname[s] != s_free && sname[s] != s_recycle) np = 1; // o3 calling normal code: fatten the arguments - ifcall = s; + ifcall = s; effect_argument_base = 0; np = args(tags, ft, bs, np); - storage = argfact; + storage = argfact; origin = argorigin; origins = argorigin_mask; temporary = argtemporary; owner = argowner; owner_fact = argowner_fact; if (original < ubase && (sname[original] == s_free || sname[original] == s_recycle)) af_end(storage, cp); - if (sflag[s] & F_GEN) s = ginstance(s, bs); + if (sflag[s] & F_GEN) { int template = s; s = ginstance(s, bs); effect_specialize(template, s, requirements, edges); } if (sname[s] == s_free && xgive) { // the last pointer: straight back to the size-class list rs = lookup(s_recycle); if (rs >= 0) s = rs; @@ -4871,8 +6059,8 @@ int call(int s) { if (sflag[s] & F_EXT) { lit(sval[s][-1]); op(620); } // an extern: the native call opcode, with the block's address else { fnref(s); op(O_CALL); } if (!o3on && (sflag[s] & F_O3) && ft) { // non-o3 called o3: fatten the result here, not inside the callee - rt = subst(ft[4], bs); - if (hasptr(rt)) { lit(rt); callname(s_fatten); } + rhs_type = subst(ft[4], bs); + if (hasptr(rhs_type)) { lit(rhs_type); callname(s_fatten); } } xstr = 0; if (sname[s] == s_cat_s || sname[s] == s_slice_s || sname[s] == s_n_to_s || sname[s] == s_new_s) { @@ -4881,10 +6069,28 @@ int call(int s) { lv = 0; lvk = LV_NONE; xzero = 0; xbor = -1; xconst = 0; xty = tword; if (ft) xty = subst(ft[4], bs); + if (xstr || xty == tbstr) { xstr = 1; xty = tbstr; } xfloat = 0; if (isfl(xty)) xfloat = 1; xlenloc = -1; xstamp = 0; if (o3on && (sflag[s] & F_O3) && ft && hasptr(xty)) xstamp = 1; + borrow_depth--; effect_value = 0; + if (!ret && (isptr(xty) || xty == tword)) { + effect_value = origins; + if (owner >= 0 && (isptr(xty) || (effect_get(original, 0) & EF_IDENTITY))) xbor = owner; + if (owner_fact && (effect_get(original, 0) & EF_IDENTITY)) xaf = owner_fact; + if (origin >= 0 && owner < 0) { int oi = lbase; while (oi < nloc) { if (lrskind[oi] == RS_PARAM && oi - lbase == origin) { xbor = oi; break; } oi++; } } + if (temporary) { int ai = 0, aliases = 0; while (ai < np) { aliases = aliases | effect_get(original, ai); ai++; } if (aliases & (EF_ALIAS | EF_UNKNOWN)) xaf = af_new(-1, 5, cp); } + } + if_value = 0; + if (ifs && !dry) { + word result_node = if_get(original); + if (result_node) { + if (sname[original] == s_new && !result_node[3]) result_node[3] = tword; + else if_join(result_node + 3, xty); + if_value = result_node + 3; + } + } if (ret) return C_FRESH; return C_PLAIN; } @@ -4905,6 +6111,7 @@ int enhcall(int s) { } } expect(s_rp); + vf_effect(); if (n != nin) serrn(sname[s], "wrong number of inputs"); lv = 0; lvk = LV_NONE; xty = tword; xzero = 0; xbor = -1; xlenloc = -1; xstamp = 0; if (sflag[s] & F_FALLBACK) { fnref(sfix[s] - 1); op(O_CALL); return C_PLAIN; } @@ -4913,15 +6120,47 @@ int enhcall(int s) { if (nout > 1) serrn(sname[s], "leaves several values; use it from asm"); return C_PLAIN; } -// ---- closures ---- +//: ---- closures ---- + // fn(a, b) { ... } is a function whose last, hidden parameter is a pointer to // its environment. the closure value is a heap block: word 0 is the body's // entry, and the captured variables start at word 3. the block itself is never // run as code. callable_code distinguishes plain function addresses, including // linked heap code segments, from closure blocks. a plain function is called // as it is. a closure is called with block + 3 as its env, at the entry in -// word 0. a captured variable is copied in when the closure is made; the body -// reads and writes its own copy. +// word 0. a capture is a shared cell. the enclosing function and every closure +// read and write the same value; cells remain reachable through escaping closures. +// scan with the compiler lexer, restoring its source and lookahead. only functions containing a closure +// need cells; their scalars get cells from declaration time, so loop back edges and earlier references agree. +int capture_scan() { + word old, f; int depth = 1, found = 0, docs = docs_on, record = grec; + if (isp < 0 || R_SRC[isp * 32] == 0 || !is_op(s_lb)) return 0; + old = R_SRC + isp * 32; + src_push(1, capture_dummy, 0, 0); old = R_SRC + (isp - 1) * 32; f = R_SRC + isp * 32; + f[0] = old[0]; f[1] = old[1]; f[2] = old[2]; f[3] = old[3]; f[4] = 0; pq_n = f[14]; + docs_on = 0; grec = 0; dry++; + while (depth) { + next(); if (tk == T_EOF) break; + if (is_op(s_lb)) depth++; + else if (is_op(s_rb)) depth--; + else if (is_id(s_fn)) { peek(); if (pk_tk == T_OP && (pk_tv == s_lp || pk_tv == s_colon)) found = 1; } + } + dry--; src_pop(); docs_on = docs; grec = record; return found; +} +int capture_cell(int i, int copy) { + if (!capture_enabled || o3on || (ltag[i] & 16) || larr[i] || local_cell[i]) return 0; + if (copy) { laddr(lslot[i]); op(O_LD); } + lit(1); lit(ltype[i]); callname(s_new_t); callname(intern("share")); + if (copy) { lit(0); op(O_PICK); laddr(lslot[i]); op(O_ST); op(O_ST); } + else { laddr(lslot[i]); op(O_ST); } + local_cell[i] = 1; return 0; +} +int capture_prepare() { + int i = lbase; capture_enabled = capture_scan(); + if (!capture_enabled) return 0; + while (i < nloc) { if (lrskind[i] == RS_PARAM) capture_cell(i, 1); i++; } + return 0; +} // grow closure frames and capture rows while keeping closure and capture indices. int reserve_closures(int need) { int cap = closure_cap; @@ -4938,22 +6177,22 @@ int reserve_closures(int need) { closure_cap = cap; return cap; } int reserve_captures(int need) { - int cap = capture_width, i = 0; word names, locals, via; + int cap = capture_width, i = 0; char** names; word locals, via; if (need <= cap) return cap; while (cap < need) cap = cap * 2; names = chk_resize(0, 0, closure_cap * cap); locals = chk_resize(0, 0, closure_cap * cap); via = chk_resize(0, 0, closure_cap * cap); while (i < nclo) { - copy_w(names + i * cap, clo_name + i * capture_width, clo_n[i]); - copy_w(locals + i * cap, clo_loc + i * capture_width, clo_n[i]); - copy_w(via + i * cap, clo_via + i * capture_width, clo_n[i]); i++; + rt.copy_w(names + i * cap, clo_name + i * capture_width, clo_n[i]); + rt.copy_w(locals + i * cap, clo_loc + i * capture_width, clo_n[i]); + rt.copy_w(via + i * cap, clo_via + i * capture_width, clo_n[i]); i++; } clo_name = names; clo_loc = locals; clo_via = via; capture_width = cap; return cap; } // find or create the capture slot for variable n in the innermost closure // being compiled: if the name is one of the enclosing function's locals, it is -// recorded (once) and the body reads it from its own environment copy. +// recorded once and the body reads its shared cell through the environment. // a name from further out is captured through each closure in between: the // closure that encloses this one captures it too, and this one copies it from // that closure's environment (clo_via is the slot there, -1 for a local). @@ -4961,12 +6200,17 @@ word clocap(int n) { return clocap_at(nclo - 1, n); } // the slot of the name n in closure c's captures, adding it when the closure has // not captured it yet. returns the capture index, or -1 when no such name is visible. word clocap_at(int c, int n) { - int i = 0, j, base = c * capture_width, r; + int i = 0, j, base = c * capture_width, r, root; while (i < clo_n[c]) { if (clo_name[base + i] == n) return i; i++; } j = clo_hi[c]; while (j > clo_lo[c]) { j--; if (lname[j] != n) continue; + effect_note(effect_origin(C_VAR + j), EF_RETAIN); + root = lroot(j); local_captured[j] = 1; local_captured[root] = 1; + if (lafact[j] && (lafact[j][2] == 1 || lafact[j][2] == 4 || lafact[j][2] == 5)) serrn(n, "a stack or temporary borrow cannot escape through a closure"); + if (root != j && lname[root]) { clocap_at(c, lname[root]); base = c * capture_width; } + if (ltag[j] & 16) serrn(n, "an o3 variable cannot be captured by reference; use an ordinary local"); if (larr[j]) serrn(n, "an array can't be captured; capture a pointer to it"); reserve_captures(clo_n[c] + 1); base = c * capture_width; clo_name[base + clo_n[c]] = n; clo_loc[base + clo_n[c]] = j; clo_via[base + clo_n[c]] = 0 - 1; clo_n[c]++; @@ -4979,6 +6223,18 @@ word clocap_at(int c, int n) { clo_name[base + clo_n[c]] = n; clo_loc[base + clo_n[c]] = clo_loc[(c - 1) * capture_width + r]; clo_via[base + clo_n[c]] = r; clo_n[c]++; return clo_n[c] - 1; } +// closure environments retain precise capture-cell types for runtime reflection. +word closure_layout(int closure_index, int count) { + word t = tnew(TY_STRUCT, 3 + count, 0, 0), fields = talloc((3 + count) * 3); int i = 0, local; + t[3] = 3 + count; t[4] = fields; + while (i < 3 + count) { + fields[i * 3 + 2] = i; + if (i < 3) fields[i * 3 + 1] = tword; + else { local = clo_loc[closure_index * capture_width + i - 3]; fields[i * 3 + 1] = tptr(ltype[local]); } + i++; + } + return t; +} // fn(params) { body } as an expression. the body is compiled in place behind // a jump (hop) as a nested function with a hidden last parameter, env. when // the code that creates the closure runs, it allocates the block (code, then @@ -4986,18 +6242,26 @@ word clocap_at(int c, int n) { // variable in. ctx_push / ctx_pop save and restore the enclosing function's // compiler state around the body. int closure() { - int np = 0; [20]word pn; [20]word pty; int hop, entry, i, t, envl, ncap, c, result; + int np = 0; char*[20] pn; word[20] pty; int hop, entry, i, t, envl, ncap, c, result, declared = 0, tags = 0, tag; word[20] ptags; + word expected = closure_expected; closure_expected = 0; reserve_closures(nclo + 1); if (o3on) serr("closures can't be made inside an o3 region"); next(); + // fn:type(args) names the result; returns are coerced and checked against it + if (accept(s_colon)) declared = ptype0(); expect(s_lp); if (!is_op(s_rp)) { while (1) { pty[np] = tword; + if (expected && expected[0] == TY_FN && np < expected[3]) pty[np] = expected[7 + np]; if (!is_op(s_dot) && ((tk == T_OP && tv == s_lk) || (tk == T_ID && (tv == s_fn || istypename(tv))))) { - pty[np] = ptype(); + pty[np] = ptype(); if (pty[np][0] == TY_ARR) pty[np] = decay(pty[np]); } - pnoffs[np] = toff; rs_note(RS_PARAM, toff); pn[np] = name(); + tag = 0; + if (accept(s_dot)) { if (is_id(s_m)) tag = 1; else if (is_id(s_d)) tag = 2; else die("a parameter tag is .m or .d"); next(); } + ptags[np] = tag; if (np < 12) tags = tags | (tag << (np + np)); else if (tag) die("only the first 12 parameters can be tagged"); + if (expected && expected[0] == TY_FN && !tag && (isptr(pty[np]) || pty[np][0] == TY_DYN || pty[np][0] == TY_FN)) ptags[np] = 1; + pnoffs[np] = toff; pnplaces[np] = tpack(); rs_note(RS_PARAM, toff); pn[np] = name(); np++; if (np >= 20) { die("too many parameters"); break; } if (!accept(s_comma)) break; @@ -5010,11 +6274,14 @@ int closure() { hop = hole(); op(O_JMP); entry = here; fn_begin(1, 0); - prologue(); + if (declared) fty = declared; + prologue(); borrow_prologue(); i = 0; while (i < np) { - t = newloc(pn[i]); lrskind[t] = RS_PARAM; lrsoff[t] = pnoffs[i]; fslots++; lsmark(1, pn[i]); + t = newloc(pn[i]); lrskind[t] = RS_PARAM; lrsoff[t] = pnoffs[i]; lpos[t] = pnplaces[i]; fslots++; lsmark(1, pn[i]); ltype[t] = pty[i]; + if (fchk && ptags[i] == 1) lstate[t] = S_BORROW; + if (fchk && ptags[i] == 2) { lstate[t] = S_M; lown[t] = 1; } if (isfl(pty[i])) setflt(t, 1); i++; } @@ -5025,21 +6292,28 @@ int closure() { body(); result = clo_ret[nclo - 1]; if (result <= 0 || live) result = tword; + if (declared) result = declared; epilogue(); nclo--; c = nclo; ncap = clo_n[c]; setlit(hop, here); ctx_pop(); + if (probe_off >= 0 && probe_has == 1 && tpend <= probe_off && !ginst) probe_snap(); // past the closure: what the function around it has // the block: [push env][push body][jump] and the captured values // the block: word 0 is the body's entry, words 3 on are the captured values. // a call through the closure pushes block + 3 as the env and jumps to word 0. - lit(3 + ncap); callname(s_new); + lit(3 + ncap); lit(closure_layout(c, ncap)); callname(s_new_t); lit(entry); lit(1); op(O_PICK); op(O_ST); i = 0; while (i < ncap) { t = clo_loc[c * capture_width + i]; - if (clo_via[c * capture_width + i] < 0) { touch(t); vaddr(t); op(O_LD); } + if (clo_via[c * capture_width + i] < 0) { + touch(t); effect_note_mask(effect_mask(C_VAR + t), EF_RETAIN); ev_put(t, EV_CAPTURE, -1); + if (cmode && !dry) { lvescaped[t] = 1; lvkind[t] = 0; } + if (!isalias(t)) { vaddr(t); op(O_LD); callname(intern("share")); op(O_DROP); } + vaddr(t); + } else { vaddr(clo_env[c - 1]); op(O_LD); lit(clo_via[c * capture_width + i]); op(O_ADD); op(O_LD); } lit(1); op(O_PICK); lit(3 + i); op(O_ADD); op(O_ST); i++; @@ -5053,10 +6327,211 @@ int closure() { xstr = 0; xlenloc = -1; xstamp = 0; - xty = tfn(result, np, pty, np + np); + xty = tfn(result, np, pty, np + np + tags * 64); + return C_FRESH; +} + +//: ---- array initializer lists and compound literals ---- +//: keep original token places while measuring and replaying an initializer. measurement never evaluates +//: value expressions: values run once, in source order, after zero filling the storage. +word array_tokens() { + word q = chk_resize(0, 0, 97); int cap = 32, n = 0, depth = 0; + while (tk != T_EOF) { + if (n == cap) { cap = cap * 2; q = chk_resize(q, n * 3 + 1, cap * 3 + 1); } + q[1 + n * 3] = tk; q[2 + n * 3] = tv; q[3 + n * 3] = tpack(); n++; + if (is_op(s_lb)) depth++; + if (is_op(s_rb)) depth--; + next(); if (!depth) break; + } + if (depth) die("unterminated array initializer"); + q[0] = n; return q; +} +word array_string_tokens(word t) { + int count, i = 0, place = tpack(); char* text = tv; word q; + count = rt.len_w(text); + if (t[3] >= 0 && t[3] == count) count--; // the array has no room for the optional terminating zero + q = chk_resize(0, 0, (count + 3) * 6 + 1); q[0] = 1; + q[1] = T_OP; q[2] = s_lb; q[3] = place; + while (i <= count) { + if (i) { q[1 + q[0] * 3] = T_OP; q[2 + q[0] * 3] = s_comma; q[3 + q[0] * 3] = place; q[0]++; } + q[1 + q[0] * 3] = T_NUM; q[2 + q[0] * 3] = text[i]; + q[3 + q[0] * 3] = place; q[0]++; i++; + } + q[1 + q[0] * 3] = T_OP; q[2 + q[0] * 3] = s_rb; q[3 + q[0] * 3] = place; q[0]++; + next(); return q; +} +word array_initializer_tokens(word t) { + if (tk == T_STR || tk == T_BSTR) return array_string_tokens(t); + return array_tokens(); +} +int array_skip_value() { + int depth = 0; + while (tk != T_EOF) { + if (!depth && (is_op(s_comma) || is_op(s_rb))) break; + if (is_op(s_lp) || is_op(s_lk) || is_op(s_lb)) depth++; + if (is_op(s_rp) || is_op(s_rk) || is_op(s_rb)) depth--; + next(); + } + return 0; +} +int ar_flat; word ar_target; +int array_scalar_count(word t) { + if (t[0] != TY_ARR) return 1; + if (t[3] < 0) return 0; + return t[3] * array_scalar_count(t[4]); +} +// a chain [row][column] selects a subarray or scalar; flat positions resume after that subobject. +int array_designator(word t) { + int index, stride; word part = t; + ar_flat = 0; + while (accept(s_lk)) { + index = cexpr(); expect(s_rk); + if (index < 0) { serr("a negative array designator"); index = 0; } + if (part[0] != TY_ARR) { serr("too many indices in array designator"); break; } + if (part[3] >= 0 && index >= part[3]) serr("array designator index is out of range"); + stride = array_scalar_count(part[4]); if (!stride) { serr("array designator needs a complete element type"); stride = 1; } + ar_flat = ar_flat + index * stride; part = part[4]; + } + expect(s_eq); ar_target = part; return 0; +} +word array_shape(word t) { + int flat = 0, count = 0, width, stride, end; word elem = t[4], row, target; + expect(s_lb); + while (tk != T_EOF && !is_op(s_rb)) { + stride = array_scalar_count(elem); if (!stride) stride = 1; + target = elem; + if (is_op(s_lk)) { array_designator(t); flat = ar_flat; target = ar_target; } + if (is_op(s_lb)) { + if (target[0] != TY_ARR) { serr("nested initializer needs an array element type"); array_skip_value(); flat++; } + else { + row = array_shape(target); + if (elem[0] == TY_ARR && elem[3] < 0) { elem = row; stride = array_scalar_count(elem); } + width = array_scalar_count(row); flat = flat + width; + } + } else { array_skip_value(); flat++; } + end = (flat + stride - 1) / stride; if (end > count) count = end; + if (t[3] >= 0 && count > t[3]) serr("too many elements in array initializer"); + if (!accept(s_comma)) break; + } + expect(s_rb); + if (t[3] >= 0) count = t[3]; + return tarr(elem, count); +} +word array_measure(word q, word t) { + word result; + src_push(1, q, 0, 0); next(); result = array_shape(t); src_pop(); + if (treadonly(t)) result = tconst(result); + return result; +} +int array_address(int base, int local, int offset) { + if (local) laddr(base + offset); else lit(base + offset); + return 0; +} +// flatten a scalar position while preserving each packed row's independent word alignment. +int array_store_scalar(word t, int flat, int base, int local, int offset) { + int c, at, stride, bytes, value, address, shift, mask; word elem = t; + while (elem[0] == TY_ARR) { + stride = array_scalar_count(elem[4]); if (!stride) stride = 1; + at = flat / stride; flat = flat % stride; + if (elem[4][0] == TY_ARR) offset = offset + at * tsz(elem[4]); + else { flat = at; elem = elem[4]; break; } + elem = elem[4]; + } + if (isagg(elem)) { serr("aggregate array elements need nested initializer lists"); array_skip_value(); return 0; } + if (local) { + c = expr(); rval(); plainuse(c); fcoerce(elem); + if (!compat(elem, xty)) tcerr("array initializer", elem, xty); + if (tbytes(elem)) { array_address(base, 1, offset); lit(8); op(15); lit(flat * tbytes(elem)); op(O_ADD); pkst(elem); } + else { array_address(base, 1, offset + flat * tsz(elem)); op(O_ST); } + } else { + if (isfl(elem) && tk == T_FLT) { value = tv; next(); if (elem == tf32) value = rt.zf2s(value); } + else { value = cnorm(elem, cexpr()); if (isfl(elem)) { value = vf_i2f(value); if (elem == tf32) value = rt.zf2s(value); } } + bytes = tbytes(elem); + if (bytes) { + address = base + offset + (flat * bytes >> 3) + hdelta; shift = ((flat * bytes) & 7) * 8; + mask = ((1 << (bytes * 8)) - 1) << shift; *address = (*address & ~mask) | ((value << shift) & mask); + } else *(base + offset + flat * tsz(elem) + hdelta) = value; + } + return 0; +} +// globals use constant expressions, locals use ordinary expressions. brace elision walks scalar elements. +int array_suboffset(word t, int flat, word target) { + int offset = 0, stride, at; + while (t != target && t[0] == TY_ARR) { + stride = array_scalar_count(t[4]); if (!stride) stride = 1; + at = flat / stride; flat = flat % stride; offset = offset + at * tsz(t[4]); t = t[4]; + } + return offset; +} +int array_values(word t, int base, int local, int offset) { + int flat = 0, total = array_scalar_count(t), stride = array_scalar_count(t[4]), dest, width; + word target; + expect(s_lb); + while (tk != T_EOF && !is_op(s_rb)) { + target = t[4]; + if (is_op(s_lk)) { array_designator(t); flat = ar_flat; target = ar_target; } + if (flat < 0 || flat >= total) { serr("array initializer index is out of range"); array_skip_value(); } + else if (target[0] == TY_ARR && is_op(s_lb)) { + // the selected subarray starts at its scalar position, converted to an in-place word offset. + dest = array_suboffset(t, flat, target); + array_values(target, base, local, offset + dest); flat = flat + array_scalar_count(target); + } else { array_store_scalar(t, flat, base, local, offset); flat++; } + if (!accept(s_comma)) break; + } + expect(s_rb); return 0; +} +int array_initialize(word q, word t, int base, int local) { + if (local && tsz(t)) { array_address(base, 1, 0); lit(0); lit(tsz(t)); callname(intern("fill_w")); op(O_DROP); } + src_push(1, q, 0, 0); next(); array_values(t, base, local, 0); src_pop(); + return 0; +} +int array_literal_tokens(word q, word t) { + if (!infn) { + int base, cells; t = array_measure(q, t); cells = tsz(t); reserve_code(cells); base = here; + while (cells) { emit(0); cells--; } array_initialize(q, t, base, 0); rt.free(q); + if_value = 0; + xty = t; lv = 0; lvk = LV_NONE; xbor = -1; xaf = 0; xconst = 1; xcval = base; + return C_PLAIN; + } + int i = newloc(intern("<array literal>")), place = tpack(); + t = array_measure(q, t); ltype[i] = t; larr[i] = 1; fslots = fslots + tsz(t); + array_initialize(q, t, lslot[i], 1); + lafact[i] = af_new(t[3], 4, place); laaddr[i] = lafact[i]; if (lafact[i]) lafact[i][5] = i; + laddr(lslot[i]); xty = t; lv = 0; lvk = LV_NONE; xbor = i; xaf = lafact[i]; + xzero = 0; xfloat = 0; xstr = 0; xconst = 0; xvk = 0; rt.free(q); + return C_PLAIN; +} +// a typed growable initializer owns a heap handle, filled from temporary in-place values. +int array_dynamic_literal(word t) { + word q = array_tokens(), shape = array_measure(q, tarr(t[4], -1)); + int slot = fslots, cells = tsz(shape), place = tpack(); fslots = fslots + cells; + if (!infn) serr("a growable array initializer requires function scope"); + array_initialize(q, shape, slot, 1); rt.free(q); + if (tbytes(t[4])) lit(0 - tbytes(t[4])); else lit(tsz(t[4])); + lit(shape[3]); lit(t); callname(s_dnew); vf_effect(); + lit(0); op(O_PICK); lit(2); op(O_ADD); op(O_LD); laddr(slot); lit(cells); callname(intern("copy_w")); op(O_DROP); + xty = t; lv = 0; lvk = LV_NONE; xbor = -1; xaf = af_new(shape[3], 3, place); + xzero = 0; xfloat = 0; xstr = 0; xconst = 0; xvk = 0; xstamp = 1; newsz = -1; newleaf = 0; return C_FRESH; } -// ---- the expression grammar ---- +int array_literal(word t) { word q = array_tokens(); return array_literal_tokens(q, t); } +// statement-bearing braces keep block-expression semantics; expression-only braces are array literals. +int brace_primary() { + word q = array_tokens(); int k = 0, depth = 0, block = 0, c; + while (k < q[0]) { + if (q[1 + k * 3] == T_OP) { + if (q[2 + k * 3] == s_lb) depth++; + if (q[2 + k * 3] == s_rb) depth--; + if (q[2 + k * 3] == s_semi && depth == 1) block = 1; + } + k++; + } + if (!block) return array_literal_tokens(q, tarr(tint, -1)); + src_push(1, q, 0, 0); next(); c = blockexpr(); src_pop(); rt.free(q); return c; +} + +//: ---- the expression grammar ---- + // precedence, loosest to tightest: // expr (assignment, ?:) -> bin (binary operators by precedence) -> // unary (prefix operators, casts, recasts) -> postfix (index, field, call, @@ -5074,25 +6549,27 @@ int closure() { // the analyzer tracks returns C_VAR + its index so the caller can tell which // variable the value is. int primary() { - int n, i, s, c, k; [8]word bs0; int ft, mn, ms; + effect_value = 0; xprojection = 0; if_value = 0; + int n, i, s, c, k; word[8] bs0; int ft, mn, ms; lv = 0; lvk = LV_NONE; xzero = 0; xbor = -1; - xconst = 0; xfloat = 0; xstr = 0; xlenloc = -1; xstamp = 0; xaf = 0; - if (tk == T_FLT) { op(OP_FLIT); emit(tv); xfloat = 1; xty = tfloat; next(); return C_PLAIN; } + xconst = 0; xfloat = 0; xstr = 0; xlenloc = -1; xstamp = 0; xaf = 0; xvk = 0; + if (tk == T_FLT) { op(OP_FLIT); emit(tv); xvk = 2; xvv = tv; xfloat = 1; xty = tfloat; next(); return C_PLAIN; } if (tk == T_NUM) { if (cmode && tv >= 100 && tglue != 3 && !nomagic && (tv & (tv - 1)) && (tv & (tv + 1))) { magic(tv); } - lit(tv); xzero = tv == 0; xconst = 1; xcval = tv; xty = tint; next(); return C_PLAIN; + lit(tv); xvk = 1; xvv = tv; xzero = tv == 0; xconst = 1; xcval = tv; xty = tint; next(); return C_PLAIN; } - if (tk == T_BSTR) { bstrlit(tv); xstr = 1; xty = tword; next(); return C_PLAIN; } - if (tk == T_STR) { strlit(tv); next(); xty = tptr(tchar); return C_PLAIN; } + if (tk == T_BSTR) { xvk = 4; xvv = vf_string(tv); bstrlit(tv); xstr = 1; xty = tbstr; next(); return C_PLAIN; } + if (tk == T_STR) { xvk = 3; xvv = vf_string(tv); strlit(tv); next(); xty = tptr(tchar); return C_PLAIN; } if (tk == T_OP) { + if (rt.eq_w(tv, "$")) { next(); return quote_tokens(); } if (tv == s_lp) { next(); c = expr(); expect(s_rp); return c; } - if (tv == s_lb) return blockexpr(); + if (tv == s_lb) return brace_primary(); diefound("expected an expression"); lit(0); xty = tword; return C_PLAIN; } if (tk != T_ID) { diefound("expected an expression"); lit(0); xty = tword; return C_PLAIN; } - if (tv == s_fn) { peek(); if (pk_tk == T_OP && pk_tv == s_lp) return closure(); } + if (tv == s_fn) { peek(); if (pk_tk == T_OP && (pk_tv == s_lp || pk_tv == s_colon)) return closure(); } n = tv; callpos = tpack(); i = findloc(n); if (i < 0 && nclo) { @@ -5101,7 +6578,7 @@ int primary() { rs_note(RS_LOCAL, 0 - 1); next(); i = nclo - 1; - vaddr(clo_env[i]); op(O_LD); lit(k); op(O_ADD); + vaddr(clo_env[i]); op(O_LD); lit(k); op(O_ADD); op(O_LD); i = clo_loc[i * capture_width + k]; xty = ltype[i]; xfloat = getflt(i); if (isfl(xty)) xfloat = 1; xstr = getstr(i); lv = 1; lvk = LV_OTHER; @@ -5109,9 +6586,12 @@ int primary() { } } if (i >= 0) { - luse(i); - rs_note(lrskind[i], lrsoff[i]); + if (if_local) if_value = if_local[i]; + lreadpos[i] = tpack(); + luse(i); effect_note_mask(effect_mask(C_VAR + i), EF_READ); + rs_note(lrskind[i], lrsoff[i]); os_at(toff, i); xaf = lafact[i]; af_record(toff, xaf); + if (cmode) { xvk = lvkind[i]; xvv = lvvalue[i]; vf_at(rsn - 1, xvk, xvv); } next(); if (!fchk && tracked(i)) { serrn(n, "is tracked by the analyzer and can't be used in unchecked code"); @@ -5121,13 +6601,14 @@ int primary() { if (larr[i]) return C_PLAIN; xfloat = getflt(i); if (isfl(ltype[i])) xfloat = 1; - xstr = getstr(i); + xstr = getstr(i); if (xty == tbstr) xstr = 1; lv = 1; lvk = LV_LOCAL; lvi = i; - if (isalias(i)) { touch(i); xbor = lroot(i); return C_PLAIN; } - if (tracked(i)) return C_VAR + i; + if (isalias(i)) { touch(i); xbor = lroot(i); return C_VAR + i; } + if (tracked(i) || lrskind[i] == RS_PARAM || (local_origin_mask && local_origin_mask[i])) return C_VAR + i; return C_PLAIN; } - s = lookup(n); + s = source_lookup(n); + if (s < 0 && rt.eq_w(n, "new")) s = source_lookup(s_new); if (s >= 0) rs_put(toff, RS_SYM, rs_sym(s), s); if (s >= 0 && skind[s] == K_KW) { if (!(sflag[s] & F_EXPR)) dien(n, "can't be used inside an expression"); @@ -5136,10 +6617,14 @@ int primary() { if (s >= 0 && skind[s] == K_TYPE) { next(); if (is_op(s_dot)) { // Struct.name(args): a static function of the struct + c = toff + toklen; next(); - mn = name(); + if (probe_off >= 0 && cmode && !dry && !ginst && incdepth == 0 && probe_off >= c && ((tk == T_ID && probe_off <= toff + toklen) || (tk != T_ID && probe_off <= toff))) { probe_static = n; probe_has = 3; chkabort = 1; } + k = toff; mn = name(); ms = lookup(methname(n, mn)); + if (ms >= 0) rs_put(k, RS_SYM, rs_sym(ms), ms); if (ms >= 0 && skind[ms] == K_CONST) { // Enum.MEMBER + xvk = 1; xvv = sval[ms]; lit(sval[ms]); xty = tint; if (stype[ms]) xty = stype[ms]; @@ -5165,6 +6650,7 @@ int primary() { next(); xty = tword; if (s >= 0 && skind[s] == K_CONST) { + xvk = 1; xvv = sval[s]; lit(sval[s]); xty = tint; if (stype[s]) xty = stype[s]; @@ -5174,11 +6660,12 @@ int primary() { return C_PLAIN; } if (s >= 0 && skind[s] == K_GVAR) { + if (cmode) { vf_symbols(); xvk = svkind[s]; xvv = svvalue[s]; vf_at(rsn - 1, xvk, xvv); } lit(sval[s]); if (stype[s]) xty = stype[s]; - if (isfl(xty)) xfloat = 1; + if (isfl(xty)) xfloat = 1; if (xty == tbstr) xstr = 1; lv = 1; - lvk = LV_GLOBAL; + lvk = LV_GLOBAL; lvi = s; return C_PLAIN; } if (s >= 0 && skind[s] == K_GARR) { lit(sval[s]); if (stype[s]) xty = stype[s]; return C_PLAIN; } @@ -5194,7 +6681,7 @@ int primary() { } closeangle(); if (!is_op(s_lp)) { // f<int> alone is that instance, as a function value - fill_w(bs0, 0, 8); copy_w(bs0, gbind, ngbind); ngbind = 0; + rt.fill_w(bs0, 0, 8); rt.copy_w(bs0, gbind, ngbind); ngbind = 0; ft = stype[s]; s = ginstance(s, bs0); fnref(s); @@ -5212,7 +6699,9 @@ int primary() { // is the result; dlt is +1 or -1. it also updates the loop-bound facts: a // decrement or a store of an unknown value ends any proof about the variable. int incdec(int post, int dlt) { + require_mutable(xty); int t; + if (cmode && !dry) { if (lvk == LV_LOCAL) lvkind[lvi] = 0; else vf_effect(); } xvk = 0; if (lv && lvk == LV_LOCAL) lafact[lvi] = 0; xaf = 0; if (!lv) die("++ and -- need something to assign to"); if (lvk == LV_LOCAL && fchk && tracked(lvi) && !isalias(lvi)) { @@ -5252,21 +6741,53 @@ int incdec(int post, int dlt) { // name is followed by ( it is a method call instead. a pointer-valued field // of a tracked variable is a borrow of that variable (xbor). word field(int c) { + int readonly = treadonly(xty); + if_value = 0; int n, fieldoff; word st; - af_access(xaf, tpack()); xaf = 0; int i, b = -1, k = 0, scanned = 0, ms; + word borrowed = xaf, base_mask = effect_mask(c); af_use(xaf, tpack(), c); xaf = 0; xvk = 0; int i, b = xbor, k = 0, scanned = 0, ms, de, pmatch = 0; + de = toff + toklen; next(); + // is the place asked about on this member name (or just after the dot, with none typed) + if (probe_off >= 0 && cmode && !dry && !ginst && incdepth == 0 && probe_off >= de) { + if (tk == T_ID) { if (probe_off <= toff + toklen) pmatch = 1; } + else if (probe_off <= toff) pmatch = 1; + } n = name(); fieldoff = nameoff; + st = xty; + if (st[0] == TY_ARR || st[0] == TY_DYN) { + if (pmatch) probe_member(st); + if (is_op(s_lp) && array_method(n)) { xaf = borrowed; return array_member(c, n); } + if (is_op(s_lp)) { + ms = receiver_method(st, n); + if (ms == -2) serrn(n, "matches more than one receiver method"); + if (ms >= 0) { + rval(); plainuse(c); + if (c >= C_VAR) b = lroot(c - C_VAR); + if (st[0] == TY_ARR && stype[ms][7][0] == TY_DYN) { + int slot = fslots; fslots = fslots + 4; + laddr(slot + 2); op(O_ST); + lit(st[3]); laddr(slot); op(O_ST); lit(st[3]); laddr(slot + 1); op(O_ST); + if (tbytes(st[4])) lit(0 - tbytes(st[4])); else lit(tsz(st[4])); + laddr(slot + 3); op(O_ST); laddr(slot); st = tdyn(st[4], 1); + } + return methcall(ms, st, borrowed, b, base_mask); + } + } + } rval(); if (c == C_FRESH) cerr("use a field of a new object only after binding it to a name"); else if (c >= C_VAR) { touch(c - C_VAR); b = c - C_VAR; } st = xty; if (st[0] == TY_PTR) st = st[4]; + if (pmatch) probe_member(st); if (is_op(s_lp)) { // e.name(...): a method, unless the struct has a field of that name ms = 0 - 1; if (st[0] == TY_STRUCT) { if (tfindfield(st, n) < 0) ms = methof(st, n); } else if (st == tword || isnum(st)) { if (fieldscan(n) == 0) ms = methscan(n); } if (ms == 0 - 2) { serrn(n, "is a method of more than one struct"); ms = 0 - 1; } - if (ms >= 0) return methcall(ms, st); + if (ms >= 0) { + return methcall(ms, st, borrowed, b, base_mask); + } } if (st[0] != TY_STRUCT && (st == tword || isnum(st))) { k = fieldscan(n); @@ -5274,9 +6795,9 @@ word field(int c) { else if (k > 1) { if (!panic && !dry) { errat(); - put_c('.'); - put_s(n); - put_s(" is a field of more than one struct"); + rt.put_c('.'); + rt.put_s(n); + rt.put_s(" is a field of more than one struct"); errend(); } xty = tword; lv = 1; lvk = LV_OTHER; xbor = -1; return C_PLAIN; @@ -5285,31 +6806,35 @@ word field(int c) { if (st[0] != TY_STRUCT) { if (!panic && !dry) { errat(); - put_c('.'); - put_s(n); - put_s(": that's a "); + rt.put_c('.'); + rt.put_s(n); + rt.put_s(": that's a "); tprint(xty); - put_s(", not a struct or a pointer to one"); + rt.put_s(", not a struct or a pointer to one"); errend(); } xty = tword; lv = 1; lvk = LV_OTHER; xbor = -1; return C_PLAIN; } if (scanned) i = fldi; else i = tfindfield(st, n); if (i < 0) { - if (!panic && !dry) { errat(); tprint(st); put_s(" has no field "); put_s(n); errend(); } + if (!panic && !dry) { errat(); tprint(st); rt.put_s(" has no field "); rt.put_s(n); errend(); } xty = tword; lv = 1; lvk = LV_OTHER; xbor = -1; return C_PLAIN; } rs_put(fieldoff, RS_FIELD, field_decl(st, n), st[4][i * 3 + 1]); k = st[4][i * 3 + 2]; xty = st[4][i * 3 + 1]; + if (readonly) xty = tconst(xty); if (k < 0) { // a packed field: the lvalue is a byte address, of the packed twin lit(8); op(15); k = 0 - k - 1; if (k) { lit(k); op(O_ADD); } xty = tpk(xty); } else if (k) { lit(k); op(O_ADD); } - if (xty[0] == TY_PTR) { if (b >= 0) xbor = lroot(b); } + if (isref(xty)) { if (b >= 0) xbor = lroot(b); } else xbor = -1; + if (borrowed && borrowed[2] == 5) xaf = borrowed; + effect_value = base_mask; + xprojection = af_project(borrowed, k, tsz(xty), tpack(), 0); lv = 1; xzero = 0; if (b >= 0) { lvk = LV_ELEM; lvi = b; } else lvk = LV_OTHER; return C_PLAIN; @@ -5371,19 +6896,23 @@ int range_jz(int ix, int src, int fld) { // compared with the length). a plain pointer or fixed array is just base + // index * size, and a constant index against a known length is checked at // compile time. -int postfix() { - int c = primary(), b, n, t, et; word ft; [8]word bs; int k, ci, was, al, ix, enh, cz, cj, place; word storage; +int postfix() { return postfix_tail(primary()); } +int postfix_tail(int c) { + int b, n, t, et; word ft; word[8] bs; int k, ci, was, al, ix, enh, cz, cj, place, offset, temporary, call_owner, readonly; word storage, storage_mask, call_origins; while (1) { if (tk == T_ID && tv == s_as) { // e as T: same bits, another type - next(); t = ptype(); rval(); xaf = 0; + next(); t = ptype(); rval(); if (!xaf || xaf[2] != 5) xaf = 0; vf_coerce(t, 0); if (tbytes(t) && !xfloat && !samesz(t, xty)) { tnorm(t); xconst = 0; } // x as u8 cuts the value to size + if_value = 0; xty = t; xfloat = 0; lvk = LV_NONE; if (isfl(t)) xfloat = 1; continue; } if (tk != T_OP) return c; if (tv == s_lk) { - storage = xaf; place = tpack(); af_access(storage, place); + readonly = treadonly(xty); + xvk = 0; + storage = xaf; storage_mask = effect_mask(c); offset = -1; n = -1; place = tpack(); af_use(storage, place, c); was = xstr; ci = 0; if (lv && lvk == LV_LOCAL) lptr[lvi] = 1; @@ -5395,13 +6924,13 @@ int postfix() { if (accept(s_colon)) { c = expr(); rval(); plainuse(c); expect(s_rk); - callname(s_slice_s); - lv = 0; lvk = LV_NONE; xty = tword; xfloat = 0; xstr = 1; xzero = 0; xbor = -1; + callname(s_slice_s); xvk = 0; + lv = 0; lvk = LV_NONE; xty = tbstr; xfloat = 0; xstr = 1; xzero = 0; xbor = -1; c = C_PLAIN; continue; } expect(s_rk); - callname(s_at_s); + callname(s_at_s); xvk = 0; lv = 0; lvk = LV_NONE; xty = tint; xfloat = 0; xstr = 0; xzero = 0; xbor = -1; c = C_PLAIN; continue; @@ -5412,7 +6941,7 @@ int postfix() { if (xty[0] == TY_DYN && tbytes(xty[4])) { // packed elements: the runtime checks and gives the byte address et = xty[4]; c = expr(); rval(); plainuse(c); - af_index(storage, place); + if (xconst) offset = xcval; af_index(storage, place); tcneed(isnum(xty) || xty == tword, "an index has to be a number", xty); expect(s_rk); callname(intern("datb")); @@ -5426,7 +6955,7 @@ int postfix() { ix = -1; if (lv && lvk == LV_LOCAL) ix = lvi; rval(); plainuse(c); - af_index(storage, place); + if (xconst) offset = xcval; af_index(storage, place); tcneed(isnum(xty) || xty == tword, "an index has to be a number", xty); expect(s_rk); k = lookup(s_outside); @@ -5472,7 +7001,7 @@ int postfix() { n = 0 - 1; if (xty[0] == TY_ARR && xty[3] > 0) n = xty[3]; xconst = 0; c = expr(); rval(); plainuse(c); - if (n < 0) af_index(storage, place); + if (xconst) offset = xcval; if (n < 0) af_index(storage, place); if (n >= 0 && xconst && (xcval < 0 || xcval >= n)) { serr("the index is out of range for this array"); } @@ -5487,9 +7016,14 @@ int postfix() { op(O_ADD); } } - xaf = 0; - lv = 1; xty = et; xzero = 0; xconst = 0; xlenloc = -1; xstamp = 0; - if (xty[0] == TY_PTR) { if (b >= 0) xbor = lroot(b); } + xprojection = af_project(storage, offset, 1, place, 1); + if (xprojection && n >= 0 && offset >= 0) { xprojection[0] = n - offset; xprojection[13] = 0 - offset; xprojection[14] = 1; } + xaf = 0; if (storage && storage[2] == 5) xaf = storage; + effect_value = 0; if (isptr(et) || et == tword) effect_value = storage_mask; + if_value = 0; + if (readonly) et = tconst(et); + lv = 1; xty = et; xzero = 0; xconst = 0; xvk = 0; xlenloc = -1; xstamp = 0; + if (isref(xty)) { if (b >= 0) xbor = lroot(b); } else xbor = -1; if (b >= 0) { lvk = LV_ELEM; lvi = b; } else lvk = LV_OTHER; c = C_PLAIN; @@ -5501,16 +7035,19 @@ int postfix() { c = field(c); xfloat = 0; if (isfl(xty)) xfloat = 1; - xstr = 0; + xstr = xty == tbstr; continue; } if (tv == s_lp) { // call through a value + effect_note_mask(effect_mask(c), EF_READ); rval(); plainuse(c); ft = 0; if (xty[0] == TY_FN) ft = xty; else tcneed(xty == tword, "only functions can be called", xty); - fill_w(bs, 0, 8); - n = args(0, ft, bs, o3on); + rt.fill_w(bs, 0, 8); + ifcall = -1; effect_argument_base = 0; + int call_tags = 0; if (ft) call_tags = ft[5] >> 6; + n = args(call_tags, ft, bs, o3on); temporary = argtemporary; call_owner = argowner; call_origins = argorigin_mask; // raw code can live in a heap segment; a closure has an environment block lit(n); op(O_PICK); callname(intern("callable_code")); cz = hole(); op(O_JZ); @@ -5522,9 +7059,12 @@ int postfix() { op(O_CALL); setlit(cj, here); op(O_SWAP); op(O_DROP); + if_value = 0; lv = 0; lvk = LV_NONE; c = C_PLAIN; xzero = 0; xbor = -1; xconst = 0; xty = tword; if (ft) xty = subst(ft[4], bs); + if (ft && (ft[5] & 1)) c = C_FRESH; + else if (isptr(xty) || xty == tword) { effect_value = call_origins; if (isptr(xty)) xbor = call_owner; if (temporary) xaf = af_new(-1, 5, place); } xfloat = 0; if (isfl(xty)) xfloat = 1; xstr = 0; continue; } @@ -5538,7 +7078,7 @@ int postfix() { // tag), casts like (int)x and (node*)x, dereference, address-of, prefix ++ and // --, and any unary operator a program added (- ! ~ built in). int unary() { - int c, un, t, place, constant, value; word storage; + int c, un, t, place, constant, value; word storage, dependency; if (tk == T_ID && (tv == s_u || tv == s_m || tv == s_d)) { // u*x, d*x, m*x: recast only when * is glued peek(); if (pk_tk == T_OP && pk_tv == s_star && pq_gl[0]) { @@ -5553,22 +7093,26 @@ int unary() { next(); t = ptype(); expect(s_rp); - c = unary(); rval(); plainuse(c); xaf = 0; + if (t[0] == TY_ARR && is_op(s_lb)) return postfix_tail(array_literal(t)); + if (t[0] == TY_DYN && is_op(s_lb)) return postfix_tail(array_dynamic_literal(t)); + c = unary(); dependency = effect_mask(c); rval(); plainuse(c); effect_value = dependency; if (!xaf || xaf[2] != 5) xaf = 0; vf_coerce(t, 1); if (isfl(t)) { if (!xfloat) { op(OP_I2F); xfloat = 1; } if (t == tf32) { xenh("zf2s"); xenh("zs2f"); } } else if (xfloat && isnum(t)) { op(OP_F2I); xfloat = 0; } else xfloat = 0; if (tbytes(t) && !samesz(t, xty)) { tnorm(t); xconst = 0; } // (u8)x cuts the value to size + if_value = 0; xty = t; lv = 0; lvk = LV_NONE; xzero = 0; xbor = -1; return C_PLAIN; } if (tk == T_OP) { if (tv == s_star) { - place = tpack(); next(); c = unary(); - af_access(xaf, place); xaf = 0; + place = tpack(); next(); c = unary(); dependency = effect_mask(c); + storage = xaf; af_use(xaf, place, c); xaf = 0; if (storage && storage[2] == 5) xaf = storage; xvk = 0; if (lv && lvk == LV_LOCAL) lptr[lvi] = 1; rval(); plainuse(c); tcneed(xty[0] == TY_PTR || xty == tword, "only pointers can be dereferenced", xty); - xty = telem(xty); + if_value = 0; + xty = telem(xty); effect_value = 0; if (isptr(xty) || xty == tword) effect_value = dependency; if (tbytes(xty)) { lit(8); op(15); xty = tpk(xty); } // the first packed element: a byte address lv = 1; lvk = LV_OTHER; @@ -5581,10 +7125,11 @@ int unary() { } if (tv == s_amp) { place = tpack(); next(); c = unary(); - storage = 0; + storage = xprojection; if (xaf && xaf[2] == 5) storage = xaf; if (lv && lvk == LV_LOCAL) { + if (cmode && !dry) { lvescaped[lvi] = 1; lvkind[lvi] = 0; } storage = laaddr[lvi]; - if (!storage) { storage = af_new(1, 1, place); laaddr[lvi] = storage; if (storage) storage[5] = lvi; } + if (!storage) { int storage_kind = 1; if (local_cell[lvi]) storage_kind = 2; storage = af_new(1, storage_kind, place); laaddr[lvi] = storage; if (storage) storage[5] = lvi; } af_forget(); } else if (isagg(xty)) storage = 0; @@ -5593,8 +7138,10 @@ int unary() { if (lv && lvk == LV_LOCAL && (ltag[lvi] & 16)) serr("an o3 variable has no address"); if (lv && lvk == LV_LOCAL) ltag[lvi] = ltag[lvi] | LREAD | LASG; if (c >= C_VAR) touch(c - C_VAR); - lv = 0; lvk = LV_NONE; xzero = 0; xbor = -1; xfloat = 0; xstr = 0; - xty = tptr(xty); xaf = storage; + if (lvk == LV_ELEM && lvi >= 0) xbor = lroot(lvi); + lv = 0; lvk = LV_NONE; xzero = 0; xfloat = 0; xstr = 0; + if_value = 0; + xty = tptr(xty); xaf = storage; xvk = 0; return C_PLAIN; } if (tv == s_inc) { next(); unary(); return incdec(0, 1); } @@ -5603,6 +7150,9 @@ int unary() { if (un >= 0) { next(); c = unary(); rval(); plainuse(c); constant = xconst && !xfloat && uname[un] == s_minus; value = xcval; xaf = 0; + if (uname[un] == s_minus && xvk == 1) xvv = 0 - xvv; + else if (uname[un] == s_minus && xvk == 2) xvv = vf_negf(xvv); + else xvk = 0; if (un <= 1) ftest(); if (uname[un] == s_minus && xfloat) { op(OP_FLIT); @@ -5616,6 +7166,7 @@ int unary() { if (un <= 1) xty = tbool; else if (xty != tword && !isfl(xty)) { tcneed(isnum(xty), "this operator needs a number", xty); xty = tint; } } + if (un <= 1) if_value = 0; xstr = 0; xconst = constant; if (constant) xcval = 0 - value; xlenloc = -1; xstamp = 0; lv = 0; lvk = LV_NONE; xzero = 0; xbor = -1; @@ -5629,8 +7180,11 @@ int unary() { // level tighter and emit its code sequence from the operator table. && and || // short-circuit with jumps; floats get their own opcodes; a * or / by a // constant that is a power of two gets a shift hint in check mode. +int if_comparison(int name) { + return name == s_eqeq || name == s_ne || name == s_lt || name == s_gt || name == s_le || name == s_ge; +} int bin(int minp) { - int c = unary(), b, p, a, j, lt, lb, rb, src, lf, rf, ls, rs, fresh, top = 0, il = -1, opp; + int c = unary(), b, p, a, j, lt, lb, rb, src, lf, rf, ls, rs, fresh, top = 0, il = -1, opp; word borrowed, rightborrow, left_mask, right_mask, infer_left, infer_right; if (fortest == 1) { fortest = 2; top = 1; ft_nops = 0; } while (tk == T_OP) { b = binfind(tv); @@ -5639,9 +7193,10 @@ int bin(int minp) { if (p < minp) return c; lb = -1; if (c >= C_VAR && tracked(c - C_VAR)) lb = lroot(c - C_VAR); else lb = xbor; - lf = xfloat; ls = xstr; fresh = 0; + borrowed = xaf; left_mask = effect_mask(c); lf = xfloat; ls = xstr; fresh = 0; il = -1; if (top && lv && lvk == LV_LOCAL) il = lvi; + infer_left = if_value; infer_right = 0; opp = tpack(); next(); rval(); plainuse(c); lt = xty; @@ -5669,7 +7224,7 @@ int bin(int minp) { setlit(j, here); xfloat = 0; xstr = 0; } else { - c = bin(p + 1); + c = bin(p + 1); infer_right = if_value; rightborrow = xaf; right_mask = effect_mask(c); rb = -1; if (c >= C_VAR && tracked(c - C_VAR)) rb = lroot(c - C_VAR); else rb = xbor; if (top) { ft_ir = -1; if (lv && lvk == LV_LOCAL) ft_ir = lvi; } @@ -5691,12 +7246,19 @@ int bin(int minp) { } else if ((lf || rf) && isfop(bname[b])) { femit(bname[b], lf, rf); xstr = 0; } else { emitseq(bseq[b], blen[b]); xfloat = 0; xstr = 0; } if (bname[b] == s_plus || bname[b] == s_minus) { + if ((!borrowed || borrowed[2] != 5) && rightborrow && rightborrow[2] == 5) borrowed = rightborrow; if (lb >= 0) src = lb; else src = rb; if (bname[b] == s_minus && lb >= 0 && rb >= 0) src = -1; // pointer difference is a distance } } if (blen[b] < 0) af_forget(); - xty = bintype(b, lt, xty); + if (xstr) xty = tbstr; else xty = bintype(b, lt, xty); + if_value = 0; + if (blen[b] > 0 && !if_comparison(bname[b]) && !xstr && !lf && !rf && (xty == tint || xty == tword) && (infer_left || infer_right)) { + if_link(infer_left, infer_right); + if_value = infer_left; if (!if_value) if_value = infer_right; + if_join(if_value, xty); + } if (src >= 0 && xty[0] != TY_PTR && xty != tword) src = -1; xbor = src; if (top) { @@ -5704,12 +7266,15 @@ int bin(int minp) { if (ft_nops > 1) ft_iv = -1; else if (bname[b] == s_lt && blen[b] > 0 && il >= 0 && ft_ir >= 0 && !lf && !rf && !ls && !rs) { ft_iv = il; ft_bd = ft_ir; } } - lv = 0; lvk = LV_NONE; c = C_PLAIN; xzero = 0; xconst = 0; xlenloc = -1; xstamp = 0; xaf = 0; + lv = 0; lvk = LV_NONE; c = C_PLAIN; xzero = 0; xconst = 0; xlenloc = -1; xstamp = 0; xaf = 0; xvk = 0; + effect_value = 0; if (isptr(xty) || xty == tword) effect_value = left_mask | right_mask; if (fresh) c = C_FRESH; + if (borrowed && borrowed[2] == 5 && (isptr(xty) || xty == tword) && (bname[b] == s_plus || bname[b] == s_minus)) xaf = borrowed; } return c; } -// ---- assignment, conditionals and blocks as expressions ---- +//: ---- assignment, conditionals and blocks as expressions ---- + int storekeep(int t) { if (mdbg && !dry && !ispk(t)) { lit(1); op(O_PICK); callname(s_check); op(O_DROP); } op(O_SWAP); lit(1); op(O_PICK); op(O_SWAP); @@ -5720,20 +7285,20 @@ int storekeep(int t) { // restored for the second side) and then joined, so a variable can't be given // away on one side only. both sides are fresh objects, or neither is. int ternary(int c) { - int a, b, s0, sa, livea, live0 = live, n0 = nloc, ca, cb, ta, ba, bb; + int a, b, s0, sa, livea, live0 = live, n0 = nloc, ca, cb, ta, ba, bb; word temp_a, temp_b, mask_a, mask_b; rval(); plainuse(c); ftest(); next(); a = hole(); op(O_JZ); s0 = snap_push(); - ca = expr(); rval(); ta = xty; ba = xbor; + ca = expr(); rval(); ta = xty; ba = xbor; temp_a = xaf; mask_a = effect_mask(ca); if (ca != C_FRESH) plainuse(ca); livea = live; sa = snap_push(); b = hole(); op(O_JMP); expect(s_colon); setlit(a, here); snap_restore(s0); live = live0; - cb = expr(); rval(); bb = xbor; + cb = expr(); rval(); bb = xbor; temp_b = xaf; mask_b = effect_mask(cb); if (cb != C_FRESH) plainuse(cb); join2(sa, livea, b, live, n0, "ends the two sides of ?: in different states"); snap_pop(s0); @@ -5741,8 +7306,11 @@ int ternary(int c) { if (!compat(ta, xty) && !compat(xty, ta)) tcerr("the two sides of ?:", ta, xty); if (ta != tword && !isfl(ta)) xty = ta; else if (isfl(ta) || isfl(xty)) { xty = tfloat; xfloat = 1; } + if_value = 0; + effect_value = mask_a | mask_b; if (ba != bb) xbor = -1; - lv = 0; lvk = LV_NONE; xzero = 0; + if (temp_a && temp_a[2] == 5) xaf = temp_a; else if (temp_b && temp_b[2] == 5) xaf = temp_b; + lv = 0; lvk = LV_NONE; xzero = 0; xvk = 0; if (ca == C_FRESH && cb == C_FRESH) return C_FRESH; if (ca == C_FRESH || cb == C_FRESH) cerr("both sides of ?: have to be new objects, or neither"); return C_PLAIN; @@ -5761,7 +7329,7 @@ int expr() { if (tv == s_q) return ternary(c); if (tv == s_eq) { if (!lv) die("this can't be assigned to"); - k = lvk; i = lvi; lt = xty; + k = lvk; i = lvi; lt = xty; require_mutable(lt); if (isagg(lt)) serr("a whole struct or array can't be assigned; assign its fields"); next(); a = o3on && k == LV_LOCAL && (ltag[i] & 16) && !larr[i]; @@ -5773,8 +7341,10 @@ int expr() { ltag[i] = ltag[i] | LASG; setflt(i, xfloat); setstr(i, xstr); - note_store(i); af_store(i); + note_store(i); af_store(i); vf_store(i); } + if (cmode && !dry && k == LV_GLOBAL) { vf_symbols(); svkind[i] = xvk; svvalue[i] = xvv; } + else if (k != LV_LOCAL) vf_effect(); if (!compat(lt, xty)) tcerr("assignment", lt, xty); if (a) { lit(0); op(O_PICK); vaddr(i); op(O_ST); o3refresh(i); } else { @@ -5790,7 +7360,7 @@ int expr() { if (lvk == LV_LOCAL && fchk && tracked(lvi) && !isalias(lvi)) { cerrn(lvi, "owns an object; compound assignment would lose it"); } - lt = xty; + lt = xty; require_mutable(lt); fk = -1; lf = xfloat; if (lvk == LV_LOCAL) fk = lvi; if (isfl(lt)) lf = 1; @@ -5806,7 +7376,7 @@ int expr() { else { emitseq(bseq[abin[a]], blen[abin[a]]); xfloat = 0; } fcoerce(lt); if (fk >= 0) { - lafact[fk] = 0; xaf = 0; + lafact[fk] = 0; xaf = 0; if (cmode && !dry) lvkind[fk] = 0; ltag[fk] = ltag[fk] | LASG; setflt(fk, xfloat); setstr(fk, 0); if (forstep && fk == ft_iv && bname[abin[a]] == s_plus && xconst && xcval == 1) fs_ok = 1; @@ -5814,6 +7384,7 @@ int expr() { else putfact(fk, -1, -1); bpoison(fk); } + if (fk < 0) vf_effect(); xvk = 0; k = bintype(abin[a], lt, xty); if (!compat(lt, k)) tcerr("compound assignment", lt, k); if (i) { lit(0); op(O_PICK); vaddr(i - 1); op(O_ST); } else storekeep(lt); @@ -5848,7 +7419,7 @@ int blockexpr() { tail = 1; t = xty; } next(); - if (tail) c = handup(c); else { lit(0); xbor = -1; xfloat = 0; xstr = 0; } + if (tail) c = handup(c); else { lit(0); xbor = -1; xfloat = 0; xstr = 0; xvk = 1; xvv = 0; } if (xbor >= save) xbor = -1; // that local dies with this block; the pointer leaves untracked block_close(save); fdepth--; bxdepth--; @@ -5856,7 +7427,8 @@ int blockexpr() { return c; } -// ---- constant expressions ---- +//: ---- constant expressions ---- + // constant expressions, evaluated while compiling: numbers, named constants, // addresses of globals and functions, unary minus and ~, parentheses, and the // operators + - * | & << >>. used for array lengths, enum values, const @@ -5904,7 +7476,8 @@ int cexpr() { return v; } -// ---- statements ---- +//: ---- statements ---- + // after a syntax error: skip to the end of the statement // statements. stmt is the dispatcher: a keyword statement (if, while, return, // a parse word...), a declaration, an empty statement, a block, or an @@ -5918,7 +7491,11 @@ int resync() { } // compile one statement. int stmt() { + word before = borrow_slots; int result; emit_statement_hooks(); result = stmt_inner(); borrow_cleanup_since(before); return result; +} +int stmt_inner() { int s, c; + if (probe_off >= 0) probe_stmt(); if (panic) return resync(); if (tk == T_ID && findloc(tv) < 0) { s = lookup(tv); @@ -5940,6 +7517,7 @@ int block() { expect(s_lb); fdepth++; while (!is_op(s_rb)) { if (tk == T_EOF) { die("missing }"); break; } stmt(); } + if (probe_off >= 0 && !chkabort && !dry && !ginst && incdepth == 0 && ctxsp <= nclo && toff >= probe_off) { probe_snap(); chkabort = 1; } block_close(save); // before the }, so its line is the one reported next(); fdepth--; @@ -5988,29 +7566,29 @@ int lt_note(int o, int t) { lt_off = chk_resize(lt_off, lt_n, ltcap); lt_at = chk_resize(lt_at, lt_n, ltcap); lt_len = chk_resize(lt_len, lt_n, ltcap); lt_type = chk_resize(lt_type, lt_n, ltcap); } - sm = outmode; sb = outbuf; sn = outn; sc = outcap; - outmode = 3; outn = 0; tprint(t); need = ltxn + outn; + sm = rt.outmode; sb = rt.outbuf; sn = rt.outn; sc = rt.outcap; + rt.outmode = 3; rt.outn = 0; tprint(t); need = ltxn + rt.outn; if (!ltxt || need > ltxtcap) { while (need > ltxtcap) ltxtcap = ltxtcap * 2; ltxt = chk_resize(ltxt, ltxn, ltxtcap); } - outmode = 2; outbuf = ltxt; outn = ltxn; outcap = ltxtcap; + rt.outmode = 2; rt.outbuf = ltxt; rt.outn = ltxn; rt.outcap = ltxtcap; tprint(t); - sl = outn; - outmode = sm; outbuf = sb; outn = sn; outcap = sc; + sl = rt.outn; + rt.outmode = sm; rt.outbuf = sb; rt.outn = sn; rt.outcap = sc; lt_off[lt_n] = o; lt_at[lt_n] = ltxn; lt_len[lt_n] = sl - ltxn; lt_type[lt_n] = t; lt_n++; ltxn = sl; return 0; } // compile a local declaration, typed or let-style. int localdecl(int typed) { - int n, i, c, k, t = 0, tg = 0, pp, kc, kp = ifkp, gty = 0, gok = !typed && ifon && !dry, o; + int n, i, c, k, t = 0, tg = 0, pp, kc, kp = ifkp, gty = 0, gok = !typed && ifon && !dry, o, vr; word iq, it; if (typed) { dtype(); t = dty; tg = dtag; if (t == tvoid) serr("a variable can't be void"); } if ((o3on || o3one) && !typed) serr("o3 variables need a type"); while (1) { pp = tpack(); o = toff; - rs_note(RS_LOCAL, o); + rs_note(RS_LOCAL, o); vr = rsn - 1; n = name(); i = newloc(n); lrsoff[i] = o; @@ -6023,10 +7601,17 @@ int localdecl(int typed) { if (gok) { if (!gty) gty = tint; else if (gty != tint) gok = 0; } } else if (t && isagg(t)) { larr[i] = 1; ltype[i] = t; fslots = fslots + tsz(t); } else { fslots++; if (t) ltype[i] = t; if (isfl(t)) setflt(i, 1); if (o3on) { o3promote(i); o3mkcache(i); } } + capture_cell(i, 0); if (accept(s_eq)) { - if (larr[i]) serr("an array or a struct can't be initialized with ="); + if (larr[i] && ltype[i][0] == TY_ARR && (is_op(s_lb) || tk == T_STR || tk == T_BSTR)) { + iq = array_initializer_tokens(ltype[i]); it = array_measure(iq, ltype[i]); + fslots = fslots + tsz(it) - tsz(ltype[i]); + ltype[i] = it; array_initialize(iq, it, lslot[i], 1); rt.free(iq); + } else { + if (larr[i]) serr("an array or a struct needs an initializer list"); newsz = -1; newleaf = 0; if (tk == T_NUM) nomagic = 1; - c = expr(); rval(); nomagic = 0; kc = xconst && !xfloat; + if (t && t[0] == TY_DYN && is_op(s_lb)) c = array_dynamic_literal(t); else c = expr(); + rval(); nomagic = 0; kc = xconst && !xfloat; if (kc) ltag[i] = ltag[i] | LCONST; if (c == C_FRESH) { lnsz[i] = newsz; lleaf[i] = newleaf; } if (t) fcoerce(t); @@ -6035,7 +7620,7 @@ int localdecl(int typed) { o3refresh(i); setflt(i, xfloat); setstr(i, xstr); - note_store(i); af_store(i); + note_store(i); af_store(i); vf_store(i); if (t) { if (!compat(t, xty)) tcerr("initializer", t, xty); } else if (!isnum(xty)) { if (xty[0] == TY_ARR) ltype[i] = decay(xty); @@ -6044,9 +7629,12 @@ int localdecl(int typed) { } assigncheck(LV_LOCAL, i, c); if (gok) { k = xty; if (isagg(k)) k = ltype[i]; if (!ifpure(k) || (gty && gty != k)) gok = 0; else gty = k; } + } } else if (gok && !larr[i]) gok = 0; // no initializer: nothing says what it is + if (larr[i] && ltype[i][0] == TY_ARR && ltype[i][3] < 0) serr("T[] needs an initializer to determine its length"); if (larr[i]) { k = 1; if (ltype[i][0] == TY_ARR) k = ltype[i][3]; lafact[i] = af_new(k, 1, pp); laaddr[i] = lafact[i]; if (lafact[i]) lafact[i][5] = i; } - af_declaration(o, lafact[i]); + af_declaration(o, lafact[i]); os_at(o, i); + if (cmode) vf_at(vr, lvkind[i], lvvalue[i]); kc = tword; if (ltype[i]) kc = ltype[i]; lt_note(((pp >> 40) & 0x1FFFFF) - 1, kc); @@ -6073,7 +7661,7 @@ int kw_if() { int a, b, c, s0, sa, live0, livea, n0 = nloc, nt, cj; stmtonly(); next(); - expect(s_lp); c = expr(); rval(); plainuse(c); ftest(); expect(s_rp); + expect(s_lp); c = borrow_condition(); expect(s_rp); a = hole(); op(O_JZ); live0 = live; s0 = snap_push(); @@ -6172,7 +7760,7 @@ int kw_while() { peek(); if (pk_tk == T_OP && pk_tv == s_rp) inf = 1; } - c = expr(); rval(); plainuse(c); ftest(); expect(s_rp); + c = borrow_condition(); expect(s_rp); a = hole(); op(O_JZ); if (!inf && live) sc = snap_push(); loop_push(c0, s0); @@ -6190,7 +7778,7 @@ int kw_while() { // if it holds and the bound is len(a) of a fixed array, the body is told that // a[i] is in range (lpfact), and its bounds check disappears. int kw_for() { - int save = nloc, s0, sc = -1, c0, a, b, st, c, inf = 0; + int save = nloc, s0, sc = -1, c0, a, b, st, c, inf = 0; word borrow_mark = borrow_slots; stmtonly(); next(); expect(s_lp); @@ -6198,12 +7786,13 @@ int kw_for() { if (is_id(s_var)) kw_var(); else if (tstart()) localdecl(1); else { if (!is_op(s_semi)) { c = expr(); rval(); op(O_DROP); discard(c); } expect(s_semi); } + borrow_cleanup_since(borrow_mark); s0 = snap_push(); c0 = here; ft_iv = -1; fs_ok = 0; fortest = 1; if (!is_op(s_semi)) { if (tk == T_NUM && tv) { peek(); if (pk_tk == T_OP && pk_tv == s_semi) inf = 1; } - c = expr(); rval(); plainuse(c); ftest(); + c = borrow_condition(); } else { lit(1); inf = 1; xfloat = 0; } fortest = 0; expect(s_semi); @@ -6211,8 +7800,9 @@ int kw_for() { if (!inf && live) sc = snap_push(); b = hole(); op(O_JMP); st = here; - forstep = 1; + forstep = 1; borrow_mark = borrow_slots; if (!is_op(s_rp)) { c = expr(); rval(); op(O_DROP); discard(c); } + borrow_cleanup_since(borrow_mark); forstep = 0; expect(s_rp); jmp(c0); @@ -6247,7 +7837,7 @@ int kw_do() { next(); expect(s_lp); if (tk == T_NUM && tv) { peek(); if (pk_tk == T_OP && pk_tv == s_rp) inf = 1; } - c = expr(); rval(); plainuse(c); ftest(); + c = borrow_condition(); expect(s_rp); expect(s_semi); if (!inf && live) sc = snap_push(); @@ -6312,7 +7902,7 @@ int casebody(int s0, int live0) { int kw_switch() { int sl = tline; int save = nloc, tmp, c, s0, live0, defent = -1, skip, nxt, bj, v, a = -1, st, tg = 0, cov = 0, gi, nb = 0, j, arm, bi; - [8]word pbn; [8]word pbl; + word[8] pbn; word[8] pbl; stmtonly(); next(); expect(s_lp); c = expr(); st = xty; rval(); plainuse(c); expect(s_rp); @@ -6328,6 +7918,13 @@ int kw_switch() { if (tk == T_EOF) { die("missing }"); break; } if (is_id(s_case)) { next(); + if (tk == T_ID && rt.eq_w(tv, "_")) { + next(); expect(s_colon); + if (defent >= 0) serr("a switch has one catch-all case"); + skip = hole(); op(O_JMP); defent = here; + casebody(s0, live0); setlit(skip, here); + continue; + } bj = 0; gi = -1; peek(); @@ -6358,6 +7955,7 @@ int kw_switch() { nxt = hole(); op(O_JMP); fixall(bj, here); for (j = 0; j < nb && j < pm_nf[gi]; j++) { + capture_cell(pbl[j], 0); laddr(lslot[tmp]); op(O_LD); lit(pm_off[gi * 8 + j]); op(O_ADD); op(O_LD); vaddr(pbl[j]); op(O_ST); } @@ -6412,6 +8010,10 @@ int kw_return() { else { c = expr(); rval(); returned = xty; + if (!nclo && !xzero) { + if (!fn_return_type) fn_return_type = returned; + else if (fn_return_type != returned) fn_return_type = 0 - 1; + } if_ret(xty, xzero); if (fty && fty != tvoid) fcoerce(fty); if (fty == tvoid) tcerr("return value", tvoid, xty); @@ -6423,9 +8025,10 @@ int kw_return() { } expect(s_semi); retcheck(c); + borrow_cleanup(); exit_vars(1, 0); epichain = fixnew(here, epichain); - lit(MW - 1); op(O_JMP); + lit(rt.MW - 1); op(O_JMP); live = 0; return 0; } @@ -6458,7 +8061,7 @@ int kw_o3() { if (no3pend >= o3pendcap) { o3pend = chk_resize(o3pend, no3pend * 67, o3pendcap * 2 * 67); o3pendcap = o3pendcap * 2; } { p = o3pend + no3pend * 67; p[0] = lo; p[1] = here; p[2] = no3sl; - copy_w(p + 3, o3sl, no3sl); copy_w(p + 35, o3rg, no3sl); no3pend++; + rt.copy_w(p + 3, o3sl, no3sl); rt.copy_w(p + 35, o3rg, no3sl); no3pend++; } local_drop(save); fdrop(save); // the hidden cache variables i = lbase; @@ -6487,12 +8090,13 @@ int kw_unchecked() { return 0; } -// ---- asm blocks: raw vm words, one per token ---- -// numbers push, -n pushes a negative number, #n emits a raw word, one-char -// tokens from the mnemonic table are opcodes, names push the address of a -// variable or function, :name defines a label for this block. -// (labels are local to the block: :name defines one, a later name refers to it, -// and references to labels not yet defined are resolved when the block ends) +//: ---- asm blocks: raw vm words, one per token ---- + +//: numbers push, -n pushes a negative number, #n emits a raw word, one-char +//: tokens from the mnemonic table are opcodes, names push the address of a +//: variable or function, :name defines a label for this block. +//: (labels are local to the block: :name defines one, a later name refers to it, +//: and references to labels not yet defined are resolved when the block ends) word asm_names = R_ASM, asm_sites = R_ASM + ASM_LABELS; word asm_refs = R_ASM + ASM_LABELS + ASM_LABELS, asm_fixes = R_ASM + ASM_LABELS + ASM_LABELS + ASM_REFS; int asm_labelcap = ASM_LABELS, asm_refcap = ASM_REFS; @@ -6549,7 +8153,11 @@ int asmblock() { if (tk != T_ID) die("bad token in asm"); i = findloc(tv); if (i >= 0 && (ltag[i] & 16) && !larr[i]) serr("asm can't reach an o3 variable"); - if (i >= 0) { luse(i); ltag[i] = ltag[i] | LREAD | LASG; laddr(lslot[i]); next(); continue; } + if (i >= 0) { luse(i); ltag[i] = ltag[i] | LREAD | LASG; vaddr(i); next(); continue; } + if (nclo) { + k = clocap(tv); + if (k >= 0) { vaddr(clo_env[nclo - 1]); op(O_LD); lit(k); op(O_ADD); op(O_LD); next(); continue; } + } k = -1; i = 0; while (i < nl) { if (ln[i] == tv) k = i; i++; } if (k >= 0) { lit(la[k]); next(); continue; } @@ -6576,27 +8184,29 @@ int contiguous_asm() { word q = grab(), wrapped; int n = q[0], i = 0, need = 2, entry, at; while (i < n) { need = need + 16; - if (q[1 + i * 3] == T_STR || q[1 + i * 3] == T_BSTR) need = need + len_w(q[2 + i * 3]); + if (q[1 + i * 3] == T_STR || q[1 + i * 3] == T_BSTR) need = need + rt.len_w(q[2 + i * 3]); i++; } reserve_code(need); entry = here; wrapped = chk_resize(0, 0, n * 3 + 7); wrapped[0] = n + 2; wrapped[1] = T_OP; wrapped[2] = s_lb; wrapped[3] = tpack(); - copy_w(wrapped + 4, q + 1, n * 3); at = 4 + n * 3; + rt.copy_w(wrapped + 4, q + 1, n * 3); at = 4 + n * 3; wrapped[at] = T_OP; wrapped[at + 1] = s_rb; wrapped[at + 2] = tpack(); src_push(1, wrapped, 0, 0); next(); asmblock(); src_pop(); op(O_RET); - free(d*q); return entry; + rt.free(d*q); return entry; } // asm as a keyword (statement or expression): the block's own stack effect is // the program's responsibility, so the result is a plain word int kw_asm() { + if (effect_current) effect_current[0] = 2; next(); asmblock(); xcls = C_PLAIN; return 0; } -// ---- quote: code as data ---- +//: ---- quote: code as data ---- + // a token list is an object: [n] then kind, value, line for each token. names, // strings and operators are interned pointers. // grab reads the { ... } after quote or times (a parse word) into a token list, @@ -6616,14 +8226,14 @@ word .u grab() { next(); } next(); - q = new(n * 3 + 1); + q = rt.new(n * 3 + 1); q[0] = n; - copy_w(q + 1, R_QBUF, n * 3); + rt.copy_w(q + 1, R_QBUF, n * 3); return u*q; } -unchecked word .u qnew(word src) { int n = src[0] * 3 + 1; word q = new(n); copy_w(q, src, n); return q; } -word .u qdup(.m q) { int n = q[0] * 3 + 1; word r = new(n); copy_w(r, q, n); return u*r; } -// emit n words copied from the array q. +word .u qnew(word src) { int n = src[0] * 3 + 1; word q = rt.new(n); rt.copy_w(q, src, n); return q; } +word .u qdup(.m q) { int n = q[0] * 3 + 1; word r = rt.new(n); rt.copy_w(r, q, n); return u*r; } +// emit n words copied from a well-formed token buffer q; references in the emitted words must outlive the image. int wcopyto(.m q, int n) { int i = 0; reserve_code(n); while (i < n) { emit(q[i]); i++; } @@ -6632,22 +8242,22 @@ int wcopyto(.m q, int n) { // quote { tokens }: the list is emitted into the code behind a jump, and the // expression's value is a fresh copy of it (qnew), so a program can edit its // copy without touching the original -int kw_quote() { +int quote_tokens() { word q; int j, a, n; - next(); q = grab(); j = hole(); op(O_JMP); n = q[0] * 3 + 1; reserve_code(n); a = here; wcopyto(q, n); - free(d*q); + rt.free(d*q); setlit(j, here); lit(a); callname(s_qnew); xcls = C_FRESH; return 0; } +int kw_quote() { next(); return quote_tokens(); } // tokens of a string, as a token list -word .u tokens(int s) { +word*.u tokens(char* s) { int n = 0; word q; src_push(2, s, 0, 0); next(); @@ -6657,20 +8267,21 @@ word .u tokens(int s) { next(); } src_pop(); - q = new(n * 3 + 1); + q = rt.new(n * 3 + 1); q[0] = n; - copy_w(q + 1, R_QBUF, n * 3); + rt.copy_w(q + 1, R_QBUF, n * 3); return u*q; } // feed a token list into the input: the parser reads it next, then carries // on where it was. the list is freed once it has been read. -unchecked int feed(.d q) { +int feed(.d q) { src_push(1, q, 1, 1); next(); return 0; } -// ---- reading a unit and scanning it for signatures ---- +//: ---- reading a unit and scanning it for signatures ---- + // a unit from stdin is read whole first (4 chars to a word), then scanned // once for function signatures, so a function's tags are known before its first // call even when it's defined further down. @@ -6678,21 +8289,22 @@ unchecked int feed(.d q) { // literals, see srcend) or end of input, packing 4 // characters to a word, so prescan can run over it and then the real parse can // run over the same text -unchecked word readsrc() { +word .u readsrc() { int cap = 4090; word b; int n = 0, c; word nb; int w; - b = new(4090); + b = rt.new(4090); qst = 0; qp = 0; qesc = 0; while (!asm { ? }) { c = asm { , } & 255; + if (c == 255 && asm { ? }) break; // the end of a slow pipe, found by the read itself // srcend only has anything to decide at a quote, a slash or a star (or // inside a comment or literal); every other character in plain code just // clears the "previous char was a slash" memory if (qst || c == 34 || c == 39 || c == '/' || c == '*' || c == 96) { if (srcend(c)) break; } else qp = 0; if ((n >> 2) + 2 >= cap) { - nb = new(cap + cap + 6); - copy_w(nb, b, cap); - free(b); + nb = rt.new(cap + cap + 6); + rt.copy_w(nb, b, cap); + rt.free(b); b = nb; cap = cap + cap + 6; } @@ -6701,7 +8313,7 @@ unchecked word readsrc() { b[w] = b[w] | (c << ((n & 3) * 8)); n++; } - b[n >> 2] = b[n >> 2] & (pow2[(n & 3) * 8] - 1); // the 0 at the end + b[n >> 2] = b[n >> 2] & (rt.pow2[(n & 3) * 8] - 1); // the 0 at the end if ((n & 3) == 0) b[n >> 2] = 0; b[(n >> 2) + 1] = 0; return b; @@ -6744,6 +8356,7 @@ int prescan_fn(int ret, int n, int o3) { if (!is_op(s_lb) && !is_op(s_semi)) return 0; // not a function: an enhancement, a call in a parse word... if (!dnm) dnm = n; s = lookup(n); + if (s >= 0 && skind[s] == K_KW) return 0; if (s < 0 || s < ubase || skind[s] != K_FN || (sflag[s] & F_DEF)) s = fwdfn(n); if (!(sflag[s] & F_SIG)) { ssig[s] = ret + np + np + tags * 64; @@ -6754,7 +8367,7 @@ int prescan_fn(int ret, int n, int o3) { } // scan ahead, without emitting code, to learn what a block of tokens contains. int prescan(int kind, int obj) { - int depth = 0, sp = panic, sn = nerr, pend = 0, po3 = 0, n, dep; + int depth = 0, sp = panic, sn = nerr, pend = 0, po3 = 0, n, dep; char* oldmodule = module_name; word oldimports = module_imports; dry++; src_push(kind, obj, 0, 0); next(); @@ -6762,6 +8375,8 @@ int prescan(int kind, int obj) { if (is_op(s_lb)) depth++; else if (is_op(s_rb)) { depth--; pend = 0; po3 = 0; } else if (depth <= 0) { + if (is_id(s_type)) { while (tk != T_EOF && !is_op(s_semi)) next(); if (is_op(s_semi)) next(); pend = 0; po3 = 0; continue; } + if (tk == T_ID && rt.eq_w(tv, "namespace")) { kw_namespace(); continue; } if (is_op(s_semi)) { pend = 0; po3 = 0; } else if (is_op(s_dot)) { peek(); if (pk_tk == T_ID && pk_tv == s_u) pend = 1; } else if (is_id(s_o3)) po3 = 1; @@ -6794,31 +8409,33 @@ int prescan(int kind, int obj) { next(); } src_pop(); - dry--; panic = sp; nerr = sn; + dry--; panic = sp; nerr = sn; module_name = oldmodule; module_imports = oldimports; return 0; } // compile a whole unit from stdin int unit_stdin() { int src = readsrc(); + module_name = 0; module_imports = 0; prescan(3, src); src_push(3, src, 0, 0); next(); unit_top(); unit_end(ubase); src_pop(); - free(src); + rt.free(src); return 0; } -// ---- nested compiles ---- -// compile() and load() can run while another function is being compiled (from -// a parse word). the function context is saved, the new code is emitted in -// place with a jump around it, and then everything is put back. -// the context frame is 34 words: everything about the function being compiled -// that a nested compile would otherwise clobber (locals, depth, frame size, -// loop stack position, the expression-state globals). closures use the same -// pair. -const int CTX_WORDS = 34; +//: ---- nested compiles ---- + +//: compile() and load() can run while another function is being compiled (from +//: a parse word). the function context is saved, the new code is emitted in +//: place with a jump around it, and then everything is put back. +//: the context frame is 34 words: everything about the function being compiled +//: that a nested compile would otherwise clobber (locals, depth, frame size, +//: loop stack position, the expression-state globals). closures use the same +//: pair. +const int CTX_WORDS = 48; int ctx_push() { af_forget(); word f; @@ -6827,8 +8444,9 @@ int ctx_push() { f[0] = nloc; f[1] = lbase; f[2] = fdepth; f[3] = fslots; f[4] = fchk; f[5] = fret; f[6] = live; f[7] = bxdepth; f[8] = epichain; f[9] = fframe; f[10] = infn; f[11] = lsp; f[12] = loopbase; f[13] = lv; f[14] = lvk; f[15] = lvi; f[16] = kw_expr; f[17] = xcls; f[18] = snaptop; - f[19] = fty; f[20] = ntp; f[21] = xty; copy_w(f + 22, tpn, 8); - f[30] = finsym; f[31] = finfer; f[32] = ifcur; f[33] = ifcall; + f[19] = fty; f[20] = ntp; f[21] = xty; rt.copy_w(f + 22, tpn, 8); + f[30] = finsym; f[31] = finfer; f[32] = ifcur; f[33] = ifcall; f[34] = effect_current; f[35] = borrow_slots; f[36] = borrow_pending; f[37] = borrow_depth; f[38] = borrow_jump; f[39] = borrow_entry; f[40] = capture_enabled; f[41] = effect_value; f[42] = xprojection; f[43] = effect_floor; f[44] = module_name; f[45] = module_imports; + f[46] = fn_return_type; f[47] = if_value; return 0; } // pop the innermost function context. @@ -6846,13 +8464,16 @@ int ctx_pop() { live = f[6]; bxdepth = f[7]; epichain = f[8]; fframe = f[9]; infn = f[10]; lsp = f[11]; loopbase = f[12]; lv = f[13]; lvk = f[14]; lvi = f[15]; kw_expr = f[16]; xcls = f[17]; snaptop = f[18]; - fty = f[19]; ntp = f[20]; xty = f[21]; copy_w(tpn, f + 22, 8); - finsym = f[30]; finfer = f[31]; ifcur = f[32]; ifcall = f[33]; + fty = f[19]; ntp = f[20]; xty = f[21]; rt.copy_w(tpn, f + 22, 8); + finsym = f[30]; finfer = f[31]; ifcur = f[32]; ifcall = f[33]; effect_current = f[34]; borrow_slots = f[35]; borrow_pending = f[36]; borrow_depth = f[37]; borrow_jump = f[38]; borrow_entry = f[39]; capture_enabled = f[40]; effect_value = f[41]; xprojection = f[42]; effect_floor = f[43]; module_name = f[44]; module_imports = f[45]; xvk = 0; + fn_return_type = f[46]; if_value = f[47]; return 0; } -// ---- functions and the top level ---- -// ---- a function, start to finish ---- +//: ---- functions and the top level ---- + +//: ---- a function, start to finish ---- + // fndef reads the header, then: fn_begin resets the per-function state, // prologue emits the frame setup, the parameters are declared and stored from // the stack into their slots, body parses the statements, and epilogue emits @@ -6861,23 +8482,25 @@ int ctx_pop() { // that epilogue fills with fslots. // fn_begin: chk is 0 for unchecked, ret is 1 for a .u return int fn_begin(int chk, int ret) { - lbase = nloc; fdepth = 1; fslots = 0; fchk = chk && check_on; fret = ret; fty = 0; finfer = 0; + capture_enabled = 0; effect_value = 0; effect_current = 0; borrow_slots = 0; borrow_pending = 0; borrow_depth = 0; borrow_jump = -1; borrow_entry = 0; + if (cmode && svkind && !dry) rt.fill_w(svkind, 0, nsym); xvk = 0; + lbase = nloc; fdepth = 1; fslots = 0; fchk = chk && check_on; fret = ret; fty = 0; finfer = 0; fn_return_type = 0; live = 1; bxdepth = 0; epichain = 0; infn = 1; loopbase = lsp; lv = 0; lvk = LV_NONE; xbor = -1; return 0; } // allocate the frame, retaining the caller's pointer until any growth finishes. int prologue() { int k, busy, done, helper = !nclo && (sname[finsym] == s_frame_grow || sname[finsym] == s_frame_leave); - lit(ZFP); op(O_LD); + lit(rt.ZFP); op(O_LD); if (!helper) { lit(0); op(O_PICK); } - fframe = hole(); op(O_SUB); lit(ZFP); op(O_ST); + fframe = hole(); op(O_SUB); lit(rt.ZFP); op(O_ST); if (helper) return 0; lit(sval[lookup(s_frame_busy)]); op(O_LD); busy = hole(); op(O_JZ); done = hole(); op(O_JMP); setlit(busy, here); - lit(ZFP); op(O_LD); lit(sval[lookup(s_frame_floor)]); op(O_LD); + lit(rt.ZFP); op(O_LD); lit(sval[lookup(s_frame_floor)]); op(O_LD); lit(sval[lookup(s_frame_guard)]); op(O_LD); op(O_ADD); op(O_LT); k = hole(); op(O_JZ); - lit(ZFP); op(O_LD); lit(1); op(O_PICK); lit(ZFP); op(O_ST); - callname(s_frame_grow); lit(ZFP); op(O_ST); + lit(rt.ZFP); op(O_LD); lit(1); op(O_PICK); lit(rt.ZFP); op(O_ST); + callname(s_frame_grow); lit(rt.ZFP); op(O_ST); busy = hole(); op(O_JMP); setlit(k, here); setlit(done, here); op(O_DROP); setlit(busy, here); return 0; @@ -6887,14 +8510,14 @@ int epilogue() { int k, node, done, helper = !nclo && (sname[finsym] == s_frame_grow || sname[finsym] == s_frame_leave); lit(0); fixall(epichain, here); - lit(ZFP); op(O_LD); lit(fslots); op(O_ADD); lit(ZFP); op(O_ST); + lit(rt.ZFP); op(O_LD); lit(fslots); op(O_ADD); lit(rt.ZFP); op(O_ST); if (!helper) { lit(sval[lookup(s_frame_node)]); op(O_LD); node = hole(); op(O_JZ); - lit(ZFP); op(O_LD); lit(sval[lookup(s_frame_top)]); op(O_LD); op(O_SUB); k = hole(); op(O_JZ); + lit(rt.ZFP); op(O_LD); lit(sval[lookup(s_frame_top)]); op(O_LD); op(O_SUB); k = hole(); op(O_JZ); done = hole(); op(O_JMP); setlit(k, here); - callname(s_frame_leave); lit(ZFP); op(O_ST); setlit(done, here); setlit(node, here); + callname(s_frame_leave); lit(rt.ZFP); op(O_ST); setlit(done, here); setlit(node, here); } - op(O_RET); + op(O_RET); borrow_initializer(); setlit(fframe, fslots); infn = 0; local_drop(lbase); @@ -6905,11 +8528,15 @@ int epilogue() { // checks: hints, unused m objects, anything still owned is freed, and a .u // function that falls off the end is an error int body() { + int closed, endplace; + capture_prepare(); borrow_begin(); expect(s_lb); while (!is_op(s_rb)) { if (tk == T_EOF) { die("missing } at the end of a function"); break; } stmt(); } + closed = is_op(s_rb); endplace = toff + toklen; + if (probe_off >= 0 && !chkabort && !dry && !ginst && incdepth == 0 && ctxsp <= nclo && toff >= probe_off) { probe_snap(); chkabort = 1; } local_hints(lbase); unused_check(lbase); if (ifcur) if_fin(); @@ -6919,6 +8546,7 @@ int body() { if (fret) cerr("a function returning .u has to end with a return"); } next(); + if (probe_off >= 0 && !chkabort && closed && nclo == 0 && probe_has == 1 && endplace <= probe_off && !ginst && ctxsp == 0) { probe_has = 0; probe_n = 0; } // the place is past this function return 0; } // the rest of a function, after its return type and name: @@ -6928,7 +8556,7 @@ int body() { // into stype, checked against any earlier prototype or prescan result, and // where the o3 region for an o3 function is handed to the core) int fndef(int chk, int rty, int ret, int n) { - int s, np = 0, tags = 0, t, sig, i; [24]word pn; [24]word pt; [24]word pty; int typed = 0, ft, o3f = o3nxt, lo, hi, pk; + int s, np = 0, tags = 0, t, sig, i; char*[24] pn; word[24] pt; word[24] pty; int typed = 0, ft, explicit_rty = !!rty, o3f = o3nxt, lo, hi, pk; o3nxt = 0; if (rty || ntp) typed = 1; if (!dnm) dnm = n; @@ -6937,6 +8565,7 @@ int fndef(int chk, int rty, int ret, int n) { s = addsym(n, K_FN, 0); sflag[s] = 0; } + def_note(s, nameoff); ifcur = 0; if (ifon && ifkk && !ginst && !ntp && !dry) { if (ifkk > 0 && !rty) if_begin(s, ifkk); else if_begin(s, 0); @@ -6950,15 +8579,16 @@ int fndef(int chk, int rty, int ret, int n) { pk = 0; if (!is_op(s_dot) && ((tk == T_OP && tv == s_lk) || (tk == T_ID && (tv == s_fn || istypename(tv))))) { pty[np] = ptype(); typed = 1; pk = 1; + if (pty[np][0] == TY_ARR) pty[np] = decay(pty[np]); if (isagg(pty[np])) { if (ginst) pty[np] = tptr(pty[np]); else if (!stubgen) serr("structs and arrays are passed by pointer"); // (an extern stub takes its struct by value, which is an address here) } } if (ifcur) { - if (pk) if_param(np, tpack(), 0); - else if (is_op(s_dot)) if_param(np, tpack(), 2); - else if_param(np, tpack(), 1); + if (pk) if_param(np, tpack(), 0, pty[np]); + else if (is_op(s_dot)) if_param(np, tpack(), 2, pty[np]); + else if_param(np, tpack(), 1, pty[np]); } if (accept(s_dot)) { // .m lends, .d takes over if (tv == s_m) { @@ -6966,7 +8596,7 @@ int fndef(int chk, int rty, int ret, int n) { } else if (tv == s_d) t = 2; else die("a parameter tag is .m or .d"); next(); } - pnoffs[np] = toff; rs_note(RS_PARAM, toff); pn[np] = name(); + pnoffs[np] = toff; pnplaces[np] = tpack(); rs_note(RS_PARAM, toff); pn[np] = name(); if (o3f && pty[np] == tword) serr("o3 parameters need a type"); if (np >= 12 && t) die("only the first 12 parameters can be tagged"); pt[np] = t; @@ -6991,6 +8621,7 @@ int fndef(int chk, int rty, int ret, int n) { if (stype[s] && stype[s] != ft) serrn(n, "doesn't match its earlier declaration"); stype[s] = ft; } + if (ifcur && explicit_rty) if_join(ifcur + 3, rty); sig_attach(n, rty, np, pn, pty, typed); if (accept(s_semi)) { sflag[s] = sflag[s] | F_DECL; ntp = 0; return s; } if (sflag[s] & F_DEF) serrn(n, "is defined twice"); @@ -7003,7 +8634,7 @@ int fndef(int chk, int rty, int ret, int n) { sval[s] = here; // the name still has an address and a type, so reflection can ask for them; it halts if it is ever jumped to sflag[s] = sflag[s] | F_DEF; fixall(sfix[s], here); sfix[s] = 0; - lit(MW - 1); op(O_JMP); + lit(rt.MW - 1); op(O_JMP); return s; } sval[s] = here; @@ -7011,14 +8642,17 @@ int fndef(int chk, int rty, int ret, int n) { fixall(sfix[s], here); sfix[s] = 0; fn_begin(chk, ret); finsym = s; finfer = !ret; - if (typed) fty = rty; + effect_current = effect_record(s); rt.fill_w(effect_current, 0, 50); effect_current[0] = 1; if (!chk || stubgen) effect_current[0] = 2; effect_current[1] = np; + i = 0; while (i < np) { if (pt[i] == 2) { effect_current[2 + i] = EF_CONSUME; effect_current[26 + i] = EF_CONSUME; } i++; } + if (typed && explicit_rty) fty = rty; no3pend = 0; if (o3f) { o3on = 1; o3reset(); sflag[s] = sflag[s] | F_O3; } - prologue(); + prologue(); borrow_prologue(); i = 0; while (i < np) { - t = newloc(pn[i]); lrskind[t] = RS_PARAM; lrsoff[t] = pnoffs[i]; fslots++; lsmark(1, pn[i]); + t = newloc(pn[i]); lrskind[t] = RS_PARAM; lrsoff[t] = pnoffs[i]; lpos[t] = pnplaces[i]; fslots++; lsmark(1, pn[i]); ltype[t] = pty[i]; lptr[t] = 0; + if (ifcur && if_local && i < IFP) if_local[t] = ifcur + 8 + 3 * i; lt_note(pnoffs[i], pty[i]); if (o3f) { ltag[t] = 16; o3promote(t); } if (isfl(pty[i])) setflt(t, 1); @@ -7033,6 +8667,10 @@ int fndef(int chk, int rty, int ret, int n) { i = 0; while (i < np) { t = lbase + i; o3mkcache(t); o3refresh(t); i++; } } body(); + if (!explicit_rty && fn_return_type > 0 && fn_return_type != tword) { + stype[s] = tfn(fn_return_type, np, pty, ssig[s]); + sig_attach(n, fn_return_type, np, pn, pty, 1); + } hi = here; epilogue(); if (o3f) { @@ -7044,10 +8682,12 @@ int fndef(int chk, int rty, int ret, int n) { } o3flush(); ntp = 0; ifcur = 0; + fn_remember(s, n); return s; } -// let g = 1, buf[10]; int g = 3; [10]int buf; point origin; -// ---- globals, constants, enhancements ---- +//: let g = 1, buf[10]; int g = 3; int[10] buf; point origin; +//: ---- globals, constants, enhancements ---- + // a global's storage is emitted in place among the code (one word, or the // whole array), and the symbol's value is its address. a string initializer // points at characters emitted right after. @@ -7059,7 +8699,7 @@ int gdecl(int typed) { } // compile the rest of a global declaration: its type, name and initial value. int gdecl2(int t, int tg, int n) { - int a, ng = nameoff; word k, arraytype; int s, kp = ifgk, gty = 0, gok = ifon && !t && !dry; + int a, ng = nameoff; word k, arraytype, iq; int s, kp = ifgk, gty = 0, gok = ifon && !t && !dry, vr, vk; word vv; ifgk = 0; if (!dnm) dnm = n; if (tg) serr("tags go on locals and parameters, not globals"); @@ -7073,28 +8713,45 @@ int gdecl2(int t, int tg, int n) { while (k) { emit(0); k--; } if (gok) { if (!gty) gty = tint; else if (gty != tint) gok = 0; } } else if (t && isagg(t)) { - k = tsz(t); reserve_code(k); - s = addsym(n, K_GARR, here); rs_put(ng, RS_SYM, rs_sym(s), s); stype[s] = t; + arraytype = t; iq = 0; + if (t[0] == TY_ARR && accept(s_eq)) { if (!is_op(s_lb) && tk != T_STR && tk != T_BSTR) serr("an array needs an initializer list"); else { iq = array_initializer_tokens(t); arraytype = array_measure(iq, t); } } + if (arraytype[0] == TY_ARR && arraytype[3] < 0) serr("T[] needs an initializer to determine its length"); + k = tsz(arraytype); reserve_code(k); a = here; + s = addsym(n, K_GARR, here); rs_put(ng, RS_SYM, rs_sym(s), s); stype[s] = arraytype; while (k) { emit(0); k--; } + if (iq) { array_initialize(iq, arraytype, a, 0); rt.free(iq); } } else { reserve_code(1); a = here; - s = addsym(n, K_GVAR, a); rs_put(ng, RS_SYM, rs_sym(s), s); + s = addsym(n, K_GVAR, a); rs_put(ng, RS_SYM, rs_sym(s), s); vr = rsn - 1; vk = 1; vv = 0; if (t) stype[s] = t; emit(0); if (accept(s_eq)) { if (tk == T_STR) { - reserve_code(len_w(tv) + 1); *(a + hdelta) = here; + vk = 3; vv = vf_string(tv); if (!t) stype[s] = tptr(tchar); + reserve_code(rt.len_w(tv) + 1); *(a + hdelta) = here; k = tv; while (*k) { emit(*k); k++; } emit(0); next(); if (gok) { if (!gty) gty = tptr(tchar); else if (gty != tptr(tchar)) gok = 0; } } + else if (tk == T_BSTR) { + vk = 4; vv = vf_string(tv); *(a + hdelta) = bstrdata(tv); next(); + if (!t) stype[s] = tbstr; else if (t != tbstr) tcerr("initializer", t, tbstr); + if (gok) { if (!gty) gty = tbstr; else if (gty != tbstr) gok = 0; } + } + else if (is_op(s_lb) || (is_op(s_lp) && caststart())) { + arraytype = tarr(tint, -1); + if (accept(s_lp)) { arraytype = ptype(); expect(s_rp); } + if (arraytype[0] != TY_ARR || !is_op(s_lb)) serr("a global compound literal needs an array initializer"); + else { array_literal(arraytype); vv = xcval; *(a + hdelta) = vv; if (!t) stype[s] = decay(xty); } + } else { - *(a + hdelta) = cnorm(t, cexpr()); + vv = cnorm(t, cexpr()); *(a + hdelta) = vv; if (gok) { if (!gty) gty = tint; else if (gty != tint) gok = 0; } } } else if (gok) gok = 0; // no initializer: nothing says what it is + vf_at(vr, vk, vv); } if (stype[s]) lt_note(ng, stype[s]); else lt_note(ng, tword); if (!accept(s_comma)) break; @@ -7118,10 +8775,10 @@ int cnorm(word t, int v) { // const A = 1, B = A + 2; // const [type] NAME = e, NAME = e; the type is optional and is read and dropped, as in zc0 int gconst() { - int n; + int n, off, s; next(); if (tk == T_ID) { peekat(0); if (pk_tk == T_ID || (pk_tk == T_OP && pk_tv == s_star)) ptype(); } - while (1) { n = name(); if (!dnm) dnm = n; expect(s_eq); addsym(n, K_CONST, cexpr()); if (!accept(s_comma)) break; } + while (1) { off = toff; n = name(); if (!dnm) dnm = n; expect(s_eq); s = addsym(n, K_CONST, cexpr()); rs_put(off, RS_SYM, rs_sym(s), s); if (!accept(s_comma)) break; } expect(s_semi); return 0; } @@ -7130,7 +8787,7 @@ int gconst() { // lays out the descriptor the install opcode reads, and wires it in right // away (an image wires its own in when it boots). int enhdef() { - int n, np = 0, nout = 1, dsc, na, pay, len = 0, kind = 0, i; word p; int s, slot, r; [8]word pn; + int n, np = 0, nout = 1, dsc, na, pay, len = 0, kind = 0, i; word p; int s, slot, r; char*[8] pn; next(); // enhance 2 divmod(a, b) if (tk == T_NUM) { @@ -7143,13 +8800,13 @@ int enhdef() { if (!is_op(s_rp)) { while (1) { if (np == 8) die("an enhancement takes at most 8 inputs"); - pnoffs[np] = toff; rs_note(RS_PARAM, toff); pn[np] = name(); + pnoffs[np] = toff; pnplaces[np] = tpack(); rs_note(RS_PARAM, toff); pn[np] = name(); np++; if (!accept(s_comma)) break; } } expect(s_rp); - len = 7 + np; i = 0; while (i < np) { len = len + len_w(pn[i]) + 1; i++; } + len = 7 + np; i = 0; while (i < np) { len = len + rt.len_w(pn[i]) + 1; i++; } reserve_code(len); len = 0; dsc = here; i = 0; while (i < 7) { emit(0); i++; } na = here; i = 0; while (i < np) { emit(0); i++; } i = 0; @@ -7170,7 +8827,7 @@ int enhdef() { if (accept(s_else)) { r = lookup(tv); if (r < 0) r = fwdfn(tv); next(); sfix[s] = r + 1; } expect(s_semi); if (!hdelta) { - r = install(dsc); + r = rt.install(dsc); if (r < 0) { if (!sfix[s]) serrn(n, "the enhancement didn't install (see install's codes in core.c)"); sflag[s] = sflag[s] | F_FALLBACK; @@ -7196,7 +8853,7 @@ int topsync() { // after a function's name: <T, U> declares its type parameters. any the return // type used were taken on trust; now they get their real numbers int genparams() { - int i, j, n = 0; [8]word ps; [8]word bs; + int i, j, n = 0; char*[8] ps; word[8] bs; if (!is_op(s_lt)) { if (ntp) serrn(tpn[0], "isn't a type"); ntp = 0; return 0; } next(); while (1) { @@ -7206,7 +8863,7 @@ int genparams() { if (!accept(s_comma)) break; } closeangle(); - fill_w(bs, 0, 8); + rt.fill_w(bs, 0, 8); i = 0; while (i < ntp) { j = 0; @@ -7215,7 +8872,7 @@ int genparams() { i++; } if (ntp && dty) dty = subst(dty, bs); - copy_w(tpn, ps, n); ntp = n; + rt.copy_w(tpn, ps, n); ntp = n; if (ginst) ntp = 0; return 0; } @@ -7300,8 +8957,12 @@ int topitem1() { if (is_id(s_memdebug)) return kw_memdebug(); if (is_id(s_enum)) return kw_enum(); if (is_id(s_include)) return kw_include(); + if (tk == T_ID && rt.eq_w(tv, "namespace")) return kw_namespace(); if (is_id(s_enhance)) return enhdef(); - if (is_id(s_const)) return gconst(); + if (is_id(s_const)) { + peekat(0); + if (pk_tk != T_ID || pk_tv != s_fixed) return gconst(); + } if (tk == T_ID) { s = lookup(tv); if (s >= 0 && skind[s] == K_KW && (sflag[s] & F_TOP)) { kwcall(s, 0); return 0; } @@ -7322,7 +8983,8 @@ int topitem1() { die("expected a declaration"); return 0; } -// ---- generic functions: templates and instances ---- +//: ---- generic functions: templates and instances ---- + // which type parameter does name n stand for while an instance is compiled int gfind(int n) { int i = 0; @@ -7356,12 +9018,12 @@ word hasvar(word t) { return 0; } // add the current token to the item being recorded -unchecked int grec_add() { +int grec_add() { word nb; if (gn * 3 + 4 > gcap) { - nb = new(gcap + gcap); - copy_w(nb, gbuf, gcap); - free(gbuf); + nb = rt.new(gcap + gcap); + rt.copy_w(nb, gbuf, gcap); + rt.free(gbuf); gbuf = nb; gcap = gcap + gcap; } @@ -7370,10 +9032,10 @@ unchecked int grec_add() { return 0; } // the template entry for generic function s, made on first sight of its header -unchecked int ginit() { +int ginit() { gtab = 1; gt_sym = chk_resize(0, 0, gtcap); gt_tok = chk_resize(0, 0, gtcap); gt_np = chk_resize(0, 0, gtcap); - gt_nm = chk_resize(0, 0, gtcap * 8); + gt_nm = chk_resize(0, 0, gtcap * 8); gt_module = chk_resize(0, 0, gtcap); gt_imports = chk_resize(0, 0, gtcap); gt_source = chk_resize(0, 0, gtcap); gi_t = chk_resize(0, 0, gicap); gi_sym = chk_resize(0, 0, gicap); gi_done = chk_resize(0, 0, gicap); gi_ar = chk_resize(0, 0, gicap * 8); return 0; @@ -7384,6 +9046,7 @@ int reserve_generics(int need) { while (cap < need) cap = cap * 2; gt_sym = chk_resize(gt_sym, gt_n, cap); gt_tok = chk_resize(gt_tok, gt_n, cap); gt_np = chk_resize(gt_np, gt_n, cap); gt_nm = chk_resize(gt_nm, gt_n * 8, cap * 8); + gt_module = chk_resize(gt_module, gt_n, cap); gt_imports = chk_resize(gt_imports, gt_n, cap); gt_source = chk_resize(gt_source, gt_n, cap); gtcap = cap; return cap; } int reserve_instances(int need) { @@ -7395,7 +9058,7 @@ int reserve_instances(int need) { gicap = cap; return cap; } // make the generic instance of the function or struct s for its type arguments. -unchecked int gtmake(int s) { +int gtmake(int s) { int k = 0; if (!gtab) ginit(); while (k < gt_n) { @@ -7405,10 +9068,10 @@ unchecked int gtmake(int s) { if (k == gt_n) { if (gt_n >= gtcap) reserve_generics(gt_n + 1); gt_n++; - gt_sym[k] = s; gt_tok[k] = 0; + gt_sym[k] = s; gt_tok[k] = 0; gt_module[k] = module_name; gt_imports[k] = module_imports; gt_source[k] = curinc; } gt_np[k] = ntp; - copy_w(gt_nm + k * 8, tpn, ntp); + rt.copy_w(gt_nm + k * 8, tpn, ntp); return k; } // skip a function body without compiling it: from the { to its matching } @@ -7426,18 +9089,98 @@ int gskip() { return 0; } // the item just read defined generic function gnew: keep its tokens -unchecked int gattach(word buf, int n) { +int gattach(word .d buf, int n) { n--; // the last token belongs to the next item - if (nerr) { free(buf); return 0; } + if (nerr) { rt.free(buf); return 0; } buf[0] = n; gt_tok[gnew] = buf; return 0; } +// retain one ordinary function's source tokens and the namespace needed to compile it again. +int fn_remember(int s, int name) { + int count = gn - 1, i = 0, j, depth, namepos = -1, open = -1, close = -1; word* q; word meta; + if (!grec || !gbuf || fn_method_compile || dry || ginst || nerr || (sflag[s] & F_GEN) || count <= 0) return 0; + while (i < count && open < 0) { + if (gbuf[1 + i * 3] == T_ID && gbuf[2 + i * 3] == name) { + j = i + 1; + if (gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_lt) { + depth = 1; j++; + while (j < count && depth) { + if (gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_lt) depth++; + else if (gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_gt) depth--; + j++; + } + } + if (j < count && gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_lp) { + depth = 1; j++; + while (j < count && depth) { + if (gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_lp) depth++; + else if (gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_rp) depth--; + j++; + } + while (j < count && !(gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_lb)) j++; + if (j < count) { namepos = i; open = j; } + } + } + i++; + } + if (open < 0) return 0; + depth = 1; j = open + 1; + while (j < count && depth) { + if (gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_lb) depth++; + else if (gbuf[1 + j * 3] == T_OP && gbuf[2 + j * 3] == s_rb) depth--; + if (depth) j++; + } + if (depth) return 0; + close = j; + q = rt.new(1 + count * 3); q[0] = count; rt.copy_w(q + 1, gbuf + 1, count * 3); + meta = rt.new(6); meta[0] = q; meta[1] = module_name; meta[2] = module_imports; + meta[3] = namepos; meta[4] = open; meta[5] = close; fnmeta[s] = meta; + return 1; +} +// compile one retained source function for a syntax callback's host context. +// calls into emitted source are cloned recursively; existing host bindings retain +// their original globals and entry addresses. +int syntax_host_function(int symbol) { + word meta = fnmeta[symbol], source, full; int clone, count, name; + char[40] generated; char[24] reversed; int sequence = fnpatchseq, digits = 0, i = 0; + if (syntax_host_symbols[symbol]) return syntax_host_symbols[symbol] - 1; + if (!meta || (sflag[symbol] & F_GEN)) { serr("an emitted syntax dependency requires retained nongeneric source"); return -1; } + fnpatchseq++; + rt.copy_w(generated, "__run_syntax_", 13); + if (!sequence) reversed[digits++] = '0'; + while (sequence) { reversed[digits++] = '0' + sequence % 10; sequence = sequence / 10; } + while (i < digits) { generated[13 + i] = reversed[digits - i - 1]; i++; } + generated[13 + digits] = 0; name = intern(generated); + source = meta[0]; count = source[0]; full = rt.new(1 + count * 3); full[0] = count; + rt.copy_w(full + 1, source + 1, count * 3); full[2 + meta[3] * 3] = name; + if (meta[1]) clone = fwdfn(methname(meta[1], name)); else clone = fwdfn(name); + syntax_host_symbols[symbol] = clone + 1; + compile_list_context(full, 1, meta[1], meta[2]); rt.free(full); + return clone; +} +// source syntax executes in resident host code even when its library's runtime +// functions are emitted into a compiler-free image. staged symbols never enter +// the image blob; executable storage stays rooted by the ordinary code blocks. +int syntax_resident(int symbol) { + int saved_here = here, saved_delta = hdelta, saved_limit = hlimit, saved_ptop = ptop, saved_vislo = vislo; + int saved_active = image_active, saved_chunk = code_chunk, from = nsym, clone, entry = -1, top; + word block = chk_resize(0, 0, 4096); + syntax_host_symbols = chk_resize(0, 0, from); syntax_host_floor = ubase; syntax_host_from = from; + top = block + block[-5] - rt.HDR; + block[0] = rt.code_blocks; block[1] = block + 3; block[2] = top; rt.code_blocks = m*block; + if (rt.syscall(9, 9, top, 0) < top) fatal("syntax code address reservation failed"); + here = block + 3; hdelta = 0; hlimit = top; image_active = 0; vislo = 0; + clone = syntax_host_function(symbol); if (clone >= 0 && (sflag[clone] & F_DEF)) entry = sval[clone]; + popsyms(from); syntax_host_floor = 0; syntax_host_from = 0; rt.free(syntax_host_symbols); syntax_host_symbols = 0; + here = saved_here; hdelta = saved_delta; hlimit = saved_limit; ptop = saved_ptop; vislo = saved_vislo; image_active = saved_active; code_chunk = saved_chunk; + return entry; +} // the symbol for template s with these type arguments, made on first use and // queued to be compiled once the template's tokens are in. a type parameter // that nothing bound is a plain word. word ginstance(int s, word bs) { - int k = 0, t = 0 - 1, np, i, j, ns; word nm; [24]word dg; int nd; word a; + int k = 0, t = 0 - 1, np, i, j, ns; char* nm; char[24] dg; int nd; char* a; if (dry || !gtab) return s; while (k < gt_n) { if (gt_sym[k] == s) { t = k; break; } @@ -7462,7 +9205,7 @@ word ginstance(int s, word bs) { } if (gi_n >= gicap) reserve_instances(gi_n + 1); i = 0; j = 0; - a = sname[s]; nm = chk_resize(0, 0, len_w(a) + np * 18 + 3); + a = sname[s]; nm = chk_resize(0, 0, rt.len_w(a) + np * 18 + 3); while (a[j]) { nm[i] = a[j]; i++; j++; } nm[i] = '<'; i++; k = 0; @@ -7479,23 +9222,25 @@ word ginstance(int s, word bs) { ssig[ns] = ssig[s]; k = gi_n; gi_n++; gi_t[k] = t; gi_sym[k] = ns; gi_done[k] = 0; - copy_w(gi_ar + k * 8, bs, np); + rt.copy_w(gi_ar + k * 8, bs, np); return ns; } // compile one instance: replay the template's tokens with its type // parameters bound -unchecked int gcompile(int k) { - int t = gi_t[k]; +int gcompile(int k) { + int t = gi_t[k]; char* oldmodule = module_name; word oldimports = module_imports; char* oldsource = curinc; int olddepth = incdepth; + curinc = gt_source[t]; if (curinc) incdepth++; + module_name = gt_module[t]; module_imports = gt_imports[t]; ginst = 1; gng = gt_np[t]; - copy_w(gnm, gt_nm + t * 8, 8); - copy_w(gib, gi_ar + k * 8, 8); + rt.copy_w(gnm, gt_nm + t * 8, 8); + rt.copy_w(gib, gi_ar + k * 8, 8); ginsn = sname[gi_sym[k]]; src_push(1, gt_tok[t], 0, 0); next(); topitem0(); if (tk != T_EOF) die("unexpected tokens after a generic function"); src_pop(); - ginst = 0; gng = 0; + ginst = 0; gng = 0; module_name = oldmodule; module_imports = oldimports; curinc = oldsource; incdepth = olddepth; return 0; } // compile the instances asked for so far whose templates have been read, and @@ -7522,25 +9267,30 @@ int gdrain() { int topitem() { int sb = gbuf, sn = gn, sc = gcap, sr = grec, sg = gnew, rb, rn; if (ginst) return topitem0(); - gbuf = new(256); gcap = 256; gn = 0; grec = 1; gnew = 0 - 1; + gbuf = rt.new(256); gcap = 256; gn = 0; grec = 1; gnew = 0 - 1; grec_add(); topitem0(); rb = gbuf; rn = gn; grec = sr; gbuf = sb; gn = sn; gcap = sc; - if (gnew >= 0) gattach(rb, rn); else free(rb); + if (gnew >= 0) gattach(rb, rn); else rt.free(rb); gnew = sg; gdrain(); + if (chk_hook && cmode && isp == 0 && !chkabort) chk_hook(); return 0; } // a unit is compiled item by item; unit_end then reports any function that // was called but never defined, and stops if any error was found int unit_top() { - while (tk != T_EOF) topitem(); + if (effect_floor < ubase) effect_floor = ubase; + while (tk != T_EOF) { + if (probe_off >= 0 && incdepth == 0 && toff > probe_off) { probe_n = 0; probe_has = 0; chkabort = 1; break; } + topitem(); + } return 0; } // the edit distance between two names (levenshtein), or 99 when one is too long word editdist(word a, word b) { - int la = len_w(a), lb = len_w(b), i, j, x, y, z, last, row[65]; + int la = rt.len_w(a), lb = rt.len_w(b), i, j, x, y, z, last, row[65]; if (la > 60 || lb > 60) return 99; j = 0; while (j <= lb) { row[j] = j; j++; } i = 1; @@ -7563,7 +9313,7 @@ word editdist(word a, word b) { // the defined function, global or constant whose name is closest to n, if it // is close enough to be a likely typo; 0 when none is int similar(int n) { - int j = 0, best = 0, bd = 99, dist, k, lim = len_w(n) / 3; + int j = 0, best = 0, bd = 99, dist, k, lim = rt.len_w(n) / 3; if (lim < 1) lim = 1; if (lim > 3) lim = 3; while (j < nsym) { @@ -7584,6 +9334,7 @@ int similar(int n) { // end a compile unit from token from onward, and report what it defined. int unit_end(int from) { int i = from, best; + effect_finish(from); panic = 0; while (i < nsym) { // a bodyless declaration (fn f();) promises a definition elsewhere. in check mode that @@ -7593,8 +9344,8 @@ int unit_end(int from) { best = similar(sname[i]); if (!dry) { errat(); - put_s(sname[i]); put_s(" is used but never defined"); - if (best) { put_s("; did you mean '"); put_s(best); put_s("'?"); } + rt.put_s(sname[i]); rt.put_s(" is used but never defined"); + if (best) { rt.put_s("; did you mean '"); rt.put_s(best); rt.put_s("'?"); } errend(); } } @@ -7609,9 +9360,9 @@ int unit_end(int from) { // (this is what eval, compile and load are built on. a function body made this // way is emitted behind a jump when another function is being compiled, so // compile() works inside a parse word.) -unchecked int compile_list(int q, int top) { +int compile_list_context(int q, int top, char* module, word imports) { int hop = -1, entry = 0, from = nsym; - ctx_push(); ifcur = 0; ifcall = -1; + ctx_push(); ifcur = 0; ifcall = -1; effect_floor = from; module_name = module; module_imports = imports; if (infn) { hop = hole(); op(O_JMP); } if (top) prescan(1, q); src_push(1, q, 0, 0); @@ -7620,7 +9371,7 @@ unchecked int compile_list(int q, int top) { else { entry = here; fn_begin(1, 0); - prologue(); + prologue(); borrow_prologue(); borrow_begin(); while (tk != T_EOF) stmt(); unused_check(lbase); if (fchk && live) exit_vars(1, 1); @@ -7633,6 +9384,50 @@ unchecked int compile_list(int q, int top) { ctx_pop(); return entry; } +int compile_list(int q, int top) { return compile_list_context(q, top, module_name, module_imports); } +// return an independent copy of a retained function body, excluding its braces. +word*.u function_tokens(int addr) { + int s = 0, open, close, count; word meta, source, result; + while (s < nsym && !(skind[s] == K_FN && sval[s] == addr && fnmeta[s])) s++; + if (s >= nsym) return 0; + meta = fnmeta[s]; source = meta[0]; open = meta[4]; close = meta[5]; count = close - open - 1; + result = rt.new(1 + count * 3); result[0] = count; + if (count) rt.copy_w(result + 1, source + 1 + (open + 1) * 3, count * 3); + return result; +} +// compile a changed body under a fresh name, check its signature, then redirect +// the original entry so existing calls and function values use the new code. +int replace_function(int addr, word* body_tokens) { + int s = 0, open, close, namepos, olderr = nerr, oldpanic = panic, oldabort = chkabort; + int errors, generated_id, new_symbol, count, i = 0, out = 0, cap, diag, seq, digits = 0, published; + char[40] generated; char[24] reversed; word meta, source, full, imports; char* module; + while (s < nsym && !(skind[s] == K_FN && sval[s] == addr && fnmeta[s])) s++; + if (s >= nsym || !body_tokens || body_tokens[0] < 0) return 0; + meta = fnmeta[s]; source = meta[0]; open = meta[4]; close = meta[5]; namepos = meta[3]; module = meta[1]; imports = meta[2]; + count = source[0] - (close - open - 1) + body_tokens[0]; + full = rt.new(1 + count * 3); full[0] = count; + i = 0; while (i <= open) { rt.copy_w(full + 1 + out * 3, source + 1 + i * 3, 3); i++; out++; } + i = 0; while (i < body_tokens[0]) { rt.copy_w(full + 1 + out * 3, body_tokens + 1 + i * 3, 3); i++; out++; } + i = close; while (i < source[0]) { rt.copy_w(full + 1 + out * 3, source + 1 + i * 3, 3); i++; out++; } + seq = fnpatchseq; fnpatchseq++; + rt.copy_w(generated, "__run_patch_", 12); generated[12] = 0; + if (!seq) { reversed[digits] = '0'; digits++; } + else { while (seq > 0) { reversed[digits] = '0' + seq % 10; digits++; seq = seq / 10; } } + i = 0; while (i < digits) { generated[12 + i] = reversed[digits - i - 1]; i++; } + generated[12 + digits] = 0; generated_id = intern(generated); full[2 + namepos * 3] = generated_id; + cap = 4096; diag = rt.new(cap); chk_begin(diag, cap); + compile_list_context(full, 1, module, imports); + errors = nerr || chkabort || panic; + chk_end(); nerr = olderr; panic = oldpanic; chkabort = oldabort; + if (module) new_symbol = lookup_raw(methname(module, generated_id)); else new_symbol = lookup_raw(generated_id); + if (!errors && new_symbol >= 0 && skind[new_symbol] == K_FN && (ssig[new_symbol] == ssig[s]) && (stype[new_symbol] == stype[s])) { + setlit(addr, sval[new_symbol]); *(addr + 1 + hdelta) = O_JMP + O_JMP; + // subsequent transformations start from the body that was published. + if (fnmeta[new_symbol]) fnmeta[s] = fnmeta[new_symbol]; + published = 1; + } else published = 0; + rt.free(diag); rt.free(full); return published; +} // compile a token list into a function and return its address. int compile(int q) { return compile_list(q, 0); } // compile a token list and run its top-level code. @@ -7640,13 +9435,14 @@ int load(int q) { compile_list(q, 1); return 0; } // compile a token list and run it, returning its result. int eval(int q) { int f = compile_list(q, 0); return f(); } // compile the source string s and run it, returning its result. -int evals(int s) { +int evals(char* s) { word q = tokens(s); int r = eval(q); - free(d*q); + rt.free(d*q); return r; } -// ---- symbol blob ---- +//: ---- symbol blob ---- + // an image ends with its own symbol table so the genome knows its names: // kind, value, signature, name chars, 0 ... then a 0 kind. // emit the names defined since symbol index from (functions, globals, @@ -7655,7 +9451,7 @@ int blob_emit(int from) { int i = from, table, words = 1; word p; int b, k; while (i < nsym) { if (skind[i] == K_DOC) { b = sval[i]; words = words + 2 + (b[0] >> 3); } - if ((skind[i] >= K_FN && skind[i] <= K_CONST) || skind[i] == K_ENH || skind[i] == K_DOC) words = words + 4 + len_w(sname[i]); + if ((skind[i] >= K_FN && skind[i] <= K_CONST) || skind[i] == K_ENH || skind[i] == K_DOC) words = words + 4 + rt.len_w(sname[i]); i++; } reserve_code(words); i = from; @@ -7693,19 +9489,19 @@ int blob_emit(int from) { // cut is by name, the functions after chk_mark, and everything from the // server's marker const (LSP_SERVER, the first thing src/lsp/server.zc declares) int blob_load(word p, int hide) { - int s, n, past = 0, after = 0, skip, mkn = intern("chk_mark"), srvn = intern("LSP_SERVER"); + int s, n, past = 0, after = 0, skip, mkn = intern("run.chk_mark"), srvn = intern("lsp.LSP_SERVER"); while (*p) { n = intern(p + 3); if (hide && n == srvn) after = 1; - skip = hide && (after || (past && p[0] == K_FN)); + skip = 0; if (hide && n == mkn) past = 1; if (!skip) { s = addsym(n, p[0], p[1]); ssig[s] = p[2]; sflag[s] = F_DEF | F_SIG; - if (p[0] == K_ENH) { if (p[1] >= nenh) nenh = p[1] + 1; install(p[2] >> 8); } + if (p[0] == K_ENH) { if (p[1] >= nenh) nenh = p[1] + 1; rt.install(p[2] >> 8); } } - p = p + 3 + len_w(p + 3) + 1; + p = p + 3 + rt.len_w(p + 3) + 1; } return 0; } @@ -7720,7 +9516,7 @@ int build_image_grow(int org, int initial) { image_buffer = chk_resize(0, 0, initial); image_active = 1; image_org = org; image_capacity = initial; ubase = nsym; vislo = nsym; nenh = 0; here = org; hdelta = image_buffer - org; hlimit = org + initial; - lit(FTOP); lit(ZFP); op(O_ST); fnref(fwdfn(s_start)); op(O_JMP); emit(0); emit(0); + lit(rt.FTOP); lit(rt.ZFP); op(O_ST); fnref(fwdfn(s_start)); op(O_JMP); emit(0); emit(0); unit_stdin(); n = blob_emit(ubase); image_buffer[5] = n; @@ -7734,28 +9530,29 @@ int build_image_grow(int org, int initial) { // retain the explicit destination interface for callers that already own storage. // use build_image_grow and image_words when the output must grow independently. int build_image(int org, int dst) { - int n = build_image_grow(org, 4096), words = syscall(9, 0, 0, 0); + int n = build_image_grow(org, 4096), words = rt.syscall(9, 0, 0, 0); if (dst < 0 || n > words - dst) fatal("image destination too small"); - copy_w(dst, image_words, n); + rt.copy_w(dst, image_words, n); return n; } -// ---- setup ---- +//: ---- setup ---- + // the collector's roots for the compiler's half of the memory map: the program // area (globals live between functions), the genome's globals, the input source // frames, the live frame stack and the live data stack. rt.zc declares this // and calls it from gc(). int gc_roots() { - int end = ptop; if (end > CODE_END) end = CODE_END; - markrange(0, end); - if (!hdelta && here > ptop && ptop < CODE_END) { end = here; if (end > CODE_END) end = CODE_END; markrange(ptop, end); } // globals of a unit still compiling - markrange(GORG, GORG[6]); - markrange(R_IH, R_POOLEND); - markrange(sval, sval + nsym); - markrange(R_TY, R_TYEND); - markrange(R_SRC, R_SRC + R_SRC_cap); - gc_frames(); - markrange(dsp(), data_top); + int end = ptop; if (end > rt.CODE_END) end = rt.CODE_END; + rt.markrange(0, end); + if (!hdelta && here > ptop && ptop < rt.CODE_END) { end = here; if (end > rt.CODE_END) end = rt.CODE_END; rt.markrange(ptop, end); } // globals of a unit still compiling + rt.markrange(rt.GORG, rt.GORG[6]); + rt.markrange(R_IH, R_POOLEND); + rt.markrange(sval, sval + nsym); + rt.markrange(R_TY, R_TYEND); + rt.markrange(R_SRC, R_SRC + R_SRC_cap); + rt.gc_frames(); + rt.markrange(rt.dsp(), rt.data_top); return 0; } // start-up in two parts. zc_init clears the intern table and interns the @@ -7764,11 +9561,13 @@ int gc_roots() { // the vm's own one-letter mnemonics (see seqasm), so "<=" is "~ < 1 ^": swap, // less-than, flip the bit. int zc_init() { + receiver_methods = 0; closure_expected = 0; + effect_floor = 0; effect_symbols = 0; effect_edges = 0; effect_requirements = 0; effect_current = 0; pooltop = R_POOL; poolend = R_POOLEND; poolblocks = 0; namecount = 0; - fill_w(namehash, 0, hashcap); + rt.fill_w(namehash, 0, hashcap); nsym = 0; nfixu = 0; fixfree = 0; nbin = 0; nun = 0; nasg = 0; nkop = 0; seqtop = 0; - lsp = 0; ctxsp = 0; snaptop = 0; nloc = 0; lbase = 0; npm = 0; npe = 0; fill_w(lkeys, 0, lkeycap); lkeyn = 0; + lsp = 0; ctxsp = 0; snaptop = 0; nloc = 0; lbase = 0; npm = 0; npe = 0; rt.fill_w(lkeys, 0, lkeycap); lkeyn = 0; s_lp = intern("("); s_rp = intern(")"); s_lb = intern("{"); s_rb = intern("}"); s_lk = intern("["); s_rk = intern("]"); s_comma = intern(","); s_semi = intern(";"); s_eq = intern("="); s_arrow = intern("->"); s_inc = intern("++"); s_dec = intern("--"); @@ -7787,20 +9586,20 @@ int zc_init() { s_fn = intern("fn"); s_var = intern("let"); s_const = intern("const"); s_else = intern("else"); s_as = intern("as"); s_unchecked = intern("unchecked"); s_immediate = intern("immediate"); s_m = intern("m"); s_d = intern("d"); s_u = intern("u"); s_static = intern("static"); - s_start = intern("start"); + s_start = intern("run.start"); s_main = intern("main"); - s_free = intern("free"); - s_recycle = intern("recycle"); + s_free = intern("rt.free"); + s_recycle = intern("rt.recycle"); s_do = intern("do"); s_while = intern("while"); s_enum = intern("enum"); s_include = intern("include"); - s_memdebug = intern("memdebug"); s_scrub = intern("scrub"); s_alloc = intern("alloc_line"); - s_check = intern("checkptr"); s_memset = intern("mem_set"); - s_own = intern("own"); s_qnew = intern("qnew"); s_fovf = intern("frame_overflow"); - s_frame_floor = intern("frame_floor"); s_frame_top = intern("frame_top"); s_frame_node = intern("frame_node"); - s_frame_busy = intern("frame_busy"); s_frame_guard = intern("frame_guard"); - s_frame_grow = intern("frame_grow"); s_frame_leave = intern("frame_leave"); + s_memdebug = intern("memdebug"); s_scrub = intern("rt.scrub"); s_alloc = intern("rt.alloc_line"); + s_check = intern("rt.checkptr"); s_memset = intern("rt.mem_set"); + s_own = intern("rt.own"); s_qnew = intern("run.qnew"); s_fovf = intern("rt.frame_overflow"); + s_frame_floor = intern("rt.frame_floor"); s_frame_top = intern("rt.frame_top"); s_frame_node = intern("rt.frame_node"); + s_frame_busy = intern("rt.frame_busy"); s_frame_guard = intern("rt.frame_guard"); + s_frame_grow = intern("rt.frame_grow"); s_frame_leave = intern("rt.frame_leave"); s_enhance = intern("enhance"); s_micro = intern("micro"); s_vm = intern("vm"); return 0; } @@ -7838,8 +9637,13 @@ int zc_tables() { add_kw("for", kw_for, F_STMT); add_kw("do", kw_do, F_STMT); add_kw("enum", kw_enum, F_STMT | F_TOP); - add_kw("include", kw_include, F_TOP); + add_kw("include", kw_include, F_TOP | F_STMT); + add_kw("run.method", kw_method, F_TOP | F_STMT); + add_kw("run.share_value", kw_share_value, F_EXPR); + add_kw("run.syntax", kw_syntax, F_TOP | F_STMT); + add_kw("run.statement_hook", kw_statement_hook, F_TOP | F_STMT); add_kw("memdebug", kw_memdebug, F_STMT | F_TOP); + add_kw("hints", kw_hints, F_STMT | F_TOP); add_kw("switch", kw_switch, F_STMT); add_kw("break", kw_break, F_STMT); add_kw("continue", kw_continue, F_STMT); @@ -7859,33 +9663,33 @@ int zc_tables() { add_kw("resize", kw_resize, F_EXPR); return 0; } -// the default map's type records sit in the last 16384 words of the heap. -// the type list is the first words of that range, so it cannot be kept aside -// as a buddy block: types_init would overwrite the header and the allocator -// would hand the range out again. cut the heap off at R_TY until the unit -// has been compiled, then put the tail back for the program. +//: the default map's type records sit in the last 16384 words of the heap. +//: the type list is the first words of that range, so it cannot be kept aside +//: as a buddy block: types_init would overwrite the header and the allocator +//: would hand the range out again. cut the heap off at R_TY until the unit +//: has been compiled, then put the tail back for the program. int heldk; // the size class of the type area, for the memory map. int type_class() { int k = 0, n = R_TYEND - R_TY; - while (k < 31 && pow2[k] < n) k++; - if (pow2[k] != n) return 0 - 1; + while (k < 31 && rt.pow2[k] < n) k++; + if (rt.pow2[k] != n) return 0 - 1; return k; } // keep the type area out of the collector's reach while it is in use. int shield_types() { int k; heldk = 0 - 1; - if (R_TY <= HEAP_LO || R_TY >= HEAP_HI || R_TYEND > HEAP_HI) return 0; + if (R_TY <= rt.HEAP_LO || R_TY >= rt.HEAP_HI || R_TYEND > rt.HEAP_HI) return 0; k = type_class(); if (k < 0) return 0; - if (R_TY + pow2[k] != R_TYEND) return 0; - if (heap_reset(R_TY)) heldk = k; + if (R_TY + rt.pow2[k] != R_TYEND) return 0; + if (rt.heap_reset(R_TY)) heldk = k; return 0; } // let the collector see the type area again. int unshield_types() { - if (heldk >= 0) { heap_uncut(R_TY, heldk); heldk = 0 - 1; } + if (heldk >= 0) { rt.heap_uncut(R_TY, heldk); heldk = 0 - 1; } return 0; } // the genome's entry point, run when the image boots: set up the heap and @@ -7894,35 +9698,36 @@ int unshield_types() { // call main(). int start() { int s, f; - heap_init(); chk_heap = 1; + rt.heap_init(); chk_heap = 1; shield_types(); zc_init(); - blob_load(GORG[5], 0); + blob_load(rt.GORG[5], 0); zc_tables(); types_init(); - here = 0; hdelta = 0; hlimit = CODE_END; ptop = 0; + here = 0; hdelta = 0; hlimit = rt.CODE_END; ptop = 0; ubase = nsym; vislo = 0; // a genome that carries the language server runs it instead of compiling stdin - s = lookup(intern("lsp_main")); - if (s >= 0 && skind[s] == K_FN && (sflag[s] & F_DEF)) { f = sval[s]; f(); halt(); } + s = lookup(intern("lsp.lsp_main")); + if (s >= 0 && skind[s] == K_FN && (sflag[s] & F_DEF)) { f = sval[s]; f(); rt.halt(); } unit_stdin(); if (here > ptop) ptop = here; - scrub(); // compilation's stale words would keep quarantined blocks alive - gc(); + rt.scrub(); // compilation's stale words would keep quarantined blocks alive + rt.gc(); // the type area stays out of the allocator. main() may compile again // (the self-host does) and a later new() would otherwise reuse it. - if (mdbg_flags) mem_set(mdbg_flags); + if (mdbg_flags) rt.mem_set(mdbg_flags); // a unit that carries src/lib/std.zc has a libinit to fill argc/argv/envc/envp // with. it runs before main, the same way lsp_main runs instead of stdin. - s = lookup(intern("libinit")); + s = lookup(intern("std.libinit")); if (s >= 0 && skind[s] == K_FN && (sflag[s] & F_DEF)) { f = sval[s]; f(); } s = lookup(s_main); if (s >= 0 && skind[s] == K_FN && (sflag[s] & F_DEF)) { f = sval[s]; f(); } - halt(); + rt.halt(); return 0; } -// ---- check mode: the language server ---- +//: ---- check mode: the language server ---- + // the server runs this compiler, unchanged, over a document it holds in // memory. chk_unit compiles it as a whole unit, in a machine set up the way // start() sets one up (the genome's own names, the default operators and @@ -7935,9 +9740,11 @@ int start() { int chk_mark() { return 0; } // reset check mode for a new document. int chk_reset() { - code_blocks = 0; code_chunk = CODE_END; + statement_hooks = 0; included_modules = 0; + if_local = 0; if_edges = 0; if_value = 0; + rt.code_blocks = 0; code_chunk = rt.CODE_END; isp = -1; - docs_on = 0; dn = 0; dline = 0; docpending = 0; dsn = 0; dnm = 0; + docs_on = 1; dn = 0; dline = 0; docpending = 0; dsect = 0; dsn = 0; dnm = 0; pq_n = 0; pendot = 0; qdone = 0; @@ -7958,7 +9765,8 @@ int chk_reset() { bxdepth = 0; epichain = 0; fframe = 0; - xbor = 0 - 1; xaf = 0; argfact = 0; + xbor = 0 - 1; xaf = 0; argfact = 0; xvk = 0; + if (svkind) rt.fill_w(svkind, 0, symcap); mdbg = 0; mdbg_flags = 0; lv = 0; lvk = 0; lvi = 0; kw_expr = 0; xcls = 0; fty = 0; check_on = 1; tc_on = ifstrict; dry = 0; ntp = 0; tvauto = 0; ngbind = 0; xgive = 0; xconst = 0; o3on = 0; o3nxt = 0; no3pend = 0; no3sl = 0; no3free = 0; o3nreg = 0; o3na = 0; o3fn0 = 0; @@ -7974,20 +9782,22 @@ int chk_reset() { // check one document in check mode and fill the records it made. int chk_unit(int src, int buf, int cap, int sets, int lim) { chk_reset(); + module_name = 0; module_imports = 0; chk_begin(buf, cap); lsets = sets; - if (sets) fill_w(sets, 0, 4 * LSW); - if (!chk_heap) { heap_init(); chk_heap = 1; } else gc(); // the caller's blocks are reachable from its globals, the rest goes - rsn = 0; rsbase = src; - if (!rsrec) { rscap = RS_CAP; rsrec = new(RS_CAP * RS_W); rstmp = new(RS_CAP * RS_W); } + if (sets) rt.fill_w(sets, 0, 4 * LSW); + if (!chk_heap) { rt.heap_init(); chk_heap = 1; } else rt.gc(); // the caller's blocks are reachable from its globals, the rest goes + rsn = 0; evn = 0; hl_n = 0; literal_doc_n = 0; if (literal_docs) rt.fill_w(literal_docs, 0, literal_doc_cap); curinc = 0; std_done = 0; icon_done = 0; incdepth = 0; dgdrop = 0; probe_has = 0; probe_n = 0; probe_mem = 0; probe_static = 0; nameoff = 0 - 1; rsbase = src; + if (!rsrec) { rscap = RS_CAP; rsrec = rt.new(RS_CAP * RS_W); rstmp = rt.new(RS_CAP * RS_W); } ifs = 0; ifx = 0; nifx = 0; nift = 0; nif = 0; ifcur = 0; ifkk = 0; ifgk = 0; ifcall = 0 - 1; if (ifon) { ifrcap = NIF; ifxcap = IFN; iftcap = IFT; - ifs = new(symcap); ifr = chk_resize(0, 0, ifrcap); ifx = chk_resize(0, 0, ifxcap * 4); ift = chk_resize(0, 0, iftcap); - fill_w(ifs, 0, symcap); + ifs = rt.new(symcap); ifr = chk_resize(0, 0, ifrcap); ifx = chk_resize(0, 0, ifxcap * 4); ift = chk_resize(0, 0, iftcap); + if_local = chk_resize(0, 0, loccap); if_edges = 0; if_value = 0; + rt.fill_w(ifs, 0, symcap); } zc_init(); - blob_load(GORG[5], 1); + blob_load(rt.GORG[5], 1); zc_tables(); types_init(); here = 0; hdelta = 0; hlimit = lim; ptop = 0; @@ -8025,3 +9835,31 @@ int hl_end() { pq_n = 0; return 0; } +// ---- embedded std library (generated by src/tools/embed.sh from src/lib/std.zc; do not edit) ---- +bstr std_part(int i) { + if (i == 0) return b"namespace std;\ninclude rt, run;\n\n// small helpers for strings, files, the clock and byte buffers. not part of the\n// genome as declarations; its source is embedded. include it with: include std;\n// syscall numbers (SYS_OPEN and the rest) live in the runtime.\n// a character string has type char*, one character per word and a 0 at the end.\n// the Bytes group takes packed byte strings (b\"...\"), whose\n// type is bstr.\n// the helpers are grouped in structs, and each is a static function, so it is\n// called as std.Group.name(args): std.Text.itoa(n, buf), std.Host.now(),\n// std.Buf.get(b, 0).\n// the constructors arena(n) and bnew(n) and the argv helpers are plain\n// functions, because an owned result has to carry its .u tag.\n\n//: ---- the command line and the environment ----\n\n//: argc and argv, envc and envp, as passed to main by c. a zc string is\n//: one character per word, so an array of them is char*[?]. libinit() fills\n//: these and the runtime calls it before main, if the program has one (an\n//: image built by build_image, or a check-mode document, does not).\n\nint argc;\nchar*[?] argv;\nint envc;\nchar*[?] envp;\n\n// one entry of a vector syscall, as a fresh string, or 0 past the end.\n// op: the syscall that fetches an entry (SYS_ARG or SYS_ENV)\n// i: which entry\n// returns: the entry, owned by the caller; 0 when there is no entry i\nchar*.u argstr(int op, int i) {\n\tint n = rt.syscall(op, i, 0, 0);\n\tchar* s;\n\tif (n < 0) return 0;\n\ts = rt.new(n + 1);\n\trt.syscall(op, i, s, n + 1);\n\treturn s;\n}\n\n// every entry of a vector syscall, in order.\n// cntop: the syscall that counts the entries (SYS_ARGC or SYS_ENVC)\n// stop: the syscall that fetches one (SYS_ARG or SYS_ENV)\n// returns: a new array of the strings, owned by the caller\nchar*[?].u argsvec(int cntop, int stop) {\n\tint n = rt.syscall(cntop, 0, 0, 0), i = 0;\n\tchar*[?] a = rt.new(char*[?], 0);\n\twhile (i < n) { push(a, argstr(stop, i)); i++; }\n\treturn a;\n}\n\n// reads the command line and the environment into argc, argv, envc and envp.\n// it stays a plain function: the compiler looks it up by name.\n// returns: 0\nint libinit() {\n\targv = argsvec(rt.SYS_ARGC, rt.SYS_ARG);\n\tenvp = argsvec(rt.SYS_ENVC, rt.SYS_ENV);\n\targc = len(argv); envc = len(envp);\n\treturn 0;\n}\n\n//: ---- capturing output ----\n\n// capture.start() redirects everything the program writes into a string and\n// returns it. writes go to the buffer instead of the terminal until end()\n// stops it, so the buffer is a string once end() has run:\n// let buf = capture.start();\n// put_s(\"hello world!\"); // nothing on the terminal\n// capture.end(buf);\n// put_s(buf); // hello world\n// one capture at a time, and the buffer belongs to whoever holds it.\n\nstruct capture {\n\t// redirects program output into a collector-managed character buffer.\n\t// returns the buffer; call end(buffer) before reading its terminator.\n\t// returns: the buffer used for this capture\n\tstatic char* start() {\n\t\tint cap = 256;\n\t\trt.outmode = 2; rt.outn ="; + if (i == 1) return b" 0; rt.outcap = cap;\n\t\trt.outbuf = rt.new(cap);\n\t\treturn rt.outbuf;\n\t}\n\t// stops output capture, terminates the buffer, and returns the collected text.\n\t// buffer: the buffer returned by start\n\t// returns: the same buffer, now terminated\n\tstatic char* end(char* buffer) {\n\t\trt.outbuf[rt.outn] = 0;\n\t\trt.outmode = 0; rt.outbuf = 0; rt.outn = 0; rt.outcap = 0;\n\t\treturn buffer;\n\t}\n}\n\n//: ---- character strings ----\n\nstruct Text {\n\t// the number written in decimal at the start of s. a leading '-' makes it\n\t// negative and the first character that is not a digit ends it.\n\t// s: the string to read\n\t// returns: the number, 0 when s starts with no digit\n\tstatic int atoi(char* s) {\n\t\tint n = 0, i = 0, sign = 1;\n\t\tif (s[0] == '-') { sign = 0 - 1; i = 1; }\n\t\twhile (s[i] >= '0' && s[i] <= '9') { n = n * 10 + s[i] - '0'; i++; }\n\t\treturn n * sign;\n\t}\n\n\t// writes n in decimal into dst, with a 0 after it.\n\t// n: the number\n\t// dst: where the text goes; 24 words is always enough\n\t// returns: how many characters were written, not counting the 0\n\tstatic int itoa(int n, char* dst) {\n\t\tint b[24], i = 0, j = 0, neg = 0;\n\t\tif (!n) { dst[0] = '0'; dst[1] = 0; return 1; }\n\t\tif (n < 0) { neg = 1; n = 0 - n; }\n\t\tif (n < 0) { rt.copy_w(dst, \"-9223372036854775808\", 21); return 20; }\n\t\twhile (n) { b[i] = '0' + n % 10; i++; n = n / 10; }\n\t\tif (neg) { dst[0] = '-'; j = 1; }\n\t\twhile (i) { i--; dst[j] = b[i]; j++; }\n\t\tdst[j] = 0;\n\t\treturn j;\n\t}\n\n\t// joins two strings.\n\t// dst: where the result goes, with room for both and a 0; not a or b\n\t// a: the first string\n\t// b: the second string\n\t// returns: the length of the result\n\tstatic int cat(char* dst, char* a, char* b) {\n\t\tint i = 0, j = 0;\n\t\twhile (a[i]) { dst[j] = a[i]; i++; j++; }\n\t\ti = 0;\n\t\twhile (b[i]) { dst[j] = b[i]; i++; j++; }\n\t\tdst[j] = 0;\n\t\treturn j;\n\t}\n\n\t// does s begin with pfx?\n\t// s: the string to test\n\t// pfx: the beginning to look for\n\t// returns: 1 if it does (always, for an empty pfx), 0 if not\n\tstatic int starts(char* s, char* pfx) {\n\t\tint i = 0;\n\t\twhile (pfx[i]) { if (s[i] != pfx[i]) return 0; i++; }\n\t\treturn 1;\n\t}\n\n\t// does s end with suffix? an empty suffix always matches.\n\t// s: the string to test\n\t// suffix: the ending to look for\n\t// returns: 1 when it matches, 0 otherwise\n\tstatic int ends(char* s, char* suffix) {\n\t\tint n = 0, m = 0, i = 0;\n\t\twhile (s[n]) n++;\n\t\twhile (suffix[m]) m++;\n\t\tif (m > n) return 0;\n\t\twhile (i < m) { if (s[n - m + i] != suffix[i]) return 0; i++; }\n\t\treturn 1;\n\t}\n\n\t// how many characters come before the 0.\n\t// s: the string\n\t// returns: the length, not counting the 0\n\tstatic int len(char* s) {\n\t\tint n = 0;\n\t\twhile (s[n]) n++;\n\t\treturn n;\n\t}\n\n\t// are the two strings the same, character for character?\n\t// a, b: the strings to compare\n\t// returns: 1 if they are equal, 0 if not\n\tstatic int eq(char* a, char* b) {\n\t\tint i = 0;\n\t\twhile (a[i] && a[i] == b[i]) i++;\n\t\treturn a[i] == b[i];\n\t}\n\n\t// copies s into dst, with a 0 after it.\n\t// dst: where th"; + if (i == 2) return b"e copy goes, with room for s and the 0\n\t// s: the string to copy\n\t// returns: the length of the copy\n\tstatic int copy(char* dst, char* s) {\n\t\tint i = 0;\n\t\twhile (s[i]) { dst[i] = s[i]; i++; }\n\t\tdst[i] = 0;\n\t\treturn i;\n\t}\n\n\t// where pat first appears in s.\n\t// s: the string to search\n\t// pat: the string to look for; an empty pat is found at 0\n\t// returns: the index of the first match, or -1 when there is none\n\tstatic int find(char* s, char* pat) {\n\t\tint i = 0, j;\n\t\tif (!pat[0]) return 0;\n\t\twhile (s[i]) {\n\t\t\tj = 0;\n\t\t\twhile (pat[j] && s[i + j] == pat[j]) j++;\n\t\t\tif (!pat[j]) return i;\n\t\t\ti++;\n\t\t}\n\t\treturn 0 - 1;\n\t}\n\n\t// changes the ASCII letters of s to lower case, in place.\n\t// s: the string to change\n\t// returns: the length of s\n\tstatic int lower(char* s) {\n\t\tint i = 0;\n\t\twhile (s[i]) {\n\t\t\tif (s[i] >= 'A' && s[i] <= 'Z') s[i] = s[i] + 32;\n\t\t\ti++;\n\t\t}\n\t\treturn i;\n\t}\n\n\t// changes the ASCII letters of s to upper case, in place.\n\t// s: the string to change\n\t// returns: the length of s\n\tstatic int upper(char* s) {\n\t\tint i = 0;\n\t\twhile (s[i]) {\n\t\t\tif (s[i] >= 'a' && s[i] <= 'z') s[i] = s[i] - 32;\n\t\t\ti++;\n\t\t}\n\t\treturn i;\n\t}\n\n\t// removes the spaces, tabs and line breaks from both ends of s, in place.\n\t// s: the string to change\n\t// returns: the length of what is left\n\tstatic int trim(char* s) {\n\t\tint n = 0, a = 0, b, i = 0, c;\n\t\twhile (s[n]) n++;\n\t\twhile (a < n && (s[a] == ' ' || s[a] == '\\t' || s[a] == '\\n' || s[a] == '\\r')) a++;\n\t\tb = n;\n\t\twhile (b > a) {\n\t\t\tc = s[b - 1];\n\t\t\tif (!(c == ' ' || c == '\\t' || c == '\\n' || c == '\\r')) break;\n\t\t\tb--;\n\t\t}\n\t\tn = b - a;\n\t\twhile (i < n) { s[i] = s[i + a]; i++; }\n\t\ts[n] = 0;\n\t\treturn n;\n\t}\n}\n\n//: ---- numbers ----\n\n// plain integer helpers. none of them can fail: an out of range result wraps\n// the same way the arithmetic does.\n\nstruct Num {\n\t// the absolute value of n, using wrapping word arithmetic. the minimum\n\t// word remains negative because its positive value cannot fit in a word.\n\t// n: the value\n\t// returns: its absolute value, or the minimum word\n\tstatic int abs(int n) {\n\t\tif (n < 0) return 0 - n;\n\t\treturn n;\n\t}\n\n\t// the smaller of a and b.\n\t// a, b: the values to compare\n\t// returns: the smaller value (a when they are equal)\n\tstatic int min(int a, int b) {\n\t\tif (a < b) return a;\n\t\treturn b;\n\t}\n\n\t// the larger of a and b.\n\t// a, b: the values to compare\n\t// returns: the larger value (a when they are equal)\n\tstatic int max(int a, int b) {\n\t\tif (a > b) return a;\n\t\treturn b;\n\t}\n\n\t// v, kept between lo and hi.\n\t// v: the value\n\t// lo, hi: the bounds, lo first; lo must not exceed hi\n\t// returns: v clamped to the bounds\n\tstatic int clamp(int v, int lo, int hi) {\n\t\tif (v < lo) return lo;\n\t\tif (v > hi) return hi;\n\t\treturn v;\n\t}\n\n\t// the greatest common divisor of a and b, computed with word arithmetic.\n\t// a, b: the values\n\t// returns: their greatest common divisor\n\tstatic int gcd(int a, int b) {\n\t\tif (a < 0) a = 0 - a;\n\t\tif (b < 0) b = 0 - b;\n\t\tint t;\n\t\twhile (b) { t "; + if (i == 3) return b"= a % b; a = b; b = t; }\n\t\treturn a;\n\t}\n\n\t// b raised to the power e. e below 0 gives 1; the result wraps as a word.\n\t// b: the base\n\t// e: the exponent\n\t// returns: b to the power e, or 1 when e is negative\n\tstatic int pow(int b, int e) {\n\t\tint r = 1;\n\t\twhile (e > 0) { r = r * b; e--; }\n\t\treturn r;\n\t}\n\n\t// the whole square root of n, rounded down. n below 0 gives 0.\n\t// n: the value\n\t// returns: the greatest integer whose square is at most n, or 0 for n <= 0\n\tstatic int isqrt(int n) {\n\t\tint lo = 0, hi = 3037000499, mid;\n\t\tif (n <= 0) return 0;\n\t\twhile (lo < hi) {\n\t\t\tmid = lo + (hi - lo + 1) / 2;\n\t\t\tif (mid <= n / mid) lo = mid; else hi = mid - 1;\n\t\t}\n\t\treturn lo;\n\t}\n\n\t// -1, 0 or 1 according to the sign of n.\n\t// n: the value\n\t// returns: -1 when n is negative, 0 when it is zero, and 1 when it is positive\n\tstatic int sign(int n) {\n\t\tif (n < 0) return 0 - 1;\n\t\tif (n > 0) return 1;\n\t\treturn 0;\n\t}\n}\n\n//: ---- the host ----\n\n//: the argument block for Host.raw: the number, then its six arguments.\nint rawblk[7];\n\nstruct Host {\n\t// calls any linux system call on the host. this is the escape hatch below\n\t// the named helpers: the number is the kernel's, so the meaning of each\n\t// argument depends on the call. a pointer argument is an address in the\n\t// program's memory, and only the host reads it, so a string or buffer has\n\t// to be passed as its address.\n\t// it runs the call on the real machine, not inside the vm, so a program\n\t// that uses it can do whatever the process can do. only use it where that\n\t// is wanted.\n\t// nr: the syscall number (x86-64 and arm64 numbers differ, so check the host)\n\t// a..f: the six arguments, unused ones 0\n\t// returns: the kernel's result, -errno on failure, or -1 on unsupported hosts\n\tstatic int raw(int nr, int a, int b, int c, int d, int e, int f) {\n\t\trawblk[0] = nr; rawblk[1] = a; rawblk[2] = b; rawblk[3] = c;\n\t\trawblk[4] = d; rawblk[5] = e; rawblk[6] = f;\n\t\treturn rt.syscall(rt.SYS_RAW, rawblk, 0, 0);\n\t}\n\n\t// the value of an environment variable.\n\t// name: the variable's name\n\t// dst: where the value goes, a zero-terminated character string\n\t// n: how many words dst has\n\t// returns: the length of the value, -1 if the variable is not set\n\tstatic int env(char* name, char* dst, int n) { return rt.syscall(rt.SYS_GETENV, name, dst, n); }\n\n\t// the clock.\n\t// returns: the time in seconds since 1970\n\tstatic int now() { return rt.syscall(rt.SYS_TIME, 0, 0, 0); }\n\n\t// writes a string to a file, replacing what was there.\n\t// path: the file's name\n\t// s: the text to write\n\t// returns: how many bytes were written, -1 if the file could not be opened\n\tstatic int writefile(char* path, char* s) {\n\t\tint fd = rt.syscall(rt.SYS_OPEN, path, 1, 0), n;\n\t\tif (fd < 0) return -1;\n\t\tn = rt.syscall(rt.SYS_WRITE, fd, s, rt.len_w(s));\n\t\trt.syscall(rt.SYS_CLOSE, fd, 0, 0);\n\t\treturn n;\n\t}\n}\n\n//: ---- byte strings ----\n\n// the same helpers for packed byte strings. character-string helpers above are unchanged.\n// a"; + if (i == 4) return b" packed byte string has type bstr; len_s reads its stored byte length.\n\nstruct Bytes {\n\t// reads a decimal number at the start of s; a leading '-' makes it\n\t// negative, and the first non-digit ends the number.\n\t// s: the byte string\n\t// returns: the number, or 0 when s starts with no digit\n\tstatic int atoi(bstr s) { return rt.s_to_n(s); }\n\n\t// like Text.itoa, but the text is made fresh.\n\t// n: the number\n\t// returns: a new byte string\n\tstatic bstr itoa(int n) { return rt.n_to_s(n); }\n\n\t// like Text.starts, for byte strings.\n\t// s: the byte string to test\n\t// pfx: the beginning to look for\n\t// returns: 1 if s begins with pfx, 0 if not\n\tstatic int starts(bstr s, bstr pfx) {\n\t\tint i = 0, n = rt.len_s(s), m = rt.len_s(pfx);\n\t\tif (m > n) return 0;\n\t\twhile (i < m) { if (rt.at_s(s, i) != rt.at_s(pfx, i)) return 0; i++; }\n\t\treturn 1;\n\t}\n\n\t// the number of bytes in a packed byte string.\n\t// s: the byte string\n\t// returns: its stored length\n\tstatic int len(bstr s) { return rt.len_s(s); }\n\n\t// are two packed byte strings equal, byte for byte.\n\t// a, b: the byte strings to compare\n\t// returns: 1 if equal, 0 otherwise\n\tstatic int eq(bstr a, bstr b) {\n\t\tint i = 0;\n\t\tif (rt.len_s(a) != rt.len_s(b)) return 0;\n\t\twhile (i < rt.len_s(a)) { if (rt.at_s(a, i) != rt.at_s(b, i)) return 0; i++; }\n\t\treturn 1;\n\t}\n\n\t// where pat first appears in s.\n\t// s: the byte string to search\n\t// pat: the byte string to find; an empty pat is found at 0\n\t// returns: its byte index, or -1 when absent\n\tstatic int find(bstr s, bstr pat) {\n\t\tint i = 0, j, n = rt.len_s(s), m = rt.len_s(pat);\n\t\tif (!m) return 0;\n\t\twhile (i + m <= n) {\n\t\t\tj = 0;\n\t\t\twhile (j < m && rt.at_s(s, i + j) == rt.at_s(pat, j)) j++;\n\t\t\tif (j == m) return i;\n\t\t\ti++;\n\t\t}\n\t\treturn 0 - 1;\n\t}\n\n\t// does s end with suffix? an empty suffix always matches.\n\t// s: the byte string to test\n\t// suffix: the ending to look for\n\t// returns: 1 when it matches, 0 otherwise\n\tstatic int ends(bstr s, bstr suffix) {\n\t\tint n = rt.len_s(s), m = rt.len_s(suffix), i = 0;\n\t\tif (m > n) return 0;\n\t\twhile (i < m) { if (rt.at_s(s, n - m + i) != rt.at_s(suffix, i)) return 0; i++; }\n\t\treturn 1;\n\t}\n\n\t// like Host.env, for byte strings.\n\t// name: the variable's name, a byte string\n\t// dst: the packed byte string whose payload receives the value\n\t// n: the maximum number of bytes to write\n\t// returns: the length of the value, -1 if the variable is not set\n\tstatic int env(bstr name, bstr dst, int n) { return rt.bgetenv(name, dst, n); }\n\n\t// like Host.writefile, for a byte string.\n\t// path: the file's name, a byte string\n\t// s: the bytes to write\n\t// returns: how many bytes were written, -1 if the file could not be opened\n\tstatic int writefile(bstr path, bstr s) {\n\t\tint fd = rt.bopen(path, 1), n;\n\t\tif (fd < 0) return -1;\n\t\tn = rt.bwrite(fd, s, rt.len_s(s));\n\t\trt.syscall(rt.SYS_CLOSE, fd, 0, 0);\n\t\treturn n;\n\t}\n}\n\n//: ---- arenas ----\n\n// a region that is freed all at once.\n// n: its capacity in words; value"; + if (i == 5) return b"s below 1 allocate one word\n// returns: the arena, owned by the caller\nint .u arena(int n) { return rt.arena_new(n); }\n\nstruct Arena {\n\t// takes n zeroed words from an arena.\n\t// a: the arena\n\t// n: how many words\n\t// returns: the first word, or 0 when n words do not fit\n\tstatic int alloc(word a, int n) { return rt.arena_alloc(a, n); }\n\n\t// gives every block of an arena back at once. pointers returned by alloc\n\t// before the reset are no longer valid.\n\t// a: the arena\n\t// returns: 0\n\tstatic int reset(word a) { return rt.arena_reset(a); }\n}\n\n//: ---- growable byte buffers ----\n\n// a byte buffer stores its length and capacity in two words, followed by the\n// bytes, packed.\n// n: the capacity in bytes; a negative value is treated as 0\n// returns: an empty buffer with that capacity, owned by the caller\nword .u bnew(int n) {\n\tint words;\n\tword p;\n\tif (n < 0) n = 0;\n\twords = (n + 7) >> 3;\n\tp = rt.new(2 + words);\n\tp[0] = 0;\n\tp[1] = n;\n\treturn u*p;\n}\n\nstruct Buf {\n\t// how many bytes a buffer holds.\n\t// b: the buffer\n\t// returns: its length\n\tstatic int length(word b) { return b[0]; }\n\n\t// how many bytes a buffer can hold.\n\t// b: the buffer\n\t// returns: its capacity\n\tstatic int capacity(word b) { return b[1]; }\n\n\t// one byte of a buffer.\n\t// b: the buffer\n\t// i: the position\n\t// returns: the byte, 0 when i is outside the buffer\n\tstatic int get(word b, int i) { if (i < 0 || i >= b[0]) return 0; return rt.ld8(rt.addr_b(b + 2) + i); }\n\n\t// sets one byte of a buffer. setting the byte at the current length makes the\n\t// buffer one longer.\n\t// b: the buffer\n\t// i: the position, below the capacity\n\t// v: the value; only its low 8 bits are kept\n\t// returns: the byte stored, 0 when i is outside the capacity\n\tstatic int set(word b, int i, int v) {\n\t\tif (i < 0 || i >= b[1]) return 0;\n\t\trt.st8(rt.addr_b(b + 2) + i, v & 255);\n\t\tif (i >= b[0]) b[0] = i + 1;\n\t\treturn v & 255;\n\t}\n}\n\n//: array algorithms borrow their input and visit elements in index order.\n//: map and filter return independent handles. reference elements are shared\n//: with the collector, so freeing the input does not invalidate the result.\n\n// the first index equal to value, or -1 when no element matches.\nint array_find<T>(fixed T[?].m source, T value) {\n\tint length = len(source);\n\tfor (int i = 0; i < length; i++) if (source[i] == value) return i;\n\treturn -1;\n}\n// true when an element equals value.\nint array_contains<T>(fixed T[?].m source, T value) { return array_find(source, value) >= 0; }\n// the first index whose predicate is true, or -1 when none matches.\nint array_find_if<T>(fixed T[?].m source, int(T).m predicate) {\n\tint length = len(source);\n\tfor (int i = 0; i < length; i++) if (predicate(source[i])) return i;\n\treturn -1;\n}\n// true when any element satisfies predicate; false for an empty array.\nint array_any<T>(fixed T[?].m source, int(T).m predicate) { return array_find_if(source, predicate) >= 0; }\n// true when every element satisfies predicate, including an "; + if (i == 6) return b"empty array.\nint array_all<T>(fixed T[?].m source, int(T).m predicate) {\n\tint length = len(source);\n\tfor (int i = 0; i < length; i++) if (!predicate(source[i])) return 0;\n\treturn 1;\n}\n// the number of elements satisfying predicate.\nint array_count<T>(fixed T[?].m source, int(T).m predicate) {\n\tint length = len(source);\n\tint count = 0;\n\tfor (int i = 0; i < length; i++) if (predicate(source[i])) count++;\n\treturn count;\n}\n// apply transform once to each element and return the transformed array.\n// the result element type follows the callback's return type.\nU[?].u array_map<T,U>(fixed T[?].m source, U(T).m transform) {\n\tint length = len(source);\n\tU[?] result = rt.new(U[?]);\n\tfor (int i = 0; i < length; i++) push(result, run.share_value(transform(source[i])));\n\treturn result;\n}\n// copy elements satisfying predicate into a new array, preserving order.\nT[?].u array_filter<T>(fixed T[?].m source, int(T).m predicate) {\n\tint length = len(source);\n\tT[?] result = rt.new(T[?]);\n\tfor (int i = 0; i < length; i++) if (predicate(source[i])) push(result, run.share_value(source[i]));\n\treturn result;\n}\n// fold elements from left to right, starting with initial; an empty array\n// returns initial without calling combine.\nA array_reduce<T,A>(fixed T[?].m source, A initial, A(A,T).m combine) {\n\tint length = len(source);\n\tfor (int i = 0; i < length; i++) initial = combine(initial, source[i]);\n\treturn initial;\n}\n\nrun.method(\"find\", array_find);\nrun.method(\"contains\", array_contains);\nrun.method(\"find_if\", array_find_if);\nrun.method(\"any\", array_any);\nrun.method(\"all\", array_all);\nrun.method(\"count\", array_count);\nrun.method(\"map\", array_map);\nrun.method(\"filter\", array_filter);\nrun.method(\"reduce\", array_reduce);\n"; + return 0; +} +// ---- end embedded std library ---- +// ---- embedded icon library (generated by src/tools/embed.sh from src/lib/icon.zc; do not edit) ---- +bstr icon_part(int i) { + if (i == 0) return b"namespace icon;\ninclude rt, run;\n\n// structured runtime transformations of compiler token lists.\n// token storage follows run.tokens: count, then kind, value and source line.\n// edits create independent lists; publication recompiles a function only after\n// delimiter validation. captures match complete balanced token ranges.\n\nint icon_error;\n\nint icon_eq(word* a, int ai, word* b, int bi, int n) {\n\tint i = 0;\n\twhile (i < n) {\n\t\tif (a[1 + (ai + i) * 3] != b[1 + (bi + i) * 3] || a[2 + (ai + i) * 3] != b[2 + (bi + i) * 3]) return 0;\n\t\ti++;\n\t}\n\treturn 1;\n}\nint icon_value(word* q, int i) { if (i < 0 || i >= q[0]) return 0; return q[2 + i * 3]; }\nint icon_kind(word* q, int i) { if (i < 0 || i >= q[0]) return 0; return q[1 + i * 3]; }\nint icon_is(word* q, int i, char* s) {\n\tint kind = run.T_OP; if (s[0] >= 'a' && s[0] <= 'z') kind = run.T_ID;\n\treturn icon_kind(q, i) == kind && icon_value(q, i) == run.intern(s);\n}\nint icon_open(int v) { return v == run.intern(\"(\") || v == run.intern(\"[\") || v == run.intern(\"{\"); }\nint icon_close(int v) { return v == run.intern(\")\") || v == run.intern(\"]\") || v == run.intern(\"}\"); }\nint icon_pair(int a, int b) {\n\treturn (a == run.intern(\"(\") && b == run.intern(\")\")) || (a == run.intern(\"[\") && b == run.intern(\"]\")) || (a == run.intern(\"{\") && b == run.intern(\"}\"));\n}\nint icon_declaration(word* q, int start, int binding) {\n\tint first = icon_value(q, start), i = start;\n\tif (icon_kind(q, start) != run.T_ID || (first != run.intern(\"let\") && first != run.intern(\"const\") && !run.istypename(first))) return 0;\n\twhile (i < binding) { if (icon_is(q, i, \"(\") || icon_is(q, i, \"=\")) return 0; i++; }\n\treturn binding > start;\n}\n// locate a simple declaration at a statement boundary, including nested blocks.\n// result records [type start, binding position, semicolon, statement start].\nint icon_locate_declaration(word* q, int id, word* result) {\n\tint start = 0, i, depth, v, binding, comma, type_start;\n\twhile (start < q[0]) {\n\t\tif (start && !icon_is(q, start - 1, \";\") && !icon_is(q, start - 1, \"{\") && !icon_is(q, start - 1, \"}\")) { start++; continue; }\n\t\ttype_start = start; if (icon_is(q, type_start, \"const\")) type_start++;\n\t\tif (icon_kind(q, type_start) != run.T_ID || (!icon_is(q, type_start, \"let\") && !run.istypename(icon_value(q, type_start)))) { start++; continue; }\n\t\ti = type_start; depth = 0; binding = -1; comma = 0;\n\t\twhile (i < q[0]) {\n\t\t\tv = icon_value(q, i);\n\t\t\tif (!depth && v == id && icon_kind(q, i) == run.T_ID && binding < 0 && icon_declaration(q, type_start, i) && (icon_is(q, i + 1, \"=\") || icon_is(q, i + 1, \";\") || icon_is(q, i + 1, \",\") || icon_is(q, i + 1, \"(\"))) binding = i;\n\t\t\tif (icon_kind(q, i) == run.T_OP) {\n\t\t\t\tif (icon_open(v)) depth++;\n\t\t\t\telse if (icon_close(v)) { if (!depth) break; depth--; }\n\t\t\t\tif (!depth && v == run.intern(\",\")) comma = 1;\n\t\t\t\tif (!depth && v == run.intern(\";\")) {\n\t\t\t\t\tif (binding >= 0) {\n\t\t\t\t\t\tif (comma || icon_is(q, binding + 1, \"(\")) return 0;\n\t\t\t\t\t\tresult[0] = type_start; result[1] = bindin"; + if (i == 1) return b"g; result[2] = i; result[3] = start; return 1;\n\t\t\t\t\t}\n\t\t\t\t\tbreak;\n\t\t\t\t}\n\t\t\t}\n\t\t\ti++;\n\t\t}\n\t\tstart++;\n\t}\n\treturn 0;\n}\nint icon_balanced(word* q, int a, int b) {\n\tword* stack = rt.new(b - a + 1); int n = 0, i = a, v, valid = 1;\n\twhile (i < b) {\n\t\tv = icon_value(q, i);\n\t\tif (icon_kind(q, i) == run.T_OP) {\n\t\t\tif (icon_open(v)) { stack[n] = v; n++; }\n\t\t\telse if (icon_close(v)) { if (!n || !icon_pair(stack[n - 1], v)) { valid = 0; break; } n--; }\n\t\t}\n\t\ti++;\n\t}\n\tif (n) valid = 0;\n\trt.free(stack);\n\treturn valid;\n}\nword* .u icon_slice(word* q, int a, int b) {\n\tword* r = rt.new(1 + (b - a) * 3); int i = 0; r[0] = b - a;\n\twhile (i < (b - a) * 3) { r[1 + i] = q[1 + a * 3 + i]; i++; }\n\treturn r;\n}\nword* .u icon_splice(word* q, int a, int b, word* insert) {\n\tint n = q[0] - (b - a) + insert[0], i = 0, j = 0;\n\tword* r = rt.new(1 + n * 3); r[0] = n;\n\twhile (i < a * 3) { r[1 + j] = q[1 + i]; i++; j++; }\n\ti = 0; while (i < insert[0] * 3) { r[1 + j] = insert[1 + i]; i++; j++; }\n\ti = b * 3; while (i < q[0] * 3) { r[1 + j] = q[1 + i]; i++; j++; }\n\treturn r;\n}\n// capture classes use integer literals, identifiers, expressions, statements\n// or blocks. a named capture records its exact token range for substitution.\nint icon_class(int name) {\n\tif (name == run.intern(\"int\")) return 1;\n\tif (name == run.intern(\"name\")) return 2;\n\tif (name == run.intern(\"expr\")) return 3;\n\tif (name == run.intern(\"stmt\")) return 4;\n\tif (name == run.intern(\"block\")) return 5;\n\treturn 0;\n}\nint icon_pattern(word* q) {\n\tint i = 0;\n\tif (!q || q[0] <= 0 || !icon_balanced(q, 0, q[0])) return 0;\n\twhile (i < q[0]) {\n\t\tif (icon_is(q, i, \"$\")) {\n\t\t\tif (icon_kind(q, i + 1) != run.T_ID || !icon_class(icon_value(q, i + 1))) return 0;\n\t\t\ti += 2;\n\t\t\tif (icon_is(q, i, \":\")) { if (icon_kind(q, i + 1) != run.T_ID || icon_class(icon_value(q, i + 1))) return 0; i += 2; }\n\t\t} else i++;\n\t}\n\treturn 1;\n}\nint icon_precedence(int name) {\n\tif (name == run.intern(\"||\")) return 1;\n\tif (name == run.intern(\"&&\")) return 2;\n\tif (name == run.intern(\"|\")) return 3;\n\tif (name == run.intern(\"^\")) return 4;\n\tif (name == run.intern(\"&\")) return 5;\n\tif (name == run.intern(\"==\") || name == run.intern(\"!=\")) return 6;\n\tif (name == run.intern(\"<\") || name == run.intern(\"<=\") || name == run.intern(\">\") || name == run.intern(\">=\")) return 7;\n\tif (name == run.intern(\"<<\") || name == run.intern(\">>\")) return 8;\n\tif (name == run.intern(\"+\") || name == run.intern(\"-\")) return 9;\n\tif (name == run.intern(\"*\") || name == run.intern(\"/\") || name == run.intern(\"%\")) return 10;\n\treturn 0;\n}\n// recognize expression structure before matching a variable-length capture.\n// this excludes statement keywords and incomplete operator sequences.\nint icon_expression(word* q, int start, int end, int minimum) {\n\tint i = start, v, kind, priority, next;\n\tif (i >= end) return -1;\n\tv = icon_value(q, i); kind = icon_kind(q, i);\n\tif (kind == run.T_OP && (v == run.intern(\"!\") || v == run.intern(\"~\") || v == run.intern(\"+\") || v == run.intern(\"-\") || v == run.intern(\"*"; + if (i == 2) return b"\") || v == run.intern(\"&\"))) {\n\t\ti = icon_expression(q, i + 1, end, 11); if (i < 0) return -1;\n\t} else if (icon_is(q, i, \"(\")) {\n\t\ti = icon_expression(q, i + 1, end, 1); if (i < 0 || !icon_is(q, i, \")\")) return -1; i++;\n\t} else if (kind == run.T_NUM || kind == run.T_FLT || kind == run.T_STR || kind == run.T_BSTR || kind == run.T_ID) {\n\t\tif (kind == run.T_ID && (v == run.intern(\"return\") || v == run.intern(\"let\") || v == run.intern(\"if\") || v == run.intern(\"while\") || v == run.intern(\"for\") || v == run.intern(\"else\") || v == run.intern(\"break\") || v == run.intern(\"continue\") || v == run.intern(\"do\") || v == run.intern(\"switch\") || v == run.intern(\"const\"))) return -1;\n\t\ti++;\n\t} else return -1;\n\twhile (i < end) {\n\t\tif (icon_is(q, i, \"(\") || icon_is(q, i, \"[\")) {\n\t\t\tv = icon_value(q, i); i++;\n\t\t\tif (v != run.intern(\"(\") || !icon_is(q, i, \")\")) {\n\t\t\t\twhile (1) {\n\t\t\t\t\ti = icon_expression(q, i, end, 1); if (i < 0) return -1;\n\t\t\t\t\tif (v == run.intern(\"(\") && icon_is(q, i, \",\")) { i++; continue; } break;\n\t\t\t\t}\n\t\t\t}\n\t\t\tif (i >= end || icon_kind(q, i) != run.T_OP || !icon_pair(v, icon_value(q, i))) return -1; i++; continue;\n\t\t}\n\t\tif (icon_is(q, i, \".\")) { if (i + 1 >= end || icon_kind(q, i + 1) != run.T_ID) return -1; i += 2; continue; }\n\t\tif (icon_is(q, i, \"++\") || icon_is(q, i, \"--\")) { i++; continue; }\n\t\tif (minimum == 1 && icon_is(q, i, \"?\")) {\n\t\t\ti = icon_expression(q, i + 1, end, 1); if (i < 0 || !icon_is(q, i, \":\")) return -1;\n\t\t\ti = icon_expression(q, i + 1, end, 1); if (i < 0) return -1; continue;\n\t\t}\n\t\tpriority = 0; if (icon_kind(q, i) == run.T_OP) priority = icon_precedence(icon_value(q, i)); if (!priority || priority < minimum) break;\n\t\tnext = icon_expression(q, i + 1, end, priority + 1); if (next < 0) return -1; i = next;\n\t}\n\treturn i;\n}\nint icon_range(word* q, int a, int b, int kind) {\n\tif (a < 0 || a >= b || b > q[0]) return 0;\n\tif (kind == 1) return b == a + 1 && icon_kind(q, a) == run.T_NUM;\n\tif (kind == 2) return b == a + 1 && icon_kind(q, a) == run.T_ID;\n\tif (kind == 3) return icon_expression(q, a, b, 1) == b;\n\tif (kind == 5) return icon_is(q, a, \"{\") && icon_mate(q, a) == b - 1;\n\tif (kind == 4) return icon_statement_end(q, a) == b;\n\treturn icon_expression(q, a, b, 1) == b;\n}\n// recursively match a pattern with backtracking at balanced capture boundaries.\n// captures contain count followed by [name, start, end, class] records.\nint icon_match(word* q, int at, word* p, int pi, word* caps) {\n\tint kind, name, next, end, old, i;\n\tif (pi == p[0]) return at + 1;\n\tif (at >= q[0]) return 0;\n\tif (!icon_is(p, pi, \"$\")) {\n\t\tif (!icon_eq(q, at, p, pi, 1)) return 0;\n\t\treturn icon_match(q, at + 1, p, pi + 1, caps);\n\t}\n\tif (pi + 1 >= p[0]) return 0;\n\tname = icon_value(p, pi + 1); kind = icon_class(name); next = pi + 2;\n\tif (!kind) return 0;\n\tif (icon_is(p, next, \":\")) { name = icon_value(p, next + 1); next += 2; if (!name) return 0; }\n\telse name = 0;\n\told = caps[0]; end = at + 1;\n\tif (kind == 4) end = icon_statement_end(q, at);\n\tif (kind == 5) { if (!ic"; + if (i == 3) return b"on_is(q, at, \"{\")) return 0; end = icon_mate(q, at) + 1; }\n\tif (end <= at) return 0;\n\twhile (end <= q[0]) {\n\t\tif (icon_range(q, at, end, kind)) {\n\t\t\ti = 0;\n\t\t\twhile (i < old) {\n\t\t\t\tif (name && caps[1 + i * 4] == name) {\n\t\t\t\t\tif (end - at != caps[3 + i * 4] - caps[2 + i * 4] || !icon_eq(q, at, q, caps[2 + i * 4], end - at)) break;\n\t\t\t\t}\n\t\t\t\ti++;\n\t\t\t}\n\t\t\tif (i == old) {\n\t\t\t\tcaps[0] = old + 1; caps[1 + old * 4] = name; caps[2 + old * 4] = at; caps[3 + old * 4] = end; caps[4 + old * 4] = kind;\n\t\t\t\ti = icon_match(q, end, p, next, caps);\n\t\t\t\tif (i) return i;\n\t\t\t\tcaps[0] = old;\n\t\t\t}\n\t\t}\n\t\tif (kind != 3) break;\n\t\tend++;\n\t}\n\treturn 0;\n}\nint icon_capture(word* caps, int name, int ordinal) {\n\tint c = 0, kind = icon_class(name), k = 0;\n\twhile (c < caps[0]) {\n\t\tif (!kind) { if (caps[1 + c * 4] == name) return c + 1; }\n\t\telse if (caps[4 + c * 4] == kind) { if (k == ordinal) return c + 1; k++; }\n\t\tc++;\n\t}\n\treturn 0;\n}\nword* .u icon_expand(word* q, word* replacement, word* caps) {\n\tint n = 0, i = 0, j = 0, c, name, kind, k, seen[6], a, b;\n\tk = 0; while (k < 6) { seen[k] = 0; k++; }\n\twhile (i < replacement[0]) {\n\t\tif (icon_is(replacement, i, \"$\")) {\n\t\t\tif (icon_kind(replacement, i + 1) != run.T_ID) { icon_error = 3; return 0; }\n\t\t\tname = icon_value(replacement, i + 1); kind = icon_class(name); c = icon_capture(caps, name, seen[kind]);\n\t\t\tif (!c) { icon_error = 3; return 0; } c--;\n\t\t\tn += caps[3 + c * 4] - caps[2 + c * 4]; if (kind) seen[kind]++; i += 2;\n\t\t} else { n++; i++; }\n\t}\n\tword* r = rt.new(1 + n * 3); i = 0;\n\tk = 0; while (k < 6) { seen[k] = 0; k++; }\n\twhile (i < replacement[0]) {\n\t\tif (icon_is(replacement, i, \"$\")) {\n\t\t\tname = icon_value(replacement, i + 1); kind = icon_class(name); c = icon_capture(caps, name, seen[kind]) - 1;\n\t\t\tif (kind) seen[kind]++;\n\t\t\ta = caps[2 + c * 4] * 3; b = caps[3 + c * 4] * 3;\n\t\t\twhile (a < b) { r[1 + j] = q[1 + a]; a++; j++; }\n\t\t\ti += 2;\n\t\t} else {\n\t\t\tk = 0; while (k < 3) { r[1 + j] = replacement[1 + i * 3 + k]; j++; k++; } i++;\n\t\t}\n\t}\n\tr[0] = j / 3;\n\tif (!icon_balanced(r, 0, r[0])) { rt.free(r); icon_error = 1; return 0; }\n\treturn r;\n}\n\n// resumable frames retain their own token list and explicit binding maps.\n// [tokens, position, state, result, error, locals, globals, blocks, steps].\n\n// locate a binding in a frame map by its interned name.\nint icon_binding(word map, int name) {\n\twhile (map) { if (map[0] == name) return map; map = map[2]; }\n\treturn 0;\n}\nint icon_read(word binding) { if (binding[3]) return *binding[3]; return binding[1]; }\nint icon_store(word*.m frame, int column, int name, int value) {\n\tword entry = icon_binding(frame[column], name);\n\tif (!entry) {\n\t\tword* node = rt.new(4); node[0] = name; node[1] = value; node[2] = frame[column]; node[3] = 0; frame[column] = node; return value;\n\t}\n\tentry[1] = value; if (entry[3]) *entry[3] = value; return value;\n}\nint icon_declare(word*.m frame, int name, int value) {\n\tword* node = rt.new(4); node[0] = name; node[1] = value; node[2] = frame[5]; node[3] = 0; frame[5] = no"; + if (i == 4) return b"de; return value;\n}\nint icon_fail(word frame) { frame[2] = 2; frame[4] = 5; icon_error = 5; return 0; }\nint icon_end(word* q, int start) {\n\tint depth = 0, i = start, v;\n\twhile (i < q[0]) {\n\t\tv = icon_value(q, i);\n\t\tif (icon_kind(q, i) == run.T_OP) {\n\t\t\tif (icon_open(v)) depth++;\n\t\t\telse if (icon_close(v)) { if (!depth) return i; depth--; }\n\t\t\tif (!depth && v == run.intern(\";\")) return i;\n\t\t}\n\t\ti++;\n\t}\n\treturn i;\n}\nint icon_mate(word* q, int start) {\n\tint depth = 1, i = start + 1, v;\n\twhile (i < q[0]) {\n\t\tv = icon_value(q, i);\n\t\tif (icon_kind(q, i) == run.T_OP) {\n\t\t\tif (icon_open(v)) depth++; else if (icon_close(v)) depth--;\n\t\t\tif (!depth) return i;\n\t\t}\n\t\ti++;\n\t}\n\treturn -1;\n}\n// find one statement boundary; control-flow captures require braced bodies.\nint icon_statement_end(word* q, int start) {\n\tint close, end;\n\tif (icon_is(q, start, \"{\")) { close = icon_mate(q, start); if (close < 0) return -1; return close + 1; }\n\tif (icon_is(q, start, \"if\") || icon_is(q, start, \"while\")) {\n\t\tif (!icon_is(q, start + 1, \"(\")) return -1;\n\t\tclose = icon_mate(q, start + 1);\n\t\tif (close < 0 || !icon_is(q, close + 1, \"{\")) return -1;\n\t\tend = icon_mate(q, close + 1); if (end < 0) return -1; end++;\n\t\tif (icon_is(q, start, \"if\") && icon_is(q, end, \"else\")) {\n\t\t\tif (!icon_is(q, end + 1, \"{\")) return -1;\n\t\t\tclose = icon_mate(q, end + 1); if (close < 0) return -1; return close + 1;\n\t\t}\n\t\treturn end;\n\t}\n\tend = icon_end(q, start);\n\tif (!icon_is(q, end, \";\")) return -1;\n\treturn end + 1;\n}\n// construct a compiler return body in one allocation, retaining source lines.\nword* .u icon_eval_tokens(word* q, int start, int end) {\n\tint count = end - start;\n\tword* expr = rt.new(7 + count * 3); expr[0] = count + 2;\n\texpr[1] = run.T_ID; expr[2] = run.intern(\"return\"); expr[3] = 1;\n\tif (count) rt.copy_w(expr + 4, q + 1 + start * 3, count * 3);\n\texpr[4 + count * 3] = run.T_OP; expr[5 + count * 3] = run.intern(\";\"); expr[6 + count * 3] = 1;\n\treturn expr;\n}\n// scalar frame substitution cannot preserve lvalue operations or implicit\n// program-global reads. reject these before compiling or executing a range.\nint icon_frame_expression(word frame, int start, int end) {\n\tword* q = frame[0]; int i = start, s; word binding;\n\tif (!icon_range(q, start, end, 3)) return 0;\n\twhile (i < end) {\n\t\tif (icon_is(q, i, \"++\") || icon_is(q, i, \"--\")) return 0;\n\t\tif (icon_kind(q, i) == run.T_ID && !icon_is(q, i - 1, \".\")) {\n\t\t\tbinding = icon_binding(frame[5], icon_value(q, i));\n\t\t\tif (!binding) binding = icon_binding(frame[6], icon_value(q, i));\n\t\t\tif (!binding) {\n\t\t\t\ts = run.lookups(icon_value(q, i));\n\t\t\t\tif (s < 0 || (run.skind[s] != run.K_FN && run.skind[s] != run.K_CONST)) return 0;\n\t\t\t}\n\t\t}\n\t\ti++;\n\t}\n\treturn 1;\n}\nint icon_frame_eval(word frame, int start, int end) {\n\tword* expr = icon_eval_tokens(frame[0], start, end);\n\tint i = 1; word binding;\n\twhile (i < expr[0] - 1) {\n\t\tif (icon_kind(expr, i) == run.T_ID && !icon_is(expr, i - 1, \".\")) {\n\t\t\tbinding = icon_binding(frame[5], icon_value(expr, i));\n\t\t\tif (!bi"; + if (i == 5) return b"nding) binding = icon_binding(frame[6], icon_value(expr, i));\n\t\t\tif (binding) { expr[1 + i * 3] = run.T_NUM; expr[2 + i * 3] = icon_read(binding); }\n\t\t}\n\t\ti++;\n\t}\n\tint result = run.eval(expr); rt.free(expr); return result;\n}\nint icon_block(word*.m frame, int open, int resume) {\n\tint close = icon_mate(frame[0], open);\n\tif (close < 0) return icon_fail(frame);\n\tword block = rt.new(4); block[0] = close; block[1] = resume; block[2] = frame[7]; block[3] = frame[5]; frame[7] = block; frame[1] = open + 1; return 1;\n}\n\nstruct icon {\n\t// the last transformation status: 0 success, 1 invalid delimiters,\n\t// 2 invalid range or declaration, 3 unresolved replacement capture,\n\t// 4 unavailable function metadata, 5 unsupported stepping syntax,\n\t// 6 rejected function publication.\n\tstatic int error() { return icon_error; }\n\t// check token delimiters without compiling or executing the list.\n\tstatic int validate(word* q) { icon_error = 0; if (!q || q[0] < 0 || !icon_balanced(q, 0, q[0])) { icon_error = 1; return 0; } return 1; }\n\t// return an independent token list with [start, end) replaced by insert.\n\tstatic word* .u splice(word* q, int start, int end, word* insert) {\n\t\ticon_error = 0;\n\t\tif (!q || !insert || q[0] < 0 || insert[0] < 0 || start < 0 || end < start || end > q[0]) { icon_error = 2; return 0; }\n\t\tword* r = icon_splice(q, start, end, insert);\n\t\tif (!icon_balanced(r, 0, r[0])) { rt.free(r); icon_error = 1; return 0; }\n\t\treturn r;\n\t}\n\t// replace each nonoverlapping pattern match. inserted tokens are not\n\t// searched again. the input and templates remain unchanged.\n\tstatic word* .u rewrite(word* q, word* pattern, word* replacement) {\n\t\ticon_error = 0;\n\t\tif (!q || q[0] < 0 || !replacement || replacement[0] < 0 || !icon_pattern(pattern) || !icon_balanced(q, 0, q[0])) { icon_error = 1; return 0; }\n\t\tword* result = icon_slice(q, 0, q[0]); word* caps = rt.new(1 + pattern[0] * 4); word* insert, next;\n\t\tint at = 0, end;\n\t\twhile (at < result[0]) {\n\t\t\tcaps[0] = 0; end = icon_match(result, at, pattern, 0, caps);\n\t\t\tif (!end) { at++; continue; }\n\t\t\tinsert = icon_expand(result, replacement, caps);\n\t\t\tif (!insert) { rt.free(result); rt.free(caps); return 0; }\n\t\t\tnext = icon_splice(result, at, end - 1, insert); at += insert[0];\n\t\t\trt.free(insert); rt.free(result); result = next;\n\t\t}\n\t\trt.free(caps); return result;\n\t}\n\t// transform retained function tokens and publish the compiled replacement.\n\t// references to the original entry continue to use the published version.\n\tstatic int find_replace(word function, word* pattern, word* replacement) {\n\t\ticon_error = 0;\n\t\tif (!replacement || replacement[0] < 0 || !icon_pattern(pattern)) { icon_error = 1; return 0; }\n\t\tword* q = run.function_tokens(function);\n\t\tif (!q) { icon_error = 4; return 0; }\n\t\tword* caps = rt.new(1 + pattern[0] * 4); int i = 0, end, count = 0;\n\t\twhile (i < q[0]) { caps[0] = 0; end = icon_match(q, i, pattern, 0, caps); if (end) { count++; i = end - 1; } else i++; }\n\t\trt.free(caps);\n\t\tif (!count) { rt.free(q); "; + if (i == 6) return b"return 0; }\n\t\tword* changed = icon.rewrite(q, pattern, replacement); rt.free(q);\n\t\tif (!changed) return 0;\n\t\tint published = run.replace_function(function, changed); rt.free(changed);\n\t\tif (!published) { icon_error = 6; return 0; }\n\t\treturn count;\n\t}\n\t// remove a complete declaration statement containing the requested name.\n\t// declarations containing multiple bindings are rejected to preserve them.\n\tstatic word* .u undeclare(word* q, char* name) {\n\t\ticon_error = 0;\n\t\tint declaration[4];\n\t\tif (!q || q[0] < 0 || !name || !icon_locate_declaration(q, run.intern(name), declaration)) { icon_error = 2; return 0; }\n\t\tword* empty = rt.new(1); empty[0] = 0;\n\t\tword* result = icon.splice(q, declaration[3], declaration[2] + 1, empty); rt.free(empty); return result;\n\t}\n\t// change the type of the first simple declaration of name.\n\tstatic word* .u change_type(word* q, char* name, word* type) {\n\t\ticon_error = 0;\n\t\tint declaration[4];\n\t\tif (!q || q[0] < 0 || !name || !icon_locate_declaration(q, run.intern(name), declaration)) { icon_error = 2; return 0; }\n\t\treturn icon.splice(q, declaration[0], declaration[1], type);\n\t}\n\t// insert a scoped statement block at the given token boundary.\n\tstatic word* .u insert_block(word* q, int at, word* body) {\n\t\ticon_error = 0;\n\t\tif (!q || !body || body[0] < 0 || at < 0 || at > q[0]) { icon_error = 2; return 0; }\n\t\tword* block = rt.new(7 + body[0] * 3); int i = 0; block[0] = body[0] + 2;\n\t\tblock[1] = run.T_OP; block[2] = run.intern(\"{\"); block[3] = 1;\n\t\twhile (i < body[0] * 3) { block[4 + i] = body[1 + i]; i++; }\n\t\tblock[4 + i] = run.T_OP; block[5 + i] = run.intern(\"}\"); block[6 + i] = 1;\n\t\tword* result = icon.splice(q, at, at, block); rt.free(block); return result;\n\t}\n\t// evaluate a selected expression using the compiler rather than a text\n\t// evaluator. the range must contain one complete expression.\n\tstatic int calculate(word* q, int start, int end) {\n\t\ticon_error = 0;\n\t\tif (!q || start < 0 || end > q[0] || !icon_range(q, start, end, 3)) { icon_error = 2; return 0; }\n\t\tword* expr = icon_eval_tokens(q, start, end);\n\t\tint value = run.eval(expr); rt.free(expr); return value;\n\t}\n\t// start a resumable evaluation with independent tokens and binding storage.\n\t// stepping executes statements through the compiler; no native frame is\n\t// suspended or inspected. scalar bindings are visible between statements.\n\tstatic word* .u begin(word* q) {\n\t\tif (!icon.validate(q)) { icon_error = 1; return 0; }\n\t\tword* frame = rt.new(9); int i = 0; while (i < 9) { frame[i] = 0; i++; } frame[0] = icon_slice(q, 0, q[0]); return frame;\n\t}\n\t// create an explicit frame from a function's retained body. argument values\n\t// are supplied through set_local before stepping starts.\n\tstatic word* .u begin_function(int function) {\n\t\tword* q = run.function_tokens(function);\n\t\tif (!q) { icon_error = 4; return 0; }\n\t\tword* frame = icon.begin(q); rt.free(q); return frame;\n\t}\n\t// set an isolated global binding, or update storage explicitly bound to it.\n\tstatic int set_gl"; + if (i == 7) return b"obal(word frame, char* name, int value) { icon_error = 0; return icon_store(frame, 6, run.intern(name), value); }\n\t// update the innermost local binding, creating a binding when absent.\n\tstatic int set_local(word frame, char* name, int value) { icon_error = 0; return icon_store(frame, 5, run.intern(name), value); }\n\t// bind one existing scalar global explicitly. reads observe its current\n\t// value, and subsequent frame assignments update its program storage.\n\tstatic int bind_global(word frame, char* name) {\n\t\tint s = run.lookups(name); icon_error = 0;\n\t\tif (s < 0 || run.skind[s] != run.K_GVAR) { icon_error = 2; return 0; }\n\t\tword type = run.stype[s];\n\t\tif (type && type != run.tint && type != run.tword) { icon_error = 2; return 0; }\n\t\ticon_store(frame, 6, run.intern(name), *run.sval[s]);\n\t\tword binding = icon_binding(frame[6], run.intern(name)); binding[3] = run.sval[s]; return 1;\n\t}\n\t// read the innermost local binding; a missing binding sets status 2.\n\tstatic int local(word frame, char* name) { icon_error = 0; word b = icon_binding(frame[5], run.intern(name)); if (!b) { icon_error = 2; return 0; } return icon_read(b); }\n\t// read an isolated or explicitly connected global binding.\n\tstatic int global(word frame, char* name) { icon_error = 0; word b = icon_binding(frame[6], run.intern(name)); if (!b) { icon_error = 2; return 0; } return icon_read(b); }\n\t// frame state: 0 resumable, 1 complete, 2 unsupported statement.\n\tstatic int state(word frame) { return frame[2]; }\n\t// the returned value after completion; zero before a return is executed.\n\tstatic int result(word frame) { return frame[3]; }\n\t// token index of the next statement or control-flow decision.\n\tstatic int position(word frame) { return frame[1]; }\n\t// number of executed statements and control-flow decisions.\n\tstatic int steps(word frame) { return frame[8]; }\n\t// number of retained local binding records, including active shadowed records.\n\tstatic int local_count(word frame) { word b = frame[5]; int n = 0; while (b) { n++; b = b[2]; } return n; }\n\t// interned name of a local record; zero when index is outside the map.\n\tstatic char* local_name(word frame, int index) { word b = frame[5]; while (b && index > 0) { index--; b = b[2]; } if (!b || index < 0) return 0; return b[0]; }\n\t// execute one statement or control-flow decision. returns 1 after progress,\n\t// 0 after completion or an unsupported statement. blocks, if and while\n\t// require braces; declarations contain one scalar binding.\n\tstatic int step(word frame) {\n\t\tif (!frame || frame[2]) return 0;\n\t\ticon_error = 0;\n\t\tword* q = frame[0]; int at = frame[1], end, name, value, column = 5, v, close, open, alternate;\n\t\tword block, binding;\n\t\twhile (frame[7] && at == frame[7][0]) {\n\t\t\tblock = frame[7];\n\t\t\twhile (frame[5] != block[3]) { binding = frame[5]; frame[5] = binding[2]; rt.free(binding); }\n\t\t\tat = block[1]; frame[7] = block[2]; rt.free(block); frame[1] = at;\n\t\t}\n\t\tif (at >= q[0]) { frame[2] = 1; return 0; }\n\t\tframe[8]++;\n\t\tif (icon_is(q"; + if (i == 8) return b", at, \";\")) { frame[1] = at + 1; return 1; }\n\t\tif (icon_is(q, at, \"{\")) { close = icon_mate(q, at); return icon_block(frame, at, close + 1); }\n\t\tif (icon_is(q, at, \"if\") || icon_is(q, at, \"while\")) {\n\t\t\tif (!icon_is(q, at + 1, \"(\")) return icon_fail(frame);\n\t\t\tclose = icon_mate(q, at + 1); open = close + 1;\n\t\t\tif (close < 0 || !icon_is(q, open, \"{\")) return icon_fail(frame);\n\t\t\tend = icon_mate(q, open); if (end < 0) return icon_fail(frame);\n\t\t\talternate = end + 1;\n\t\t\tif (icon_is(q, at, \"if\") && icon_is(q, alternate, \"else\")) {\n\t\t\t\tif (!icon_is(q, alternate + 1, \"{\")) return icon_fail(frame);\n\t\t\t}\n\t\t\tif (!icon_frame_expression(frame, at + 2, close)) return icon_fail(frame);\n\t\t\tvalue = icon_frame_eval(frame, at + 2, close);\n\t\t\tif (icon_is(q, at, \"while\")) { if (value) return icon_block(frame, open, at); frame[1] = end + 1; return 1; }\n\t\t\tif (icon_is(q, alternate, \"else\")) {\n\t\t\t\tclose = icon_mate(q, alternate + 1);\n\t\t\t\tif (value) return icon_block(frame, open, close + 1);\n\t\t\t\treturn icon_block(frame, alternate + 1, close + 1);\n\t\t\t}\n\t\t\tif (value) return icon_block(frame, open, end + 1);\n\t\t\tframe[1] = end + 1; return 1;\n\t\t}\n\t\tend = icon_end(q, at);\n\t\tif (end >= q[0] || !icon_is(q, end, \";\")) return icon_fail(frame);\n\t\tif (icon_is(q, at, \"return\")) {\n\t\t\tframe[3] = 0;\n\t\t\tif (end > at + 1) { if (!icon_frame_expression(frame, at + 1, end)) return icon_fail(frame); frame[3] = icon_frame_eval(frame, at + 1, end); }\n\t\t\tframe[1] = end + 1; frame[2] = 1; return 1;\n\t\t}\n\t\tif (icon_is(q, at, \"let\") || icon_is(q, at, \"int\") || icon_is(q, at, \"word\")) {\n\t\t\tif (icon_kind(q, at + 1) != run.T_ID) return icon_fail(frame);\n\t\t\tname = icon_value(q, at + 1); value = 0;\n\t\t\tif (end > at + 2) { if (!icon_is(q, at + 2, \"=\") || !icon_frame_expression(frame, at + 3, end)) return icon_fail(frame); value = icon_frame_eval(frame, at + 3, end); }\n\t\t\ticon_declare(frame, name, value); frame[1] = end + 1; return 1;\n\t\t}\n\t\tname = icon_value(q, at); binding = icon_binding(frame[5], name);\n\t\tif (!binding) { binding = icon_binding(frame[6], name); column = 6; }\n\t\tif (binding && icon_kind(q, at) == run.T_ID) {\n\t\t\tv = icon_value(q, at + 1);\n\t\t\tif (end == at + 2 && (v == run.intern(\"++\") || v == run.intern(\"--\"))) {\n\t\t\t\tvalue = icon_read(binding) + 1; if (v == run.intern(\"--\")) value = icon_read(binding) - 1;\n\t\t\t} else {\n\t\t\t\tif (v != run.intern(\"=\") && v != run.intern(\"+=\") && v != run.intern(\"-=\") && v != run.intern(\"*=\") && v != run.intern(\"/=\")) return icon_fail(frame);\n\t\t\t\tif (!icon_frame_expression(frame, at + 2, end)) return icon_fail(frame);\n\t\t\t\tvalue = icon_frame_eval(frame, at + 2, end);\n\t\t\t\tif (v == run.intern(\"+=\")) value = icon_read(binding) + value;\n\t\t\t\tif (v == run.intern(\"-=\")) value = icon_read(binding) - value;\n\t\t\t\tif (v == run.intern(\"*=\")) value = icon_read(binding) * value;\n\t\t\t\tif (v == run.intern(\"/=\")) value = icon_read(binding) / value;\n\t\t\t}\n\t\t\ticon_store(frame, column, name, value); frame[1] = end + 1; return 1;\n\t\t}\n\t\treturn icon_fail(frame);\n\t}\n\t// execute at most budg"; + if (i == 9) return b"et statements. returning 0 leaves the frame resumable;\n\t// callers can inspect bindings, edit them, then continue with another budget.\n\tstatic int resume(word frame, int budget) { while (budget > 0 && !frame[2]) { icon.step(frame); budget--; } return frame[2]; }\n\t// release a frame and all storage it owns.\n\tstatic int close(.d frame) {\n\t\tword b, next; int column = 5;\n\t\twhile (column <= 7) { b = frame[column]; while (b) { next = b[2]; rt.free(b); b = next; } column++; }\n\t\trt.free(frame[0]); rt.free(frame); return 0;\n\t}\n}\n\n// public entry points for the icon namespace.\n\n// status of the most recent transformation or binding operation; see docs/ICON.md.\nint error() { return icon.icon.error(); }\n// check delimiter structure only; does not compile or execute tokens.\nint validate(word* q) { return icon.icon.validate(q); }\n// return a new token list with [start, end) replaced; caller owns the result.\nword* .u splice(word* q, int start, int end, word* insert) { return icon.icon.splice(q, start, end, insert); }\n// return a new list with nonoverlapping matches replaced; inputs remain unchanged.\nword* .u rewrite(word* q, word* pattern, word* replacement) { return icon.icon.rewrite(q, pattern, replacement); }\n// publish a transformed function body; returns matches, or 0 on no match or failure.\nint find_replace(word function, word* pattern, word* replacement) { return icon.icon.find_replace(function, pattern, replacement); }\n// remove the first complete single-binding declaration named name; caller owns the result.\nword* .u undeclare(word* q, char* name) { return icon.icon.undeclare(q, name); }\n// replace the type prefix of the first simple declaration named name; caller owns the result.\nword* .u change_type(word* q, char* name, word* type) { return icon.icon.change_type(q, name, type); }\n// insert a scoped block containing body at token boundary at; caller owns the result.\nword* .u insert_block(word* q, int at, word* body) { return icon.icon.insert_block(q, at, body); }\n// compile and evaluate the expression in [start, end); expression calls retain their effects.\nint calculate(word* q, int start, int end) { return icon.icon.calculate(q, start, end); }\n// create a frame with copied tokens and isolated bindings; caller closes the frame.\nword* .u begin(word* q) { return icon.icon.begin(q); }\n// create a frame from retained function tokens; set arguments as locals before stepping.\nword* .u begin_function(int function) { return icon.icon.begin_function(function); }\n// set an isolated global binding, creating it when absent; returns the stored value.\nint set_global(word frame, char* name, int value) { return icon.icon.set_global(frame, name, value); }\n// set the innermost local binding, creating it when absent; returns the stored value.\nint set_local(word frame, char* name, int value) { return icon.icon.set_local(frame, name, value); }\n// connect an int or word program global to the frame; later frame writes update it.\nint bind_global(word frame, char* name) { ret"; + if (i == 10) return b"urn icon.icon.bind_global(frame, name); }\n// read a local binding; missing names return 0 and set error status 2.\nint local(word frame, char* name) { return icon.icon.local(frame, name); }\n// read a frame global binding; missing names return 0 and set error status 2.\nint global(word frame, char* name) { return icon.icon.global(frame, name); }\n// return frame state: 0 resumable, 1 complete, or 2 unsupported statement.\nint state(word frame) { return icon.icon.state(frame); }\n// return the value produced by return, or 0 before a return is executed.\nint result(word frame) { return icon.icon.result(frame); }\n// return the token index of the next statement or control-flow decision.\nint position(word frame) { return icon.icon.position(frame); }\n// return the number of statements and control-flow decisions executed.\nint steps(word frame) { return icon.icon.steps(frame); }\n// count retained local records, including shadowed records in active blocks.\nint local_count(word frame) { return icon.icon.local_count(frame); }\n// return the zero-based local record name, or 0 when index is out of range.\nchar* local_name(word frame, int index) { return icon.icon.local_name(frame, index); }\n// execute scalar declarations and assignments, standalone ++/--, return, blocks,\n// or braced if/else/while; direct program globals need bind_global first.\n// unsupported syntax sets error status 5.\nint step(word frame) { return icon.icon.step(frame); }\n// execute at most budget steps and return the frame state; frame bindings are scalar int/word values.\nint resume(word frame, int budget) { return icon.icon.resume(frame, budget); }\n// release the frame and owned storage; consumes frame and returns 0.\nint close(.d frame) { return icon.icon.close(d*frame); }\n"; + return 0; +} +// ---- end embedded icon library ---- |