diff options
| author | 2026-10-14 15:52:01 +0330 | |
|---|---|---|
| committer | 2026-10-14 15:52:01 +0330 | |
| commit | 7c54e48bca711a6596be4bb17fcaa4330b9928e3 (patch) | |
| tree | fd426755976847e368fe298db1c7c757f58799b9 | |
| parent | todo: record each agent's state at the end of the session and the merge order (diff) | |
line tables and backtraces: port agent/line-tables onto main, op 56 on the x86-64 jit, docs, measurements, tests; default genome reservation 0x68000
31 files changed, 634 insertions, 48 deletions
@@ -272,6 +272,25 @@ zy: bad heap pointer the line is the `new` that made the block. a double free prints and returns; it does not smash the block. `rt.memreport()` prints one line per recorded object that is still live, `leak: line 12 3 words`, and returns how many. the word count is the block's payload, which is a power-of-two chunk minus the 5 word header, not the size you asked for. the table holds 64 sites. freed blocks stay in it until the storage is reused, which is how a double free still knows the line. nothing here changes programs that never say `memdebug`. +### line tables and backtraces + +a fault names the place in the source it came from, and the calls that led there. the compiler records, for every function it compiles, which source line each stretch of its code came from, and the runtime's fatal reports print the call stack after their first line, which stays as it was: + +``` +zy: index 4 out of range (length 4) + at <stdin>:5 in get + at <stdin>:11 in sum + at <stdin>:34 in main +``` + +the innermost call comes first. `rt.trace()` prints the same lines without stopping. the reports that print one are an out-of-range index, a bad heap pointer, a use after free, a double free, running out of heap memory and a frame overflow. a source read from stdin is named `<stdin>`; an included file goes by its own name. a stack deeper than 28 frames shows the innermost 24, a line `... N more frames` (N counts return-stack words, not only calls), and the outermost 4. + +`lines on;` and `lines off;` at the top level turn the recording on and off for the functions that follow. it is on for a program and off for the genome (`src/tools/build.zc` says `lines off;`), so frames in the runtime and the compiler are left out of a trace. methods, closures and `run.evals` code get their own entries; check mode records nothing. each function's table is a block among the code, [next, entry, end, name, file, count, then count words of offset << 32 | line], chained from `rt.line_blocks`, newest first. an image built with `build_image` carries its own chain, and zc0 makes the same tables for a run-only image: `src/tools/automap.sh` puts a comment `//:lines NAME` in front of the program, which turns them on and counts lines from the next one. + +the runtime finds the frames through gate op 56, which returns the return-stack pointer as of the gate store (the word after the top return address). the interpreter, both jits, llvm, the microcore (and so wasm and bare metal) and the javascript vm answer it. what the trace cannot see: o3 frames, which are not on the return stack; a function whose code straddles two code segments, which is not recorded; and stack overflows the core reports itself. + +the cost: the genome is 395477 words with lines off and would be 410937 (+3.9%) with them on. a run-only image grows by the size of its program's tables, for example 31051 to 31119 words for `tests/programs/trace_run.zc`. compile time is not measurably different: `examples/mandel.zc` compiles and runs in about 22 ms either way on the x86-64 jit, and the genome compiles itself in about 1.2 s either way. + bit 4 makes the collector treat a word that falls inside a quarantined payload as a pointer, up to 64 words back to the header. an exact pointer already did this. turn it on when you want `p + k` to keep the block. it can also pin a block that a stale integer happens to land in, so it stays off unless you set the bit. where paths join (after an `if`, a loop, a `switch` or `?:`) every variable has to be in the same state on every path that reaches the join, with one exception. a variable that became d, or was given away, on some path is dead afterwards, and the compiler frees it on the paths where it's still alive, on their way in. a loop's exit state is the join of its `break`s and its normal exit, so giving something away and breaking is fine. what a loop can't do is give something away on a path that comes back around to the top, since the next pass would use it dead. a u object doesn't get the dead treatment: it still has to leave. @@ -553,12 +572,12 @@ the initial layout reserves code, compiler tables, the genome, and stacks below 0x080000..0x08FFFF type records (65536 words) 0x090000 frame pointer word (ZFP); 0x090002 is the syscall gate 0x090100..0x113FFF compiler data: interned names, symbol table (16384), locals, fixups, tables -0x114100..0x163FFF genome: 0x114100 entry, then the symbol table address and the image end (327680 words reserved) -0x164100..0x1A3FFF frame stack, grows down from 0x1A4100 (262144 words) -0x1A4100..0x1B40FF return stack, grows up -0x1B4100..0x1BC0FF mark stack -0x1BC100..0x1C4FF0 data stack, grows down -0x1C5000..0x224FFF the heap to start with (0x60000 words), then it grows up to MW +0x114100..0x17C0FF genome: 0x114100 entry, then the symbol table address and the image end (425984 words reserved) +0x17C100..0x1BC0FF frame stack, grows down from 0x1BC100 (262144 words) +0x1BC100..0x1CC0FF return stack, grows up +0x1CC100..0x1D40FF mark stack +0x1D4100..0x1DCFF0 data stack, grows down +0x1DD000..0x23CFFF the heap to start with (0x60000 words), then it grows up to MW ``` the numbers describe the default build's initial reservations. compact compiler and run-only maps start with smaller reservations and also support growth. `build/bin/genmap` writes matching layouts (see "smaller machines"); `--preset fixed` retains the older bounded heap for compatibility. the host answers `core.syscall(9, k)` with facts about the map (0 total words, 1 heap start, 2 heap end for now, 3 bytes). argument 4 requests memory growth, and argument 5 changes the growth limit within the host's backing constraints. @@ -324,6 +324,28 @@ of the open list. the frame window (top 24 plus last 4) prints "... N more frames" with a raw word count. merge note: main has the x86-64 jit since this branched; op 56 must also work there (it reads `cb_r`, check the x64 hook sets it). + resumed on work/line-tables (ported onto main fa35813): done. the x86-64 jit + needed no change: its store hook publishes s and r into S before a gate + store, so `cb_r` is exact there (comment at `hook` in core.c); + `tests/programs/trace_jit.zc` traces from translated code and agrees on zy, + zyi and zym, also with ZYNOO3=1. docs: README "line tables and backtraces" + (after memory debug), SPEC 5.15, 8.3 (`rt.trace`), 8.4 and appendix a. + measurements: genome 388100 words at fa35813, 395477 with this work (+7377, + +1.9%) with lines off, 410937 (+3.9% more) with lines on; run-only images + grow by their tables (`trace_run.zc` 31051 to 31119 words, `gc.zc` 30822 to + 30857); `examples/mandel.zc` compiles and runs in about 22 ms on the x86-64 + jit with lines on, off, and on main, and the genome compiles itself in + about 1.2 s either way (no measurable difference). the genome no longer fit + the default reservation (0x60000 words), so `preset_default` in genmap.c now + reserves 0x68000 and the map constants in rt.zc moved by 0x8000 (README map + table updated). side effect: the heap starts higher, where many genome code + words look like heap addresses to the conservative collector (it scans + GORG..GORG[6], code included), so `tests/programs/gc.zc` freed 969 of 1000 + instead of 998; its threshold is now 950. `make parity` puts `//:lines + <stdin>` before each program, so zc0 images trace like zy. not checked: the + javascript vm on the final build (no tsc on the box), the full suite. + open: scan only the genome's globals as roots, not its code; whether the + genome should ever carry lines (+15460 words). - bounds checking with `unbounded` (`agents/bounds-checking`, 2 commits, last 46aa911). done, builds to the fixpoint: zc0 accepts and ignores `unbounded` in all three positions; the compiler has the prefix (any order with @@ -470,7 +492,7 @@ brief someone without re-reading the code. - [ ] (partly done, see "agent work at the end of the session") bounds checking on by default, `unbounded` to turn it off (function prefix, block, top-level `unbounded;`, mirroring `unchecked`), implied by `o3`. architecture: indexing is compiled in the postfix `[` handling of the expressions part; growable arrays already check (`dat`, `range_jz`, the `outside` enhancement, `proven()` removing checks a loop test already did). fixed arrays `T[n]` with a variable index get the same inline check against the constant `n`. pointers get a runtime call (for example `rt.bchk(p, i)`) that checks only when `p` is the start of a live heap block (header check word `p ^ MAGIC`, `paylen`), skipping constant index 0 and sizes the analyzer knows (`lnsz`). reports go through `oob`. the runtime, core, compiler parts, server, `run.zc` and `build.zc` start with `unbounded;`, which zc0 accepts and ignores (genome bytes unchanged). keep std, icon and actor checked unless a function touches raw memory on purpose. measure interpreter time before and after. - [ ] after the fuzzing work, add more general functionality to the standard library. - [ ] (partly done, see "agent work at the end of the session") struct destructors: a struct may declare `fn drop()`, called wherever the compiler frees an owned value of that type, before the memory is released. architecture: the compiler frees owned locals in `freevar` (analyzer part), called from `exit_vars`, reassignment and explicit `rt.free`/`rt.recycle` calls (`s_free`, `s_recycle` handling in the call code); when the local's static type is a pointer to a struct with a `Name.drop` method, emit a call to it first. objects freed by the collector or inside unchecked code do not run `drop`: document that. first users: file descriptors in `std.Host`, processes in `actor.zc`. -- [ ] (partly done, see "agent work at the end of the session") runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing ` at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it. +- [x] (done on work/line-tables, see "agent work at the end of the session") runtime source positions. architecture: at each statement start (`stmt()`) record (code address `here`, line); per function a permanent data block [entry, end, name, file, count, pairs...] linked into a runtime list (`rt.line_blocks`), or appended to run-only images by `build_image` like the symbol blob. `lines on;`/`lines off;` like `docs on;`; on for user programs, off for the genome unless the growth is small. a new syscall op returns the return-stack pointer (the return stack is in vm memory, 0x1A4100..0x1B40FF, growing up); implement it in `src/core/core.c`, `src/micro/micro.c`, `src/js/tinyvm.ts` and the wasm port. `rt.trace()` walks the return stack, printing ` at file:line in function` for words inside a known function range. the runtime's fatal reports (`oob`, bad heap pointer, double free, out of memory, frame overflow) print the trace after their unchanged first line. check mode skips all of it. - [ ] borrow returns tied to an argument. inference already works for a callee defined before the call: `char* skip(char* s) { ...; return s + i; }` makes `let r = skip(t); rt.free(t); r[0]` an error. the gap: when the call comes before the callee's definition, nothing is reported and the program reads freed memory. extend the deferred effect requirements (`effect_require`, `effect_finish` at unit end) so a forward-called function whose return turns out to alias a parameter produces the same error. for bodyless declarations, externs and function types, add an unobtrusive spelling, for example a `borrows s` clause after the parameter list (`char* skip(char* s) borrows s;`), checked against the body when one exists. - [ ] variadic arguments and `std.print`. design: a pack is passed as one hidden argument, a caller-built read-only growable-array handle (`fixed T[?]` view: [len, cap, items, element size] in the caller's frame, so `len(a)` and `a[i]` work unchanged and nothing is allocated). `...int a` is a typed pack; several packs per function when their element types differ (`void f(...int a, ...char b)`): each extra argument goes to the pack whose element type matches its static type, an ambiguous match is an error, packs come after the fixed parameters. `...any args` packs each value with a kind code (int, float, char, string, bstr, pointer, ...) and its type record: `any` is a small built-in record { kind, value, type }. `...*any args` makes the call comma-free: after the fixed arguments, the parser reads unary expressions until `)`, commas optional, so `std.print("x = " name ", age " age "\n");` works and binary expressions must be parenthesized. `std.print(...*any args)` prints each argument by its kind (`put_n`, the float printer, `put_s`, `put_bs`, a character for `char`, hex for pointers). - [ ] x86-64 jit phase 2: compile o3 regions to x86-64 (port `rcompile` and the region register allocation; regions now run on the region interpreter and are 2 to 4 times slower than their plain versions on this jit) and inline enhancements (lower the optimized micro-ops to x86 registers as the arm64 path does, instead of calling `xrun`). validate with `tests/drivers/jit_x64_test.sh` and `tests/bench/bench.sh`, which now has an x86-64 jit column. add the driver to `tests/test.sh` on x86-64 linux. diff --git a/docs/SPEC.md b/docs/SPEC.md index 3ad93c8..5618a5e 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -74,7 +74,7 @@ a name that looks like `x.y` where `x` is not a known module is the three tokens a keyword marked only S used inside an expression is the error `NAME: can't be used inside an expression`. a keyword that is not E and is used where an expression is expected is the same error. -1.3.5 these words are recognised by position by the parser, are not in the keyword table, and are treated as ordinary names elsewhere: `fn`, `const`, `fixed`, `static`, `struct`, `type`, `typecheck`, `docs`, `namespace`, `enhance`, `immediate`, `as`, `case`, `default`, `on`, `off`, `small`, `micro`, `vm`, `this`, `new`, and the tag names `u`, `m`, `d`. `struct`, `type`, `typecheck`, `docs`, `namespace` and `enhance` are top-level only (5.1.2). +1.3.5 these words are recognised by position by the parser, are not in the keyword table, and are treated as ordinary names elsewhere: `fn`, `const`, `fixed`, `static`, `struct`, `type`, `typecheck`, `docs`, `lines`, `namespace`, `enhance`, `immediate`, `as`, `case`, `default`, `on`, `off`, `small`, `micro`, `vm`, `this`, `new`, and the tag names `u`, `m`, `d`. `struct`, `type`, `typecheck`, `docs`, `namespace` and `enhance` are top-level only (5.1.2). 1.3.6 the type names `int word char bool void float f64 f32 bstr i8 u8 byte i16 u16 i32 u32` are ordinary names bound to types (2.1). a local variable with such a name shadows the type name where a declaration or expression could start. @@ -659,9 +659,9 @@ int y = 7; asm { y @ 1 + y ! } // y is 8 ### 5.1 items -5.1.1 a unit is a sequence of items. an item is: a function definition or prototype, a global variable, `const`, `struct`, `type`, `enum`, `include`, `embed`, `namespace`, `typecheck`, `docs`, `memdebug`, `hints`, `enhance`, `extern`, `unchecked;`, or a use of a top-level parse word (7.2). +5.1.1 a unit is a sequence of items. an item is: a function definition or prototype, a global variable, `const`, `struct`, `type`, `enum`, `include`, `embed`, `namespace`, `typecheck`, `docs`, `lines`, `memdebug`, `hints`, `enhance`, `extern`, `unchecked;`, or a use of a top-level parse word (7.2). -5.1.2 `struct`, `type`, `typecheck`, `docs`, `namespace`, `enhance` and `extern` are valid only at the top level. `enum`, `embed`, `memdebug`, `hints` and `const` also appear as statements in functions. `include` is top-level only (5.12.7). +5.1.2 `struct`, `type`, `typecheck`, `docs`, `lines`, `namespace`, `enhance` and `extern` are valid only at the top level. `enum`, `embed`, `memdebug`, `hints` and `const` also appear as statements in functions. `include` is top-level only (5.12.7). 5.1.3 the unit is scanned once for function signatures before it is compiled, so calls may precede definitions and prototypes are optional (an untyped function called before its tags are declared that later declares `.m`/`.d` tags is error: f: was called before its tags were declared; put a prototype above the first call). @@ -787,6 +787,14 @@ rt.put_n(util.twice(1)); // error: module access requires `include util;` `docs on;` keeps doc comments of what follows in the image; `docs off;` stops. no effect on a program's behaviour. +### 5.15 lines + +5.15.1 `lines on;` records a line table for each function, method, closure and evaluated unit compiled after it; `lines off;` stops. a program starts with lines on. `lines` followed by anything other than `on` or `off` is an ordinary name. + +5.15.2 a line table changes no result of a program, only what a trace prints (8.4). a function compiled with lines off shows in a trace by its address and nearest name, or not at all when it belongs to the genome. + +5.15.3 the comment `//:lines NAME` names the source after it NAME, counts its lines from the next line, and turns lines on from the next top-level item. + ## 6. ownership and lifetimes this section describes the analyzer: a pass that runs inside the compiler on every function (including code compiled at run time by `run.eval` and friends), tracks heap objects through local variables, inserts frees, and reports violations. it can be switched off (6.12). what it proves is limited to what is stated here. @@ -1050,19 +1058,20 @@ each error is a line `zc: line N: MESSAGE`. after the unit, a summary line `zc: | `rt.len_w(s)`, `rt.eq_w(a, b)`, `rt.copy_w(dst, src, n)`, `rt.fill_w(dst, v, n)` | word-string length, equality, word copy, word fill | | `rt.len_s rt.at_s rt.slice_s rt.cat_s rt.eq_s rt.s_to_n rt.n_to_s rt.put_bs` | packed byte strings | | `rt.gc() rt.memreport() rt.syscall rt.readfile` | collector, leak report, host gate, file read | +| `rt.trace()` | write the current call stack, innermost call first, one line ` at FILE:LINE in NAME` per frame with a line table (8.4); returns 0 | heap objects have a 5 word header before the pointer (size, flags, check word, owner, type); `rt.new` words are zeroed. output is the process's standard output, one byte per `put_c`. ### 8.4 run-time errors -the program prints `zy: index I out of range (length N)` and stops on an out-of-range `T[?]` index; `zy: use after free from line N`, `zy: double free from line N`, `zy: bad heap pointer` under `memdebug`. the machine does not trap on integer division by zero (3.4.3) or on a stray write inside memory. +the program prints `zy: index I out of range (length N)` and stops on an out-of-range `T[?]` index; `zy: use after free from line N`, `zy: double free from line N`, `zy: bad heap pointer` under `memdebug`. these reports, `zc: out of heap memory` and a frame overflow are followed by the call stack in the form `rt.trace()` prints (5.15); a frame without a line table shows as ` at 0xADDR in NAME`, frames of the genome are left out, and a stack of more than 28 frames shows the innermost 24, ` ... N more frames` and the outermost 4. the machine does not trap on integer division by zero (3.4.3) or on a stray write inside memory. ## appendix a: what zc0 accepts zc0 (`src/tools/zc0.c`) is the stage-0 bootstrap compiler. it reads source on stdin and writes a genome image; to run a program through it use `make run PROG=file.zc` (the source starts with `namespace;` and defines `main`). it has no types (they are read and dropped), no analyzer, no checks. in the zc0 subset: - lexing: comments, decimal/hex integers (but see appendix b for octal and saturation), floats, character literals, word strings, `b"..."`, the operator set `== != <= >= << >> && || ++ -- += -= *= /= %= &= |= ^= -> <<= >>=`, the source-ending backtick, `rt.`/`run.`/`core.`/`lsp.`/`std.`/`icon.` joined names. -- top level: `namespace`, `include` (the names are read and ignored; resident modules only), `embed`, `unchecked`, `fn`, `.u name(...)`, typed functions and globals, `const`, `enhance`, `let`/`var` globals, prototypes. +- top level: `namespace`, `include` (the names are read and ignored; resident modules only), `embed`, `unchecked`, `lines on;`/`lines off;` and the `//:lines NAME` comment (line tables as in 5.15), `fn`, `.u name(...)`, typed functions and globals, `const`, `enhance`, `let`/`var` globals, prototypes. - statements: blocks, declarations (typed or `let`/`var`, arrays `x[n]` and `T[n]`), `unchecked stmt`, `if`/`else`, `while`, `for`, `break`, `continue`, `return`, `asm`, expression statements. - expressions: all binary operators with the c precedences, assignment and compound assignment, unary `- ! ~ * & ++ --`, recasts `u*x m*x d*x` (no code), postfix `[]`, `[i:j]` and `[i]` on byte strings, calls (including through a value), `++ --`, `x as name`, `asm { }`, float arithmetic and comparisons, byte strings (`+`, comparisons). diff --git a/src/compiler/base.zc b/src/compiler/base.zc index d83dc9d..2f802b9 100644 --- a/src/compiler/base.zc +++ b/src/compiler/base.zc @@ -224,6 +224,16 @@ char[1100] tbuf_initial; char* tbuf = tbuf_initial; int tbufcap = 1100; // collects them in dbuf, dlast is the line of the last one, and dline the // line of the token that follows them. only when docs are on. int docs_on; // docs on; keeps them: a symbol "doc name" holds a byte string +// line tables: lines on; (the default) records, for each function compiled, its entry, its end and the +// source line of each statement, as a line block among the code that rt.line_blocks chains (see +// rt.zc). lt_buf holds the pairs of the functions being compiled, innermost last, each as a frame +// [outer frame, entry, name, file, 0, offset << 32 | line ...]; lt_base is the innermost frame. +// lt_head is the newest block of an image being built, whose own rt.line_blocks it will become. +int lines_on = 1; +// //:lines NAME (see mark_line): the markers the lexer has met, [lines before the file, NAME] each, and the +// one in force, which a top-level item applies when it is reached +word lt_marks; int lt_nmarks, lt_markcap, lt_next_mark; char* lt_mark_file; int lt_mark_base; +word lt_buf, lt_head, lt_file_at; int lt_top, lt_cap, lt_base = 0 - 1, lt_file_img; char* lt_file; char[2100] dbuf_initial; char* dbuf = dbuf_initial; int docbufcap = 2100; int dn, dlast, dline, docpending, dsect; char[2100] dsnap_initial; char* dsnap = dsnap_initial; int docsnapcap = 2100; int dsn, dnm; char[300] dnmb_initial; char* dnmb = dnmb_initial; int docnamecap = 300; int pk_tk, pk_tv, pq_n; word[4] pq_tk; word[4] pq_tv; word[4] pq_gl; word[4] pq_ln; // up to 4 tokens of lookahead @@ -299,7 +309,7 @@ word tdstk; int ntd, tdcap; word readonly_types; int dty, dtag; // result of dtype() int s_typedef; -int s_lt, s_gt, s_shr, s_le, s_ge, s_eqeq, s_ne, s_plus, s_struct, s_type, s_typecheck, s_docs, s_small, s_std, s_icon, s_on, s_off, s_new, s_new_t; +int s_lt, s_gt, s_shr, s_le, s_ge, s_eqeq, s_ne, s_plus, s_struct, s_type, s_typecheck, s_docs, s_lines, s_small, s_std, s_icon, s_on, s_off, s_new, s_new_t; int s_o3, s_lfa, s_outside, s_new_raw, s_new_oz, s_oob; int s_ld8, s_st8, s_ld16, s_st16, s_ld32, s_st32; // ---- o3 region state ---- @@ -1222,6 +1232,83 @@ int hole() { lit(rt.MW - 1); return a; } +// ---- line tables ---- + +// make room for n more words of pairs. +int lt_room(int n) { + int cap = lt_cap; + if (lt_top + n <= cap) return 0; + if (cap < 256) cap = 256; + while (lt_top + n > cap) cap = cap * 2; + lt_buf = chk_resize(lt_buf, lt_top, cap); lt_cap = cap; + return 0; +} +// start recording the function whose code begins at entry, named name, with its header on line. +// returns the frame to give lt_end, or -1 when no table is being made (lines off, a dry run, +// check mode). +int lt_begin(int entry, char* name, int line) { + int b; + if (!lines_on || dry || cmode) return 0 - 1; + lt_room(8); + b = lt_top; + lt_buf[b] = lt_base; lt_buf[b + 1] = entry; lt_buf[b + 2] = name; lt_buf[b + 3] = curinc; lt_buf[b + 4] = 0; + if (!curinc) { lt_buf[b + 3] = lt_mark_file; lt_buf[b + 4] = lt_mark_base; } // the main source may be marked as a file of its own + lt_top = b + 5; lt_base = b; + lt_buf[lt_top] = line - lt_buf[b + 4]; lt_top++; // the code before the first statement belongs to the header's line + return b; +} +// a statement begins at the current token, and its code at here. +int lt_stmt() { + int off, n; + if (lt_base < 0 || dry) return 0; + off = here - lt_buf[lt_base + 1]; + if (off < 0) return 0; + lt_room(2); + if (lt_top > lt_base + 5 && (lt_buf[lt_top - 1] >> 32) == off) lt_top--; // same place: the later statement is the one + n = tline - lt_buf[lt_base + 4]; + if (lt_top > lt_base + 5 && (lt_buf[lt_top - 1] & 4294967295) == n) return 0; + lt_buf[lt_top] = off * 4294967296 + n; lt_top++; + return 0; +} +// the file name in a block: the include's path, or <stdin> for the main source. one copy of the +// bytes is shared by the blocks that follow it. +int lt_file_data(char* f) { + if (!f) f = "<stdin>"; + if (lt_file_at && lt_file_img == image_active && rt.eq_w(lt_file, f)) return lt_file_at; + lt_file_at = bstrdata(f); lt_file_img = image_active; + lt_file = chk_resize(0, 0, rt.len_w(f) + 1); rt.copy_w(lt_file, f, rt.len_w(f) + 1); + return lt_file_at; +} +// finish the function begun at frame b, whose code ends at end: put its line block among the code +// and chain it. a block is [next, entry, end, name, file, count, pairs...]. +int lt_end(int b, int end) { + int entry, count, i = 0, name, file, blk, need; word head; char* nm; char* fl; + if (b < 0) return 0; + entry = lt_buf[b + 1]; nm = lt_buf[b + 2]; fl = lt_buf[b + 3]; count = lt_top - (b + 5); + if (!nm) nm = "<fn>"; + if (!fl) fl = "<stdin>"; + if (end >= entry && !dry) { + need = 8 + count + ((rt.len_w(nm) + 7) >> 3) + 2 + ((rt.len_w(fl) + 7) >> 3) + 2; + reserve_code(need); + name = bstrdata(nm); file = lt_file_data(fl); + if (image_active) head = lt_head; else head = rt.line_blocks; + blk = here; + emit(head); emit(entry); emit(end); emit(name); emit(file); emit(count); + while (i < count) { emit(lt_buf[b + 5 + i]); i++; } + if (image_active) lt_head = blk; else rt.line_blocks = blk; + } + lt_top = b; lt_base = lt_buf[b]; + return 0; +} +// at the top level, a marker that comes before the item takes effect: lines are on, in its file. +int lt_apply_marks() { + if (incdepth || ginst) return 0; + while (lt_next_mark < lt_nmarks && lt_marks[lt_next_mark * 2] < tline) { + lt_mark_base = lt_marks[lt_next_mark * 2]; lt_mark_file = lt_marks[lt_next_mark * 2 + 1]; + lines_on = 1; lt_next_mark++; + } + return 0; +} // forward references. calling a function that is not defined yet creates a // symbol with F_DEF clear whose value is the place of its first use (for the // "never defined" error). each call site emits a placeholder literal and adds diff --git a/src/compiler/builtins.zc b/src/compiler/builtins.zc index 53d8597..5d7bc9c 100644 --- a/src/compiler/builtins.zc +++ b/src/compiler/builtins.zc @@ -507,6 +507,15 @@ int kw_docs() { next(); expect(s_semi); return 0; } +// lines on; records line tables for the functions that follow, lines off; stops (see lt_begin) +int kw_lines() { + next(); + if (is_id(s_on)) { + lines_on = 1; + } else if (is_id(s_off)) lines_on = 0; else die("lines on or lines off"); + next(); expect(s_semi); + return 0; +} // hints on; / hints small; / hints off; from here to the end of the document (or the next such line) int kw_hints() { int level = 2, at = toff; @@ -1167,7 +1176,7 @@ int types_init() { s_le = intern("<="); s_ge = intern(">="); s_eqeq = intern("=="); s_ne = intern("!="); s_plus = intern("+"); - s_struct = intern("struct"); s_type = intern("type"); s_typedef = intern("typedef"); s_typecheck = intern("typecheck"); s_docs = intern("docs"); s_std = intern("std"); s_icon = intern("icon"); s_small = intern("small"); + s_struct = intern("struct"); s_type = intern("type"); s_typedef = intern("typedef"); s_typecheck = intern("typecheck"); s_docs = intern("docs"); s_lines = intern("lines"); s_std = intern("std"); s_icon = intern("icon"); s_small = intern("small"); s_on = intern("on"); s_off = intern("off"); s_new = intern("rt.new"); diff --git a/src/compiler/decl.zc b/src/compiler/decl.zc index 19b1811..cc84603 100644 --- a/src/compiler/decl.zc +++ b/src/compiler/decl.zc @@ -280,7 +280,7 @@ int body() { // into stype, checked against any earlier prototype or prescan result, and // where the o3 region for an o3 function is handed to the core) int fndef(int chk, int rty, int ret, int n) { - int s, np = 0, tags = 0, t, sig, i; char*[24] pn; word[24] pt; word[24] pty; int typed = 0, ft, explicit_rty = !!rty, o3f = o3nxt, lo, hi, pk; + int s, np = 0, tags = 0, t, sig, i; char*[24] pn; word[24] pt; word[24] pty; int typed = 0, ft, explicit_rty = !!rty, o3f = o3nxt, lo, hi, pk, lt, lend, lfl = tline; o3nxt = 0; if (rty || ntp) typed = 1; if (!dnm) dnm = n; @@ -365,6 +365,7 @@ int fndef(int chk, int rty, int ret, int n) { sflag[s] = sflag[s] | F_DEF; fixall(sfix[s], here); sfix[s] = 0; fn_begin(chk, ret); + lt = lt_begin(sval[s], sname[s], lfl); finsym = s; finfer = !ret; effect_current = effect_record(s); rt.fill_w(effect_current, 0, 50); effect_current[0] = 1; if (!chk || stubgen) effect_current[0] = 2; effect_current[1] = np; i = 0; while (i < np) { if (pt[i] == 2) { effect_current[2 + i] = EF_CONSUME; effect_current[26 + i] = EF_CONSUME; } i++; } @@ -397,6 +398,7 @@ int fndef(int chk, int rty, int ret, int n) { } hi = here; epilogue(); + lend = here; if (o3f) { o3on = 0; o3fs = fslots; @@ -405,6 +407,7 @@ int fndef(int chk, int rty, int ret, int n) { o3region(lo, hi, o3sl, o3rg, no3sl); } o3flush(); + lt_end(lt, lend); ntp = 0; ifcur = 0; fn_remember(s, n); return s; @@ -567,7 +570,7 @@ int topsync() { if (is_op(s_lb)) depth++; else if (is_op(s_rb)) { depth--; if (depth < 0) depth = 0; } else if (!depth && tk == T_ID && (tv == s_fn || tv == s_var || tv == s_const || tv == s_struct || tv == s_type || tv == s_typedef - || tv == s_enhance || tv == s_typecheck || tv == s_docs || tv == s_unchecked || tv == s_immediate || tv == s_o3 + || tv == s_enhance || tv == s_typecheck || tv == s_docs || tv == s_lines || tv == s_unchecked || tv == s_immediate || tv == s_o3 || tv == s_memdebug || tv == s_enum || tv == s_include)) break; next(); } @@ -628,6 +631,7 @@ int fndone(int s, int imm) { // ( or < means function, otherwise a global. int topitem0() { int r, i; + lt_apply_marks(); dsn = 0; dnm = 0; if (docs_on && dn && dline == tline && !ginst) { if (dn > docsnapcap) { docsnapcap = dn; dsnap = chk_resize(dsnap, 0, docsnapcap); } @@ -679,6 +683,7 @@ int topitem1() { if (typedef_ahead()) return kw_typedef(); if (is_id(s_typecheck)) return kw_typecheck(); if (is_id(s_docs)) return kw_docs(); + if (is_id(s_lines)) { peek(); if (pk_tk == T_ID && (pk_tv == s_on || pk_tv == s_off)) return kw_lines(); } if (is_id(s_memdebug)) return kw_memdebug(); if (is_id(s_enum)) return kw_enum(); if (is_id(s_include)) return kw_include(); @@ -1334,7 +1339,7 @@ int unit_end(int from) { // way is emitted behind a jump when another function is being compiled, so // compile() works inside a parse word.) int compile_list_context(int q, int top, char* module, word imports) { - int hop = -1, entry = 0, from = nsym; + int hop = -1, entry = 0, from = nsym, lt = 0 - 1; ctx_push(); ifcur = 0; ifcall = -1; effect_floor = from; module_name = module; module_imports = imports; if (infn) { hop = hole(); op(O_JMP); } if (top) prescan(1, q); @@ -1344,12 +1349,14 @@ int compile_list_context(int q, int top, char* module, word imports) { else { entry = here; fn_begin(1, 0); + lt = lt_begin(entry, "<eval>", tline); prologue(); borrow_prologue(); borrow_begin(); while (tk != T_EOF) stmt(); defer_run(1, 1); unused_check(lbase); if (fchk && live) exit_vars(1, 1); epilogue(); + lt_end(lt, here); } src_pop(); unit_end(from); diff --git a/src/compiler/expr.zc b/src/compiler/expr.zc index 40de6ef..f41109e 100644 --- a/src/compiler/expr.zc +++ b/src/compiler/expr.zc @@ -647,7 +647,7 @@ word closure_layout(int closure_index, int count) { // variable in. ctx_push / ctx_pop save and restore the enclosing function's // compiler state around the body. int closure() { - int np = 0; char*[20] pn; word[20] pty; int hop, entry, i, t, envl, ncap, c, result, declared = 0, tags = 0, tag; word[20] ptags; + int np = 0; char*[20] pn; word[20] pty; int hop, entry, i, t, envl, ncap, c, result, declared = 0, tags = 0, tag, lt, lfl = tline; word[20] ptags; word expected = closure_expected; closure_expected = 0; reserve_closures(nclo + 1); if (o3on) serr("closures can't be made inside an o3 region"); @@ -679,6 +679,7 @@ int closure() { hop = hole(); op(O_JMP); entry = here; fn_begin(1, 0); + lt = lt_begin(entry, "<closure>", lfl); if (declared) fty = declared; prologue(); borrow_prologue(); i = 0; @@ -699,6 +700,7 @@ int closure() { if (result <= 0 || live) result = tword; if (declared) result = declared; epilogue(); + lt_end(lt, here); nclo--; c = nclo; ncap = clo_n[c]; diff --git a/src/compiler/image.zc b/src/compiler/image.zc index dfe5f77..45cd928 100644 --- a/src/compiler/image.zc +++ b/src/compiler/image.zc @@ -62,25 +62,48 @@ int blob_load(word p, int hide) { } return 0; } +// the name of the function whose code holds address a, for the frames of a backtrace that have +// no line table (rt.frame_namer is set to this when the compiler starts). the nearest function +// at or below a, or 0. +char* frame_symbol(int a) { + int i = 0, best = 0 - 1; + while (i < nsym) { + if (skind[i] == K_FN && (sflag[i] & F_DEF) && sval[i] <= a && sval[i] > 0 && (best < 0 || sval[i] > sval[best])) best = i; + i++; + } + if (best < 0) return 0; + return sname[best]; +} +// an image built with line tables gets the head of its chain in its own rt.line_blocks, the +// global the runtime reads, as that global's initial value. +int lt_image_head() { + int s = lookup(intern("rt.line_blocks")); + if (lt_head && s >= 0 && skind[s] == K_GVAR) *(sval[s] + hdelta) = lt_head; + return 0; +} // compile the rest of stdin for org into a growing output buffer. the image's // addresses remain logical addresses at org while its temporary storage grows. // image_words and image_size expose the finished image; initial is its first // buffer reservation. the matching host layout must still fit the finished image. int build_image_grow(int org, int initial) { int sh = here, sd = hdelta, sl = hlimit, sub = ubase, svl = vislo, sns = nsym, sen = nenh, n; + word saved_head = lt_head; char* saved_file = lt_file; word saved_file_at = lt_file_at; char* saved_mark = lt_mark_file; int saved_base = lt_mark_base, saved_nm = lt_next_mark, saved_nmarks = lt_nmarks; word saved_buffer = image_buffer; int saved_active = image_active, saved_org = image_org, saved_capacity = image_capacity; if (org < 0 || initial < 7 || initial > 0x7fffffff - org) fatal("bad image reservation"); image_buffer = chk_resize(0, 0, initial); image_active = 1; image_org = org; image_capacity = initial; ubase = nsym; vislo = nsym; nenh = 0; here = org; hdelta = image_buffer - org; hlimit = org + initial; + lt_head = 0; lt_file_at = 0; lt_mark_file = 0; lt_mark_base = 0; lt_next_mark = 0; lt_nmarks = 0; lit(rt.FTOP); lit(rt.ZFP); op(O_ST); fnref(fwdfn(s_start)); op(O_JMP); emit(0); emit(0); unit_stdin(); + lt_image_head(); n = blob_emit(ubase); image_buffer[5] = n; image_buffer[6] = here; n = here - org; image_words = image_buffer; image_size = n; popsyms(sns); ubase = sub; vislo = svl; here = sh; hdelta = sd; hlimit = sl; nenh = sen; + lt_head = saved_head; lt_file = saved_file; lt_file_at = saved_file_at; lt_mark_file = saved_mark; lt_mark_base = saved_base; lt_next_mark = saved_nm; lt_nmarks = saved_nmarks; image_buffer = saved_buffer; image_active = saved_active; image_org = saved_org; image_capacity = saved_capacity; return n; } diff --git a/src/compiler/lex.zc b/src/compiler/lex.zc index 0b509e4..0348cae 100644 --- a/src/compiler/lex.zc +++ b/src/compiler/lex.zc @@ -366,10 +366,31 @@ int fieldscan(int n) { // a // comment, with ch on its second slash. one that starts a line joins the // doc comment being collected, if the line above was part of it, or starts a // new one; a comment after code on the same line is not documentation. +// a comment //:lines NAME (ch is its colon) says that what follows is the file NAME, with its lines counted +// from the next one, and turns line tables on from the next top-level item. src/tools/automap.sh writes one +// before a program it joins to the runtime. the rest of the line is read here and the marker kept: the +// source is read once to find the signatures and again to compile, and it counts only when compiling reaches it. +int mark_line() { + char[300] b; int n = 0, i, base; word name; + while (ch >= 0 && ch != 10) { if (n < 298) { b[n] = ch; n++; } nextch(); } + while (n && (b[n - 1] == 13 || b[n - 1] == 32)) n--; + b[n] = 0; + if (n > 7 && b[1] == 'l' && b[2] == 'i' && b[3] == 'n' && b[4] == 'e' && b[5] == 's' && b[6] == 32) { + base = line; + if (ch == 10) base = line - 1; + if (!lt_nmarks || base > lt_marks[lt_nmarks * 2 - 2]) { + if (lt_nmarks * 2 + 2 > lt_markcap) { lt_marks = chk_resize(lt_marks, lt_nmarks * 2, lt_markcap * 2 + 16); lt_markcap = lt_markcap * 2 + 16; } + name = chk_resize(0, 0, n); i = 7; + while (i <= n) { name[i - 7] = b[i]; i++; } + lt_marks[lt_nmarks * 2] = base; lt_marks[lt_nmarks * 2 + 1] = name; lt_nmarks++; + } + } + return 0; +} int doccomment(int nl) { int cl = line, start; nextch(); - if (ch == ':') { while (ch >= 0 && ch != 10) nextch(); dn = 0; dsect = 1; dlast = cl; docpending = 1; return 0; } + if (ch == ':') { mark_line(); dn = 0; dsect = 1; dlast = cl; docpending = 1; return 0; } if (!nl) { while (ch >= 0 && ch != 10) nextch(); return 0; } if (dlast != cl - 1) { dn = 0; dsect = 0; } if (ch == 32) nextch(); @@ -451,7 +472,11 @@ int lex1() { if (ch != '/') break; nextch(); if (ch == '/') { - if (docs_on) doccomment(nl); else while (ch >= 0 && ch != 10) nextch(); + if (docs_on) doccomment(nl); + else { + nextch(); + if (ch == ':') mark_line(); else while (ch >= 0 && ch != 10) nextch(); + } tglue = 0; continue; } if (ch == '*') { diff --git a/src/compiler/setup.zc b/src/compiler/setup.zc index 1d8ce6f..9ed938b 100644 --- a/src/compiler/setup.zc +++ b/src/compiler/setup.zc @@ -168,6 +168,7 @@ int start() { blob_load(rt.GORG[5], 0); zc_tables(); types_init(); + rt.frame_namer = sval[lookup(intern("run.frame_symbol"))]; here = 0; hdelta = 0; hlimit = rt.CODE_END; ptop = 0; ubase = nsym; vislo = 0; // a genome that carries the language server runs it instead of compiling stdin diff --git a/src/compiler/stmt.zc b/src/compiler/stmt.zc index 21d5801..4dafe4f 100644 --- a/src/compiler/stmt.zc +++ b/src/compiler/stmt.zc @@ -13,7 +13,7 @@ int resync() { } // compile one statement. int stmt() { - word before = borrow_slots; int result; emit_statement_hooks(); result = stmt_inner(); borrow_cleanup_since(before); return result; + word before = borrow_slots; int result; lt_stmt(); emit_statement_hooks(); result = stmt_inner(); borrow_cleanup_since(before); return result; } int stmt_inner() { int s, c; diff --git a/src/core/core.c b/src/core/core.c index b571981..2a0c467 100644 --- a/src/core/core.c +++ b/src/core/core.c @@ -293,6 +293,7 @@ static Cell hostcall(Cell op, Cell a, Cell b, Cell c) #else return -1; #endif + case 56: return cb_r; /* the return stack pointer as of the gate store: the word after the top return address */ case 1: if (zcstr(a, path, 1024) < 0) return -1; if (b == 1) return open(path, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, 0644); @@ -3608,7 +3609,9 @@ static void syncicache(JW *a, JW *b) #endif } -/* store hook for words that are covered by a translation */ +/* store hook for words that are covered by a translation. both jits reach it with s and r + published in S[1] and S[2] (the x86-64 jit's emit_store calls publish_stacks first), so a + gate store sees the exact stacks: cb_r is what op 56 returns to rt.trace() */ static int hook(Cell a, Cell v) { int j, hit = 0; diff --git a/src/js/tinyvm.ts b/src/js/tinyvm.ts index f779487..f52d4af 100644 --- a/src/js/tinyvm.ts +++ b/src/js/tinyvm.ts @@ -466,6 +466,8 @@ export class TinyVM { } return -1n; } + case 56: // the return stack pointer: the word after the top return address + return BigInt(this.rr); case 42: // poll: input is waiting return this.inputPos < this.inputBuffer.length ? 1n : 0n; case 41: { // shortest float text into a word buffer diff --git a/src/lsp/server.zc b/src/lsp/server.zc index 372724f..c3f9f4f 100644 --- a/src/lsp/server.zc +++ b/src/lsp/server.zc @@ -1704,6 +1704,7 @@ char* kwtext() { if (hname_is("typecheck")) return "typecheck on; / typecheck off;~turns on type checks for the code that follows, enums included."; if (hname_is("hints")) return "hints on; / hints small; / hints off;~controls subsequent hint diagnostics: informational ranges by default, compact markers with small, or no hints with off."; if (hname_is("docs")) return "docs on; / docs off;~keeps the doc comments above declarations in the image, so the language server can show them on hover."; + if (hname_is("lines")) return "lines on; / lines off;~records, for the functions that follow, the source line of each statement, so a run-time fault prints a backtrace (at file:line in function) and rt.trace() can print the call stack. on by default for a program. the language server never builds the tables."; if (hname_is("micro")) return "enhance name(args) micro \"recipe\";~the body of an enhancement, written as micro-ops on registers r0 to r15. the inputs arrive in r0 and up, and the outputs leave from r0."; if (hname_is("vm")) return "enhance name(args) vm { ... };~the body of an enhancement, written as stack machine asm. it becomes one opcode when the core lifts it."; if (hname_is("enhance")) return "enhance name(args) micro \"...\" or vm { ... };~installs a custom opcode at compile time, with up to 8 inputs and 8 outputs. an enhancement with one output can be called like a function."; @@ -2575,7 +2576,7 @@ int hover_source_decl(int i) { if (!init && prev >= 0 && hs_word(i, prev) && prev > head) { candidate = prev; if (first < 0) first = prev; // directives and type aliases are not variable declarations. - if (!fm_is(i, head, "include") && !fm_is(i, head, "docs") && !fm_is(i, head, "hints") && !fm_is(i, head, "typecheck") && !fm_is(i, head, "type") && !fm_is(i, head, "enhance")) matched = hs_match(i, candidate); + if (!fm_is(i, head, "include") && !fm_is(i, head, "docs") && !fm_is(i, head, "lines") && !fm_is(i, head, "hints") && !fm_is(i, head, "typecheck") && !fm_is(i, head, "type") && !fm_is(i, head, "enhance")) matched = hs_match(i, candidate); } if (hs_equal(i, t)) init = 1; else { diff --git a/src/micro/micro.c b/src/micro/micro.c index f598c80..23994dd 100644 --- a/src/micro/micro.c +++ b/src/micro/micro.c @@ -530,6 +530,9 @@ static int run_from(int p, int s, int r) Cell dst = m[a + 2], cap = m[a + 3]; if (dst < 0 || cap <= n || (UCell)dst > mw || (UCell)cap > mw - (UCell)dst) result = -1; else { for (i = 0; i < n; i++) m[dst + i] = (unsigned char)text[i]; m[dst + n] = 0; result = n; } + } else if (b == 56) { + /* the return stack pointer: the word after the top return address */ + result = r; } else if (b == 42) { /* poll: a streaming port suspends once, so its host can deliver input that is waiting */ result = micro_syscall(m, b, 0, 0, 0); diff --git a/src/runtime/rt.zc b/src/runtime/rt.zc index ef5a5c6..4e52cb5 100644 --- a/src/runtime/rt.zc +++ b/src/runtime/rt.zc @@ -34,20 +34,20 @@ include run; // across targets and image sizes. // @map-rt -const MW = 0x1D5000; +const MW = 0x1DD000; const CODE_END = 0x80000; -const HEAP_LO = 0x1D5000; -const HEAP_HI = 0x235000; +const HEAP_LO = 0x1DD000; +const HEAP_HI = 0x23D000; const HEAP_GROW = 0x1; const ZFP = 0x90000; const GORG = 0x114100; -const FBOT = 0x174100; -const FTOP = 0x1B4100; -const RS0 = 0x1B4100; -const MARK0 = 0x1C4100; +const FBOT = 0x17C100; +const FTOP = 0x1BC100; +const RS0 = 0x1BC100; +const MARK0 = 0x1CC100; const MARKN = 0x8000; -const DSLO = 0x1CC100; -const DSHI = 0x1D4FF0; +const DSLO = 0x1D4100; +const DSHI = 0x1DCFF0; const MAGIC = 0x2B7E1516; // @end-map @@ -670,6 +670,7 @@ int scan_ptr(int p, int n) { // pointer (so this can run at all) and stops int frame_overflow() { put_s("zy: frame stack overflow\n"); + trace_at(1); halt(); return 0; } @@ -714,6 +715,7 @@ word frame_leave() { int oom() { outmode = 0; outgrow = 0; put_s("zc: out of heap memory\n"); + trace_at(1); halt(); return 0; } @@ -825,6 +827,7 @@ unchecked int free(.d p) { if (isobj(p) && (p[-4] & F_SHARE)) return 0; if ((dbg & 1) && quar(p)) { put_s("zy: double free from line "); put_n(site_line_of(p)); put_c(10); + trace_at(1); return 0; } if (isobj(p)) { @@ -840,6 +843,7 @@ unchecked int recycle(.d p) { word b; int n, i = 0; word w; if ((dbg & 1) && quar(p)) { put_s("zy: double free from line "); put_n(site_line_of(p)); put_c(10); + trace_at(1); return 0; } if (!isobj(p) || (p[-4] & F_SHARE)) return 0; @@ -986,6 +990,99 @@ unchecked int dgrow(word h, int need) { free(old); return 0; } +// ---- line tables and backtraces ---- + +// with lines on (the default for a program) the compiler records, for every +// function it compiles, a line block among the code: [next, entry, end, name, +// file, count, then count words of offset << 32 | line]. name and file are +// packed byte strings (length word, then the bytes). each word after the +// count says that the code from entry + offset up to the next word's offset +// came from that source line. blocks are chained through line_blocks, newest +// first. a function nested in another (a closure) has its own block and the +// innermost block holding an address is the one that names it. the compiler +// fills line_blocks when it compiles in the same memory as the program; the +// compiler of an image writes the head into the image's own line_blocks. +// frame_namer, when the compiler is resident, is a function that returns the +// name (a zero-terminated character string) of the function holding an +// address, for frames in code without lines. +word line_blocks, frame_namer; +// op 56 of the syscall gate: the return stack pointer, the word after the top return address. +const int SYS_RSP = 56; +// a trace shows this many of the innermost frames, then skips to the last few +const int TRACE_HEAD = 24, TRACE_TAIL = 4; +// the line block of the innermost function whose code holds address a, or 0 +word line_block_of(int a) { + word b = line_blocks, best = 0; + while (b) { + if (b[1] <= a && a < b[2] && (!best || b[1] > best[1])) best = b; + b = b[0]; + } + return best; +} +// the source line of address a within its line block b: the last recorded line at or before it +int line_of(word b, int a) { + int i = 0, off = a - b[1], n = b[5], line = 0; word w; + while (i < n) { + w = b[6 + i]; + if ((w >> 32) > off) break; + line = w & 0xFFFFFFFF; + i++; + } + return line; +} +// write the hexadecimal form of a number +int put_hex(int n) { + char[20] b; int i = 0, d; + put_s("0x"); + if (!n) { put_c('0'); return 0; } + while (n && i < 16) { d = n & 15; if (d < 10) b[i++] = '0' + d; else b[i++] = 'a' + d - 10; n = n >> 4; } + while (i) put_c(b[--i]); + return 0; +} +// write the line for one frame, given the return address a that is on the +// return stack: its function, file and line when the compiler recorded them, +// else the address and the nearest name. code of the runtime and the compiler +// that has no lines is left out, as are words that are not return addresses. +// returns 1 when a line was written. +int trace_frame(int a) { + word b; char* name = 0; + if (a <= 0) return 0; + b = line_block_of(a - 1); + if (b) { + put_s(" at "); put_bs(b[4]); put_c(':'); put_n(line_of(b, a - 1)); put_s(" in "); put_bs(b[3]); put_c(10); + return 1; + } + if (a >= GORG && a < GORG[6]) return 0; + if (!callable_code(a)) return 0; + if (frame_namer) name = frame_namer(a); + put_s(" at "); put_hex(a); + if (name) { put_s(" in "); put_s(name); } + put_c(10); + return 1; +} +// print the call stack, innermost call first, one line per frame, from the +// return stack. skip is how many words at the top of the stack belong to the +// runtime's own calls and are left out: a function that calls trace_at passes 1 +// to leave out the return into itself. this reads the pointer with the gate +// store inline, since a call to syscall would add a frame of its own. +int trace_at(int skip) { + word g = ZFP + 2; int r, lo, shown = 0, left; word rs; + g[1] = 0; g[2] = 0; g[3] = 0; g[0] = SYS_RSP; + r = g[0]; + lo = return_low; + if (r <= lo || r - lo > 0x7FFFFFF) return 0; + r = r - skip; + while (r > lo && shown < TRACE_HEAD) { r--; rs = r; if (trace_frame(rs[0])) shown++; } + left = r - lo; + if (left > TRACE_TAIL + 8) { + put_s(" ... "); put_n(left - TRACE_TAIL); put_s(" more frames\n"); + r = lo + TRACE_TAIL; + } + while (r > lo) { r--; rs = r; trace_frame(rs[0]); } + return 0; +} +// print the current call stack without stopping +int trace() { trace_at(1); return 0; } // the out-of-range report for T[?] indexing (dat, dpop) int oob(int i, int n) { put_s("zy: index "); @@ -993,6 +1090,7 @@ int oob(int i, int n) { put_s(" out of range (length "); put_n(n); put_s(")\n"); + trace_at(1); halt(); return 0; } @@ -1276,9 +1374,10 @@ int checkptr(int w) { if (!(dbg & 1) || !w) return 0; if (w < HEAP_LO || w >= hstop) return 0; p = obj_base(w); - if (!p) { put_s("zy: bad heap pointer\n"); halt(); return 0; } + if (!p) { put_s("zy: bad heap pointer\n"); trace_at(1); halt(); return 0; } if (!(p[-4] & 1)) { put_s("zy: use after free from line "); put_n(site_line_of(p)); put_c(10); + trace_at(1); halt(); } return 0; diff --git a/src/tools/automap.sh b/src/tools/automap.sh index ff559ac..b4aceb7 100755 --- a/src/tools/automap.sh +++ b/src/tools/automap.sh @@ -35,8 +35,18 @@ lay() { # $1 = genome slot, or empty for the preset's own $CC -O1 -w -fwrapv src/tools/zc0.c -I"$O/map" -Isrc/core -o "$ZC0" $CC -O1 -w -fwrapv src/tools/mkgenome.c -I"$O/map" -o "$O/.mkgenome" } +# the files after the map's own. a run-only image gets line tables for its program (so a fault +# prints a backtrace): zc0 reads the comment //:lines NAME as "from here on, file NAME, count +# its lines from the next one". the files under src/ are the runtime's and have none. +feed() { + local f + for f in "$@"; do + if [ "$P" = run ]; then case "$f" in src/*) ;; *) printf '\n//:lines %s\n' "$f" ;; esac; fi + cat "$f" + done +} build() { - if [ -f "$O/map/compiler.zc" ]; then cat "$O/map/rt.zc" src/core/core.zc "$O/map/compiler.zc" "$@"; else cat "$O/map/rt.zc" src/core/core.zc "$@"; fi | "$ZC0" > "$O/image.bin" + if [ -f "$O/map/compiler.zc" ]; then { cat "$O/map/rt.zc" src/core/core.zc "$O/map/compiler.zc"; feed "$@"; }; else { cat "$O/map/rt.zc" src/core/core.zc; feed "$@"; }; fi | "$ZC0" > "$O/image.bin" "$O/.mkgenome" < "$O/image.bin" > "$O/genome.bin" } # with DOCS=1 the genome is made again with zc0 built for docs (ZC0DOCS), so the @@ -44,7 +54,7 @@ build() { # docofs(). the image is the same code either way; the docs are stored after it. docs() { $CC -O1 -w -fwrapv -DZC0DOCS src/tools/zc0.c -I"$O/map" -Isrc/core -o "$O/.zc0docs" - if [ -f "$O/map/compiler.zc" ]; then cat "$O/map/rt.zc" src/core/core.zc "$O/map/compiler.zc" "$@"; else cat "$O/map/rt.zc" src/core/core.zc "$@"; fi | "$O/.zc0docs" > "$O/image.bin" + if [ -f "$O/map/compiler.zc" ]; then { cat "$O/map/rt.zc" src/core/core.zc "$O/map/compiler.zc"; feed "$@"; }; else { cat "$O/map/rt.zc" src/core/core.zc; feed "$@"; }; fi | "$O/.zc0docs" > "$O/image.bin" "$O/.mkgenome" < "$O/image.bin" > "$O/genome.bin" } len() { echo $(( $(wc -c < "$1") / 8 )); } diff --git a/src/tools/build.zc b/src/tools/build.zc index b0e3da0..b628a33 100644 --- a/src/tools/build.zc +++ b/src/tools/build.zc @@ -6,6 +6,9 @@ include rt, run; docs on; +// the genome has no line tables: its code is the runtime's and the compiler's, and a program +// built into an image turns them on for itself (lines on;) +lines off; // writes the compiled genome image to stdout as little-endian 64-bit words. int main() { int n = run.build_image_grow(rt.GORG, 4096), i = 0, w; diff --git a/src/tools/genmap.c b/src/tools/genmap.c index d515778..6d8db55 100644 --- a/src/tools/genmap.c +++ b/src/tools/genmap.c @@ -242,7 +242,7 @@ static void preset_default(Spec *s) s->compiler = 1; s->heap_top = 1; s->code_end = 0x80000; s->types = 65536; s->heap = 0x60000; s->total = 0x8000000; - s->genome = 0x60000; s->frame = 0x40000; s->ret = 0x10000; s->mark = 0x8000; + s->genome = 0x68000; s->frame = 0x40000; s->ret = 0x10000; s->mark = 0x8000; s->data = 0x7FF0; tables_big(s); } diff --git a/src/tools/parity.sh b/src/tools/parity.sh index 00245a8..2b9b870 100755 --- a/src/tools/parity.sh +++ b/src/tools/parity.sh @@ -63,7 +63,9 @@ KNOWN=tests/parity.known mkdir -p "$W" # what every image starts with, and how many lines that is (zc0 counts from the top) SRCS="src/runtime/rt.zc src/core/core.zc src/runtime/run.zc" -{ cat $SRCS; echo 'namespace;'; } > "$W/prefix.new" +# the marker turns zc0's line tables on for the program, named <stdin> as under zy, so faults +# and rt.trace() print the same backtrace on both sides (src/tools/automap.sh does the same) +{ cat $SRCS; echo 'namespace;'; echo '//:lines <stdin>'; } > "$W/prefix.new" if cmp -s "$W/prefix.new" "$W/prefix.zc"; then rm "$W/prefix.new"; else mv "$W/prefix.new" "$W/prefix.zc"; fi nprefix=$(wc -l < "$W/prefix.zc") # what the compiled image depends on besides the program diff --git a/src/tools/zc0.c b/src/tools/zc0.c index 040a4ea..5846e22 100644 --- a/src/tools/zc0.c +++ b/src/tools/zc0.c @@ -73,6 +73,13 @@ static int esc(void) switch (c) { case 'n': return 10; case 't': return 9; case 'r': return 13; case '0': return 0; case 'e': return 27; } return c; } +/* line tables, as the self-hosted compiler makes them with lines on (see rt.line_blocks in rt.zc). + they are off until a program says lines on; or a marker comment //:lines NAME comes by, which + src/tools/automap.sh puts in front of the program of a run-only image: it turns them on, names + the file, and counts the lines from the next one. */ +static int zl_on, zl_base, zl_head, zl_entry, zl_open, zl_fline, zl_file_at, zl_n, zl_cap; +static char zl_file[256] = "<stdin>", zl_file_done[256], zl_name[64]; +static int64_t *zl_pairs; /* doc comments, as the self-hosted compiler reads them with docs on: the // lines directly above a declaration, up to a blank line or any other text */ static char dbuf[2100], dsnap[2100], dnm[64]; static int dn, dlast, dline, dsn, docpending; @@ -82,7 +89,15 @@ static void doccomment(void) /* pos is on the first slash */ while (k >= 0 && (src[k] == ' ' || src[k] == '\t')) k--; bol = k < 0 || src[k] == '\n'; pos += 2; - if (src[pos] == ':') { dn = 0; dlast = cl; docpending = 0; while (src[pos] && src[pos] != '\n') pos++; return; } + if (src[pos] == ':') { + if (!strncmp(src + pos, ":lines ", 7)) { + int n = 0; + pos += 7; + while (src[pos] && src[pos] != '\n' && src[pos] != '\r' && n < 255) zl_file[n++] = src[pos++]; + zl_file[n] = 0; zl_on = 1; zl_base = cl; + } + dn = 0; dlast = cl; docpending = 0; while (src[pos] && src[pos] != '\n') pos++; return; + } if (!bol) { while (src[pos] && src[pos] != '\n') pos++; return; } if (dn && dlast != cl - 1) dn = 0; if (src[pos] == ' ') pos++; @@ -500,6 +515,42 @@ static int bstrwords(const unsigned char *b, long n) if ((n & 7) == 0) emit(0); return a; } +/* line tables. zl_begin starts recording the function whose code begins at entry, zl_stmt notes + that a statement starts at the current line, and zl_end puts the function's line block among + the code, as lt_end does in the resident compiler: [next, entry, end, name, file, count, then + count words of offset << 32 | line]. */ +static void zl_begin(int entry, const char *name, int fline) +{ + if (!zl_on) { zl_open = 0; return; } + zl_open = 1; zl_entry = entry; zl_n = 0; zl_fline = fline - zl_base; + strncpy(zl_name, name, 63); zl_name[63] = 0; + if (zl_n == zl_cap) { zl_cap = zl_cap ? zl_cap * 2 : 1024; zl_pairs = realloc(zl_pairs, (size_t)zl_cap * sizeof *zl_pairs); } + zl_pairs[zl_n++] = zl_fline; +} +static void zl_stmt(void) +{ + int off = here - zl_entry; + if (!zl_open) return; + if (zl_n == zl_cap) { zl_cap *= 2; zl_pairs = realloc(zl_pairs, (size_t)zl_cap * sizeof *zl_pairs); } + if (zl_n && (zl_pairs[zl_n - 1] >> 32) == off) zl_n--; + if (zl_n && (zl_pairs[zl_n - 1] & 0xffffffffLL) == line - zl_base) return; + zl_pairs[zl_n++] = (int64_t)off * 4294967296LL + (line - zl_base); +} +static void zl_end(void) +{ + int end = here, name, file, blk, i; + if (!zl_open) return; + zl_open = 0; + name = bstrwords((const unsigned char *)zl_name, (long)strlen(zl_name)); + if (!zl_file_at || strcmp(zl_file, zl_file_done)) { + zl_file_at = bstrwords((const unsigned char *)zl_file, (long)strlen(zl_file)); strcpy(zl_file_done, zl_file); + } + file = zl_file_at; + blk = here; + emit(zl_head); emit(zl_entry); emit(end); emit(name); emit(file); emit(zl_n); + for (i = 0; i < zl_n; i++) emit(zl_pairs[i]); + zl_head = blk; +} /* b"...": the words inline, jumped over, then the address pushed */ static void bstrlit(void) { @@ -691,6 +742,7 @@ static void patchbreaks(void) { int i; for (i = 0; i < nbrk[nloop]; i++) setlit( static void stmt(void) { int a, b, c, d; + zl_stmt(); if (accept(";")) return; if (is("{")) { block(); return; } { @@ -759,7 +811,7 @@ static void stmt(void) /* ---- top level ---- */ static void fndef(int ret, int rstr) /* rstr: the return type is bstr */ { - char *n = ident(); Sym *s; int np = 0, tags = 0, t, frame, i, helper, busy, done, check, node; + char *n = ident(); int zfl = line; Sym *s; int np = 0, tags = 0, t, frame, i, helper, busy, done, check, node; if (dsn && !dnm[0]) strcpy(dnm, n); char pn[16][64]; int pstr[16]; char full_name[256]; qualify(full_name, sizeof full_name, n); s = find_raw(full_name); @@ -794,6 +846,7 @@ static void fndef(int ret, int rstr) /* rstr: the return type is bstr */ } #endif nlo = 0; nslot = 0; nepi = 0; infn = 1; + zl_begin(here, s->name, zfl); for (i = 0; i < np; i++) { strcpy(lo[nlo].name, pn[i]); lo[nlo].arr = 0; lo[nlo].flt = 0; lo[nlo].dstr = lo[nlo].str = pstr[i & 15]; lo[nlo++].slot = nslot++; } /* retain the caller's pointer until the frame has its final segment. */ helper = !find("frame_busy") || !strcmp(s->name, "rt.frame_grow") || !strcmp(s->name, "rt.frame_leave"); @@ -823,6 +876,7 @@ static void fndef(int ret, int rstr) /* rstr: the return type is bstr */ op(O_RET); setlit(frame, nslot); infn = 0; + zl_end(); } /* enhance name(params) [-> n] micro "recipe"; or ... vm { asm }; emits the install descriptor; the booted genome installs it */ @@ -953,6 +1007,11 @@ static void top1(void) expect(";"); return; } if (accept("include")) { do { ident(); } while (accept(",")); expect(";"); return; } + if (is("lines")) { /* lines on; and lines off; (anything else is a name) */ + Lex L; lexsave(&L); next(); + if (is("on") || is("off")) { zl_on = is("on"); next(); expect(";"); return; } + lexrest(&L); + } if (accept("embed")) { embeddecl(0); return; } { if (accept("unchecked")) { top1(); return; } @@ -1088,6 +1147,8 @@ int main(int argc, char **argv) setlit(fx[i].at, s->val); } if (!(st = find("start")) || !st->defd) { fprintf(stderr, "zc0: no start\n"); return 1; } + /* the line tables are reached through rt.line_blocks, which starts out as the newest block */ + if (zl_head && (st = find_raw("rt.line_blocks"))) img[st->val - GORG] = zl_head; /* symbol blob: kind, value, sig, name chars, 0 ... then a 0 kind */ img[5] = here; for (i = 0; i < nsy; i++) { diff --git a/tests/drivers/runtime_image_test.py b/tests/drivers/runtime_image_test.py index 6c44349..64aa353 100644 --- a/tests/drivers/runtime_image_test.py +++ b/tests/drivers/runtime_image_test.py @@ -17,6 +17,7 @@ args = parser.parse_args() cases = [ ('enhancements', 'include rt; fn main() { rt.put_n(17 % 5); rt.put_c(32); rt.put_n(17 / 5); }', b'2 3'), ('array algorithms', Path('examples/learn/31-array-algorithms.zc').read_text(), Path('examples/learn/31-array-algorithms.out').read_bytes()), + ('line tables', '\n//:lines trace.zc\n' + Path('tests/programs/trace.zc').read_text(), Path('tests/programs/trace.expected').read_bytes().replace(b'<stdin>', b'trace.zc')), ('owned pop', 'include rt; int*.u take() { int*[?] a = rt.new(int*[?]); int* p = rt.new(int); *p = 8; a.push(p); return a.pop(); } fn main() { let p = take(); rt.put_n(rt.isobj(p)); rt.put_c(32); rt.put_n(*p); }', b'1 8'), # main may be void: run.zc's prototype of main must not fix its type (docs/SPEC.md 5.2.5) ('void main', Path('tests/programs/void_main.zc').read_text(), b'void main\n'), diff --git a/tests/programs/gc.zc b/tests/programs/gc.zc index 7d2441b..44fde5d 100644 --- a/tests/programs/gc.zc +++ b/tests/programs/gc.zc @@ -12,7 +12,7 @@ fn main() { churn(1000); rt.put_n(rt.live_objects() - base); rt.put_c(' '); f = rt.gc(); - rt.put_n(f >= 990); rt.put_c(' '); // conservative: a stale word or two may keep one alive + rt.put_n(f >= 950); rt.put_c(' '); // conservative: stale words, and genome code words that happen to look like heap addresses, keep a few alive rt.put_n(keep[0]); rt.put_c(' '); // the one a global still points at survives churn(200000); // 3.2m words through a 393k word heap: needs the automatic collector rt.put_n(rt.gc_runs > 1); rt.put_c(' '); diff --git a/tests/programs/trace.expected b/tests/programs/trace.expected new file mode 100644 index 0000000..2c7dce3 --- /dev/null +++ b/tests/programs/trace.expected @@ -0,0 +1,9 @@ + at <stdin>:18 in where + at <stdin>:23 in outer + at <stdin>:30 in main +1 +10 +zy: index 4 out of range (length 4) + at <stdin>:5 in get + at <stdin>:11 in sum + at <stdin>:34 in main diff --git a/tests/programs/trace.zc b/tests/programs/trace.zc new file mode 100644 index 0000000..c283f75 --- /dev/null +++ b/tests/programs/trace.zc @@ -0,0 +1,36 @@ +namespace; +include rt; + +int get(int[?] t, int i) { + return t[i]; +} + +int sum(int[?] t, int n) { + int s = 0, i = 0; + while (i < n) { + s = s + get(t, i); + i++; + } + return s; +} + +int where() { + rt.trace(); + return 1; +} + +int outer() { + int r = where(); + return r; +} + +int main() { + int[?] t = rt.new(int[?]); + t.push(1); t.push(2); t.push(3); t.push(4); + rt.put_n(outer()); + rt.put_c(10); + rt.put_n(sum(t, 4)); + rt.put_c(10); + rt.put_n(sum(t, 5)); + return 0; +} diff --git a/tests/programs/trace_jit.expected b/tests/programs/trace_jit.expected new file mode 100644 index 0000000..68e2eb4 --- /dev/null +++ b/tests/programs/trace_jit.expected @@ -0,0 +1,6 @@ + at <stdin>:9 in down + at <stdin>:12 in down + at <stdin>:12 in down + at <stdin>:12 in down + at <stdin>:18 in main +20000 diff --git a/tests/programs/trace_jit.zc b/tests/programs/trace_jit.zc new file mode 100644 index 0000000..e864c50 --- /dev/null +++ b/tests/programs/trace_jit.zc @@ -0,0 +1,24 @@ +namespace; +include rt; + +// a hot path: the jits have translated down() and the loop in main long before +// the last call, so the return stack pointer comes from translated code (op 56 +// reads it at the gate store, cb_r on both jits) +int down(int n, int last) { + if (n == 0) { + if (last) rt.trace(); + return 1; + } + return down(n - 1, last) + 1; +} + +int main() { + int i = 0, s = 0; + while (i < 5000) { + s = s + down(3, i == 4999); + i++; + } + rt.put_n(s); + rt.put_c(10); + return 0; +} diff --git a/tests/programs/trace_more.expected b/tests/programs/trace_more.expected new file mode 100644 index 0000000..b35d42d --- /dev/null +++ b/tests/programs/trace_more.expected @@ -0,0 +1,38 @@ + at <stdin>:6 in pt.get + at <stdin>:31 in main +3 + at <stdin>:34 in <closure> + at <stdin>:11 in apply + at <stdin>:32 in main +9 + at <stdin>:14 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:15 in rec + ... 35 more frames + at <stdin>:15 in rec + at <stdin>:15 in rec + at <stdin>:37 in main + at ADDR in quiet + at <stdin>:24 in loud + at <stdin>:38 in main diff --git a/tests/programs/trace_more.zc b/tests/programs/trace_more.zc new file mode 100644 index 0000000..512f93f --- /dev/null +++ b/tests/programs/trace_more.zc @@ -0,0 +1,40 @@ +namespace; +include rt; +struct pt { + int x; + int get() { + rt.trace(); + return this.x; + } +}; +int apply(fn(int) f, int v) { + return f(v); +} +int rec(int n) { + if (n == 0) { rt.trace(); return 0; } + return rec(n - 1) + 1; +} +lines off; +int quiet(int n) { + rt.trace(); + return n; +} +lines on; +int loud(int n) { + return quiet(n) + + 1; +} +int main() { + pt* p = rt.new(pt); + p.x = 3; + int k = 5; + rt.put_n(p.get()); rt.put_c(10); + rt.put_n(apply(fn(int a) { + int b = a + k; + rt.trace(); + return b; + }, 4)); rt.put_c(10); + rec(60); + loud(2); + return 0; +} diff --git a/tests/programs/trace_run.zc b/tests/programs/trace_run.zc new file mode 100644 index 0000000..1d42e26 --- /dev/null +++ b/tests/programs/trace_run.zc @@ -0,0 +1,36 @@ +namespace; +include rt; + +int get(word t, int i) { + if (i >= 4) rt.oob(i, 4); return t[i]; +} + +int sum(word t, int n) { + int s = 0, i = 0; + while (i < n) { + s = s + get(t, i); + i++; + } + return s; +} + +int where() { + rt.trace(); + return 1; +} + +int outer() { + int r = where(); + return r; +} + +int main() { + word t = rt.new(4); + t[0] = 1; t[1] = 2; t[2] = 3; t[3] = 4; + rt.put_n(outer()); + rt.put_c(10); + rt.put_n(sum(t, 4)); + rt.put_c(10); + rt.put_n(sum(t, 5)); + return 0; +} diff --git a/tests/test.sh b/tests/test.sh index 22fdf30..ec51838 100755 --- a/tests/test.sh +++ b/tests/test.sh @@ -166,9 +166,12 @@ hello world!42 v" "$l a full capture is still terminated inside its buffer" chk "$($vm < tests/programs/sys.zc)" "2 zy time" "$l syscalls: write, read, time" chk "$($vm < tests/programs/memleak.zc)" $'leak: line 8 3 words\n1' "$l memdebug names a live allocation" - chk "$($vm < tests/programs/uaf.zc)" $'zc: line 10: warning: use after free: this storage\'s lifetime ended on line 9\nzy: use after free from line 7' "$l memdebug: use after free" - chk "$($vm < tests/programs/uaf_in.zc)" "zy: use after free from line 7" "$l memdebug: use after free through an interior pointer" - chk "$($vm < tests/programs/doublefree.zc)" $'zy: double free from line 7\nok' "$l memdebug: double free is reported and does nothing" + chk "$($vm < tests/programs/uaf.zc)" $'zc: line 10: warning: use after free: this storage\'s lifetime ended on line 9\nzy: use after free from line 7\n at <stdin>:10 in main' "$l memdebug: use after free" + chk "$($vm < tests/programs/uaf_in.zc)" $'zy: use after free from line 7\n at <stdin>:11 in main' "$l memdebug: use after free through an interior pointer" + chk "$($vm < tests/programs/doublefree.zc)" $'zy: double free from line 7\n at <stdin>:9 in main\nok' "$l memdebug: double free is reported and does nothing" + chk "$($vm < tests/programs/trace.zc 2>&1 | md5sum)" "$(md5sum < tests/programs/trace.expected)" "$l rt.trace() and a fault print the call stack with file, line and function" + chk "$($vm < tests/programs/trace_more.zc 2>&1 | sed 's/0x[0-9a-f]*/ADDR/' | md5sum)" "$(md5sum < tests/programs/trace_more.expected)" "$l backtraces: methods, closures, lines off, a deep stack is cut to its ends" + chk "$($vm < tests/programs/trace_jit.zc 2>&1 | md5sum)" "$(md5sum < tests/programs/trace_jit.expected)" "$l rt.trace() from code the jit has translated: op 56 reads the return stack pointer there too" chk "$($vm < tests/programs/scrub.zc)" $'zc: line 9: warning: use after free: this storage\'s lifetime ended on line 8\n57005' "$l memdebug scrub poisons a freed payload" chk "$($vm < tests/programs/interior.zc)" "0" "$l an interior pointer keeps a quarantined block" chk "$($vm < tests/programs/wide.zc)" "1099511627776 1 1 1 -1" "$l 64-bit shifts and equality" @@ -193,7 +196,7 @@ ABC" "$l o3 lifts byte loads and stores" chk "$($vm < tests/programs/manyerrors.zc | tr '\n' '/')" "zc: line 6: expected an expression, found ';'/zc: line 7: x: owned object is never used/zc: line 8: type error: initializer: expected char*, got int/zc: line 9: expected ';', found '}'/zc: 4 errors/" "$l several errors in one run" chk "$($vm < tests/programs/overflow_return.zc 2>&1)" "zy: return stack overflow" "$l return stack overflow is caught" chk "$($vm < tests/programs/overflow_data.zc 2>&1)" "zy: data stack overflow" "$l data stack overflow is caught" - chk "$($vm < tests/programs/overflow_frame.zc 2>&1)" "zc: out of heap memory" "$l frame growth stops at a configured memory limit" + chk "$($vm < tests/programs/overflow_frame.zc 2>&1 | head -1)" "zc: out of heap memory" "$l frame growth stops at a configured memory limit" chk "$($vm < examples/mandel.zc 2>&1 | md5sum | cut -c1-32)" "f41aecbf2c844168050413cfa832db0e" "$l example: mandelbrot with a fixed-point enhancement" chk "$($vm < examples/bf.zc 2>&1 | md5sum | cut -c1-32)" "223570983752b17f7ece32e93d5e19e4" "$l example: brainfuck compiled at runtime" chk "$($vm < examples/dump.zc 2>&1 | md5sum | cut -c1-32)" "757774b00f6b3cd0bfbcba66448a8619" "$l example: struct printer through reflection" @@ -216,10 +219,10 @@ ABC" "$l o3 lifts byte loads and stores" chk "$($vm < tests/programs/o3licm.zc)" "3184200 7950 315 50 260 -56" "$l o3 hoists loop invariants and shares values between blocks" chk "$($vm < tests/programs/o3deep.zc)" "30000" "$l o3 recursion thirty thousand deep" chk "$($vm < tests/programs/o3deep_over.zc 2>&1)" "100000" "$l o3 recursion grows past the original stack" - chk "$($vm < tests/programs/o3_bounds.zc 2>&1)" "zy: index 7 out of range (length 5)" "$l o3 bounds check" + chk "$($vm < tests/programs/o3_bounds.zc 2>&1 | head -1)" "zy: index 7 out of range (length 5)" "$l o3 bounds check" chk "$($vm < tests/programs/bounds.zc)" "55 55 8" "$l a proved +1 loop skips the bounds check" - chk "$($vm < tests/programs/bounds_lo.zc 2>&1)" "zy: index -1 out of range (length 4)" "$l a negative loop index is still checked" - chk "$($vm < tests/programs/bounds_hi.zc 2>&1)" "zy: index 4 out of range (length 4)" "$l an index that is not the induction variable is still checked" + chk "$($vm < tests/programs/bounds_lo.zc 2>&1 | head -1)" "zy: index -1 out of range (length 4)" "$l a negative loop index is still checked" + chk "$($vm < tests/programs/bounds_hi.zc 2>&1 | head -1)" "zy: index 4 out of range (length 4)" "$l an index that is not the induction variable is still checked" local r; r=$(bash tests/runerr.sh "$vm" tests/errors | tail -1) chk "$r" "rejected: $(ls tests/errors/*.zc | wc -l | tr -d ' ') passed, 0 failed" "$l analyzer rejects every bad program" r=$(bash tests/runerr.sh "$vm" tests/typeerrors | tail -1) @@ -350,7 +353,7 @@ s_runonly() { GENMAP=$T/genmap bash src/tools/automap.sh run $d src/runtime/run.zc "$input" >/dev/null || { chk 0 1 "run $1 builds"; return; } cp src/micro/micro.c src/micro/micro_port.h src/micro/port_linux.c src/micro/port_stub.c src/core/fops.h src/core/bops.h src/core/stack.h src/core/genome.h $d/ $CC -pthread -DZY_THREADS -O2 -w -fwrapv $d/micro.c $d/port_linux.c -o $T/zymr || { chk 0 1 "run $1 links"; return; } - a=$($T/zy < "$input" 2>&1 | tr '\n' '/') + a=$($T/zy < "$input" 2>&1 | sed "s#<stdin>#$input#g" | tr '\n' '/') b=$($T/zymr </dev/null 2>&1 | tr '\n' '/') chk "$b" "$a" "run $1" } @@ -360,6 +363,7 @@ s_runonly() { runonly selfpatch examples/selfpatch.zc runonly gc tests/programs/gc.zc runonly voidmain tests/programs/void_main.zc + runonly trace tests/programs/trace_run.zc # the image carries line tables: a fault and rt.trace() print the same backtrace as the resident compiler's runonly float tests/programs/float.zc runonly ftruth tests/programs/ftruth.zc runonly ftext tests/programs/ftext.zc @@ -382,6 +386,9 @@ s_runonly() { 0" "zcc: a program that never reads stdin does not wait for it" if [ "$(uname -s)" = Linux ]; then chk "$(file -b $T/zcc-hello | grep -c 'statically linked')" "1" "zcc: the binary is statically linked"; else echo "skip zcc static linkage (unavailable on this host)"; fi else chk 0 1 "zcc builds examples/hello.zc"; fi + if src/tools/zcc -o $T/zcc-trace tests/programs/trace_run.zc >/dev/null 2>&1; then + chk "$($T/zcc-trace </dev/null 2>&1 | sed "s#$PWD/##g" | md5sum)" "$($T/zy < tests/programs/trace_run.zc 2>&1 | sed 's#<stdin>#tests/programs/trace_run.zc#g' | md5sum)" "zcc: a fault in a compiled program prints its backtrace" + else chk 0 1 "zcc builds tests/programs/trace_run.zc"; fi printf 'fn main() {\n\tput_n(1\n\treturn 0;\n}\n' > $T/zcc-bad.zc src/tools/zcc -o $T/zcc-bad $T/zcc-bad.zc </dev/null >/dev/null 2>&1; r=$? chk "$r $( [ -e $T/zcc-bad ] && echo yes || echo no)" "1 no" "zcc: a bad program exits 1 and writes no binary" @@ -653,6 +660,7 @@ s_js() { chk "$(node $T/jsvm/tinyvm.js examples/fact.zc | tail -1)" "3628800" "js vm: factorial" chk "$(node $T/jsvm/tinyvm.js examples/primes.zc | tail -1)" "widest gap: 72, after 31397" "js vm: primes" chk "$(node $T/jsvm/tinyvm.js tests/programs/growone.zc)" "12 1" "js vm: a block past the initial heap grows the machine" + chk "$(node $T/jsvm/tinyvm.js tests/programs/trace.zc 2>&1 | md5sum)" "$(md5sum < tests/programs/trace.expected)" "js vm: backtraces read the return stack through gate op 56" chk "$(node $T/jsvm/tinyvm.js tests/programs/replace_owner.zc)" "$(cat tests/programs/replace_owner.expected)" "js vm: replacement ownership and cleanup" chk "$(node $T/jsvm/tinyvm.js tests/programs/embed.zc | tr '\n' '/')" "hello/world 11 42 5 255 8 -1 0 119 15 5 255 3 98 321 63/" "js vm: embed declarations read their files through node" if sh src/tools/findcc.sh --wasm >/dev/null 2>&1 && command -v wasm-ld >/dev/null; then |
