merge: seven self-hosted compiler fixes from conformance

13 files changed, 46 insertions, 33 deletions

diff --git a/TODO.md b/TODO.md
index 7a892a3..7e21881 100644
--- a/TODO.md
+++ b/TODO.md
@@ -479,8 +479,8 @@ items with an architecture note carry the design worked out so far, enough to
 brief someone without re-reading the code.
 
 - [ ] remaining zc0 gaps: string literals over 4095 bytes, `embed` of an array type inside a function, `x as T` with more than a single name (`x as int*`), zc0 silently accepting `.u` on a parameter, and integer literals (`010` is 8 in zc0, 10 in the compiler; zc0 saturates on overflow). `docs/history/43-stdlib/notes.md` still says zc0 reserves u/m/d: add a pointer forward rather than rewriting history.
-- [x] verify both compilers against `docs/SPEC.md`: `tests/conformance/<section>/` (sections 1 to 8 plus `9-appendix-b`, 657 programs), runner `tests/conformance/run.sh` (`--zc0`, `--zym`, `--zy`, `-w`), understood failures in `tests/conformance/known.txt` (class a self-hosted bug, b zc0, c spec issue), wired into the interp section of `tests/test.sh`. zy 626 ok, 31 known; zc0 221 ok, 38 known, 398 skipped (outside appendix a). appendix b.2 items 1 to 16 confirmed except b.2.8, which contradicts 5.3.2 and 3.14.2 (the spec documents the address). new class a bugs: `case member():` on a member with fields is rejected (4.7.3); top-level `o3 { }` is rejected (4.10.2, class c). open: `rt.memreport()` reports buddy capacity (`rt.new(2)` is 3 words), whether that is the spec's W; whether top-level `unchecked;` is in the zc0 subset.
-- [ ] fix the implementation bugs listed in `docs/SPEC.md` appendix b, once confirmed: `memdebug scrub;` rejected; enum members can't reference earlier members; a plain function redefinition silently shadows the first; a prototype after the definition hides it; `?:` with one int and one float branch gives garbage; `bool` not normalised; float global initializers rejected; a global initializer naming another global uses its address; `include std;` inside a function body; syntax-error recovery hiding later errors; unterminated `/*` and string literals; `x as int * y`; prototypes needing parameter names; `len(char*)` with typecheck off; duplicate `case` labels; local `const` in constant expressions; constant expressions without precedence (`1+2*3` is 9). fix the prose items the appendix lists too.
+- [x] verify both compilers against `docs/SPEC.md`: `tests/conformance/<section>/` (sections 1 to 8 plus `9-appendix-b`, 657 programs), runner `tests/conformance/run.sh` (`--zc0`, `--zym`, `--zy`, `-w`), understood failures in `tests/conformance/known.txt` (class a self-hosted bug, b zc0, c spec issue), wired into the interp section of `tests/test.sh`. zy 626 ok, 31 known; zc0 221 ok, 38 known, 398 skipped (outside appendix a). appendix b.2 items 1 to 16 confirmed except b.2.8, which is documented behavior (5.3.2, 3.14.2; the spec wins). top-level `o3 { }` is rejected (4.10.2, class c, left alone). the seven class a bugs are fixed, one commit each: a bad literal in a function's first statement (capture_scan left panic set), `u*d` with visible `u`/`d`, the bstr flag surviving a cast, the 24-parameter limit, `return e;` in a void function (now dropped, yields 0), `memdebug scrub;`, and `case member():` on a member with fields. the conformance run is in `tests/test.sh` but not in `-q`. open: `rt.memreport()` reports buddy capacity (`rt.new(2)` is 3 words), whether that is the spec's W; whether top-level `unchecked;` is in the zc0 subset.
+- [ ] fix the implementation bugs listed in `docs/SPEC.md` appendix b, once confirmed: enum members can't reference earlier members; a plain function redefinition silently shadows the first; a prototype after the definition hides it; `?:` with one int and one float branch gives garbage; `bool` not normalised; float global initializers rejected; `include std;` inside a function body; syntax-error recovery hiding later errors; unterminated `/*` and string literals; `x as int * y`; prototypes needing parameter names; `len(char*)` with typecheck off; duplicate `case` labels; local `const` in constant expressions; constant expressions without precedence (`1+2*3` is 9). (`memdebug scrub;` is fixed; a global initializer naming another global giving its address is documented behavior, 5.3.2.) fix the prose items the appendix lists too.
 - [ ] (partly done, see "agent work at the end of the session") a cross-reference report: for every global of the compiler, the functions that write it and the ones that only read it. build it from the language server's references (`textDocument/references` on `build/bin/zclsp`, so scoping is exact) and classify each reference as a write when an assignment operator, `++` or `--` follows it. output under `build/xref/`; a committed baseline and a `--check` mode that fails when a new writer of a global appears, so it can run in ci. 
 - [ ] optional, low priority: plain enums and `?:` in zc0 (both are small and produce ordinary code), so parity covers more programs. no structs in zc0 (decided).
 - [ ] group the compiler's parallel per-local arrays (`lname`, `ltype`, `lstate`, `lhome`, `lown`, `lslot`, `ltag`, `lpos`, `lnsz`, `lleaf`, ...) without structs: declare each family together in one place, with one comment block stating each array's meaning and invariant (for example when `lhome` is meaningful), route snapshot and restore of the family through one pair of functions, and use the cross-reference report to confirm nothing else writes them. then symbols and types the same way. fixpoint and analyzer behavior unchanged.
diff --git a/src/compiler/builtins.zc b/src/compiler/builtins.zc
index 5d7bc9c..d04c444 100644
--- a/src/compiler/builtins.zc
+++ b/src/compiler/builtins.zc
@@ -7,7 +7,7 @@ int kw_memdebug() {
 	stmtonly();
 	next();
 	if (is_id(s_on)) { mdbg = 1; f = 5; }
-	else if (is_id(s_scrub)) { mdbg = 1; f = 7; }
+	else if (tk == T_ID && rt.eq_w(tv, "scrub")) { mdbg = 1; f = 7; }	// s_scrub is the runtime function rt.scrub, not this word
 	else if (is_id(s_off)) { mdbg = 0; f = 0; }
 	else die("memdebug on, memdebug scrub or memdebug off");
 	next();
diff --git a/src/compiler/decl.zc b/src/compiler/decl.zc
index cc84603..0870756 100644
--- a/src/compiler/decl.zc
+++ b/src/compiler/decl.zc
@@ -326,8 +326,8 @@ int fndef(int chk, int rty, int ret, int n) {
 			pt[np] = t;
 			if (np < 12) tags = tags + (t << (np + np));
 			np++;
-			if (np >= 24) die("too many parameters");
 			if (!accept(s_comma)) break;
+			if (np >= 24) { die("too many parameters"); break; }	// a 25th would not fit pn, pt, pty
 		}
 	}
 	expect(s_rp);
diff --git a/src/compiler/expr.zc b/src/compiler/expr.zc
index f41109e..3d63e4b 100644
--- a/src/compiler/expr.zc
+++ b/src/compiler/expr.zc
@@ -538,7 +538,7 @@ int enhcall(int s) {
 // scan with the compiler lexer, restoring its source and lookahead. only functions containing a closure
 // need cells; their scalars get cells from declaration time, so loop back edges and earlier references agree.
 int capture_scan() {
- word old, f; int depth = 1, found = 0, docs = docs_on, record = grec;
+ word old, f; int depth = 1, found = 0, docs = docs_on, record = grec, sp = panic;
  if (isp < 0 || R_SRC[isp * 32] == 0 || !is_op(s_lb)) return 0;
  old = R_SRC + isp * 32;
  src_push(1, capture_dummy, 0, 0); old = R_SRC + (isp - 1) * 32; f = R_SRC + isp * 32;
@@ -550,7 +550,7 @@ int capture_scan() {
   else if (is_op(s_rb)) depth--;
   else if (is_id(s_fn)) { peek(); if (pk_tk == T_OP && (pk_tv == s_lp || pk_tv == s_colon)) found = 1; }
  }
- dry--; src_pop(); docs_on = docs; grec = record; return found;
+ dry--; src_pop(); docs_on = docs; grec = record; panic = sp; return found;	// a lexer error in the dry scan must not leave panic set for the real parse
 }
 int capture_cell(int i, int copy) {
  if (!capture_enabled || o3on || (ltag[i] & 16) || larr[i] || local_cell[i]) return 0;
@@ -1512,7 +1512,7 @@ int postfix_tail(int c) {
 // --, and any unary operator a program added (- ! ~ built in).
 int unary() {
 	int c, un, t, place, constant, value; word storage, dependency;
-	if (tk == T_ID && (tv == s_u || tv == s_m || tv == s_d)) {	// u*x, d*x, m*x: recast only when * is glued
+	if (tk == T_ID && (tv == s_u || tv == s_m || tv == s_d) && findloc(tv) < 0) {	// u*x, d*x, m*x: recast only when * is glued and the tag is not a local's name
 		peek();
 		if (pk_tk == T_OP && pk_tv == s_star && pq_gl[0]) {
 			t = S_M; if (tv == s_u) t = S_U; if (tv == s_d) t = S_D;
@@ -1536,6 +1536,7 @@ int unary() {
 		else if (xfloat && isnum(t)) { op(OP_F2I); xfloat = 0; }
 		else xfloat = 0;
 		if (tbytes(t) && !samesz(t, xty)) { tnorm(t); xconst = 0; }	// (u8)x cuts the value to size
+		xstr = t == tbstr;	// (int*)s is a word pointer, not a byte string
 		if_value = 0;
 		xty = t; lv = 0; lvk = LV_NONE; xzero = 0; xbor = -1;
 		return C_PLAIN;
diff --git a/src/compiler/stmt.zc b/src/compiler/stmt.zc
index 4dafe4f..3ecb81b 100644
--- a/src/compiler/stmt.zc
+++ b/src/compiler/stmt.zc
@@ -571,7 +571,7 @@ int kw_switch() {
 					if (!accept(s_comma)) break;
 				}
 				expect(s_rp);
-				if (nb != pm_nf[gi]) serr("a case binds every field of its member");
+				if (nb && nb != pm_nf[gi]) serr("a case binds every field of its member");	// member() with no names matches by tag (spec 4.7.3)
 				v = pm_tag[gi];
 				if (v < 63) cov = cov | (1 << v);
 				laddr(lslot[tmp]); op(O_LD); op(O_LD); lit(v); op(O_SUB);
@@ -649,7 +649,7 @@ int kw_return() {
 		}
 		if_ret(xty, xzero);
 		if (fty && fty != tvoid) fcoerce(fty);
-		if (fty == tvoid) tcerr("return value", tvoid, xty);
+		if (fty == tvoid) { tcerr("return value", tvoid, xty); op(O_DROP); lit(0); }	// typecheck off: the value is ignored (spec 4.8.3)
 		else if (fty && !compat(fty, xty)) tcerr("return value", fty, xty);
 	}
 	if (nclo) {
diff --git a/tests/conformance/4-statements/memdebug_scrub_fills.zc b/tests/conformance/4-statements/memdebug_scrub_fills.zc
index ad15e3d..eed4c2d 100644
--- a/tests/conformance/4-statements/memdebug_scrub_fills.zc
+++ b/tests/conformance/4-statements/memdebug_scrub_fills.zc
@@ -1,10 +1,10 @@
 // expect: 57005/
-// spec 4.11: memdebug scrub; fills freed payloads with 57005
+// spec 4.11: memdebug scrub; fills freed payloads with 57005 (read through asm, which the use-after-free check does not see)
 include rt;
 void pn(int x) { rt.put_n(x); rt.put_c(10); }
 memdebug scrub;
 fn main() {
-	unchecked { int* p = rt.new(int); *p = 5; rt.free(p); pn(*p); }
+	unchecked { int* p = rt.new(int); *p = 5; word a = p; rt.free(p); pn(asm { a @ @ }); }
 	return 0;
 }
 // zc0: skip
diff --git a/tests/conformance/4-statements/switch_tagged_empty_parens_with_fields.zc b/tests/conformance/4-statements/switch_tagged_empty_parens_with_fields.zc
new file mode 100644
index 0000000..789c650
--- /dev/null
+++ b/tests/conformance/4-statements/switch_tagged_empty_parens_with_fields.zc
@@ -0,0 +1,15 @@
+// expect: rect/circle/
+// spec 4.7.3: case member(): with no names matches by tag even when the member has fields
+include rt;
+enum shape { circle { int r; }, rect { int w; int h; }, dot };
+void t(shape* s) {
+	switch (s) { case circle(): rt.put_s("circle"); case rect(): rt.put_s("rect"); case dot: rt.put_s("dot"); }
+	rt.put_c(10);
+}
+fn main() {
+	shape* a = shape.rect(4, 5); shape* b = shape.circle(2);
+	t(a); t(b);
+	rt.free(a); rt.free(b);
+	return 0;
+}
+// zc0: skip
diff --git a/tests/conformance/5-declarations/global_names_global_b8.zc b/tests/conformance/5-declarations/global_names_global_b8.zc
index 7d678eb..6e11713 100644
--- a/tests/conformance/5-declarations/global_names_global_b8.zc
+++ b/tests/conformance/5-declarations/global_names_global_b8.zc
@@ -1,5 +1,5 @@
 // expect: 1/
-// spec 5.3.2 and 3.14.2: a global named in an initializer is its address (appendix b.2.8 calls this a bug)
+// spec 5.3.2 and 3.14.2: a global named in an initializer is its address (documented behavior: the spec wins over appendix b.2.8)
 include rt;
 void pn(int x) { rt.put_n(x); rt.put_c(10); }
 int a = 5; int b = a + 1;
diff --git a/tests/conformance/known.txt b/tests/conformance/known.txt
index c26b50e..13ef43c 100644
--- a/tests/conformance/known.txt
+++ b/tests/conformance/known.txt
@@ -1,22 +1,7 @@
 # understood conformance failures: PATH IMPL CLASS reason
 # class a: self-hosted compiler bug; b: zc0 deviation (appendix a/b); c: spec issue to settle
-1-source/ident_tags_as_variables_glued_star.zc zy a u*d with visible variables u and d is taken as a recast, not a multiplication (spec 1.6.3)
-1-source/lit_bstr_cast_to_int_pointer.zc zy a the bstr flag leaks into an int* local, so p[0] reads a byte
-1-source/lit_char_non_ascii.zc zy a bad character literal in the first statement of a body is swallowed by lookahead
-1-source/lit_char_two_bytes.zc zy a bad character literal in the first statement of a body is swallowed by lookahead
-1-source/lit_char_two_bytes_local_init.zc zy a bad character literal in the first statement of a body is swallowed by lookahead
-1-source/lit_float_bad_exponent.zc zy a bad float literal in the first statement of a body is swallowed by lookahead
-1-source/lit_float_bad_exponent_first_stmt_local.zc zy a bad float literal in the first statement of a body is swallowed by lookahead
-1-source/lit_float_bad_exponent_first_stmt_paren.zc zy a bad float literal in the first statement of a body is swallowed by lookahead
-1-source/lit_float_bad_exponent_in_helper.zc zy a bad float literal in the first statement of a body is swallowed by lookahead
-1-source/lit_float_bad_exponent_sign.zc zy a bad float literal in the first statement of a body is swallowed by lookahead
-2-types/typecheck_off_void_return_value_ok.zc zy a void f() { return 3; } yields 3, not 0 (spec 4.8.3)
-3-expressions/call_24_parameters.zc zy a the parameter limit is 23, not 24 (decl.zc; closures 19)
-3-expressions/call_24_parameters_prototype.zc zy a the parameter limit is 23, not 24 (decl.zc; closures 19)
-4-statements/memdebug_scrub_accepted.zc zy a memdebug scrub; is rejected (appendix b.1.1, b.2.1)
-4-statements/memdebug_scrub_fills.zc zy a memdebug scrub; is rejected (appendix b.1.1, b.2.1)
+# appendix b.2.8 (a global named in an initializer is its address) is documented behavior per 5.3.2 and 3.14.2: no entry
 4-statements/o3_outside_function_ordinary_block.zc zy c top-level o3 { } is rejected; spec 4.10.2 says it is an ordinary block
-4-statements/switch_tagged_match_by_tag.zc zy a case member(): on a member with fields is rejected (a case binds every field); spec 4.7.3 says it matches by tag
 5-declarations/enum_member_names_earlier_b2.zc zy a appendix b.2.2 confirmed: enum members can't name earlier members
 5-declarations/fn_prototype_after_definition_b4.zc zy a appendix b.2.4 confirmed: a prototype after the definition hides it
 5-declarations/fn_prototype_unnamed_params_b13.zc zy a appendix b.2.13 confirmed (message now: doesn't match its earlier declaration)
@@ -31,7 +16,7 @@
 9-appendix-b/b2_14_len_char_ptr_typecheck_off.zc zy a appendix b.2.14 confirmed
 9-appendix-b/b2_15_duplicate_case.zc zy c appendix b.2.15 confirmed; 4.7.2 says duplicates are allowed
 9-appendix-b/b2_16_local_const_in_const_expr.zc zy c appendix b.2.16 confirmed; 3.14.2 documents it
-1-source/ident_tags_as_variables_glued_star.zc zc0 b recast u*d emits no code (same misparse as zy)
+1-source/ident_tags_as_variables_glued_star.zc zc0 b u*d with visible u and d is read as a recast (no code); zy follows 1.6.3
 1-source/lit_hex.zc zc0 b literals read with strtoll base 0: octal and saturation (appendix b.1.11)
 1-source/lit_hex_wide.zc zc0 b literals read with strtoll base 0: octal and saturation (appendix b.1.11)
 1-source/lit_hex_wraps.zc zc0 b literals read with strtoll base 0: octal and saturation (appendix b.1.11)
diff --git a/tests/langerrors/bad_char_first_statement.zc b/tests/langerrors/bad_char_first_statement.zc
new file mode 100644
index 0000000..ca2ccfa
--- /dev/null
+++ b/tests/langerrors/bad_char_first_statement.zc
@@ -0,0 +1,3 @@
+// expect: bad character literal
+include rt;
+fn main() { int a = 'ab'; rt.put_n(a); return 0; }
diff --git a/tests/langerrors/bad_float_first_statement.zc b/tests/langerrors/bad_float_first_statement.zc
new file mode 100644
index 0000000..2dcca38
--- /dev/null
+++ b/tests/langerrors/bad_float_first_statement.zc
@@ -0,0 +1,3 @@
+// expect: bad float
+include rt;
+fn main() { rt.put_n(1e); return 0; }
diff --git a/tests/langerrors/params_25.zc b/tests/langerrors/params_25.zc
new file mode 100644
index 0000000..738cbcc
--- /dev/null
+++ b/tests/langerrors/params_25.zc
@@ -0,0 +1,3 @@
+// expect: too many parameters
+fn f(a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14, a15, a16, a17, a18, a19, a20, a21, a22, a23, a24, a25) { return 0; }
+fn main() { return 0; }
diff --git a/tests/test.sh b/tests/test.sh
index a1a5453..37fa71c 100755
--- a/tests/test.sh
+++ b/tests/test.sh
@@ -234,10 +234,13 @@ s_interp() {
 	T=$ROOT/interp
 	$CC -pthread -O2 -w -fwrapv -DJTYPE=int src/core/core.c src/core/io_linux.c $GI -o $T/zy
 	run $T/zy "interp"
-	# docs/SPEC.md conformance programs; failures listed in tests/conformance/known.txt do not count
-	bash tests/conformance/run.sh --zy $T/zy -w $T/conformance -q > $T/conformance.out 2>&1; r=$?
-	grep '^FAIL' $T/conformance.out
-	chk $r 0 "interp conformance suite: $(grep '^  total' $T/conformance.out | sed 's/  */ /g')"
+	# docs/SPEC.md conformance programs; failures listed in tests/conformance/known.txt do not count.
+	# not in the quick path (-q, make check): it takes about half a minute
+	if [ $QUICK = 0 ]; then
+		bash tests/conformance/run.sh --zy $T/zy -w $T/conformance -q > $T/conformance.out 2>&1; r=$?
+		grep '^FAIL' $T/conformance.out
+		chk $r 0 "interp conformance suite: $(grep '^  total' $T/conformance.out | sed 's/  */ /g')"
+	fi
 	python3 tests/drivers/array_methods_test.py $T/zy > $T/arrays.out 2>&1; r=$?
 	grep -v '^ok' $T/arrays.out
 	chk $r 0 "native array receiver, algorithm and ownership contracts"