bench: o3 guard is jit o3 collatz under half of jit plain (no interp rows needed)

2 files changed, 10 insertions, 6 deletions

diff --git a/ONBOARD.md b/ONBOARD.md
index 2ccb987..0f1c95c 100644
--- a/ONBOARD.md
+++ b/ONBOARD.md
@@ -219,7 +219,7 @@ flake), or a row keeps more checks once both sides have a count. rows missing fr
 don't fail. interp rows are left out of the table unless `INTERP=1` (bench.sh still times
 the interpreter today, so this doesn't make the run shorter yet). before timing, the suite
 runs `ZYTRACE=1` on `tests/programs/o3.zc` and refuses when zy compiles no o3 region, and
-it refuses when jit o3 collatz isn't under a fifth of interp o3; that is how a baseline of
+it refuses when jit o3 collatz isn't under half of jit plain collatz; that is how a baseline of
 interpreter numbers got saved once. `make bench-selftest` checks the pass/fail verdicts on
 doctored tables in a second, and `suite.py check --rows FILE` compares a saved table
 without measuring.
diff --git a/tests/bench/suite.py b/tests/bench/suite.py
index f7d5c75..656e877 100644
--- a/tests/bench/suite.py
+++ b/tests/bench/suite.py
@@ -9,7 +9,7 @@ it wraps tests/bench/bench.sh, which times the kernels of tests/programs/bench.z
 256-word pointer sweep of tests/bench/sweep.zc, checked (plain column) and unbounded,
 timed here on build/bin/zy the same way. interp rows are dropped unless INTERP=1. before
 timing, ZYTRACE=1 on tests/programs/o3.zc must show at least one compiled o3 region, and
-jit o3 collatz must be under a fifth of interp o3 (when both rows exist), or it refuses.
+jit o3 collatz must be under half of jit plain collatz, or it refuses.
 selftest checks the verdict logic on doctored tables in a second, without timing: the slope between 1 and N repetitions, the
 median of TRIALS slopes. bench.sh itself is run as it is (it needs bc).
 
@@ -139,11 +139,15 @@ def region_guard(zy):
 
 
 def o3_guard(rows):
-    """jit o3 collatz must be far below interp o3 collatz when both rows are there"""
+    """jit o3 collatz must be under half of jit plain collatz (needs no interp rows)"""
     d = {r["program"]: r for r in rows}
-    j, i = d.get("x86-64-jit:collatz"), d.get("interp:collatz")
-    if j and i and j["o3"] is not None and i["o3"] is not None and j["o3"] * 5 > i["o3"]:
-        sys.exit("bench: REFUSING: jit o3 collatz %.2f ms is not far below interp o3 %.2f ms; o3 regions are not compiled" % (j["o3"], i["o3"]))
+    j = d.get("x86-64-jit:collatz")
+    if j is None:
+        return
+    if j["o3"] is None or j["plain"] is None:
+        sys.exit("bench: REFUSING: x86-64-jit:collatz has no plain or o3 figure")
+    if j["o3"] * 2 >= j["plain"]:
+        sys.exit("bench: REFUSING: jit o3 collatz %.2f ms is not under half of jit plain %.2f ms; o3 regions are not compiled" % (j["o3"], j["plain"]))
 
 
 def run_all(zy):