Files

T

dimgigov a0bed68cbf docs: selfhost sema error recovery spec + plan

2026-07-28 21:09:03 +03:00

20 KiB

Raw Blame History

Self-hosted Sema Error Recovery Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Make the self-hosted Bux compiler (buxc2) report all independent semantic errors in a single run instead of stopping after the first one, using a tyUnknown error sentinel.

Architecture: Work entirely in the self-hosted compiler (src/sema.bux, src/cli.bux). Semantic errors are recorded via Sema_EmitError; error paths return a tyUnknown sentinel (through a new Sema_EmitExprError helper) so parent expressions and sibling statements keep being checked. Codegen is still skipped whenever Sema_HasError is true.

Tech Stack: Bux (self-hosted compiler source), Bash test fixtures, make selfhost (bootstrap buxc compiles src/*.bux into build/selfhost/build/buxc2).

Spec: docs/superpowers/specs/2026-07-28-selfhost-sema-error-recovery-design.md

Repo layout facts the engineer must know:

Self-hosted compiler sources: src/*.bux (entry: src/main.bux; pipeline wired in src/cli.bux).
make selfhost copies src/*.bux to build/selfhost/src/, renames main.bux → Main.bux, and builds with the bootstrap ./buxc. Result binary: build/selfhost/build/buxc2.
buxc2 check <dir> type-checks a package (needs BUX_STDLIB=<repo>/lib exported when run from a foreign directory).
Type kind constants live in src/types.bux (tyUnknown = 0, tyVoid = 1, tyBool = 2, tyStr = 9, tyInt = 14, tyFloat64 = 21, tyPointer = 22, tyNamed = 26, tyFunc = 28).
Sema_CheckStmt / Sema_CheckExpr are in src/sema.bux (statement checking starts ~line 1425, expression checking ~line 819).
Do not use if as an expression in src/*.bux — the bootstrap parser rejects it (that is exactly what Task 0 fixes in hir_lower.bux). Ternary cond ? a : b is ALSO unsupported by the bootstrap parser (postfix ? is always consumed as the try-operator); use statement form instead.

Task 0: Unblock `make selfhost` (prerequisite)

make selfhost currently fails because src/hir_lower.bux:2726 uses if as an expression, which the bootstrap parser does not support. Replace it with the supported ternary form. This task exists only so that every later task is verifiable end-to-end with buxc2.

Files:

Modify: src/hir_lower.bux:2726
Step 1: Verify the failure first

Run: cd /home/ziko/z-git/bux/bux && make selfhost 2>&1 | tail -20 Expected: FAIL — parse error expected expression at src/hir_lower.bux:2726:50 pointing at the if.

Step 2: Replace the if-expression with statement form

NOTE (corrected during execution): the bootstrap parser supports NEITHER if-expressions NOR ternary cond ? a : b — the postfix loop in bootstrap/parser.nim:794-796 consumes any ? as the try-operator, making the ternary branch unreachable. Use statement form instead.

In src/hir_lower.bux, change line 2726 from:

let operandTypeExpr: *TypeExpr = if operandExpr != null as *Expr { operandExpr.refType } else { null as *TypeExpr };

to:

var operandTypeExpr: *TypeExpr = null as *TypeExpr;
if operandExpr != null as *Expr { operandTypeExpr = operandExpr.refType; }

(operandTypeExpr is only read afterwards inside the same block, so var instead of let is semantically safe.)

Step 3: Verify selfhost now builds

Run: cd /home/ziko/z-git/bux/bux && make selfhost 2>&1 | tail -5 Expected: === Self-hosted compiler built successfully === and the binary exists:

Run: test -x build/selfhost/build/buxc2 && echo buxc2-ok Expected: buxc2-ok

Step 4: Commit

git add src/hir_lower.bux
git commit -m "fix: rewrite if-expression as statements in hir_lower (unblocks make selfhost)"

Task 1: Add `Sema_EmitExprError` sentinel helper

Introduce one explicit helper for the "record an error, mark the expression with a ? type, return tyUnknown" pattern, and convert the two existing expression error sites that already return tyUnknown (undeclared identifier, self outside method) to use it. This locks in the convention the rest of the plan relies on.

Files:

Modify: src/sema.bux (after line 94; and lines ~852, ~880)
Step 1: Write the failing test fixture

Create _test_error_recovery/ (fixture content is finalized in Task 4; for now create a minimal package):

mkdir -p _test_error_recovery/src
cat > _test_error_recovery/bux.toml <<'EOF'
[Package]
Name    = "error_recovery"
Version = "0.1.0"
Type    = "bin"

[Build]
Output = "Bin"
EOF
cat > _test_error_recovery/src/Main.bux <<'EOF'
import Std::Io::{PrintLine};

func Main() -> int {
    PrintLine(undefined_variable);
    return 0;
}
EOF

Run: cd /home/ziko/z-git/bux/bux/_test_error_recovery && BUX_STDLIB=/home/ziko/z-git/bux/bux/lib ../build/selfhost/build/buxc2 check src/Main.bux 2>&1; echo "exit=$?" Expected: exit=1 and the output contains undeclared identifier 'undefined_variable'. (This passes today; the fixture proves the sentinel path keeps working after the refactor.)

NOTE (corrected during execution): the self-hosted check command takes a FILE path, not a directory (Cli_Check reads the file directly), and single-file check mode does NOT resolve imports — so PrintLine would also appear as undeclared. The final fixture (Task 3) therefore avoids imports entirely.

Step 2: Add the helper to src/sema.bux

Immediately after Sema_EmitError (after line 94), insert:

    func Sema_EmitExprError(sema: *Sema, expr: *Expr, msg: String) -> int {
        Sema_EmitError(sema, expr.line, expr.column, msg);
        let te: *TypeExpr = bux_alloc(sizeof(TypeExpr)) as *TypeExpr;
        te.kind = tekNamed;
        te.typeName = "?";
        expr.refType = te;
        return tyUnknown;
    }

Step 3: Convert the undeclared-identifier site

In Sema_CheckExpr (ekIdent branch, ~line 849-854), change:

            let sym: Symbol = Scope_Lookup(sema.scope, expr.strValue);
            if sym.kind == 0 && !String_Eq(sym.name, expr.strValue) {
                let errMsg: String = String_Concat("undeclared identifier '", expr.strValue);
                let errMsg2: String = String_Concat(errMsg, "'");
                Sema_EmitError(sema, expr.line, expr.column, errMsg2);
                return tyUnknown;
            }

to:

            let sym: Symbol = Scope_Lookup(sema.scope, expr.strValue);
            if sym.kind == 0 && !String_Eq(sym.name, expr.strValue) {
                let errMsg: String = String_Concat("undeclared identifier '", expr.strValue);
                let errMsg2: String = String_Concat(errMsg, "'");
                return Sema_EmitExprError(sema, expr, errMsg2);
            }

Step 4: Convert the self site

In the ekSelf branch (~line 877-882), change:

            if sym.kind == 0 && !String_Eq(sym.name, "self") {
                Sema_EmitError(sema, expr.line, expr.column, "self outside method");
                return tyUnknown;
            }

to:

            if sym.kind == 0 && !String_Eq(sym.name, "self") {
                return Sema_EmitExprError(sema, expr, "self outside method");
            }

Step 5: Rebuild selfhost and re-run the fixture

Run: cd /home/ziko/z-git/bux/bux && make selfhost 2>&1 | tail -3 Expected: === Self-hosted compiler built successfully ===

Step 6: Commit

git add src/sema.bux
git add -f _test_error_recovery/
git commit -m "feat(selfhost-sema): add Sema_EmitExprError sentinel helper"

NOTE: _test_*/ is gitignored, but tracked fixtures exist (e.g. _test_error_snippet/), so git add -f is the established way to commit a new fixture.

Task 2: Arithmetic errors return the sentinel (no cascading diagnostics)

Today a failed arithmetic expression emits an error but then returns tyInt/tyFloat64, which lets follow-on code treat the bad expression as a valid number and produce derived errors. Return tyUnknown via the helper instead.

Files:

Modify: src/sema.bux (ekBinary arithmetic tail, ~lines 961-965)
Step 1: Extend the fixture with a cascade probe

Replace _test_error_recovery/src/Main.bux with (no imports — single-file check mode does not resolve them):

func Main() -> int {
    let bad: int = "a" + 1;
    let z: int = undefined_variable;
    return 0;
}

Run: cd /home/ziko/z-git/bux/bux/_test_error_recovery && BUX_STDLIB=/home/ziko/z-git/bux/bux/lib ../build/selfhost/build/buxc2 check src/Main.bux 2>&1; echo "exit=$?" Expected BEFORE the fix: exit=1, output contains arithmetic requires numeric operands and undeclared identifier 'undefined_variable'. (If extra derived errors about bad appear, this task removes them in Task 3 interplay; the key assertion is that both independent errors appear in ONE run.)

Step 2: Return the sentinel from the arithmetic error path

In Sema_CheckExpr, ekBinary branch, change:

            if !Sema_IsNumeric(left) || !Sema_IsNumeric(right) {
                Sema_EmitError(sema, expr.line, expr.column, "arithmetic requires numeric operands");
            }
            if left == tyFloat64 || right == tyFloat64 { return tyFloat64; }
            return tyInt;

to:

            if !Sema_IsNumeric(left) || !Sema_IsNumeric(right) {
                return Sema_EmitExprError(sema, expr, "arithmetic requires numeric operands");
            }
            if left == tyFloat64 || right == tyFloat64 { return tyFloat64; }
            return tyInt;

Note: Sema_IsNumeric returns true for tyUnknown (see src/sema.bux:181), so an already-broken operand does not re-trigger this error. That is the intended cascade suppression.

Step 3: Rebuild and verify both independent errors appear in one run

Run: cd /home/ziko/z-git/bux/bux && make selfhost 2>&1 | tail -3 Expected: === Self-hosted compiler built successfully ===

Run: cd /home/ziko/z-git/bux/bux/_test_error_recovery && BUX_STDLIB=/home/ziko/z-git/bux/bux/lib ../build/selfhost/build/buxc2 check src/Main.bux 2>&1 | tee /tmp/err_recovery.out; echo "exit=$?"

Run: grep -c "arithmetic requires numeric operands" /tmp/err_recovery.out && grep -c "undeclared identifier" /tmp/err_recovery.out Expected: 1 and 1 (each error reported exactly once, in the same run).

Step 4: Commit

git add src/sema.bux
git add -f _test_error_recovery/
git commit -m "feat(selfhost-sema): arithmetic errors return tyUnknown sentinel"

Task 3: Add annotation-vs-initializer assignment checking

The self-hosted sema currently never checks let x: T = init for type agreement (the bootstrap compiler reports cannot assign String to int; buxc2 silently accepts it and lets the C compiler fail later). Add the check in Sema_CheckStmt (skLet), emitting the same message shape as the bootstrap compiler, and keep registering the variable so checking continues.

Files:

Modify: src/sema.bux (new strict-numeric predicate after Sema_IsBool ~line 190; skLet branch ~lines 1443-1462)
Step 1: Extend the fixture with two assignment errors

Replace _test_error_recovery/src/Main.bux with:

func Main() -> int {
    let x: int = "hello";
    let y: bool = 42;
    let z: int = undefined_variable;
    return 0;
}

Run: cd /home/ziko/z-git/bux/bux/_test_error_recovery && BUX_STDLIB=/home/ziko/z-git/bux/bux/lib ../build/selfhost/build/buxc2 check src/Main.bux 2>&1; echo "exit=$?" Expected BEFORE the fix: exit=1, output contains undeclared identifier 'undefined_variable' but NO cannot assign lines (this is the failing assertion the task fixes).

Step 2: Add a strict numeric predicate

Sema_IsNumeric (src/sema.bux:181) deliberately treats tyNamed/tyTypeParam as numeric (operator overloading). Assignment checking must not. After Sema_IsBool (~line 190), insert:

    func Sema_IsStrictNumeric(kind: int) -> bool {
        if kind == tyInt8 || kind == tyInt16 || kind == tyInt32 || kind == tyInt64 || kind == tyInt { return true; }
        if kind == tyUInt8 || kind == tyUInt16 || kind == tyUInt32 || kind == tyUInt64 || kind == tyUInt { return true; }
        if kind == tyFloat32 || kind == tyFloat64 { return true; }
        return false;
    }

Step 3: Add a display-name helper for diagnostics

After Sema_IsStrictNumeric, insert:

    func Sema_TypeNameForDiag(te: *TypeExpr, kind: int) -> String {
        if te != null as *TypeExpr {
            if te.kind == tekPointer && te.pointerPointee != null as *TypeExpr {
                return String_Concat(te.pointerPointee.typeName, "*");
            }
            if !String_Eq(te.typeName, "") { return te.typeName; }
        }
        if kind == tyBool { return "bool"; }
        if kind == tyStr { return "String"; }
        if kind == tyInt { return "int"; }
        if kind == tyInt64 { return "int64"; }
        if kind == tyUInt { return "uint"; }
        if kind == tyFloat64 { return "float64"; }
        if kind == tyPointer { return "*void"; }
        return "?";
    }

Step 4: Emit the assignment error in skLet

In Sema_CheckStmt, skLet branch, insert the check AFTER the annotation/inference block (after the } else if stmt.child1 != null as *Expr && stmt.child1.refType != null as *TypeExpr { block closes, i.e. right before sym.isMutable = stmt.boolValue; ~line 1462):

            // Assignment check: annotation vs initializer (skip when either side is unknown)
            if stmt.refStmtType != null as *TypeExpr && initType != tyUnknown {
                let annotKind: int = Sema_ResolveType(sema, stmt.refStmtType);
                if annotKind != tyUnknown {
                    var mismatch: bool = false;
                    if initType == tyNamed && annotKind == tyNamed {
                        // Both named: kinds are equal, compare type names (when available).
                        if stmt.child1 != null as *Expr && stmt.child1.refType != null as *TypeExpr {
                            if !String_Eq(stmt.child1.refType.typeName, "") &&
                            !String_Eq(stmt.child1.refType.typeName, stmt.refStmtType.typeName) {
                                mismatch = true;
                            }
                        }
                    } else if initType != annotKind {
                        let numericOk: bool = Sema_IsStrictNumeric(initType) && Sema_IsStrictNumeric(annotKind);
                        let initIsPtr: bool = initType == tyPointer || initType == tyStr;
                        let annotIsPtr: bool = annotKind == tyPointer || annotKind == tyStr;
                        let ptrOk: bool = initIsPtr && annotIsPtr;
                        if !numericOk && !ptrOk {
                            mismatch = true;
                        }
                    }
                    if mismatch {
                        let gotName: String = Sema_TypeNameForDiag(stmt.child1.refType, initType);
                        let wantName: String = Sema_TypeNameForDiag(stmt.refStmtType, annotKind);
                        let msg: String = String_Concat("cannot assign ",
                            String_Concat(gotName, String_Concat(" to ", wantName)));
                        Sema_EmitError(sema, stmt.line, stmt.column, msg);
                    }
                }
            }

Important: the variable is still registered with the annotation type on the next lines (sym.typeKind = Sema_ResolveType(...) already ran), so subsequent uses of x type-check as the annotated type and do not cascade.

Known limitation (documented, not fixed here): struct-literal initializers (ekStructInit) do not set expr.refType, so two different named struct types are only compared by kind; name comparison only applies when child1.refType is available.

Step 5: Rebuild and verify all three independent errors appear in one run

Run: cd /home/ziko/z-git/bux/bux && make selfhost 2>&1 | tail -3 Expected: === Self-hosted compiler built successfully ===

Run: cd /home/ziko/z-git/bux/bux/_test_error_recovery && BUX_STDLIB=/home/ziko/z-git/bux/bux/lib ../build/selfhost/build/buxc2 check src/Main.bux 2>&1 | tee /tmp/err_recovery.out; echo "exit=$?"

Run: grep -c "cannot assign String to int" /tmp/err_recovery.out; grep -c "cannot assign int to bool" /tmp/err_recovery.out; grep -c "undeclared identifier 'undefined_variable'" /tmp/err_recovery.out Expected: 1, 1, 1 — three independent errors in a single run.

Also verify valid code still passes (no false positives). Use an import-free file (check mode does not resolve imports):

mkdir -p /tmp/hello_ok/src
cat > /tmp/hello_ok/src/Main.bux <<'EOF'
func Main() -> int {
    let x: int = 5;
    let y: bool = true;
    let s: String = "ok";
    return 0;
}
EOF
BUX_STDLIB=/home/ziko/z-git/bux/bux/lib /home/ziko/z-git/bux/bux/build/selfhost/build/buxc2 check /tmp/hello_ok/src/Main.bux 2>&1; echo "exit=$?"

Expected: exit=0 (no errors for valid code).

Step 6: Commit

git add src/sema.bux
git add -f _test_error_recovery/
git commit -m "feat(selfhost-sema): check annotation vs initializer type in let/var"

Task 4: Regression runner for multi-error recovery

Lock the behavior in with a runnable script so regressions are caught by one command.

Files:

Create: _test_error_recovery/run.sh
Modify: Makefile (add test-error-recovery target)
Step 1: Write the runner script

Create _test_error_recovery/run.sh:

#!/usr/bin/env bash
# Regression: buxc2 must report ALL independent semantic errors in one run.
set -euo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
ROOT="$(cd "$HERE/.." && pwd)"
BUXC2="$ROOT/build/selfhost/build/buxc2"
export BUX_STDLIB="$ROOT/lib"

if [[ ! -x "$BUXC2" ]]; then
  echo "=== building selfhost (buxc2) ==="
  (cd "$ROOT" && make selfhost)
fi
if [[ ! -x "$BUXC2" ]]; then
  echo "error: buxc2 not found at $BUXC2" >&2
  exit 1
fi

out="$(cd "$HERE" && "$BUXC2" check src/Main.bux 2>&1 || true)"
echo "$out"

grep -q "cannot assign String to int" <<<"$out"
grep -q "cannot assign int to bool" <<<"$out"
grep -q "undeclared identifier 'undefined_variable'" <<<"$out"

echo "PASS: all independent semantic errors reported in one run"

Run: chmod +x _test_error_recovery/run.sh

Step 2: Run it — must pass

Run: cd /home/ziko/z-git/bux/bux && ./_test_error_recovery/run.sh Expected: ends with PASS: all independent semantic errors reported in one run

Step 3: Add the Makefile target

In Makefile, add after the test-errors target block (after line 141):

test-error-recovery: ensure-buxc selfhost
	@echo "=== Selfhost multi-error recovery test ==="
	@chmod +x _test_error_recovery/run.sh
	@_test_error_recovery/run.sh

Step 4: Run the target

Run: cd /home/ziko/z-git/bux/bux && make test-error-recovery Expected: PASS: all independent semantic errors reported in one run

Step 5: Negative check — valid code still compiles through buxc2

Run: BUX_STDLIB=/home/ziko/z-git/bux/bux/lib /home/ziko/z-git/bux/bux/build/selfhost/build/buxc2 check /tmp/hello_ok/src/Main.bux && echo "exit=$?" Expected: exit=0

Step 6: Commit

git add Makefile
git add -f _test_error_recovery/
git commit -m "test: selfhost multi-error recovery regression fixture"

Self-review notes

Spec coverage: sentinel helper (Task 1), cascade suppression (Task 2), statement-level continuation with new assignment diagnostics (Task 3), CLI keeps skipping codegen on hasError (existing behavior, asserted by fixtures exiting 1 before codegen), regression test (Task 4). The Sema_CheckStmt early-return audit found no statement-level early returns that skip sibling statements, so no Task is needed for that; the Sema_CheckReturnLifetime internal returns only exit that helper and are correct.
Type consistency: Sema_EmitExprError(sema: *Sema, expr: *Expr, msg: String) -> int, Sema_IsStrictNumeric(kind: int) -> bool, Sema_TypeNameForDiag(te: *TypeExpr, kind: int) -> String are used with the same signatures in every task.
Known blockers/limitations: struct-literal refType is not set, so named-vs-named assignment checking is best-effort; bootstrap compiler behavior is unchanged.

20 KiB Raw Blame History

Self-hosted Sema Error Recovery Implementation Plan

Task 0: Unblock make selfhost (prerequisite)

Task 1: Add Sema_EmitExprError sentinel helper

Task 2: Arithmetic errors return the sentinel (no cascading diagnostics)

Task 3: Add annotation-vs-initializer assignment checking

Task 4: Regression runner for multi-error recovery

Self-review notes

20 KiB

Raw Blame History

Task 0: Unblock `make selfhost` (prerequisite)

Task 1: Add `Sema_EmitExprError` sentinel helper