Compare commits

...

168 Commits

Author SHA1 Message Date
dimgigov a23860be3e release: Bux v1.0.0 language freeze
ci / build (ubuntu) (push) Has been cancelled
ci / unit + fmt (push) Has been cancelled
ci / examples (push) Has been cancelled
ci / goldens + tools (push) Has been cancelled
ci / apps (push) Has been cancelled
ci / selfhost smoke (push) Has been cancelled
ci / macos smoke (push) Has been cancelled
ci / windows smoke (push) Has been cancelled
ci / CI gate (push) Has been cancelled
selfhost-loop / bootstrap determinism (push) Has been cancelled
Bump compiler banners and package version to 1.0.0, activate SEMVER policy,
and add RELEASE_v1.0.0 notes. Fix closure auto-Drop leaking outer Array
drops into nested capture bodies (iter_hof). Fmt-clean examples/src for CI.
2026-07-27 21:49:12 +03:00
dimgigov d60ce2bc3f feat: stdlib daily APIs, macro tt/type paste, riscv64 cross smoke
ci / build (ubuntu) (push) Has been cancelled
ci / unit + fmt (push) Has been cancelled
ci / examples (push) Has been cancelled
ci / goldens + tools (push) Has been cancelled
ci / apps (push) Has been cancelled
ci / selfhost smoke (push) Has been cancelled
ci / macos smoke (push) Has been cancelled
ci / windows smoke (push) Has been cancelled
ci / CI gate (push) Has been cancelled
selfhost-loop / bootstrap determinism (push) Has been cancelled
Sessions 83–87: grow Array/Map/String/Test ergonomics; delimiter-balanced
and juxta :tt macros plus $t:type fragments; expression-level $(…),* in
templates; selfhost slice lits; riscv64/aarch64 cross smoke helper and
freestanding docs. Null-safe CBE type names and String_StartsWith.
2026-07-27 21:40:11 +03:00
dimgigov a785747c37 feat: Linux/cloud platform stack (TLS, registry, static/cross, selfhost PM)
ci / build (ubuntu) (push) Has been cancelled
ci / macos smoke (push) Has been cancelled
ci / windows smoke (push) Has been cancelled
selfhost-loop / bootstrap determinism (push) Has been cancelled
ci / unit + fmt (push) Has been cancelled
ci / examples (push) Has been cancelled
ci / goldens + tools (push) Has been cancelled
ci / apps (push) Has been cancelled
ci / selfhost smoke (push) Has been cancelled
ci / CI gate (push) Has been cancelled
Ship the QUALITY_PLAN platform focus: thin/minimal runtime, --static/--target,
Nexus HTTPS/mTLS with graceful stop, lock checksums + install --locked,
selfhost registry (search/add/HTTP), containers, and CI smokes for cloud path.
2026-07-23 23:00:55 +03:00
dimgigov a939f74b1b feat: field-move Drop (partial/nested/ptr), stmt/pat macros, Windows runtime
ci / build (ubuntu) (push) Has been cancelled
ci / macos smoke (push) Has been cancelled
ci / windows smoke (push) Has been cancelled
selfhost-loop / bootstrap determinism (push) Has been cancelled
ci / unit + fmt (push) Has been cancelled
ci / examples (push) Has been cancelled
ci / goldens + tools (push) Has been cancelled
ci / apps (push) Has been cancelled
ci / selfhost smoke (push) Has been cancelled
ci / CI gate (push) Has been cancelled
Sessions 70–74: partialMovedPaths + ptrAliases in bootstrap/selfhost CBE,
remaining drops after field moves, macro stmt/pat fragments, runtime_win.c
and MinGW hello CI, examples + drop-move smoke coverage, QUALITY_PLAN update.
2026-07-21 12:29:53 +03:00
dimgigov fe3b1e8b6a feat: macros (multi-rep, hygiene), Drop field-move, lean multi-OS CI
ci / build (ubuntu) (push) Has been cancelled
ci / unit + fmt (push) Has been cancelled
ci / examples (push) Has been cancelled
ci / goldens + tools (push) Has been cancelled
ci / apps (push) Has been cancelled
ci / selfhost smoke (push) Has been cancelled
ci / macos smoke (push) Has been cancelled
ci / windows smoke (push) Has been cancelled
ci / CI gate (push) Has been cancelled
selfhost-loop / bootstrap determinism (push) Has been cancelled
Sessions 56–69: declarative macro! with rep/zip/literal/block and
unhygienic var $name binders; partial field-move skip Drop; @[Release]
polish; LSP type hierarchy; CI Nim cache + lean macOS + Windows smoke.
2026-07-20 17:19:46 +03:00
dimgigov 6f2a3b1d88 feat(selfhost): quote/graft hygiene API for macros and mono
ci / make test (push) Has been cancelled
selfhost-loop / bootstrap determinism (push) Has been cancelled
Add Ast_Graft/Clone/Quote helpers, HIR Lcx_GraftSourceFile, and force
definition-site paths on monomorphized bodies. Smoke checks Array mono
#line stays in lib while Main stays isolated.
2026-07-20 00:31:23 +03:00
dimgigov 1f20c683a1 feat(lsp): type hierarchy for interfaces and implementors (0.15)
prepareTypeHierarchy plus supertypes/subtypes via extend Type for Iface.
Smoke covers Drawable ↔ Circle/Square.
2026-07-20 00:24:38 +03:00
dimgigov 60b7b7522f ci: run make test on every PR and push to main
Add .github/workflows/ci.yml (Nim + gcc → make test). Keep selfhost-loop
as an optional/weekly job. Document CI layout; reformat hir_lower for fmt-check.
2026-07-20 00:23:04 +03:00
dimgigov e63dacfdbf fix(selfhost): green fixed-point buxc2→buxc3→buxc4
Raise scope/HIR buffer caps, parenthesize unary operands, fix fat-func
typedef and param access so selfhost-built compilers recompile src
identically (path-normalized C + stripped ELF).
2026-07-20 00:21:39 +03:00
dimgigov cb6df5443a feat(selfhost): Expr/Stmt/Block sourceFile for multi-file and macros
Stamp AST trees from Decl path; lower prefers node sourceFile over
func-level path so grafted macro nodes keep their origin for #line maps.
2026-07-20 00:12:18 +03:00
dimgigov b332e98ab3 feat(lsp): workspace-wide import path index (0.14)
Index import paths from scanWorkspace so path classification works without
open documents. Smoke renames Std::Io with Util.bux closed on disk.
2026-07-20 00:09:51 +03:00
dimgigov a296f1695d feat(lsp): textDocument/implementation for interfaces (0.13)
Go-to-implementation resolves interface types to implementing types and
interface methods to extend Type for Iface method decls (Circle+Square).
2026-07-19 23:46:17 +03:00
dimgigov fe1da8b42e ci: optional selfhost-loop workflow + determinism script
Add tools/selfhost_loop.sh with path-normalized C and stripped ELF
checks (bootstrap ×2). Wire GitHub Actions (manual/weekly/main paths)
outside default make test. Raise selfhost lexer maxTokens for large files.
2026-07-19 23:43:51 +03:00
dimgigov c82757d51c feat(selfhost): HirNode.sourceFile for mid-function #line maps
Stamp each HIR node with Decl source path; C backend tracks lastDebugFile
so #line re-emits when file or line changes. Closures inherit the
enclosing function path. Smoke checks Util_Double body maps to Util.bux.
2026-07-19 23:40:10 +03:00
dimgigov 200adf4d68 feat(lsp): module-path segment rename for Std::Io imports (0.12)
Index import A::B path segments and rename with left-prefix match so
Std::Io does not clobber Foo::Io, bare idents, or enum Color::Red.
2026-07-19 23:35:54 +03:00
dimgigov 244a3be8f3 feat(lsp): interface dispatch in call hierarchy (0.11)
Index interface methods and extend Type for Trait relations so call
hierarchy can show implementors as outgoing targets and .Method call
sites as incoming.

- interface I { func M } + extend T for I tracking
- prepare/outgoing/incoming with data Iface#Method
- smoke_lsp_iface_hierarchy (Drawable.Draw ↔ Circle / Render)
2026-07-19 23:30:52 +03:00
dimgigov 00195e2d98 feat(lsp): method, type, and self-receiver rename (0.10)
Add rtkMethod so .Method( call sites rename with the method decl, improve
type renames across extend/annotations, and clip self to the enclosing
method body.

- Len→Length includes self.Len(); Point→Vec2 hits extend + types
- self→this stays within one method (textual func bounds)
- smoke_lsp_rename_method + test-lsp wiring
2026-07-19 23:27:21 +03:00
dimgigov 8f645ad68b feat(lsp): method call hierarchy for extend Type / .Method (0.9)
Index methods inside extend/impl blocks and include .Method( call sites
in the call graph so IDEs can navigate receiver methods.

- analyzeFile: track extend/impl brace scope; kind=method + container
- Call hierarchy treats methods as callables; display Type.Method
- smoke_lsp_method_hierarchy (Scale→Len, Main→Scale)
2026-07-19 23:22:13 +03:00
dimgigov 5de8d8912d ci: selfhost smoke for move_field and multi-file #line
Add tools/smoke_selfhost.sh and make test-selfhost-smoke (via make selfhost)
to the default make test suite, covering field-move ownership and per-file
#line paths under buxc2.
2026-07-19 23:14:40 +03:00
dimgigov 70321075a6 feat(selfhost): multi-file #line paths from Decl.sourceFile
Stamp each merged .bux path onto decls, propagate to HirFunc, and emit
#line N \"path\" per function without requiring BUX_DEBUG_FILE.

- Decl.sourceFile + Cli_StampSourceFile on merge/project parse
- HirFunc.sourceFile; CBE switches currentFile per function
- Stdlib and multi-file user packages get distinct paths automatically
2026-07-19 23:13:02 +03:00
dimgigov ed07cf8921 feat(lsp): call hierarchy prepare / incoming / outgoing (0.8)
Expose LSP call hierarchy over a lightweight textual call graph of
known functions and Name( call sites across the workspace.

- prepareCallHierarchy on func decl or call site
- incomingCalls / outgoingCalls with fromRanges
- smoke_lsp_call_hierarchy + test-lsp wiring
2026-07-19 23:11:10 +03:00
dimgigov 42f9453e18 feat(lsp): deeper rename for fields, variants, qualified paths
bux-lsp 0.7 indexes struct fields and enum variants and renames only the
right binding: .member / ::Variant / field-init, without clobbering locals.

- Member scan in analyzeFile; access kinds bare/dot/colon/field-init
- classifyRenameTarget: local | member | global
- smoke_lsp_rename_deep + test-lsp wiring
2026-07-19 23:09:51 +03:00
dimgigov 94cc94f638 feat(selfhost): field-move skip Drop, #line maps, Array field mono
Bring selfhost C backend closer to bootstrap ownership and debug quality.

- Mark locals moved into struct fields / returns (no double Drop)
- Mangle Array/Set/Map/Channel field types so user structs fully emit
- Emit #line N from HIR line; BUX_DEBUG_FILE / BUX_NO_LINE controls
2026-07-19 23:07:01 +03:00
dimgigov cfb89dd72f fix(compiler): skip auto-Drop after field/let/return moves
Track locals moved by value into struct fields, let bindings, or return
values so Array/Drop types are not freed while still owned by the target.

- hir_lower: movedOutLocals + markMovedOutFromAst + shouldSkipDrop
- Nexus: drop zeroing workaround; free headers after each request
- examples/move_field.bux regression for Box { items: arr }
2026-07-19 23:03:05 +03:00
dimgigov adc5c743a6 fix: Array move ownership, selfhost -g flags, LSP workspace/symbol
Session 36: prevent UAF when Array is moved into a struct field (Nexus
headers), align selfhost cc flags with bootstrap debug/release, and add
workspace/symbol search to bux-lsp 0.6.

- Parser: zero headers after embedding so auto-drop is a no-op
- Request_WantsKeepAlive uses RequestHeader_Get again safely
- Selfhost: default -O0 -g; --release -O2 -DNDEBUG; BUX_CFLAGS
- LSP: workspace/symbol + smoke; version 0.6.0
2026-07-19 22:59:02 +03:00
dimgigov e30d8a5eb9 feat(nexus): HTTP/1.1 keep-alive (~2× throughput)
Reuse TCP connections for multiple requests, honor Connection close,
and raise wrk health RPS from ~45k to ~80k+ on the same machine.

- Loop HandleConnection up to 1000 reqs/fd with Connection keep-alive
- RawRequest_WantsKeepAlive (HTTP/1.1 default; HTTP/1.0 needs keep-alive)
- Nexus 0.3.0 banner; benches note before/after RPS
2026-07-19 22:53:59 +03:00
dimgigov abb2b54dce docs: record session 34 (LSP 0.5 refs/rename + CI wiring)
Complete QUALITY_PLAN notes for references/rename and expanded make test.
2026-07-19 22:50:20 +03:00
dimgigov cd7a82822c feat: LSP references/rename (0.5) and expand CI make test
Add workspace-aware Find All References and Rename for scoped locals and
globals, plus registry/apps/dwarf smokes in the default test target.

- bux-lsp: textDocument/references, prepareRename, rename
- make test-lsp runs smoke_lsp_rename; make test includes registry/dwarf/apps
2026-07-19 22:49:45 +03:00
dimgigov eb81856565 feat: HTTP registry, app/bench harnesses, and DWARF #line maps
Ship QUALITY_PLAN sessions 31–33: fetchable package index URLs, showcase
app and micro/nexus benchmarks, and debugger-friendly C codegen.

- Registry: BUX_REGISTRY accepts http(s) URLs (curl/wget → ~/.bux/cache)
- E.2: make test-apps smoke for nexus/boko/simpledb/jwt-pitbul
- E.5: benches/micro + C/Nim/Zig twins; make bench-nexus (wrk)
- E.4: HIR locs → #line .bux; default -O0 -g; --release -O2; make test-dwarf
2026-07-19 22:46:45 +03:00
dimgigov 53b43b0f79 feat: lifetime elision, tooling CI, registry, and LSP locals
Ship the QUALITY_PLAN stretch from ownership through ecosystem: C.1
lifetime elision (bootstrap + selfhost), bux fmt/test/doc CI hooks,
stdlib goldens, package registry (bux search/add), and LSP 0.4
position-sensitive locals with inferred let types. Full-tree format
pass plus Map/Set remove double-free fix.
2026-07-19 16:35:08 +03:00
dimgigov 3eb1ad3a82 feat: match guards, HOF inference, ownership C.2/C.3, LSP sema hover
Sessions 18–23 quality work:

- B.3c match arm guards + sequential found-flag lower (bootstrap + selfhost)
- Generic HOF type inference (Array/Iter map/filter/fold without type args)
- Pattern binding shadowing via unique C locals (__pN_src)
- Ownership C.2 exclusive &mut data-flow + C.4 goldens; *p= store-through fix
- Ownership C.3 auto-drop on early return/branches: scoped defers, move-on-return,
  Drop monomorphization, materialize return before Drop
- LSP 0.3.0: hover from real sema types
- Examples and QUALITY_PLAN session log; selfhost-loop identical
2026-07-18 21:52:14 +03:00
dimgigov 66f11d1869 feat: nested multi-field enum patterns (bootstrap + selfhost)
Multi-field variants use Enum_Variant_Payload nested types (avoids tag name clash).
Sema resolves data.Variant.Variant_i; enum field types fully resolve tuples.
Selfhost parses full type exprs in enum payloads; emit structs/tuples before enums.
Example: examples/nested_patterns.bux (Pair::Two, Box::Val((a,c)), Shape::Dot).
2026-07-18 01:33:36 +03:00
dimgigov f9185c96b2 feat: multi-stmt match arms and block-as-expression
Blocks can yield a value: last expression statement is the result.
Match arms accept block bodies, so multi-statement arms work:

  match n {
    1 => { let a = 10; a + 1 },
    _ => 0
  }

Bootstrap: lowerBlock(asExpr) lifts the trailing skExpr. Selfhost: parse
{...} as ekBlock, emit __blk_N yield temps (retTypeKind -2), and treat
only non-empty strValue as match/block yield. Nested enum/struct pattern
bindings recurse in both compilers.

Example: match_block.bux. Selfhost-loop remains binary-identical.
2026-07-18 01:24:48 +03:00
dimgigov e3ca724bfa feat: struct and tuple patterns in match (bootstrap + selfhost)
Support destructuring in match arms:
- Tuple: (a, b) binds subject._0 / _1
- Struct: Point { x: px, y: py } and shorthand Point { x, y }

Bootstrap: matchPatternBindings for pkTuple/pkStruct; register local
tuple typedefs from function bodies. Selfhost: parse, Sema_BindPattern,
Lcx_PatternBindings with scope defines. Fix operator-overload path that
crashed when typeName was null after pattern binds.

Example: examples/struct_tuple_pat.bux. Selfhost-loop IDENTICAL.
2026-07-18 01:15:44 +03:00
dimgigov eac78f28c1 feat(stdlib): generic Iter_Map/Filter/Fold with func monomorphization
Add Iter_Map<T,U>, Filter, Fold, Any, All, ForEach over fat function
pointers; keep Iter_MapInt and friends as thin aliases.

Compiler fixes required for non-int returns and capturing closures:
- bootstrap: resolve fat-func call return type under mono typeSubst
- bootstrap: type-check args of Foo<T>(...) so closures capture correctly
- selfhost: substitute type params inside tekFunc (BuxFn_U_T → concrete)
- selfhost: emit cstr fat typedefs with #ifndef redefinition guards

Example: examples/iter_generic.bux (int↔String map, fold, closures).
Selfhost-loop remains binary-identical.
2026-07-18 01:07:41 +03:00
dimgigov 26631252c0 feat(lsp): hover signatures, go-to-def, document outline
Upgrade bux-lsp to 0.2.0 with a richer lightweight symbol index:
func signatures, typed let/var/const, struct/enum/interface/type/module.
Skip comments and strings while scanning. Hover shows markdown Bux
code fences; definition resolves in-file and across workspace .bux files;
documentSymbol provides outline. Fix responses writing to stdout (stdin
was a broken pipe). didChange refreshes symbols; diagnostics still run
via buxc on open/save.
2026-07-18 01:01:22 +03:00
dimgigov db41ba4d84 feat: pattern bindings, empty closures, match-as-expr, string interp
Sessions 10–12 from QUALITY_PLAN:

- Pattern payload bindings (Some(value) => value) in bootstrap and selfhost
- Empty-param closures via || (tkPipePipe) with loop/return bodies
- Expression-form match: let x = match …; newline before arms
- f"…" string interpolation desugared to String_Concat + conversions
- Lexer preserves \{ \} for literal braces in f-strings
- Bootstrap fix: f"plain" strips the f prefix after escape processing
- Examples: pattern_matching, closure_control, match_let, string_interp

Selfhost-loop remains binary-identical; all examples and error goldens pass.
2026-07-18 00:58:44 +03:00
dimgigov f619316470 feat: full match expressions in bootstrap and selfhost
Lower match to if-else for literals, ranges, enum tags, and wildcards.
Fix bootstrap literal arms that always matched; port real arm AST/parser
and Lcx_LowerMatch to selfhost with last-expression return. Expand
pattern_matching example and add parse/use-after-move/double-mut golden
diagnostics. Selfhost-loop remains binary-identical.
2026-07-16 16:19:43 +03:00
dimgigov 2cbbccc508 docs: complete QUALITY_PLAN session log and next-day backlog
Record Iter HOF session and proposed follow-ups for the next work day.
2026-07-15 16:17:04 +03:00
dimgigov e805af6ef6 docs: sync README and plans with v0.5.x features
Document multi-instance closures, tuples, Iter HOF, diagnostics, and
updated test targets. Refresh QUALITY_PLAN diagnosis and next-day backlog.
2026-07-15 16:16:54 +03:00
dimgigov f1896183f5 feat(stdlib): Iter map/filter/fold with closures
Add int-specialized higher-order Iter helpers that accept fat function
pointers, plus an example using named funcs and capturing closures.
Fix selfhost C backend pointer field access and Array Push indexing.
2026-07-15 16:07:47 +03:00
dimgigov 9efba57b4c feat(selfhost): full tuple type support with .0/.1 field access
Parse (T, U) types and (a, b) expressions, lower to Tuple_* C structs,
and emit common tuple typedefs so buxc2 matches bootstrap tuple codegen.
2026-07-15 16:04:01 +03:00
dimgigov 61ac06ab5f feat: multi-instance closures, richer stdlib, and Rust-style diagnostics
Introduce fat function pointers (BuxFn {code, env}) so capturing closures
are heap-allocated per value in both bootstrap and selfhost. Expand
Array/Map/Set/String/Test/Result APIs, add proper tuple codegen and
error snippets with multi-char underlines, golden diagnostic tests, and
LSP diagnostics via buxc check.
2026-07-15 16:00:21 +03:00
dimgigov 94e6806dda fix(compiler): allow integer coercion in range bounds; use for loops in boko-framework
- sema: range bounds now accept compatible integer types (e.g. int..uint)
  instead of requiring exact type equality.
- hir_lower: derive loop variable type from the common range type.
- boko-framework: replace manual while loops with for loops in Query_Parse,
  Request_Parse, Path_Match and App_Run.
2026-06-17 12:08:36 +03:00
dimgigov 9c632ce389 Rewrite jwt-pitbul and boko-framework with modern Bux constructs
- jwt-pitbul: algebraic enums, Array<String>, methods on JwtAlg,
  string interpolation, structured Result/Option error handling.
- boko-framework: StringMap<String> for headers/query/path params,
  Request/Response methods, StringBuilder, for loops, algebraic enums.
- Update READMEs and bump versions to 0.2.0.
2026-06-16 10:12:52 +03:00
dimgigov aaeb01e518 feat(nexus): production modular HTTP/1.1 server + compiler fixes
- Rewrite apps/nexus with modular architecture:
  Config, Http, Errors, Parser, Router, Handlers, Server, Main
- Use algebraic enums for ParseResult/FileResult/HttpError
- Thread-pool server via Channel<ConnectionTask> and spawn
- Fix C backend type ordering for generic struct instances
  (Array_T, Iter_T) and algebraic enum struct payloads
- Collect Slice_T types from struct fields and enum payloads
- Fix match lowering for simple enums (direct value compare)
- Resolve match expression return type from first arm
- Infer element type for for-in over Array<UserStruct>
- Preserve generic type args in field access resolution
- Add fflush to PrintLine/Print for immediate server logs
- Add modern_features golden regression test
- Regenerate golden expected.c files
2026-06-15 00:54:03 +03:00
dimgigov fc0a560e60 docs(roadmap): mark concurrency done in selfhost
Green threads (lib/Task.bux) and channels (lib/Channel.bux) already work.

Verified by _test_green_threads and _test_forin_channel.
2026-06-14 17:50:40 +03:00
dimgigov e11dce56d2 docs(roadmap): mark slice bounds checking done in selfhost
Slice<T> bounds-checked indexing already works via lib/Slice.bux
2026-06-14 17:48:45 +03:00
dimgigov e5e490605a feat(selfhost): auto-drop for interface-based Drop implementations
- Extend Lcx_BuildAutoDropFree to detect TypeName_Drop methods

  registered by sema for extend Type for Drop impl blocks.

- Preserve existing @[Drop] attribute path and move semantics.

- Add design doc and implementation plan.

- Mark Destructors/Drop roadmap item done in selfhost.
2026-06-14 17:44:35 +03:00
dimgigov 2c8223f94e fix(bootstrap): avoid emitting malformed Slice_T* typedefs
- Strip pointer suffix when scanning for slice types in C backend
- Skip slice typedefs already emitted from module.structs

Fixes _test_slice
2026-06-14 17:23:58 +03:00
dimgigov 44bac83471 fix(bootstrap): implement collection for-in lowering
- Export typeToTypeExpr from sema and preserve generic type args
- Derive loop variable type from collection element type in sema
- Substitute generic struct type params on field access
- Add getCollectionElementTypeExpr helper in hir_lower
- Replace placeholder collection for-in lowering
- Add Array/Iter lowering: Array_Iter_T / Iter_HasNext_T / Iter_Next_T
- Add Channel lowering: Channel_Recv_Ok_T loop
- Register loop variable in varTypeExprs before body lowering

Fixes _test_forin_stdlib, _test_forin_channel, _test_generic_trait, _test_import, _test_mono
2026-06-14 17:17:56 +03:00
dimgigov 06db4926a7 fix(bootstrap): substitute generic type args in operator [] resolution
- Preserve type args on tkNamed in resolveType
- Resolve explicit generic call return types with concrete substitutions
- Substitute method type params when looking up operator_index_get
- Skip strict arg checks for generic function calls (deferred to inference)
- Fix generic struct monomorphization when no caller substitution map exists
- Fix parameter varTypeExprs ordering so pointer params are visible in bodies

Fixes _test_drop_trait and _test_checked_index
2026-06-14 13:15:13 +03:00
dimgigov 84df4bba9d feat: bux test runner, bitwise-not parsing, stdlib discovery hardening\n\n- Add 'bux test' runner: discovers tests/*.bux, builds each in a temp package, runs and reports PASS/FAIL.\n- Fix unary ~ parsing and C emission in self-hosted compiler.\n- Harden stdlib discovery (require Fs.bux, avoid system /lib) and lstat in directory traversal.\n- Selfhost loop passes: buxc2 builds src/ and produces identical C output + stripped binary.\n- Update PLAN.md with completed milestones. 2026-06-12 22:19:13 +03:00
dimgigov a668127721 docs: update README and PLAN for v0.5.0
- README: add @[Release], green threads, drop trait, --release flag
- README: fix Task_Join → Task_Wait, update project structure
- README: add make selfhost-loop to build instructions
- PLAN: mark selfhost loop, borrow checker, release mode as done
- PLAN: update milestones summary
2026-06-11 10:05:10 +03:00
dimgigov 9bbeb4fd1d feat(selfhost): achieve binary-identical selfhost loop
- Add -Wl,--build-id=none to C compilation flags (removes non-deterministic build ID)
- Update Makefile selfhost-loop to strip debug info before ELF comparison
- Bootstrap compiler and selfhost compiler both deterministic
2026-06-11 10:00:14 +03:00
dimgigov 0e93ea911c feat(diag): format lexer errors with Diagnostic_Print in Cli_Check and Cli_BuildProject 2026-06-11 09:51:50 +03:00
dimgigov 290cbc8f98 feat(release): add @[Release] attribute and --release CLI flag
- @[Release] disables borrow checking and bounds checking in the function
- --release passes -O3 -flto to the C compiler for optimized builds
- Selfhost loop still deterministic
2026-06-11 09:39:11 +03:00
dimgigov 224542df7f fix(examples): Task_Join -> Task_Wait to match stdlib API 2026-06-11 09:21:06 +03:00
dimgigov a619247f06 test(slice): add operator indexing test for Array and Slice 2026-06-11 01:29:09 +03:00
dimgigov a4f25bc737 feat(operator): add operator_index_get/set to Array and Slice with generic monomorphization 2026-06-11 01:28:59 +03:00
dimgigov aa2d6f9632 test(drop): verify auto-drop works without @[Checked] 2026-06-11 01:09:23 +03:00
dimgigov b549e09a3b feat(drop): remove checkedFunc gate from auto-drop 2026-06-11 01:08:57 +03:00
dimgigov 2259567ac2 docs: add Drop trait implementation plan 2026-06-11 01:08:27 +03:00
dimgigov f4f09602c6 docs: add Drop trait / destructors design document 2026-06-11 01:07:34 +03:00
dimgigov ef4aa42b0b feat: add web playground with Docker sandbox
- playground/backend/main.go — Go HTTP server with /compile endpoint
- playground/frontend/index.html — Monaco Editor with 8 code examples
- playground/sandbox/Dockerfile — isolated Docker container with buxc2
- playground/docker-compose.yml — orchestrates backend + sandbox
- playground/Makefile — build and run targets
- playground/README.md — deployment guide for VPS with nginx + SSL
2026-06-10 20:33:26 +03:00
dimgigov 8f68a9da1a docs: document --target cross-compilation flag
- Add cross-compilation to README.md features and quick start
- Add cross-compilation section to docs/BuildAndTest.md
2026-06-10 20:26:31 +03:00
dimgigov e4e4e0d8ef feat: add --target <triple> cross-compilation flag
- Parse --target flag in Cli_Run before command dispatch
- Pass targetTriple through Cli_Build, Cli_BuildProject, Cli_RunProject
- Use clang with -target when cross-compiling, cc otherwise
- Selfhost loop: C output identical
2026-06-10 20:24:46 +03:00
dimgigov 18aa232b1c docs: add cross-compilation design document 2026-06-10 20:19:17 +03:00
dimgigov c6bae8b9ee feat(diag): add parser severity filtering and source map heuristic for bux build 2026-06-10 20:15:57 +03:00
dimgigov aef0cd7936 docs: add source location error fixes design 2026-06-10 20:12:20 +03:00
dimgigov 62d92b0aa6 test: add error snippet formatting test 2026-06-10 20:08:24 +03:00
dimgigov 9a63dcca11 fix(diag): remove parser diag printing, fix String_CharAt and sourceName issues 2026-06-10 20:08:20 +03:00
dimgigov 10a2b64ac4 feat(diag): print parser diagnostics in Parser_Parse 2026-06-10 20:04:31 +03:00
dimgigov 92962fa1b0 feat(diag): format all lexer and sema errors with Diagnostic_Print 2026-06-10 20:03:56 +03:00
dimgigov b971775baf feat(diag): add Diagnostic struct and Rust-style formatter 2026-06-10 20:02:33 +03:00
dimgigov 85d15cd16d docs: add source location error messages implementation plan 2026-06-10 20:00:09 +03:00
dimgigov de3ac76581 docs: add source location error messages design document 2026-06-10 19:56:55 +03:00
dimgigov 2824d25369 test: add green thread integration test
Also fix runtime scheduler bugs:
- g_task_creating was thread-local but set on main thread and read on worker
- g_scheduler_context was global shared by all workers instead of per-thread
2026-06-10 19:48:47 +03:00
dimgigov 9cd0580832 feat(task): add full green thread API (Init, Wait, Yield, CurrentId, Shutdown) 2026-06-10 19:45:24 +03:00
dimgigov 9e547f123a fix(scheduler): make Task_Sleep yield instead of block (MVP) 2026-06-10 19:44:55 +03:00
dimgigov 211a811f32 feat(scheduler): implement green thread scheduler with work-stealing 2026-06-10 19:43:31 +03:00
dimgigov 4456bb00af docs: add green threads implementation plan 2026-06-10 19:38:24 +03:00
dimgigov 6a0a0425dd docs: add green threads / task scheduler design document 2026-06-10 19:33:53 +03:00
dimgigov 9e515c157d feat(pkg): add basic package manager (add, remove, fetch, build deps)
- manifest.bux: parse [dependencies] section with up to 8 deps
- manifest.bux: Manifest_AddDep, Manifest_RemoveDep, Manifest_ToString
- cli.bux: bux add <name> <url> — adds dependency to bux.toml
- cli.bux: bux remove <name> — removes dependency from bux.toml
- cli.bux: bux fetch — git clone/pull dependencies into deps/
- cli.bux: Cli_BuildProject now merges deps/<name>/src/*.bux before user code
- Selfhost bootstrap loop verified: C output deterministic ✓
2026-06-10 19:18:36 +03:00
dimgigov 8f8708c5df feat(drop): enforce @[Drop] attribute and add basic move semantics
- Auto-drop now only triggers for user-defined types explicitly marked
  with @[Drop] on the struct declaration.
- Added move tracking in C backend to skip auto-drop for variables
  that have been moved (via let y = x, return x, or assignment).
- Prevents double-free when ownership is transferred.
- Selfhost bootstrap loop verified: C output is deterministic.
2026-06-10 19:08:15 +03:00
dimgigov 7392013301 docs: add Phase 8.7 — Optimization & Release Mode to PLAN.md
- @[Release] attribute for zero-cost abstractions
- Auto-drop inline, dead-store elimination, -O3 by default, PGO
- Milestone M15: C-speed benchmarks vs Nim/Crystal
2026-06-10 13:51:10 +03:00
dimgigov 0b887b92d3 selfhost: for-in support for Channel<T>; fix break/continue C codegen
- lib/Channel.bux: add Channel_Recv_Ok<T> for use in for-in desugaring
- hir_lower.bux: Channel for-in desugars to while+recv_ok pattern
- c_backend.bux: emit 'break;' / 'continue;' with semicolon, fix missing
  semicolon when break/continue inside if-block
- bootstrap/hir_lower.nim: skip generic methods in vtable generation to
  avoid referencing non-existent monomorphized C functions
2026-06-10 13:46:06 +03:00
dimgigov 7d6cf7f511 selfhost: add lib/Drop.bux with interface Drop; extend methods visible in scope
- lib/Drop.bux: interface Drop { func Drop(self: *Self); }
- sema.bux: define TypeName_MethodName in scope for impl block methods
  so auto-drop (Lcx_BuildAutoDropFree) can find Drop implementations
  via extend Type for Drop, not just @[Drop] or standalone functions
- Verified: extend Buffer for Drop auto-calls Buffer_Drop in @[Checked]
2026-06-10 13:37:55 +03:00
dimgigov 0a41ce85f4 selfhost: add @[Drop] attribute for user-defined structs
- ast.bux: add isDrop field to Decl
- parser.bux: parse @[Drop] attribute and attach to struct Decls
- hir_lower.bux: auto-drop now looks for TypeName_Drop for any type
- Test: _test_drop_user verifies Buffer_Drop is auto-called
2026-06-10 13:32:40 +03:00
dimgigov d6f0a30948 selfhost: add *_Drop wrappers to stdlib types; auto-drop uses _Drop instead of _Free
- Array.bux, Channel.bux, Set.bux, Map.bux: add *_Drop<T> delegating to *_Free
- hir_lower.bux: Lcx_BuildAutoDropFree searches for _Drop suffix instead of _Free
- Skip direct lowering of generic functions (only monomorphized instances)
2026-06-10 13:25:11 +03:00
dimgigov a2236de69a feat(selfhost): add operator overloading support (+,-,==,[], etc.)
- Auto-register func Type_Method(self: T, ...) as methods in sema.
- Add method table lookup for binary operators in sema.bux ekBinary.
- Add method table lookup for operator_index_get in sema.bux ekIndex.
- Add HIR lowering for binary operators to method calls in hir_lower.bux.
- Add HIR lowering for operator_index_get/set in hir_lower.bux.
- Set expr.refType for overloaded operators so chained ops work.
2026-06-10 13:11:46 +03:00
dimgigov a4e410fb30 fix(selfhost): for-in with monomorphized Array/Iter types + golden test updates + ROADMAP cleanup
- Fix hir_lower.bux to recognize Array_int/Iter_string etc. in for-in loops
  by checking Array_/Iter_ prefix outside the empty-elemTypeName branch.
- Update all golden expected.c files to include duplicate bux_bounds_check
  declaration (matches actual C codegen).
- Remove duplicate Drop trait item from ROADMAP.md.
2026-06-10 12:57:53 +03:00
dimgigov 51bd3c172b Add Slice<T> generic struct support and fix C backend generic filtering
- Add lib/Slice.bux with Slice<T> generic struct
- Fix C backend to recognize Slice/Slice* as generic types
- Use var instead of let for uninitialized Slice<T> locals
- Selfhost loop passes, tests pass
2026-06-10 12:03:31 +03:00
dimgigov 3a7b4bd5c1 selfhost: extend auto-Drop to Channel<T>, Set<T>, Map<K,V>
In @[Checked] functions, variables of type Channel<T>, Set<T>, and
Map<K,V> now automatically get defer Free_T(&var) injected.

- Added Lcx_BuildAutoDropFree helper that matches typeName prefix
  and generates the appropriate Free instance:
  - Array_<T>  → Array_Free_<T>
  - Channel_<T> → Channel_Free_<T>
  - Set_<T>    → Set_Free_<T>
  - Map_<K>_<V> → Map_Free_<K>_<V>

Selfhost loop passes, C output deterministic.
2026-06-10 11:47:50 +03:00
dimgigov a099999e02 selfhost: auto-Drop for Array<T> in @[Checked] functions
In @[Checked] functions, variables of type Array<T> automatically get
a defer Array_Free_T(&var) injected at declaration site. This ensures
Array values are freed when they go out of scope, preventing leaks.

Implementation:
- HIR lowering detects Array_<T> types in skLet statements
- Generates Array_Free_T generic instance if needed
- Builds synthetic defer node: defer Array_Free_T(&var)
- Wraps let+defer in hBlock for proper statement chaining

Also adds checkedFunc flag to HirFunc and CEmitter for future use.
2026-06-10 11:40:11 +03:00
dimgigov 5eec4328f3 docs: update ROADMAP and PHASE8 with completed features
Mark as done:
- for x in collection iterator loops
- Trait bounds (T: Comparable)
- CTFE (compile-time const evaluation)
- Selfhost bootstrap loop (deterministic C output)
- Implicit generic type argument inference
2026-06-10 11:21:04 +03:00
dimgigov c7d4c87fcf selfhost: basic CTFE — compile-time const expression evaluator
Evaluate const declarations at compile time:
- Arithmetic: +, -, *, /, %
- Comparisons: <, <=, >, >=, ==, !=
- Bitwise: &, |, ^, <<, >>
- Logical: &&, ||
- Unary: -, !, ~
- Ternary: cond ? a : b
- Cast: (T)expr (evaluates operand)
- Cross-const references: const C = A + B

Uses multi-pass evaluation to handle forward references
(caused by parser push-front ordering).
2026-06-10 11:11:52 +03:00
dimgigov d70ccba5ca selfhost: implicit generic type argument inference for Array/Iter stdlib functions
- sema: add Sema_InferGenericArg + Sema_ExtractElemType helpers
- Infers T from first argument's collection type:
  - Array_Push(&arr, 10) → Array_Push_int (from &arr's Array<int> type)
  - Iter_HasNext(&it) → Iter_HasNext_int (from &it's Iter<int> type)
- Supports both explicit generic types (Array<int>) and mangled types (Array_int)
- Works with reference expressions (&x) by unwrapping to x's type
- Selfhost loop: C output IDENTICAL
2026-06-10 10:26:08 +03:00
dimgigov 01677001c3 selfhost: for x in collection works with non-identifier expressions + buxc2 build merges stdlib
- cli: buxc2 build now delegates to Cli_BuildProject for src/Main.bux,
  enabling stdlib merging (Array_Iter, Iter_HasNext, etc. available)
- sema: set expr.refType on ekCall from function return type
  (needed for for-loop type extraction from MakeArr()-style expressions)
- hir_lower: remove ekIdent restriction on collection-based for loops
  - supports arbitrary expressions: for x in MakeArr() { ... }
  - creates temp variable __tmp_coll_N for non-identifier iterators
  - generates proper Array_T struct instances for temp variables
- Test: for x in MakeArr() sums [10,20,30] → 60
- Selfhost loop: C output IDENTICAL
2026-06-10 10:20:16 +03:00
dimgigov bb84693acb selfhost: fix extend-for method linked list + trait bounds check before indirect call return
- parser: extend-for only linked first 2 methods; now uses lastMethod pointer
- sema: trait bounds checking was after indirect-call early return (tyFunc path),
  so Max<Circle>(...) skipped bounds check when callee had tekFunc refType.
  Moved bounds check before both indirect/direct call returns.
- Negative test: Circle without Comparable impl now correctly errors.
- Selfhost loop: C output IDENTICAL.
2026-06-10 09:02:22 +03:00
dimgigov f63cbd1bf0 selfhost: trait bounds parsing, sema checks, and generic monomorphization fix
- AST: add typeParam0Bound/typeParam1Bound to Decl, strValue2 to dkImpl
- Parser: parse <T: Trait> bounds, Self type in interfaces, extend-for blocks
- Sema: add interfaceTable/methodTable, Sema_CheckTraitBounds, Sema_TypeImplements
- HIR lower: two-pass decl iteration — collect generic funcs before lowering bodies
  Fixes Max<Circle>(...) not generating Max_Circle when caller precedes callee
- C backend: skip generic funcs in emission (only emit monomorphized instances)
- Array.bux: fix for-loop over monomorphized Array types
- All debug prints removed; selfhost loop passes (C output IDENTICAL)
2026-06-10 08:48:10 +03:00
dimgigov 499b389ba8 fix(selfhost): collection-based for-in with Std::Array + monomorphization fixes
- Fix Fmt_Format name conflict between lib/Fmt.bux and src/fmt.bux
  by renaming selfhost formatter function to Fmt_FormatSource.

- Fix collection-based for-in loops to handle already-monomorphized
  types like Array_int and Iter_string (typeArgCount=0 cases).

- Fix ekSizeOf and ekCast in HIR lowering to apply Lcx_SubstituteType
  during generic monomorphization, so sizeof(T) and *T casts emit
  correct concrete types in C output.

- Fix sema to infer stmt.refStmtType from initializer expression
  when no explicit type annotation is given, ensuring C backend
  emits correct pointer types for let data = ... as *T.
2026-06-10 01:15:21 +03:00
dimgigov 632fa1535c fix(test): update wrong-args error message expectation 2026-06-10 00:28:18 +03:00
dimgigov 40bcbeb710 Fix generic struct monomorphization and C output for imports
- Fix nested Lcx_GenerateStructInstance overwriting parent slot by
  incrementing structCount BEFORE field processing
- Fix Lcx_LowerModule to skip generic structs from hm.structs (was missing
  continue due to bootstrap compiler codegen limitation)
- Fix C backend to forward-declare ALL structs at top, removing second
  generic-only forward declaration section that came too late
- Add empty-name guard in CBE_EmitStructDef
- Import system: parser supports ::* glob imports, sema two-pass resolution
2026-06-09 23:33:44 +03:00
dimgigov b36df0f5b7 feat(selfhost): generic monomorphization + collection-based for-in loops
- Add on-demand generic function/struct monomorphization in HIR lower
  - Lcx_SubstituteType: substitute type params and mangle names (e.g. Array<int> -> Array_int)
  - Lcx_GenerateFuncInstance: clone generic func with active substitution
  - Lcx_GenerateStructInstance: create concrete struct definitions
  - Detect generic calls (ekCall) and struct inits (ekStructInit)

- Parser: propagate generic type args from ident to ekStructInit node

- Fix duplicate HIR constant: hIndexPtr and hCall both had value 18,
  causing index expressions to emit as comma expressions in C backend

- Implement collection-based for-in desugaring:
  for x in arr { body } -> __iter = Array_Iter_T(&arr);
                           while Iter_HasNext_T(&__iter) { x = Iter_Next_T(&__iter); body }

Selfhost loop remains deterministic. All golden tests and examples pass.
2026-06-09 22:33:36 +03:00
dimgigov b3141dbcd5 docs: update ROADMAP + PHASE8 for for-in loops
- Mark range-based for-in as done
- Note collection-based for-in blocked by generic monomorphization
2026-06-09 21:55:40 +03:00
dimgigov 74db8a5790 feat: for-in range loops in selfhost + bootstrap
- Parser: add parseRange for selfhost (.. / ..=)
- Sema: determine loop variable type from range bounds
- HIR Lower: desugar range for-in to while loop with counter
- Bootstrap sema: fix range-based for variable type (was unknown)
- Selfhost loop remains deterministic
2026-06-09 21:53:12 +03:00
dimgigov 6414ef236b docs: update ROADMAP and PHASE8 for closures with captures 2026-06-09 21:23:58 +03:00
dimgigov 9b473c667c closures with captures: env struct + global instance + body rewriting
- AST: captureCount + captureName0..7 + captureType0..7 in Expr
- Sema: Scope_LookupUpTo + closureScope for capture analysis
- HIR Lower: env struct generation, capture assignments in skLet,
  body rewriting (ekIdent -> hFieldAccess on env instance)
- C Backend: env struct def + global instance emission for closures
- Bootstrap: full capture support in sema, hir_lower, lir_lower, lir_c_backend
- Selfhost loop remains deterministic
2026-06-09 21:23:05 +03:00
dimgigov ba54aa240e docs: update ROADMAP and LanguageRef with closures
- Mark capture-less closures as  Done in ROADMAP
- Add closure syntax examples to LanguageRef.md Functions section
- Update PHASE8_STRATEGY.md Milestone A with closures
2026-06-09 20:25:34 +03:00
dimgigov c83f6d5994 feat: capture-less anonymous functions (closures)
Implement MVP closures — anonymous functions without captures.

Syntax:
  |a: int, b: int| -> int { return a + b; }

Changes:
- ast.bux + ast.nim: add ekClosure AST node
- parser.bux + parser.nim: parse |params| -> Ret { body }
- sema.bux + sema.nim: type-check closure params/body, return tyFunc
- hir_lower.bux + hir_lower.nim: generate __closure_N function + hAddrOf
- lir_c_backend.nim: fix function-pointer variable declaration (cParamDecl)
- C backend: closures compile to global functions with unique names

Test: _test_closure/src/Main.bux
- Closure as variable
- Closure passed to higher-order function
- Address of named function as function pointer

Both bootstrap and selfhost compilers build and pass the test.
2026-06-09 20:24:10 +03:00
dimgigov 34504d1647 docs: sync all docs with recent features
Updated documentation to reflect changes from the last 2 days:

ROADMAP.md:
- Mark string interpolation, named/default params, bux fmt/test/new/init,
  and basic borrow checker as  Done
- Update Recommended Order (next: closures)

PHASE8_STRATEGY.md:
- Update date, achievements, and milestone statuses
- Borrow checker is now  working in selfhost
- CTFE, LSP, formatter, test runner marked done where applicable

LanguageRef.md:
- Add defer, switch/case, string interpolation (f"...")
- Add &T, &mut T, own T to composite types
- Add named/default parameters and operator overloading sections
- Expand @[Checked] rules with double-mut-borrow and use-after-move
- Update keywords and operators

STRATEGY.md:
- Update date and version (0.3.0 selfhost)
- Mark 8.2 borrow checker, 8.4 CTFE, 8.7 LSP as 

BuildAndTest.md:
- Add bux init, bux fmt, bux test commands
- Expand test sections with golden tests and Std::Test
2026-06-09 19:19:53 +03:00
dimgigov 079b76ae71 feat(selfhost): add 'bux fmt' code formatter
Line-based indentation formatter that:
- Adjusts indentation based on { } brace depth (4 spaces)
- Preserves original line structure and comments
- Skips braces inside string/char literals and // comments
- Handles } at line start and mixed brace lines
- 'bux fmt <file>' formats a single file
- 'bux fmt <dir>' formats all .bux files in directory
2026-06-09 17:48:20 +03:00
dimgigov 3ce7ba51d2 feat(selfhost): use-after-move tracking in @[Checked] functions
- Add inline movedName0..7 fields to Sema (avoids dynamic allocation issues)
- Sema_AddMoved: track variable moves from let/var init expressions
- Sema_IsMoved: check if variable was moved before use
- Sema_RemoveMoved: clear moved status on reinitialization (x = value)
- Report 'use of moved value' error for moved variable access
2026-06-09 17:40:20 +03:00
dimgigov 41c0fb1d6f feat(selfhost): double mutable borrow detection in @[Checked] functions
Detect when the same variable is passed as &mut argument twice
in a single function call (e.g. Swap(&x, &x)) inside checked functions.
Matches bootstrap sema.nim behavior (lines 1078-1089).
2026-06-09 17:32:21 +03:00
dimgigov f809827bea feat(selfhost): add &T (shared ref) and &mut T (mutable ref) type syntax
- Add tekRef and tekMutRef to TypeExpr enum (ast.bux)
- Parse &T and &mut T in parserParseType, before *T handling (parser.bux)
- Map tekRef/tekMutRef to tyPointer in HIR lowering (hir_lower.bux)
- Both emit as T* in C backend (same as raw pointer)
- Borrow checker rejects write-through-pointer in @[Checked] functions
  for all pointer types (safe default)
2026-06-09 17:30:19 +03:00
dimgigov 81281cbb11 feat(selfhost): @[Checked] borrow checker — basic write-through-ptr detection
- Add isChecked field to Decl struct (ast.bux)
- Parse @[Checked] attribute in parserParseDecl (parser.bux)
- Add checkedFunc flag and movedVars to Sema struct (sema.bux)
- Enable borrow checking per-function in Sema_Analyze
- Reject assignment through raw pointer (*T) in @[Checked] functions
- Skip newlines between @[Checked] attribute and the declaration keyword
  to avoid parser seeing the attribute and function as separate decls
2026-06-09 17:25:28 +03:00
dimgigov c14578d9dd feat(selfhost): add 'bux test' command
- 'bux test [dir]' builds the project and runs the resulting binary,
  reporting 'Tests passed' or 'Tests failed (exit code N)'.
- Builds on existing Cli_BuildProject pipeline — test code goes in
  src/ alongside production code; test assertions use lib/Test.bux.
2026-06-09 17:11:31 +03:00
dimgigov 0ac254facc feat(selfhost): add 'new' and 'init' CLI commands
- 'buxc new <name>': creates a new Bux project directory with bux.toml
  and src/Main.bux. Supports both relative names and absolute paths.
- 'buxc init': initializes a Bux project in the current directory.
- Adds extern bux_getcwd() and bux_path_parent() declarations.
- Fix mkdir return value check (bux_mkdir_if_needed returns 0 on success).
- Extract package name from path for absolute path arguments.
2026-06-09 17:09:03 +03:00
dimgigov 392dd64f97 cleanup(selfhost): remove 17 dead functions across 5 files
Remove unused helper functions, factories, accessors, and merge helpers:
- c_backend.bux: CBE_Init, CBE_ClearDefers, CBE_EmitLine, CBE_Build, CBackend_Free
- hir_lower.bux: Lcx_FreshName, Lcx_LowerType, HirLower_Free
- lexer.bux: Lexer_Init, lexIsIdentKind, Lexer_TokenCount, Lexer_GetToken
- parser.bux: parserIsBinaryOp, parserPrevious, parserMergeAddDecl,
  Parser_DiagCount, Parser_Free, Parser_MergeModules
- sema.bux: Sema_TypeName

None of these functions are referenced anywhere in src/.
2026-06-09 16:14:31 +03:00
dimgigov 90e67079f4 cleanup(selfhost): remove unused Print/PrintLine externs + dead debug comments from sema.bux 2026-06-09 15:59:57 +03:00
dimgigov 265bda850d test: expand golden C-codegen tests from 2 to 8 examples
Add golden tests for: structs, generics, algebraic_enums, enums,
methods, strings. Each test captures the expected C output from the
bootstrap compiler for regression detection.

Also remove leftover 'PARSER FIX V2 ACTIVE' debug print from parser.bux.
2026-06-09 15:58:44 +03:00
dimgigov 3ed5891de2 fix(selfhost): block scoping for if/while/loop/match bodies
- Create child scope in Sema_CheckBlock for every block, matching
  bootstrap sema.nim behavior (newScope for ekBlock).
- Fixes "var scoping bug in selfhost sema" where variables declared
  with var inside if/while blocks shared parent function scope, causing
  duplicate-definition errors between sibling if/else blocks, and C
  scoping mismatch between user variables and compiled output.
- Remove stale workaround comments from lib/Json.bux (let instead of
  var inside if blocks — no longer needed).
- Update sema.bux TODO comment for type table.
2026-06-09 15:56:16 +03:00
dimgigov 64bd41067a cleanup(selfhost): remove 24 DEBUG prints + stale parser HACK
- Remove DEBUG prints from c_backend.bux, hir_lower.bux, lexer.bux, parser.bux
- Remove HACK for stray } in parserParseFuncDecl (no longer needed after
  Decl refactor in 4a86009)
- Clean selfhost check/build output
2026-06-09 15:48:38 +03:00
dimgigov 4a860095fd fix(selfhost): resolve segfault in buxc2 compiling src/
- Replace Decl's 256 inline struct fields with dynamic fields: *StructField
  array to avoid parser field limit truncation and name shifts.
- Update parser.bux and hir_lower.bux to use decl.fields[i].
- Fix uninitialized Symbol.refType in three stack-allocated Symbol sites:
  loopSym (for-loop iterator), vSym (enum variants), and HIR param sym.
  Garbage refType values (e.g. 0xFF) were leaking into scope tables,
  later copied to expr->refType, causing SIGSEGV in Lcx_LowerExpr.
2026-06-09 15:03:45 +03:00
dimgigov 0c41c7bb25 feat(selfhost): function types (func(Params) -> Ret)
- Add tekFunc AST node and TypeExprList linked list for params
- Parse func(T, U) -> R syntax in parser.bux
- Resolve tekFunc to tyFunc in sema.bux
- Track original TypeExpr via Symbol.refType for params/lets/consts/funcs
- Build C function-pointer type names in hir_lower.bux (Ret (*)(Params))
- Lower indirect calls through function-typed values as hCallIndirect
- Add CBE_CParamDecl helper in c_backend.bux to embed name inside (*)
- Emit hCallIndirect and function-pointer variable declarations
- _test_funcptr now builds and runs with selfhost buxc2
2026-06-08 23:39:15 +03:00
dimgigov 2e536488e6 feat(bootstrap): function pointer types + C emission
- Add tekFunc AST node and parser support for func(Params) -> Ret
- Add tkFunc type resolution in sema and hir_lower
- Add &FuncName address-taking yielding tkMutRef(tkFunc)
- Fix C emission: function pointer params use Ret (*name)(Params) syntax
  via cParamDecl helper that embeds the name inside (*)
- Fix forward declarations and temp declarations for function pointers
- Add funcPtrTypes table in C backend so &Fn temps get proper type
- Add _test_funcptr integration test
2026-06-08 22:31:32 +03:00
dimgigov 3f0a3901ca feat: named and default parameters
Bootstrap compiler:
- ast.nim: add exprCallArgNames to ekCall for named arg tracking
- parser.nim: detect name: value syntax in call args
- sema.nim: add resolveCallArgs helper that injects default values
  for missing params and reorders named args into param order.
  Parser already parsed default param values; sema now uses them.

Selfhost compiler:
- ast.bux: add defaultExpr to Param, argName to ExprList
- parser.bux: parse = defaultExpr in param list, detect name: value
  syntax in call arguments
- sema.bux: add Sema_ResolveCallArgs with same logic as bootstrap

Tests:
- _test_named_params verifies defaults, named args, mixed positional+named,
  and named args with defaults in both compilers.

All verifications pass: build, selfhost-loop, test-examples, test-golden.
2026-06-08 22:02:19 +03:00
dimgigov 17c896c4dc feat: string interpolation via f"..." syntax
Bootstrap compiler:
- Add ekStringInterp AST node with text/expr parts
- Lexer: recognize f"..." prefix for interpolation strings
- Parser: split f"..." into ekStringInterp, parse inner expressions
  via sub-lexer/sub-parser. Supports escaped braces \{ and \}.
- Sema: type-check interpolated expressions, return String type
- HIR lowerer: desugar ekStringInterp to chained String_Concat
  calls with automatic type conversions:
  - int/uint types -> String_FromInt
  - float types -> String_FromFloat
  - bool -> String_FromBool
  - String -> as-is

Selfhost compiler:
- ast.bux: add ekStringInterp constant (reserved for future)
- lexer.bux: accept f" prefix, treat as plain string literal
  (selfhost parser does not implement interpolation yet)

Tests:
- _test_string_interp verifies int, float, bool, String,
  multiple interpolations, plain strings, and Fmt-style {0} templates
2026-06-08 21:48:10 +03:00
dimgigov cfce2a93e5 feat: operator overloading in bootstrap compiler
- Add operatorMethodName mapping (tkPlus -> operator_add, etc.)
- Add tryLowerOperatorCall / tryLowerIndexCall in HIR lowerer
- Modify ekBinary lowering to check method table before builtins
- Modify ekAssign lowering to detect ekIndex target and emit
  operator_index_set method call when available
- Add sema type-checking for operator overloads in ekBinary
  and ekIndex paths (method-table lookup before builtin rules)
- Fix sema compile errors: minfo.params[N].typ -> minfo.params[N]
- Fix hir_lower forward decl ordering for resolveExprType
- Update golden test expected.c files for current C backend
- Update ROADMAP.md and PHASE8_STRATEGY.md status
2026-06-08 21:31:42 +03:00
dimgigov d9aceeac6e feat: switch/case statement
- Add  syntax
- Desugars to if-else chain in HIR lowering (both compilers)
- No new HIR/C backend nodes needed — reuses hIf/hBinary/hBlock
- Supports integer, char, and enum tag dispatch
- Case body can be single statement (no braces required)
2026-06-08 21:08:55 +03:00
dimgigov a67271b08c feat: defer statement support in both compilers
- Add  statement for function-level deferred execution
- Deferred expressions run in LIFO order on function exit (return or implicit)
- Bootstrap: desugar defers in hir_lower.nim (inject before return + end of func)
- Selfhost: emit defers in c_backend.bux via CEmitter defer stack
- Both: add tkDefer token, skDefer AST node, hDefer HIR node
- Parser, lexer, sema, token files updated in both bootstrap/ and src/
2026-06-08 20:59:27 +03:00
dimgigov cf92b63ba6 fix: bootstrap parser newline skipping + nexus compile errors
- Add tkNewLine skipping in bootstrap parser parsePrimary, parseUnary, parsePostfix
- Fix nexus: req.String_Len(path) -> String_Len(req.path)
- Fix nexus: bux_sb_append_char with char literal -> bux_sb_append with string
2026-06-08 20:28:37 +03:00
dimgigov 9c3d6dc723 fix: EVP_SignFinal unsigned int* type mismatch in OpenSSL calls 2026-06-08 20:24:47 +03:00
dimgigov 7d1d722cba fix: String_IsNull/String_Offset helpers + remove pointer-to-int casts
- Add String_IsNull and String_Offset wrappers in lib/String.bux
- Add bux_str_is_null and bux_str_offset runtime functions in rt/runtime.c
- Update String_Replace to use String_IsNull instead of String_Len(pos)==0
- Replace pointer-to-uint casts (as uint == 0, pointer arithmetic) across apps/
- Parser newline skipping fixes in src/parser.bux
2026-06-08 20:23:30 +03:00
dimgigov 3c7938163c fix(runtime): detach StringBuilder buffer on build to prevent use-after-free
bux_sb_build returned sb->buf and then bux_sb_free freed it,
causing dangling pointers in all callers of StringBuilder_Build
followed by StringBuilder_Free (e.g. JsonParser_ParseString, Fmt).

Fix: bux_sb_build now detaches the buffer (sets sb->buf = NULL)
before returning it, so bux_sb_free only frees the sb struct.
This transfers buffer ownership to the caller.
2026-06-08 20:10:30 +03:00
dimgigov c4e3f80405 fix(selfhost/c_backend): handle backtick raw string literals
Backtick strings were emitted directly into C code without
stripping the backticks or escaping the content, causing C syntax errors.

Fix: detect backtick-wrapped literals in CBE_EmitExpr, strip
the backticks, escape inner content, and wrap in double quotes.
2026-06-08 20:03:33 +03:00
dimgigov d253a01aa9 fix(selfhost/c_backend): emit _Tag + struct wrapper for all enums
Simple enums were emitted as plain C enums, but the rest of the codegen
(and source code) expected them to have a .tag field like algebraic enums.
This caused C compilation errors:
- 'request for member tag in something not a structure or union'
- 'field name not in record or union initializer'

Fix: always emit the _Tag typedef + struct wrapper for every enum.
For simple enums, the struct only contains the tag (no _Data union).
For algebraic enums, keep the existing _Data union + struct behavior.

This matches the bootstrap compiler's enum codegen.
2026-06-08 20:00:58 +03:00
dimgigov 2cb971a483 fix(selfhost/parser): skip newlines inside expressions
The selfhost parser did not skip tkNewLine tokens inside expression parsing,
causing multi-line expressions (e.g. if conditions spanning lines with ||)
to be truncated. When parserParseBinaryPrec hit a newline it treated it as
an invalid primary, returning a malformed literal and leaving p.pos at the
newline. parserParseBlock then failed to find '{' and its depth-counter
fallback consumed far too many tokens, causing subsequent function bodies
to be parsed into the wrong AST nodes.

Fix: add newline skipping in parserParsePrimary, parserParseUnary, and
parserParseBinaryPrec, matching the bootstrap parser's skipNewlines() calls
in parseAnd/parseOr.

Also complete the variant cap raise from 8->9 in src/sema.bux (was already
done in ast.bux, parser.bux, hir_lower.bux).
2026-06-08 19:45:05 +03:00
dimgigov b1deff7fd5 fix(selfhost/parser): skip newlines inside multi-import brace blocks
parserParseImportDecl now skips tkNewLine tokens inside {..} blocks,
matching the bootstrap parser behavior. This fixes parsing of imports
with names on separate lines, e.g.:

  import Std::String::{
      String_Eq,
      String_Len
  };

Previously the parser failed because it expected tkIdent immediately
after { but encountered tkNewLine instead.
2026-06-08 18:56:43 +03:00
dimgigov 38bc469be7 fix(bootstrap): resolve algebraic enum _Data union field types in HIR lowering
hir_lower.resolveExprType for ekField now correctly resolves variant
positional/named fields (e.g., r.data.Ok_0, r.data.Err_0) on _Data
union types, matching the existing logic in sema.checkExpr.

This fixes the runtime segfault in examples/algebraic_enums.bux where
the C backend emitted an int temporary for a String field, causing
PrintLine to dereference a garbage pointer.
2026-06-08 18:48:30 +03:00
dimgigov bebc362ae6 fix(bootstrap): simple enum codegen, crypto parser compatibility, import ordering
- Fix bootstrap sema to type simple enum variants as the enum itself
  (not _Tag), and .tag access returns the enum type.
- Fix hir_lower to lower simple enum .tag as no-op and Enum{tag:X}
  init as X directly.
- Fix sema collectGlobals to register imports AFTER real declarations,
  preventing duplicate-symbol errors when stdlib files import symbols
  defined in later files (e.g. jwt.bux importing rsa.bux).
- Rewrite lib/crypto/*.bux as single-line to work around bootstrap
  parser limitations with multi-line function calls.
- Rename 'pub' -> 'pubBuf' in ed25519.bux and jwt-pitbul/Main.bux
  to avoid reserved keyword collision.
- Remove algTag:int workaround from jwt-pitbul/Main.bux; enum
  comparisons now type-check without manual casts.
- Update .gitignore to exclude _test_*/ and backup files.
- Delete accidentally-committed _test_array/ directory.
2026-06-08 18:43:26 +03:00
dimgigov a7a1b45588 chore: add MIT license 2026-06-08 14:44:05 +03:00
dimgigov a13cee8a0f feat(simpledb): file-backed key-value database with CLI
- Add SimpleDB: plain-text key=value store with get/set/del/has/keys/count
- Fix lib/Fmt.bux: replace *(ptr+N) pointer arithmetic with ptr[N] indexing
- Fix lib/crypto/jwt.bux: use alg.tag directly instead of int-comparison
2026-06-08 12:26:24 +03:00
dimgigov 3b7fc455b0 fix(parser): infinite loop on newlines in multi-import, multi-line params, type args, struct patterns
Parser would hang (infinite loop) when encountering newline tokens
inside certain constructs because expect() returns without advancing
position when the expected token doesn't match.

Fixed 5 places in parseBaseType, parsePrimaryPattern, parseTypeParams,
parseParamList, and parseUseDecl by adding newline-skipping at the
start of each iteration.

Verified: compiler no longer hangs/crashes on all 3 real-world apps
from apps/ (boko-framework, jwt-pitbul, nexus).
2026-06-07 23:02:43 +03:00
dimgigov e7e900973f feat: crypto library, Nexus HTTP server, JWT CLI, Boko web framework
- lib/crypto/: 9-module crypto library (Hash, HMAC, Base64/URL, Random, AES, RSA, ECDSA, Ed25519, JWT)
- rt/runtime.c: +346 lines OpenSSL crypto primitives (SHA-1/384/512, HMAC, Base64URL, AES-CBC/GCM, RSA, ECDSA, Ed25519)
- apps/nexus/: multi-threaded HTTP/1.1 + HTTP/2 detect + WebSocket server
- apps/jwt-pitbul/: JWT CLI tool (sign, verify, decode, keygen)
- apps/boko-framework/: FastAPI-inspired async web framework
- test_crypto/: crypto library test suite (23 tests)
2026-06-07 22:15:00 +03:00
dimgigov a2617e9954 feat: VS Code extension, web playground, LSP server 2026-06-07 18:34:41 +03:00
dimgigov 3ffa9f7d54 feat: web playground + LSP server + docs update 2026-06-07 18:29:05 +03:00
dimgigov b3af1208e8 feat(lsp): add LSP server with completion, definition, hover, diagnostics 2026-06-07 18:20:35 +03:00
dimgigov 0d7a30835f feat: borrow expr, parser fix, borrow tests, borrow example 2026-06-07 18:09:40 +03:00
dimgigov adca52b269 docs: update README v0.4.0, update stdlib docs to 23 modules
- README: bump version to v0.4.0, add selfhost loop & borrow/shared
- Stdlib.md: update sections 1-4, add String_Chars, FromBool, Fmt, Iter
- BuildAndTest.md: add make selfhost-loop and make test-golden targets
- Standard library list updated to all 23 modules
2026-06-07 18:08:34 +03:00
dimgigov 04be9eb400 feat(stdlib): add String_Chars, String_FromBool, fix Fmt and Net
- String.bux: add String_Chars(s, index) for char-at-index access
- String.bux: add String_FromBool(b) conversion
- Fmt.bux: rewrite with correct stdlib imports (was using non-existent names)
- Net.bux: fix String_Len -> bux_strlen (String_Len not imported)
- All lib modules compile successfully
2026-06-07 18:05:04 +03:00
dimgigov bfe87a8acb feat: add borrow test example, fix borrow &mut parsing order
- examples/borrow.bux: working borrow expression example
- tests/borrow_test.nim: add 4 new tests for borrow & @[Shared]
- parser.nim: fix borrow parsing order (& before mut)
- Build verified: example compiles and runs
2026-06-07 17:50:27 +03:00
dimgigov 46814b2abc feat: selfhost-loop determinism, golden tests, sema diagnostics 2026-06-07 17:37:22 +03:00
dimgigov 9758b17b1a fix(parser): add stray } workaround in parserParseFuncDecl, enable selfhost build
This workaround consumes a stray } that parserParseBlock sometimes leaves
behind, which was causing the module handler to exit early. With this fix,
the selfhost compiler (buxc2) can now compile the entire src/ project.

- buxc2 project build: 491 user + 310 stdlib = 801 decls, no sema errors
- make selfhost-loop: builds and runs both passes
- C compilation succeeds, resulting binary works
2026-06-07 17:33:17 +03:00
dimgigov 8347599d23 fix(parser): save/restore + depth counter for reliable } consumption 2026-06-07 17:25:30 +03:00
dimgigov 4794335673 chore: clean up experimental changes, keep core parser improvements 2026-06-07 17:11:45 +03:00
dimgigov 134e8e877d fix(parser): depth counter in module handler, stray } cleanup, token count debug 2026-06-07 17:06:36 +03:00
dimgigov 88af8b09d4 fix(parser): improve parserParseBlock infinite-loop safeguard for stuck at } 2026-06-07 17:01:15 +03:00
dimgigov e41a8d31df fix(parser): use parserExpect for module closing brace 2026-06-07 16:43:14 +03:00
dimgigov a11071ab95 fix(parser): skip newlines before checking decl kind 2026-06-07 16:38:42 +03:00
dimgigov 093e08a2b9 chore(cli): remove diagnostic scan, keep FIFO merge and per-file counts 2026-06-07 16:25:23 +03:00
dimgigov 1500575982 fix(parser): replace destructive skip with single-token skip 2026-06-07 16:24:51 +03:00
dimgigov fe048eb4aa feat(cli): diagnostic scan for missing function defs, FIFO merge order, per-file counts 2026-06-07 16:20:22 +03:00
dimgigov 9dd964dc48 test: add fibonacci golden test, expand GOLDEN_TESTS 2026-06-07 16:10:05 +03:00
dimgigov e1038059c2 fix(cli): change user merge from LIFO to FIFO order, add per-file decl counts
- Cli_BuildProject now appends declarations (preserving original file order)
  instead of prepending (which reversed order)
- Added per-file declaration count debug output
- This makes merge order match Nim bootstrap behavior
2026-06-07 16:08:51 +03:00
dimgigov 1f0baefa5a feat: add selfhost-loop determinism check, golden tests, improve sema diagnostics
- Makefile: new targets `test-golden` and `selfhost-loop`
  - selfhost-loop verifies C codegen determinism (2-pass bootstrap)
  - test-golden compares C output against expected.c golden files
  - Updated clean-all to clean new build dirs
- tests/golden/hello/: first golden test (hello world)
- src/sema.bux:
  - Fix: import stubs only registered if symbol doesn't already exist
  - Improve: error message now includes identifier name
2026-06-07 15:47:48 +03:00
dimgigov 3b4456f01f selfhost: expand Decl to 256 fields, fix unary precedence, add discard/var parsing, fix diag buffer overflow
- ast.bux: Decl expanded with field0..field255, variant0..7, param0..8
- parser.bux: add parserParseUnary for correct unary precedence; handle tkDiscard statements; allow var without initializer (matching bootstrap); add diagCount bounds check in parserEmitDiag
- hir.bux: HirParam changed to pointers
- hir_lower.bux: full field chains for struct lowering
- c_backend.bux: hardened -> vs . field access; null checks in CBE_GetExprTypeName
- cli.bux: add -lcrypto to linker
2026-06-07 14:58:23 +03:00
377 changed files with 111161 additions and 8123 deletions
+458
View File
@@ -0,0 +1,458 @@
# Default PR / main CI: split Linux jobs + lean macOS + Windows bootstrap smoke.
# Full sequential suite locally: `make test`.
# Selfhost fixed-point: see selfhost-loop.yml (not on every PR).
name: ci
on:
pull_request:
push:
branches: [main]
workflow_dispatch:
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
# Pin patch for stable toolchain cache keys (was 2.0.x).
NIM_VERSION: "2.0.8"
# setup-nim-action install dir (relative to workspace)
NIM_INSTALL_DIR: ".nim_runtime"
jobs:
# ── Shared bootstrap build (Linux) ──────────────────────────────────────
build:
name: build (ubuntu)
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Cache Nim toolchain
id: cache-nim
uses: actions/cache@v4
with:
path: ${{ env.NIM_INSTALL_DIR }}
key: ${{ runner.os }}-nim-${{ env.NIM_VERSION }}-v1
- name: Install Nim
if: steps.cache-nim.outputs.cache-hit != 'true'
uses: jiro4989/setup-nim-action@v2
with:
nim-version: ${{ env.NIM_VERSION }}
nim-install-directory: ${{ env.NIM_INSTALL_DIR }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: PATH for cached Nim
if: steps.cache-nim.outputs.cache-hit == 'true'
run: |
echo "$PWD/${{ env.NIM_INSTALL_DIR }}/bin" >> "$GITHUB_PATH"
echo "$HOME/.nimble/bin" >> "$GITHUB_PATH"
- name: Cache nimcache (bootstrap)
uses: actions/cache@v4
with:
path: nimcache
key: ${{ runner.os }}-nimcache-build-${{ hashFiles('bootstrap/**/*.nim') }}-v1
restore-keys: |
${{ runner.os }}-nimcache-build-
- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
gcc make binutils libssl-dev python3
- name: Build buxc
run: |
nim -v
make build
- name: Upload buxc
uses: actions/upload-artifact@v4
with:
name: buxc-linux
path: buxc
retention-days: 3
# ── Parallel Linux suites (reuse prebuilt buxc) ─────────────────────────
unit:
name: unit + fmt
needs: build
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- name: Cache Nim toolchain
id: cache-nim
uses: actions/cache@v4
with:
path: ${{ env.NIM_INSTALL_DIR }}
key: ${{ runner.os }}-nim-${{ env.NIM_VERSION }}-v1
- name: Install Nim
if: steps.cache-nim.outputs.cache-hit != 'true'
uses: jiro4989/setup-nim-action@v2
with:
nim-version: ${{ env.NIM_VERSION }}
nim-install-directory: ${{ env.NIM_INSTALL_DIR }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: PATH for cached Nim
if: steps.cache-nim.outputs.cache-hit == 'true'
run: |
echo "$PWD/${{ env.NIM_INSTALL_DIR }}/bin" >> "$GITHUB_PATH"
echo "$HOME/.nimble/bin" >> "$GITHUB_PATH"
- name: Cache nimcache (unit tests)
uses: actions/cache@v4
with:
path: nimcache
key: ${{ runner.os }}-nimcache-unit-${{ hashFiles('bootstrap/**/*.nim', 'tests/**/*.nim') }}-v1
restore-keys: |
${{ runner.os }}-nimcache-unit-
${{ runner.os }}-nimcache-build-
- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends gcc make libssl-dev
- name: Download buxc
uses: actions/download-artifact@v4
with:
name: buxc-linux
- name: Prepare buxc
run: chmod +x buxc && ./buxc --version
- name: fmt-check + unit tests
env:
BUX_SKIP_BUILD: "1"
run: |
unset BUX_DEBUG_FILE || true
make fmt-check BUX_SKIP_BUILD=1
make test-unit BUX_SKIP_BUILD=1
examples:
name: examples
needs: build
runs-on: ubuntu-latest
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends gcc make libssl-dev
- name: Download buxc
uses: actions/download-artifact@v4
with:
name: buxc-linux
- name: Prepare buxc
run: chmod +x buxc && ./buxc --version
- name: test-examples
env:
BUX_SKIP_BUILD: "1"
run: |
unset BUX_DEBUG_FILE || true
make test-examples BUX_SKIP_BUILD=1
goldens:
name: goldens + tools
needs: build
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
gcc make binutils libssl-dev python3 gdb \
gcc-aarch64-linux-gnu
- name: Download buxc
uses: actions/download-artifact@v4
with:
name: buxc-linux
- name: Prepare buxc
run: chmod +x buxc && ./buxc --version
- name: errors + stdlib + registry + dwarf + linux-targets
env:
BUX_SKIP_BUILD: "1"
run: |
unset BUX_DEBUG_FILE || true
make test-errors BUX_SKIP_BUILD=1
make test-stdlib BUX_SKIP_BUILD=1
make test-registry BUX_SKIP_BUILD=1
make test-dwarf BUX_SKIP_BUILD=1
make test-drop-move BUX_SKIP_BUILD=1
make test-linux-targets BUX_SKIP_BUILD=1
apps:
name: apps
needs: build
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
gcc make libssl-dev openssl curl
- name: Download buxc
uses: actions/download-artifact@v4
with:
name: buxc-linux
- name: Prepare buxc
run: chmod +x buxc && ./buxc --version
- name: test-apps + nexus TLS/mTLS
env:
BUX_SKIP_BUILD: "1"
run: |
unset BUX_DEBUG_FILE || true
make test-apps BUX_SKIP_BUILD=1
make test-nexus-tls BUX_SKIP_BUILD=1
make test-nexus-mtls BUX_SKIP_BUILD=1
selfhost:
name: selfhost smoke
needs: build
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
gcc make libssl-dev openssl curl
- name: Download buxc
uses: actions/download-artifact@v4
with:
name: buxc-linux
- name: Prepare buxc
run: chmod +x buxc && ./buxc --version
- name: test-selfhost-smoke
env:
BUX_SKIP_BUILD: "1"
run: |
unset BUX_DEBUG_FILE || true
unset BUX_SELFHOST_FIXED_POINT || true
make test-selfhost-smoke BUX_SKIP_BUILD=1
- name: selfhost install + registry
env:
BUX_SKIP_BUILD: "1"
run: |
unset BUX_DEBUG_FILE || true
make test-selfhost-install BUX_SKIP_BUILD=1
make test-selfhost-registry BUX_SKIP_BUILD=1
- name: Upload selfhost artifacts on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: ci-failure-selfhost
path: |
build/selfhost/build/main.c
_test_tmp_pkg/**
if-no-files-found: ignore
# ── macOS smoke (platform matrix) — lean: cache Nim + smoke examples ───
# Full EXAMPLES / goldens / selfhost stay on Linux. macOS still builds
# bootstrap + runs unit tests + a representative example subset.
macos:
name: macos smoke
runs-on: macos-14
timeout-minutes: 35
steps:
- uses: actions/checkout@v4
- name: Cache Nim toolchain
id: cache-nim
uses: actions/cache@v4
with:
path: ${{ env.NIM_INSTALL_DIR }}
# macOS builds Nim from source — cache is the main time win
key: ${{ runner.os }}-nim-${{ env.NIM_VERSION }}-v1
- name: Install Nim
if: steps.cache-nim.outputs.cache-hit != 'true'
uses: jiro4989/setup-nim-action@v2
with:
nim-version: ${{ env.NIM_VERSION }}
nim-install-directory: ${{ env.NIM_INSTALL_DIR }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: PATH for cached Nim
if: steps.cache-nim.outputs.cache-hit == 'true'
run: |
echo "$PWD/${{ env.NIM_INSTALL_DIR }}/bin" >> "$GITHUB_PATH"
echo "$HOME/.nimble/bin" >> "$GITHUB_PATH"
- name: Cache nimcache (bootstrap + unit)
uses: actions/cache@v4
with:
path: nimcache
key: ${{ runner.os }}-nimcache-macos-${{ hashFiles('bootstrap/**/*.nim', 'tests/**/*.nim') }}-v1
restore-keys: |
${{ runner.os }}-nimcache-macos-
- name: OpenSSL (Homebrew)
run: |
# Prefer already-installed openssl@3 (common on GHA images)
if ! brew list openssl@3 &>/dev/null; then
brew install openssl@3
fi
OPENSSL_PREFIX="$(brew --prefix openssl@3)"
echo "OPENSSL_PREFIX=$OPENSSL_PREFIX" >> "$GITHUB_ENV"
# buxc passes BUX_CFLAGS through to cc (needed for Homebrew libcrypto)
echo "BUX_CFLAGS=-I${OPENSSL_PREFIX}/include -L${OPENSSL_PREFIX}/lib" >> "$GITHUB_ENV"
- name: Build buxc
run: |
nim -v
make build
- name: unit + smoke examples
run: |
unset BUX_DEBUG_FILE || true
# fmt-check is Linux-only in split CI (unit job); skip here to save time
make test-unit
make test-examples-smoke
# ── Windows smoke (bootstrap + unit + MinGW hello) ──────────────────────
# Uses rt/runtime_win.c (no pthread/OpenSSL). Full POSIX runtime is Unix-only.
windows:
name: windows smoke
runs-on: windows-latest
timeout-minutes: 35
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v4
- name: Cache Nim toolchain
id: cache-nim
uses: actions/cache@v4
with:
path: ${{ env.NIM_INSTALL_DIR }}
key: ${{ runner.os }}-nim-${{ env.NIM_VERSION }}-v1
- name: Install Nim
if: steps.cache-nim.outputs.cache-hit != 'true'
uses: jiro4989/setup-nim-action@v2
with:
nim-version: ${{ env.NIM_VERSION }}
nim-install-directory: ${{ env.NIM_INSTALL_DIR }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: PATH for cached Nim
if: steps.cache-nim.outputs.cache-hit == 'true'
run: |
echo "$PWD/${{ env.NIM_INSTALL_DIR }}/bin" >> "$GITHUB_PATH"
echo "$HOME/.nimble/bin" >> "$GITHUB_PATH"
- name: Install MinGW (gcc)
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: false
install: mingw-w64-x86_64-gcc
path-type: inherit
- name: Add MinGW64 to PATH
run: |
# setup-msys2 installs under C:/msys64 by default
for d in /c/msys64/mingw64/bin /mingw64/bin "$HOME/msys64/mingw64/bin"; do
if [[ -x "$d/gcc.exe" || -x "$d/gcc" ]]; then
echo "$d" >> "$GITHUB_PATH"
export PATH="$d:$PATH"
break
fi
done
gcc --version | head -1
- name: Cache nimcache (bootstrap + unit)
uses: actions/cache@v4
with:
path: nimcache
key: ${{ runner.os }}-nimcache-win-${{ hashFiles('bootstrap/**/*.nim', 'tests/**/*.nim') }}-v1
restore-keys: |
${{ runner.os }}-nimcache-win-
- name: Build buxc
run: |
set -e
nim -v
# MinGW gcc on PATH (from setup-msys2)
gcc --version | head -1
# Windows produces buxc.exe; keep name predictable for the smoke steps
nim c --nimcache:nimcache -o:buxc.exe -d:release --opt:size bootstrap/main.nim
./buxc.exe --version
- name: Pure Nim unit tests + CLI smoke
run: |
set -e
unset BUX_DEBUG_FILE || true
export NIMFLAGS=--nimcache:nimcache
echo "Running lexer tests..."
nim c $NIMFLAGS -r tests/lexer_test.nim
echo "Running parser tests..."
nim c $NIMFLAGS -r tests/parser_test.nim
echo "Running sema tests..."
nim c $NIMFLAGS -r tests/sema_test.nim
echo "Running HIR tests..."
nim c $NIMFLAGS -r tests/hir_test.nim
echo "Running borrow checker tests..."
nim c $NIMFLAGS -r tests/borrow_test.nim
echo "CLI smoke..."
rm -rf _test_tmp_pkg
./buxc.exe new _test_tmp_pkg
./buxc.exe --version
echo "unit+CLI: PASS"
- name: hello smoke (MinGW + runtime_win)
run: |
set -e
unset BUX_DEBUG_FILE || true
export BUX_STDLIB="$PWD/lib"
# bootstrap on Windows always picks runtime_win.c
chmod +x tools/smoke_windows_hello.sh
tools/smoke_windows_hello.sh
echo "windows smoke: PASS (bootstrap + unit + CLI + hello)"
# ── Single required status for branch protection ────────────────────────
ci-gate:
name: CI gate
if: always()
needs: [unit, examples, goldens, apps, selfhost, macos, windows]
runs-on: ubuntu-latest
steps:
- name: All jobs green?
run: |
set -e
echo "unit=${{ needs.unit.result }}"
echo "examples=${{ needs.examples.result }}"
echo "goldens=${{ needs.goldens.result }}"
echo "apps=${{ needs.apps.result }}"
echo "selfhost=${{ needs.selfhost.result }}"
echo "macos=${{ needs.macos.result }}"
echo "windows=${{ needs.windows.result }}"
for r in \
"${{ needs.unit.result }}" \
"${{ needs.examples.result }}" \
"${{ needs.goldens.result }}" \
"${{ needs.apps.result }}" \
"${{ needs.selfhost.result }}" \
"${{ needs.macos.result }}" \
"${{ needs.windows.result }}"; do
if [ "$r" != "success" ]; then
echo "CI gate failed: a required job is $r"
exit 1
fi
done
echo "CI gate: all required jobs passed"
+95
View File
@@ -0,0 +1,95 @@
# Optional / not on every PR: bootstrap selfhost determinism (make selfhost-loop).
# Default PR CI is .github/workflows/ci.yml (`make test`).
# Fixed-point gen2 vs gen3: BUX_SELFHOST_FIXED_POINT=1 (workflow_dispatch input).
name: selfhost-loop
on:
workflow_dispatch:
inputs:
fixed_point:
description: "Also run fixed-point buxc2→buxc3→buxc4 (gen2 vs gen3)"
type: boolean
default: false
schedule:
# Sundays 06:00 UTC
- cron: "0 6 * * 0"
push:
branches: [main]
paths:
- "src/**"
- "lib/**"
- "rt/**"
- "bootstrap/**"
- "tools/selfhost_loop.sh"
- "Makefile"
- ".github/workflows/selfhost-loop.yml"
concurrency:
group: selfhost-loop-${{ github.ref }}
cancel-in-progress: true
env:
NIM_VERSION: "2.0.8"
NIM_INSTALL_DIR: ".nim_runtime"
jobs:
selfhost-loop:
name: bootstrap determinism
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Cache Nim toolchain
id: cache-nim
uses: actions/cache@v4
with:
path: ${{ env.NIM_INSTALL_DIR }}
key: ${{ runner.os }}-nim-${{ env.NIM_VERSION }}-v1
- name: Install Nim
if: steps.cache-nim.outputs.cache-hit != 'true'
uses: jiro4989/setup-nim-action@v2
with:
nim-version: ${{ env.NIM_VERSION }}
nim-install-directory: ${{ env.NIM_INSTALL_DIR }}
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: PATH for cached Nim
if: steps.cache-nim.outputs.cache-hit == 'true'
run: |
echo "$PWD/${{ env.NIM_INSTALL_DIR }}/bin" >> "$GITHUB_PATH"
echo "$HOME/.nimble/bin" >> "$GITHUB_PATH"
- name: Cache nimcache
uses: actions/cache@v4
with:
path: nimcache
key: ${{ runner.os }}-nimcache-loop-${{ hashFiles('bootstrap/**/*.nim', 'src/**/*.bux') }}-v1
restore-keys: |
${{ runner.os }}-nimcache-loop-
${{ runner.os }}-nimcache-build-
- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends gcc make binutils libssl-dev
- name: Selfhost loop
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ] && [ "${{ inputs.fixed_point }}" = "true" ]; then
export BUX_SELFHOST_FIXED_POINT=1
echo "Fixed-point mode enabled (experimental)"
fi
make selfhost-loop
- name: Upload C on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: selfhost-loop-c
path: |
build/selfhost-loop-a/build/main.c
build/selfhost-loop-b/build/main.c
if-no-files-found: ignore
+11
View File
@@ -11,6 +11,7 @@ tests/lexer_test
tests/parser_test
tests/sema_test
tests/hir_test
tests/borrow_test
tests/test_generics_parse
tests/*.exe
@@ -22,3 +23,13 @@ build/
_test_tmp_pkg/
test_pkg/
examples_pkg/
# Temporary test directories
_test_*/
# Backup files
*.bak
# Log files
*.log
.nim_runtime/
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Dimitar Gigov
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+263 -16
View File
@@ -2,41 +2,63 @@ NIM := nim
SRC := bootstrap/main.nim
OUT := buxc
BUILD_DIR := build
# Project-local nimcache so CI can cache compiles (default is ~/.cache/nim).
NIMFLAGS ?= --nimcache:nimcache
EXAMPLES := hello fibonacci factorial structs enums methods algebraic_enums generics generics_struct generic_infer generic_infer2 extend_generic pattern_matching strings strings2 map result_option try_operator ownership ctfe async concurrency os_time process json iter
EXAMPLES := hello fibonacci factorial structs enums methods algebraic_enums generics generics_struct generic_infer generic_infer2 extend_generic pattern_matching strings strings2 map result_option try_operator ownership ownership_checked ownership_release drop_early_return lifetime_elision ctfe ctfe_crc async concurrency os_time process json iter trait_bounds channel sync jwt stdlib_ergonomics tuples func_ptr map_remove array_iter_extra string_extra multi_closure iter_hof closure_control match_let string_interp iter_generic generic_infer_hof struct_tuple_pat match_block nested_patterns match_guards pattern_shadow move_field move_field_partial move_field_remaining move_field_nested move_field_ptr move_cross_fn c_precedence macro_twice macro_repeat macro_nested macro_hygiene macro_unhygienic macro_stmt_pat macro_tt macro_tt_raw macro_type collections_extra
.PHONY: all build dev debug test clean clean-all test-examples selfhost
# Platform smoke (macOS CI): full EXAMPLES still runs on Linux.
EXAMPLES_SMOKE := hello ownership ownership_release strings map move_field move_field_partial move_field_remaining move_field_nested move_field_ptr move_cross_fn c_precedence macro_twice macro_repeat macro_nested macro_hygiene macro_unhygienic macro_stmt_pat macro_tt macro_tt_raw ctfe_crc
.PHONY: all build dev debug test clean clean-all test-examples test-examples-smoke selfhost test-golden test-errors test-stdlib selfhost-loop lsp fmt-check docs bench test-apps test-dwarf test-selfhost-smoke test-unit test-linux-targets ensure-buxc
all: build
build:
$(NIM) c -o:$(OUT) -d:release --opt:size $(SRC)
# Rebuild only when bootstrap sources change (CI can set BUX_SKIP_BUILD=1
# after downloading a prebuilt buxc artifact).
$(OUT): $(wildcard bootstrap/*.nim)
$(NIM) c $(NIMFLAGS) -o:$(OUT) -d:release --opt:size $(SRC)
# strip $(OUT)
build: $(OUT)
# CI parallel jobs download buxc and set BUX_SKIP_BUILD=1 to avoid rebuild.
ensure-buxc:
ifeq ($(BUX_SKIP_BUILD),1)
@test -x ./$(OUT) || (echo "error: ./$(OUT) missing (BUX_SKIP_BUILD=1)"; exit 1)
else
@$(MAKE) $(OUT)
endif
dev:
$(NIM) c -o:buxc_debug -d:debug --stackTrace:on --lineTrace:on $(SRC)
$(NIM) c $(NIMFLAGS) -o:buxc_debug -d:debug --stackTrace:on --lineTrace:on $(SRC)
debug: dev
@echo "Debug binary: buxc_debug"
test: build test-examples
# Full local / sequential suite (same coverage as split CI jobs combined).
test: build fmt-check test-examples test-errors test-stdlib test-registry test-dwarf test-drop-move test-linux-targets test-apps test-selfhost-smoke test-unit
# Nim unit tests + tiny CLI smoke (needs Nim + buxc).
test-unit: ensure-buxc
@echo "Running lexer tests..."
$(NIM) c -r tests/lexer_test.nim
$(NIM) c $(NIMFLAGS) -r tests/lexer_test.nim
@echo "Running parser tests..."
$(NIM) c -r tests/parser_test.nim
$(NIM) c $(NIMFLAGS) -r tests/parser_test.nim
@echo "Running sema tests..."
$(NIM) c -r tests/sema_test.nim
$(NIM) c $(NIMFLAGS) -r tests/sema_test.nim
@echo "Running HIR tests..."
$(NIM) c -r tests/hir_test.nim
$(NIM) c $(NIMFLAGS) -r tests/hir_test.nim
@echo "Running borrow checker tests..."
$(NIM) c -r tests/borrow_test.nim
$(NIM) c $(NIMFLAGS) -r tests/borrow_test.nim
@echo "Running integration tests..."
rm -rf _test_tmp_pkg
./$(OUT) new _test_tmp_pkg
./$(OUT) --version
test-examples: build
@for ex in $(EXAMPLES); do \
# Shared loop body for full + smoke example runners.
define run-examples
@for ex in $(1); do \
echo "=== Testing example: $$ex ==="; \
mkdir -p examples_pkg/$$ex/src; \
cp examples/$$ex.bux examples_pkg/$$ex/src/Main.bux; \
@@ -49,10 +71,23 @@ test-examples: build
echo '[Build]' >> examples_pkg/$$ex/bux.toml; \
echo 'Output = "Bin"' >> examples_pkg/$$ex/bux.toml; \
fi; \
(cd examples_pkg/$$ex && timeout 10 ../../$(OUT) run) || exit 1; \
if command -v timeout >/dev/null 2>&1; then \
(cd examples_pkg/$$ex && timeout 30 ../../$(OUT) run) || exit 1; \
else \
(cd examples_pkg/$$ex && ../../$(OUT) run) || exit 1; \
fi; \
done
endef
test-examples: ensure-buxc
$(call run-examples,$(EXAMPLES))
@echo "All examples passed!"
# Subset for macOS / quick platform smoke (Linux CI runs full EXAMPLES).
test-examples-smoke: ensure-buxc
$(call run-examples,$(EXAMPLES_SMOKE))
@echo "Smoke examples passed!"
clean:
rm -f $(OUT) buxc_debug
rm -rf $(BUILD_DIR)
@@ -62,9 +97,10 @@ clean:
rm -rf _test_cast _test_cast2 _test_cast3 _test_channel
clean-all: clean
rm -rf build/selfhost
rm -rf build/selfhost build/selfhost-loop-a build/selfhost-loop-b build/selfhost-loop-c
rm -rf tests/golden/*/build
selfhost: build
selfhost: ensure-buxc
@echo "=== Building self-hosted compiler ==="
@rm -rf build/selfhost
@mkdir -p build/selfhost/src
@@ -74,3 +110,214 @@ selfhost: build
@cd build/selfhost && ../../$(OUT) build
# strip removed for debug
@echo "=== Self-hosted compiler built successfully ==="
.PHONY: test-golden
GOLDEN_TESTS := hello fibonacci structs generics algebraic_enums enums methods strings modern_features
test-golden: ensure-buxc
@echo "=== Golden tests ==="
@passed=0; failed=0; \
for test in $(GOLDEN_TESTS); do \
gd="tests/golden/$$test"; \
if [ ! -d "$$gd" ]; then echo " SKIP $$test (no dir)"; continue; fi; \
if [ ! -f "$$gd/expected.c" ]; then echo " SKIP $$test (no expected.c)"; continue; fi; \
rm -rf "$$gd/build"; \
(cd "$$gd" && ../../../$(OUT) build > /dev/null 2>&1); \
if diff "$$gd/build/main.c" "$$gd/expected.c" > /dev/null 2>&1; then \
echo " PASS $$test"; \
passed=$$((passed + 1)); \
else \
echo " FAIL $$test — C output differs from expected"; \
failed=$$((failed + 1)); \
fi; \
done; \
echo "Golden tests: $$passed passed, $$failed failed"; \
if [ $$failed -gt 0 ]; then exit 1; fi
test-errors: ensure-buxc
@echo "=== Error diagnostic golden tests ==="
@chmod +x tests/error_golden/run.sh
@tests/error_golden/run.sh ./$(OUT)
test-stdlib: ensure-buxc
@echo "=== Stdlib golden tests ==="
@chmod +x tests/stdlib_golden/run.sh
@tests/stdlib_golden/run.sh ./$(OUT)
# Generate stdlib API docs from /// comments → docs/api/stdlib.md
docs: ensure-buxc
@mkdir -p docs/api
@./$(OUT) doc --out docs/api/stdlib.md lib/
@echo "docs/api/stdlib.md updated"
# CI: full-tree format check (lib / examples / src / tests / apps) + dirty-path smoke.
fmt-check: ensure-buxc
@echo "=== fmt --check (full tree) ==="
@./$(OUT) fmt --check lib/
@./$(OUT) fmt --check examples/
@./$(OUT) fmt --check src/
@./$(OUT) fmt --check tests/
@./$(OUT) fmt --check apps/
@echo "=== fmt --check dirty-path smoke ==="
@mkdir -p /tmp/bux_fmt_smoke
@printf 'func Main() -> int {\nreturn 0;\n}\n' > /tmp/bux_fmt_smoke/bad.bux
@if ./$(OUT) fmt --check /tmp/bux_fmt_smoke/bad.bux >/dev/null 2>&1; then \
echo "error: expected --check to fail on dirty file"; exit 1; \
fi
@echo "fmt --check passed (tree clean + dirty exits 1)"
# One-shot reformat of the same trees (run before committing style-only fixes)
.PHONY: fmt
fmt: ensure-buxc
@./$(OUT) fmt lib/
@./$(OUT) fmt examples/
@./$(OUT) fmt src/
@./$(OUT) fmt tests/
@./$(OUT) fmt apps/
@echo "Formatted lib/ examples/ src/ tests/ apps/"
# Fixed-point: bootstrap buxc → buxc2 → buxc3 (path-normalized C + stripped ELF).
# Slow; not part of default `make test`. Optional CI: .github/workflows/selfhost-loop.yml
selfhost-loop: ensure-buxc
@chmod +x tools/selfhost_loop.sh
@tools/selfhost_loop.sh
lsp: tools/bux-lsp
@echo "LSP server ready at tools/bux-lsp"
tools/bux-lsp: tools/lsp_server.nim bootstrap/*.nim
cd tools && $(NIM) c -d:release --opt:size --path:../bootstrap -o:bux-lsp lsp_server.nim
.PHONY: test-lsp
test-lsp: lsp
@echo "=== LSP unit (locals / inference) ==="
$(NIM) r --path:bootstrap tools/test_lsp_locals.nim
@echo "=== LSP hover smoke ==="
@chmod +x tools/smoke_lsp_hover.sh
@tools/smoke_lsp_hover.sh
@echo "=== LSP references / rename smoke ==="
@chmod +x tools/smoke_lsp_rename.sh
@tools/smoke_lsp_rename.sh
@echo "=== LSP workspace/symbol smoke ==="
@chmod +x tools/smoke_lsp_workspace.sh
@tools/smoke_lsp_workspace.sh
@echo "=== LSP deeper rename smoke ==="
@chmod +x tools/smoke_lsp_rename_deep.sh
@tools/smoke_lsp_rename_deep.sh
@echo "=== LSP call hierarchy smoke ==="
@chmod +x tools/smoke_lsp_call_hierarchy.sh
@tools/smoke_lsp_call_hierarchy.sh
@echo "=== LSP method call hierarchy smoke ==="
@chmod +x tools/smoke_lsp_method_hierarchy.sh
@tools/smoke_lsp_method_hierarchy.sh
@echo "=== LSP method/type/receiver rename smoke ==="
@chmod +x tools/smoke_lsp_rename_method.sh
@tools/smoke_lsp_rename_method.sh
@echo "=== LSP interface dispatch hierarchy smoke ==="
@chmod +x tools/smoke_lsp_iface_hierarchy.sh
@tools/smoke_lsp_iface_hierarchy.sh
@echo "==> LSP path rename smoke"
@chmod +x tools/smoke_lsp_rename_path.sh
@tools/smoke_lsp_rename_path.sh
@echo "==> LSP implementation smoke"
@chmod +x tools/smoke_lsp_implementation.sh
@tools/smoke_lsp_implementation.sh
@echo "==> LSP workspace import index smoke"
@chmod +x tools/smoke_lsp_workspace_imports.sh
@tools/smoke_lsp_workspace_imports.sh
@echo "==> LSP type hierarchy smoke"
@chmod +x tools/smoke_lsp_type_hierarchy.sh
@tools/smoke_lsp_type_hierarchy.sh
@echo "==> LSP type hierarchy workspace (closed multi-file)"
@chmod +x tools/smoke_lsp_type_hierarchy_ws.sh
@tools/smoke_lsp_type_hierarchy_ws.sh
.PHONY: test-registry
test-registry: ensure-buxc
@echo "=== Registry smoke (E.1 + HTTP) ==="
@chmod +x tools/smoke_registry.sh
@tools/smoke_registry.sh
# E.2 — build showcase apps + simpledb/jwt CLI smoke
.PHONY: test-apps
test-apps: ensure-buxc
@echo "=== Apps smoke (E.2) ==="
@chmod +x tools/smoke_apps.sh
@tools/smoke_apps.sh
# E.5 — micro-benchmarks (Bux + C/Nim/Zig twins)
.PHONY: bench
bench: ensure-buxc
@chmod +x tools/bench.sh
@tools/bench.sh
# E.5 — Nexus HTTP throughput (wrk); optional via BENCH_NEXUS=1 make bench
.PHONY: bench-nexus
bench-nexus: ensure-buxc
@chmod +x tools/bench_nexus.sh
@tools/bench_nexus.sh
# E.4 — DWARF / #line debugger smoke
.PHONY: test-dwarf
test-dwarf: ensure-buxc
@echo "=== DWARF / #line smoke (E.4) ==="
@chmod +x tools/smoke_dwarf.sh
@tools/smoke_dwarf.sh
# Session 75/85 — Linux / cloud / embedded: minimal, static, aarch64+riscv64 cross, CTFE CRC
.PHONY: test-linux-targets
test-linux-targets: ensure-buxc
@echo "=== Linux targets smoke (minimal / static / aarch64+riscv64 cross) ==="
@chmod +x tools/smoke_linux_targets.sh
@tools/smoke_linux_targets.sh
# Session 78 — Nexus HTTPS (self-signed) smoke
.PHONY: test-nexus-tls
test-nexus-tls: ensure-buxc
@echo "=== Nexus TLS smoke ==="
@chmod +x tools/smoke_nexus_tls.sh
@tools/smoke_nexus_tls.sh
# Session 79 — musl static (SKIP if no musl-gcc/zig)
.PHONY: test-musl-static
test-musl-static: ensure-buxc
@echo "=== musl static smoke ==="
@chmod +x tools/smoke_musl_static.sh
@tools/smoke_musl_static.sh
# Session 80 — selfhost install --locked + Nexus mTLS
.PHONY: test-selfhost-install test-nexus-mtls test-selfhost-registry
test-selfhost-install: ensure-buxc selfhost
@echo "=== Selfhost install --locked ==="
@chmod +x tools/smoke_selfhost_install.sh
@tools/smoke_selfhost_install.sh
test-nexus-mtls: ensure-buxc
@echo "=== Nexus mTLS smoke ==="
@chmod +x tools/smoke_nexus_mtls.sh
@tools/smoke_nexus_mtls.sh
# Session 81 — selfhost full registry (search / add / HTTP)
test-selfhost-registry: ensure-buxc selfhost
@echo "=== Selfhost registry ==="
@chmod +x tools/smoke_selfhost_registry.sh
@tools/smoke_selfhost_registry.sh
# Drop / field-move goldens (whole + partial field move; early-return counts)
.PHONY: test-drop-move
test-drop-move: ensure-buxc
@echo "=== Drop / field-move smoke ==="
@chmod +x tools/smoke_drop_move.sh
@tools/smoke_drop_move.sh
# Selfhost (buxc2): move_field ownership + multi-file #line (session 41/42)
# When BUX_SKIP_BUILD=1, reuse prebuilt buxc; still builds buxc2 via selfhost.
.PHONY: test-selfhost-smoke
test-selfhost-smoke: ensure-buxc selfhost
@echo "=== Selfhost smoke (move_field + multi-file #line) ==="
@chmod +x tools/smoke_selfhost.sh
@tools/smoke_selfhost.sh
@echo "=== Graft / quote hygiene smoke ==="
@chmod +x tools/smoke_graft_hygiene.sh
@tools/smoke_graft_hygiene.sh
+60 -28
View File
@@ -432,7 +432,7 @@ func ReadFile(path: String) -> Result<String, IoError> {
| `8.2.1` `own` keyword | ✅ | `own T` parsed and resolves to `T`; ready for borrow checker integration |
| `8.2.2` `borrow` / `&` | ✅ | `&T` shared reference type checked and enforced |
| `8.2.3` `mut` references | ✅ | `&mut T` mutable reference type checked and enforced |
| `8.2.4` Lifetime elision | | Simple rules for common cases; explicit `'a` for complex |
| `8.2.4` Lifetime elision | | Single-input elision + dangling return; explicit `'a` for multi-input |
| `8.2.5` Opt-in checker | ✅ | `@[Checked]` attribute enables borrow checking: writes through `&T` are rejected |
```bux
@@ -520,6 +520,34 @@ const TABLE_SIZE = Factorial(10); // Computed at compile time
| `8.6.2` Procedural macros | `#[derive(Clone)]`, `#[derive(Debug)]` |
| `8.6.3` Reflection | Compile-time type introspection for serialization |
### 8.7 — Optimization & Release Mode
**Goal:** C-speed for hot paths. `@[Release]` disables all runtime checks and inlines auto-drop.
| Task | Status | Details |
|------|--------|---------|
| `8.7.1` `@[Release]` attribute | ⏳ | Disables bounds checking, borrow checking, null checks inside the function |
| `8.7.2` Auto-drop inline | ⏳ | Emit `free`/`close` directly instead of `Type_Drop(&var)` call |
| `8.7.3` Dead-store elimination | ⏳ | C backend removes redundant temp variables (`_t1`, `_t2`) |
| `8.7.4` `-O3` by default | ⏳ | `bux build --release` passes `-O3 -flto` to C compiler |
| `8.7.5` Profile-guided optimization | ⏳ | `bux profile` + `bux build --pgo` for guided inlining |
```bux
// Default: safe but slower — bounds checks + borrow checks
@[Checked]
func SafeSum(arr: Array<int>) -> int { ... }
// Hot path: zero-cost — straight C
@[Release]
func HotLoop(data: *float64, n: int) {
for i in 0..n {
data[i] = data[i] * 2.0; // no bounds check, inlined
}
}
```
**Why this matters:** Nim compiles to C but has ARC/ORC overhead. Crystal has Boehm GC. Bux with `@[Release]` has **literally zero overhead** — the generated C is hand-written quality.
---
## Phase 9 — Ecosystem & Tooling (Week 35+)
@@ -624,36 +652,37 @@ bux/
| Selfhost fix | ✅ | Build via `build/selfhost/` (project wrapper) |
| Push to GitHub | ✅ | `ac969b3` — v0.3.0 restructure |
### 10.1 — Selfhost Loop 🔄 (v0.4.0 target)
### 10.1 — Selfhost Loop (v0.5.0 target)
**Goal:** `buxc2` can compile itself producing a binary-identical `buxc3`.
```
buxc (Nim) → src/*.bux → buxc2 ✅ (already works)
buxc2 → src/*.bux → buxc3 🔄 (needs verification)
buxc2 == buxc3 🔄 (deterministic codegen)
buxc (Nim) → src/*.bux → buxc2 ✅
buxc2 → src/*.bux → buxc3
buxc2 == buxc3 (binary-identical)
```
| Task | Status | Details |
|------|--------|---------|
| `10.1.1` Verify buxc2 can build src/ | 🔄 | Run `buxc2 build` on the selfhost project |
| `10.1.2` Deterministic C codegen | | Remove timestamps, random IDs, non-deterministic ordering |
| `10.1.3` Binary-identical loop | | `make selfhost-loop` — 2-pass bootstrap verification |
| `10.1.4` Remove hardcoded paths | | No `/home/ziko/...` paths in cli.bux |
| `10.1.1` Verify buxc2 can build src/ | | `buxc2 build` works on selfhost project |
| `10.1.2` Deterministic C codegen | | C output identical on every iteration |
| `10.1.3` Binary-identical loop | | `make selfhost-loop` passes (C + ELF parity) |
| `10.1.4` Remove hardcoded paths | | Paths resolved via `bux_getcwd()` / `bux_path_join()` |
| `10.1.5` Selfhost test in CI | ⏳ | Add to `make test` |
### 10.2 — Gradual Ownership 🔄 (v0.5.0 target) ⭐ Killer Feature
### 10.2 — Gradual Ownership (v0.5.0 target) ⭐ Killer Feature
**Goal:** `@[Checked]` functions get full borrow checking. Without this, Bux is "C with modern syntax."
**Goal:** `@[Checked]` functions get full borrow checking. `@[Release]` disables checks for zero-cost hot paths.
| Task | Status | Details |
|------|--------|---------|
| `10.2.1` `@[Checked]` attribute gate | | Enable/disable borrow checker per function |
| `10.2.2` `&T` shared reference check | | No mutation through shared refs |
| `10.2.3` `&mut T` exclusive mutable check | | No aliasing of mutable refs |
| `10.2.4` Bounds checking on slices | | Buffer overflow prevention |
| `10.2.5` Lifetime elision (simple rules) | ⏳ | 80% of cases without annotations |
| `10.2.6` Explicit lifetimes `'a` | | Only for complex cases |
| `10.2.1` `@[Checked]` attribute gate | | Enable/disable borrow checker per function |
| `10.2.2` `&T` shared reference check | | No mutation through shared refs |
| `10.2.3` `&mut T` exclusive mutable check | | No aliasing of mutable refs |
| `10.2.4` Bounds checking on slices | | `Slice_Get` / `Array_Get` with `bux_bounds_check` |
| `10.2.5` `@[Release]` zero-cost mode | ✅ | Disables borrow + bounds checks, passes `-O3 -flto` |
| `10.2.6` Lifetime elision (simple rules) | | Single-input elision; multi-input requires `'a` |
| `10.2.7` Explicit lifetimes `'a` | ✅ | Parsed + checked; multi-input + mismatch |
### 10.3 — Compiler Architecture Upgrade (v0.6.0 target)
@@ -826,10 +855,12 @@ func Main() -> int {
| **M8** | 8-9 | ✅ | **Borrow checker**, **CTFE**, **Package manager** working |
| **M9** | 8.5 | ✅ | **Trait bounds** (`<T: Comparable>`) — semantic checking implemented |
| **M10** | 10 | ✅ | **LIR backend** replaces HIR→C; 26/26 examples pass; selfhost builds |
| **M11** | 11 | 🔄 | Selfhost loop: `buxc2` compiles itself → binary-identical `buxc3` |
| **M12** | 11.3 | | `@[Checked]` borrow checker works on real code |
| **M13** | 11.4 | | `bux test`, `bux fmt`, `bux doc` shipped |
| **M14** | 11.5 | | Native x86-64 backend (no C transpiler) |
| **M11** | 11 | | **Selfhost loop**: `buxc2` compiles itself → binary-identical `buxc3` |
| **M12** | 11.3 | | `@[Checked]` borrow checker works on real code |
| **M13** | 11.4 | | `@[Release]` zero-cost mode; Rust-style error snippets |
| **M14** | 8.3 | | **Green threads** — M:N scheduler with channels |
| **M15** | 11.5 | ⏳ | Native x86-64 backend (no C transpiler) |
| **M16** | 11.4 | ⏳ | `bux test`, `bux fmt`, `bux doc` shipped |
---
@@ -888,10 +919,11 @@ func Main() -> int {
### Next Actions (Priority Order)
1.**Fix LIR backend type inference** — struct temps, undeclared vars, break/continue, duplicate declarations
2.**All 26 examples passing** — bootstrap compiler builds and runs every example
2.**All 30 examples passing** — bootstrap and self-host compilers build and run every example
3.**Selfhost build working**`make selfhost` produces working `buxc2`
4. 🔄 **Selfhost loop verification**`buxc2` compiles `src/``buxc3`
5. 🔄 **Golden tests for C codegen**Prevent future regressions in LIR → C
4. **Selfhost loop verification**`buxc2` compiles `src/``buxc3`; C output and stripped binary identical
5. **Golden tests for C codegen**`make test-golden` passes for 8 critical examples
6.**`bux test` runner** — discovers `tests/*.bux`, builds each as a temp package, and reports PASS/FAIL
---
@@ -910,9 +942,9 @@ func Main() -> int {
| Task | Status | Priority | Details |
|------|--------|----------|---------|
| `11.2.1` `buxc2 build` on `src/` | | P0 | Verify selfhost compiler can build itself |
| `11.2.2` Deterministic C codegen | | P0 | Remove non-deterministic ordering in `emitModule` |
| `11.2.3` `make selfhost-loop` | | P0 | Makefile target: buxc2 → buxc3 → binary compare |
| `11.2.1` `buxc2 build` on `src/` | | P0 | Verify selfhost compiler can build itself |
| `11.2.2` Deterministic C codegen | | P0 | Remove non-deterministic ordering in `emitModule` |
| `11.2.3` `make selfhost-loop` | | P0 | Makefile target: buxc2 → buxc3 → binary compare |
| `11.2.4` Cross-module generics in selfhost | ⏳ | P1 | `Map<K,V>` and `Array<T>` from stdlib in selfhost context |
### 11.3 — Gradual Ownership (v0.5.0) ⭐⭐
@@ -927,7 +959,7 @@ func Main() -> int {
| Task | Status | Priority | Details |
|------|--------|----------|---------|
| `11.4.1` `bux test` runner | | P0 | Built-in test framework with assertions |
| `11.4.1` `bux test` runner | | P0 | Built-in test framework with assertions |
| `11.4.2` `bux fmt` formatter | ⏳ | P1 | Auto-format Bux source |
| `11.4.3` LSP prototype | ⏳ | P2 | Autocomplete, hover types, go-to-definition |
| `11.4.4` `bux doc` generator | ⏳ | P2 | HTML from `///` doc comments |
+159 -32
View File
@@ -2,11 +2,16 @@
![Bux Language](bux-lang-01.jpeg)
> **Status:** Bootstrap compiler (`buxc`, Nim) compiles `.bux` → HIR → **LIR** → C → native binary.
> New **LIR backend** (v0.3.0) replaces direct HIR→C emission — cleaner codegen, all 26 examples passing.
> Self-hosted compiler (`buxc2`, written in Bux) compiles `.bux` → C → native binary and can build real projects.
> **Status:** **v1.0.0** — language freeze. Bootstrap (`buxc`, Nim) and self-hosted (`buxc2`, Bux) both compile `.bux` → C → native binary.
> **Selfhost loop:** deterministic C codegen + ELF verified.
> **Gradual Ownership:** `@[Checked]` borrow checker, `@[Release]` zero-cost mode, `borrow &mut` expressions.
> **Closures:** multi-instance capturing closures via fat function pointers (`BuxFn { code, env }`) in both compilers.
> **Tuples:** `(T, U)` types and `.0`/`.1` field access (bootstrap + selfhost).
> **Green Threads:** M:N scheduler with channels (Go-style goroutines without GC).
> **Examples:** 40+ programs pass (`make test-examples`). Apps: `boko-framework`, `jwt-pitbul`, `nexus`, `simpledb`.
> **Semver:** after 1.0, breaking changes require MAJOR (`docs/SEMVER.md`).
Bux is a fast, compiled, strongly-typed systems programming language. Features a C backend for native code generation, raw multi-line strings, gradual ownership, async/await, generics, algebraic enums, and a package manager.
Bux is a fast, compiled, strongly-typed systems programming language. Features a C backend for native code generation, raw multi-line strings, gradual ownership (opt-in borrow checking), multi-instance closures, async/await, generics, algebraic enums, and a package manager.
---
@@ -22,6 +27,12 @@ cd hello
# Build and run
bux run
# Build optimized release binary
bux build --release
# Cross-compile for ARM Linux (requires clang)
bux build --target aarch64-linux-gnu
```
---
@@ -129,7 +140,7 @@ func Main() -> int {
### Channels (Producer/Consumer)
```bux
import Std::Io::{PrintLine, PrintInt};
import Std::Task::{Task_Join, TaskHandle};
import Std::Task::{Task_Wait, TaskHandle};
import Std::Channel::{Channel, Channel_New, Channel_SendInt, Channel_RecvInt, Channel_Close};
func Producer(chPtr: *Channel<int>) {
@@ -159,8 +170,8 @@ func Main() -> int {
let ch: Channel<int> = Channel_New<int>(3);
let p: *void = spawn Producer(&ch);
let c: *void = spawn Consumer(&ch);
Task_Join(TaskHandle { handle: p });
Task_Join(TaskHandle { handle: c });
Task_Wait(TaskHandle { handle: p });
Task_Wait(TaskHandle { handle: c });
return 0;
}
```
@@ -177,29 +188,66 @@ func Main() -> int {
}
```
### Multi-instance closures
```bux
func MakeAdder(base: int) -> func(int) -> int {
return |a: int| -> int { return a + base; };
}
func Main() -> int {
let a10 = MakeAdder(10);
let a20 = MakeAdder(20);
// Independent capture environments
PrintInt(a10(1)); // 11
PrintInt(a20(1)); // 21
return 0;
}
```
### Iter map / filter / fold
```bux
import Std::Iter::{Array_Iter, Iter_MapInt, Iter_FilterInt, Iter_FoldInt};
func Main() -> int {
var nums: Array<int> = Array_New<int>(4);
Array_Push<int>(&nums, 1);
Array_Push<int>(&nums, 2);
Array_Push<int>(&nums, 3);
let it = Array_Iter<int>(&nums);
let doubled = Iter_MapInt(&it, |x: int| -> int { return x * 2; });
return 0;
}
```
---
## Features
| Feature | Status |
|---------|--------|
| **Types** | Primitives, pointers, slices, tuples, structs, enums, unions |
| **Types** | Primitives, pointers, slices, tuples `(T,U)` + `.0`/`.1`, structs, enums, unions |
| **Generics** | Generic functions (monomorphization) |
| **Algebraic Enums** | Enums with data (`Result`, `Option`) |
| **Pattern Matching** | `match` with guards |
| **Methods** | `extend` blocks for struct methods |
| **Interfaces** | `interface` + `extend` for trait-like behavior |
| **Error Handling** | `Result<T,E>`, `Option<T>`, and the `?` operator |
| **Standard Library** | `Io`, `Array`, `String`, `Map`, `Fs`, `Mem`, `Set`, `Path`, `Math`, `Task`, `Channel`, `Sync`, `Os`, `Time`, `Process` |
| **Error Handling** | `Result`/`Option`, `?`, `Expect`/`UnwrapOr`/`Or` helpers |
| **Closures** | Capture-less + capturing; **multi-instance** fat pointers (`BuxFn`) |
| **Function pointers** | `func(T) -> R` as fat values; named funcs via adapters |
| **Standard Library** | `Io`, `Array`, `String`, `Map`, `Set`, `Iter` (map/filter/fold), `Fs`, `Mem`, `Path`, `Math`, `Task`, `Channel`, `Sync`, `Os`, `Time`, `Process`, `Test`, … |
| **Backend** | LIR → C transpiler (clean 3-address code, then gcc/clang) |
| **Strings** | Raw multi-line backtick strings (`...`), C-string interop |
| **Gradual Ownership** | `@[Checked]` + `&T`/`&mut T` borrow checking |
| **Strings** | Raw multi-line backticks, `f"..."` interp (bootstrap), `ReplaceAll` / `IsBlank` / `Repeat` |
| **Gradual Ownership** | `@[Checked]` + `@[Release]` + `@[Shared]` + `borrow &mut` / `borrow &` |
| **Drop / RAII** | Auto-drop (`@[Drop]` / `extend … for Drop`); **field-move skips Drop** (no double-free) |
| **Green Threads** | M:N scheduler (ucontext + SIGVTALRM), work-stealing queues |
| **Async/Await** | `async func`, `spawn`, `.await` with stackful coroutines |
| **Concurrency** | `Task`/`Channel`/`Sync` (pthread-based), `bux_async_yield`/`spawn` |
| **CTFE** | `const func` — compile-time function execution |
| **Trait Bounds** | `func Max<T: Comparable>(a: T, b: T) -> T` |
| **Package Manager** | `bux add`, `bux install`, `bux.lock`, path + git deps |
| **Tooling** | `bux new`, `bux build`, `bux run`, `bux test`, `bux check` |
| **Cross-Compilation** | `--target <triple>` via clang (e.g. `aarch64-linux-gnu`) |
| **Diagnostics** | Rust-style snippets, multi-char underlines, `= help:` hints |
| **Tooling** | `bux new/build/run/test/check/fmt/doc`, LSP 0.4.0 (locals + inferred lets) |
---
@@ -207,43 +255,108 @@ func Main() -> int {
```
bux/
├── compiler/ # Compiler source code
│ ├── bootstrap/ # Bootstrap compiler (Nim)
│ ├── selfhost/ # Self-hosting compiler source (Bux)
── tests/ # Compiler unit tests (Nim)
├── library/ # Standard library
│ ├── std/ # Standard library modules (.bux)
│ ├── Io.bux, String.bux, Array.bux, Map.bux
│ ├── Fs.bux, Mem.bux, Set.bux
│ ├── Path.bux, Math.bux
│ │ └── Task.bux, Channel.bux
── runtime/ # C runtime files (runtime.c, io.c)
├── tests/ # Language integration tests
├── src/ # 🎯 Self-hosting compiler source (Bux)
│ ├── main.bux # Entry point
│ ├── lexer.bux # Tokenizer
── parser.bux # Pratt parser
│ ├── ast.bux # AST node types
│ ├── sema.bux # Type checker
│ ├── hir_lower.bux # AST → HIR lowering
│ ├── c_backend.bux # HIR → C code generator
└── cli.bux # CLI driver
├── bootstrap/ # 🔧 Bootstrap compiler (Nim)
── main.nim # Entry point
│ ├── cli.nim # CLI commands + build driver
│ └── ... # (mirrors src/ structure)
├── lib/ # 📦 Standard library (23 modules)
│ ├── Io.bux # Print, ReadFile, WriteFile
│ ├── String.bux # Full string API
│ ├── Array.bux # Generic Array<T>
│ ├── Map.bux # Generic Map<K,V> + StringMap
│ ├── Set.bux # Generic Set<T>
│ ├── Task.bux # Green threads (spawn/await)
│ ├── Channel.bux # Producer/consumer channels
│ ├── Drop.bux # Drop trait interface
│ └── ... # Math, Fs, Path, Sync, Result, ...
├── rt/ # ⚙️ C runtime
│ ├── runtime.c # Memory, scheduler, channels
│ └── io.c # File I/O wrappers
├── tests/ # 🧪 Unit tests (Nim)
├── examples/ # Example programs
├── tools/ # Additional tooling
├── apps/ # Real-world applications
├── docs/ # Documentation
├── buxs/ # Windows-compatible project root
├── README.md
├── PLAN.md # Roadmap to self-hosting
├── PLAN.md # Roadmap to v1.0.0
└── Makefile
```
---
## Documentation
| Doc | Description |
|-----|-------------|
| [`docs/LanguageRef.md`](docs/LanguageRef.md) | Language reference |
| [`docs/Stdlib.md`](docs/Stdlib.md) | Standard library API |
| [`docs/BuildAndTest.md`](docs/BuildAndTest.md) | Build, test, and tooling |
| [`docs/QUALITY_PLAN.md`](docs/QUALITY_PLAN.md) | Roadmap toward a “good” v1.0 |
| [`docs/Packages.md`](docs/Packages.md) | Package manager + registry |
| [`docs/SEMVER.md`](docs/SEMVER.md) | Versioning policy |
| [`docs/ROADMAP.md`](docs/ROADMAP.md) | Feature status (constructs) |
| [`PLAN.md`](PLAN.md) | Long-form phase plan |
---
## Build & Test
```bash
# Build bootstrap compiler (Nim → C)
make build
# Run all example programs
make test-examples
# Golden diagnostic tests (Rust-style error format)
make test-errors
# Full unit + example suite (local; CI splits the same coverage across jobs)
make test
# Individual suites (also used by .github/workflows/ci.yml in parallel):
# make test-unit / test-examples / test-errors / test-stdlib /
# test-registry / test-dwarf / test-apps / test-selfhost-smoke
# Full-tree format check (lib/ examples/ src/ tests/ apps/)
make fmt-check
# Reformat those trees
make fmt
# Stdlib behavioral goldens (Array / String / collections)
make test-stdlib
# Generate stdlib API docs from /// comments
make docs
# Build self-hosted compiler (Bux → C → native)
make selfhost
# Run all tests
make test
# Package tests (filter + summary table)
./buxc test --filter first _test_runner
# Run example programs
make test-examples
# Format / CI format check
./buxc fmt path/to/file.bux
./buxc fmt --check path/
# API docs
./buxc doc --out docs/api/stdlib.md lib/
# Package registry
./buxc search greet
make test-registry # add greet → install → build temp app
# Selfhost determinism (optional CI job; not in default `make test`)
make selfhost-loop
# Full fixed-point: buxc2 → buxc3 → buxc4 (gen2 vs gen3 C+ELF identical)
# BUX_SELFHOST_FIXED_POINT=1 make selfhost-loop
# Clean build artifacts
make clean
@@ -253,6 +366,20 @@ make clean
---
## Applications
The `apps/` directory contains real-world Bux applications that serve as integration tests for the compiler:
| App | Description | Lines |
|-----|-------------|-------|
| **boko-framework** | Async web framework (like FastAPI), multi-threaded HTTP server | ~660 |
| **jwt-pitbul** | JWT CLI tool — sign, verify, decode (HS256/384/512, RS256/384/512, ES256/384, EdDSA) | ~326 |
| **nexus** | High-performance HTTP/1.1, HTTP/2 & WebSocket server | ~550 |
The bootstrap compiler successfully parses and type-checks all three applications without hanging or crashing.
---
## Documentation
- [`docs/LanguageRef.md`](docs/LanguageRef.md) — Language reference
-2
View File
@@ -1,2 +0,0 @@
name = "test_array"
version = "0.1.0"
-33
View File
@@ -1,33 +0,0 @@
module Main {
import Std::Array::Array;
import Std::Iter::Iter;
import Std::Test::Test;
import Std::Io::Io;
func Main() -> int {
var arr: Array<int> = Array_New<int>(4);
Array_Push<int>(&arr, 10);
Array_Push<int>(&arr, 20);
Array_Push<int>(&arr, 30);
Test_AssertEqInt(Array_Len<int>(&arr) as int, 3);
Test_AssertEqInt(Array_Get<int>(&arr, 0), 10);
Test_AssertEqInt(Array_Get<int>(&arr, 1), 20);
Test_AssertEqInt(Array_Get<int>(&arr, 2), 30);
var it: Iter<int> = Array_Iter<int>(&arr);
Test_AssertTrue(Iter_HasNext<int>(&it));
Test_AssertEqInt(Iter_Next<int>(&it), 10);
Test_AssertEqInt(Iter_Next<int>(&it), 20);
Test_AssertEqInt(Iter_Next<int>(&it), 30);
Test_AssertFalse(Iter_HasNext<int>(&it));
var it2: Iter<int> = Array_Iter<int>(&arr);
let count: uint = Iter_Count<int>(&it2);
Test_AssertEqInt(count as int, 3);
Print("All array+iter tests passed!");
Array_Free<int>(&arr);
return 0;
}
}
BIN
View File
Binary file not shown.
+54
View File
@@ -0,0 +1,54 @@
/* Bux Standard Library - I/O functions */
#include <stdio.h>
#include <stdint.h>
#include <string.h>
/* PrintLine - print string with newline */
void PrintLine(const char* s) {
if (s != NULL) {
puts(s);
} else {
puts("");
}
}
/* Print - print string without newline */
void Print(const char* s) {
if (s != NULL) {
printf("%s", s);
}
}
/* PrintInt - print integer */
void PrintInt(int n) {
printf("%d", n);
}
/* PrintInt64 - print 64-bit integer */
void PrintInt64(int64_t n) {
printf("%lld", (long long)n);
}
/* PrintFloat - print float */
void PrintFloat(double f) {
printf("%g", f);
}
/* PrintBool - print boolean */
void PrintBool(int b) {
printf("%s", b ? "true" : "false");
}
/* ReadLine - read line from stdin (simplified) */
const char* ReadLine(void) {
static char buffer[1024];
if (fgets(buffer, sizeof(buffer), stdin) != NULL) {
size_t len = strlen(buffer);
if (len > 0 && buffer[len-1] == '\n') {
buffer[len-1] = '\0';
}
return buffer;
}
return "";
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+7
View File
@@ -0,0 +1,7 @@
[Package]
Name = "closure_test"
Version = "0.1.0"
Type = "bin"
[Build]
Output = "Bin"
+34
View File
@@ -0,0 +1,34 @@
module Main {
extern func PrintInt(x: int);
extern func PrintLine(msg: String);
func Apply(x: int, op: func(int) -> int) -> int {
return op(x);
}
func Double(x: int) -> int {
return x * 2;
}
func main() -> int {
// Closure as variable
let add: func(int, int) -> int = |a: int, b: int| -> int { return a + b; };
let sum: int = add(3, 4);
PrintInt(sum);
PrintLine("");
// Closure passed to function
let result: int = Apply(5, |x: int| -> int { return x * 3; });
PrintInt(result);
PrintLine("");
// Address of named function as function pointer
let d: func(int) -> int = &Double;
PrintInt(d(7));
PrintLine("");
return 0;
}
}
+15
View File
@@ -0,0 +1,15 @@
@[Drop]
struct Buffer {
ptr: *int
}
func Buffer_Drop(self: *Buffer) {
bux_free(self.ptr as *void);
}
func main() -> int {
let buf: Buffer = Buffer { ptr: bux_alloc(10 as uint * sizeof(int)) as *int };
// Buffer should be auto-dropped here via Buffer_Drop(&buf)
return 0;
}
BIN
View File
Binary file not shown.
File diff suppressed because it is too large Load Diff
+4
View File
@@ -0,0 +1,4 @@
[package]
name = "error_snippet_test"
version = "0.1.0"
pkgType = "bin"
+4
View File
@@ -0,0 +1,4 @@
func Main() -> int {
let x: int = undefined_variable;
return 0;
}
+2
View File
@@ -0,0 +1,2 @@
name = "test_funcptr"
version = "0.1.0"
+17
View File
@@ -0,0 +1,17 @@
module Main {
func Apply(x: int, op: func(int) -> int) -> int {
return op(x);
}
func Double(n: int) -> int {
return n * 2;
}
func main() -> int {
let result: int = Apply(5, &Double);
if result != 10 { return 1; }
return 0;
}
}
+4
View File
@@ -0,0 +1,4 @@
[package]
name = "green_threads_test"
version = "0.1.0"
pkgType = "bin"
+29
View File
@@ -0,0 +1,29 @@
import Std::Task;
import Std::String;
func Worker(id: int) {
PrintLine(String_Concat(String_Concat("Worker ", String_FromInt(id)), " starting"));
Task_Sleep(50);
PrintLine(String_Concat(String_Concat("Worker ", String_FromInt(id)), " done"));
}
func Main() -> int {
PrintLine("=== Green Thread Test ===");
Task_Init(4);
let h1: TaskHandle = Task_Spawn(Worker as *void, 1 as *void);
let h2: TaskHandle = Task_Spawn(Worker as *void, 2 as *void);
let h3: TaskHandle = Task_Spawn(Worker as *void, 3 as *void);
PrintLine("Waiting for tasks...");
Task_Wait(h1);
Task_Wait(h2);
Task_Wait(h3);
Task_Shutdown();
PrintLine("=== All tasks complete ===");
return 0;
}
+36
View File
@@ -0,0 +1,36 @@
import Std::Io;
import Std::Array;
import Std::Slice;
func Main() -> int {
// Test Array operator_index_get/set
let arr: Array<int> = Array_New<int>(10);
Array_Push<int>(&arr, 10);
Array_Push<int>(&arr, 20);
Array_Push<int>(&arr, 30);
// Use [] syntax (operator_index_get)
if arr[0] != 10 { return 1; }
if arr[1] != 20 { return 2; }
if arr[2] != 30 { return 3; }
// Use [] syntax (operator_index_set)
arr[1] = 99;
if arr[1] != 99 { return 4; }
// Test Slice operator_index_get/set
let s: Slice<int> = Slice_FromArray<int>(&arr);
if s[0] != 10 { return 5; }
if s[1] != 99 { return 6; }
if s[2] != 30 { return 7; }
s[2] = 42;
if s[2] != 42 { return 8; }
PrintInt(s[0]);
PrintLine("");
PrintInt(Slice_Len<int>(&s));
PrintLine("");
return 0;
}
+182
View File
@@ -0,0 +1,182 @@
# Boko Framework
**Async web framework for Bux — inspired by FastAPI.**
Boko is a lightweight, multi-threaded web framework that brings FastAPI-style routing to the Bux programming language. It handles HTTP parsing, path pattern matching, query parameter extraction, and response building so you can focus on your application logic.
This version (0.2.0) is rewritten with modern Bux: `struct` methods, generic `StringMap<String>` for headers/query/path params, `for` loops, algebraic enums, `StringBuilder`, and string interpolation.
## Quick Start
```bash
cd apps/boko-framework
../../buxc build
./build/boko-framework
```
Open `http://localhost:8080` — you'll see the demo landing page.
## How It Works
Boko follows a **single-dispatch-function** pattern. You define one function — `Boko_Router` — and the framework calls it for every incoming request:
```bux
import Boko::{Request, Response, Response_Json, Response_Html, Response_NotFound,
Path_Match,
App, App_New, App_Run};
func Boko_Router(req: Request) -> Response {
// Route: GET /
if String_Eq(req.path, "/") {
return Response_Html("<h1>Hello Boko!</h1>");
}
// Route: GET /api/health
if String_Eq(req.path, "/api/health") {
return Response_Json("{\"status\":\"ok\"}");
}
// Route: GET /users/{id} — path parameter
if Path_Match("/users/{id}", req.path, &req) {
let id: String = req.GetPathParam("id");
// Build JSON response with id
return Response_Json(...);
}
// Route: GET /search?q=... — query parameter
if String_Eq(req.path, "/search") {
let q: String = req.GetQuery("q");
return Response_Json(...);
}
return Response_NotFound();
}
func Main() -> int {
let app: App = App_New(8080, 4); // port, threads
App_Run(&app);
return 0;
}
```
## API Reference
### Request
| Field | Type | Description |
|-------|------|-------------|
| `method` | `HttpVerb` | GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS |
| `path` | `String` | Request path (without query string) |
| `body` | `String` | Request body (for POST/PUT) |
| `headers` | `StringMap<String>` | Parsed request headers |
| `query` | `StringMap<String>` | Parsed query parameters |
| `pathParams` | `StringMap<String>` | Extracted path parameters |
| Method | Returns | Description |
|--------|---------|-------------|
| `req.GetHeader(name)` | `String` | Get header value by name |
| `req.GetQuery(name)` | `String` | Get query parameter by name |
| `req.HasQuery(name)` | `bool` | Check if query parameter exists |
| `req.GetPathParam(name)` | `String` | Get extracted path parameter |
### Response
| Constructor | Content-Type | Status |
|-------------|-------------|--------|
| `Response_Html(html)` | `text/html` | 200 |
| `Response_Json(json)` | `application/json` | 200 |
| `Response_Text(text)` | `text/plain` | 200 |
| `Response_Redirect(url)` | — | 302 |
| `Response_NotFound()` | `application/json` | 404 |
| `Response_Error(code, msg)` | `application/json` | custom |
| `Response_NoContent()` | — | 204 |
### Path Matching
`Path_Match(pattern, actualPath, req)` matches a pattern with `{param}` placeholders:
```bux
// Pattern: /users/{id}/posts/{postId}
// Actual: /users/42/posts/7
// Extracts: id=42, postId=7
if Path_Match("/users/{id}/posts/{postId}", req.path, &req) {
let userId: String = req.GetPathParam("id"); // "42"
let postId: String = req.GetPathParam("postId"); // "7"
}
```
The function returns `true` if the pattern matches and populates `req.pathParams`.
### App
```bux
let app: App = App_New(8080, 4); // port 8080, 4 worker threads
App_Run(&app); // blocks, handles requests
```
## Architecture
```
Incoming connection
Net_Accept() ─── worker thread (1 of N)
Net_Recv() → raw HTTP bytes
Request_Parse() → method, path, query, headers, body
Boko_Router(req) ←── YOUR CODE
Response_Build() → HTTP/1.1 response string
Net_Send() → bytes to client
Net_Close()
```
## Endpoints in Demo App
| Method | Path | Description |
|--------|------|-------------|
| GET | `/` | Landing page (HTML) |
| GET | `/api/health` | Health check (JSON) |
| GET | `/api/info` | Framework info (JSON) |
| GET | `/hello?name=X` | Query param demo |
| GET | `/users/{id}` | Path param demo |
| GET | `/posts/{id}/comments/{cid}` | Multi path param demo |
| GET | `/redirect` | 302 redirect to `/` |
| POST | `/api/echo` | Echo request body |
## Project Structure
```
apps/boko-framework/
├── bux.toml # Package manifest
├── README.md # This file
└── src/
├── Boko.bux # Framework core (~500 lines)
└── Main.bux # Example app (~150 lines)
```
## Design Philosophy
Boko is intentionally simple. Instead of complex DSLs or code generation, it gives you:
- **One function to write** — `Boko_Router(req) -> Response`
- **Direct control** — you write plain if/else or match for routing
- **No magic** — path params, query params, headers are explicit function calls
- **Fast** — multi-threaded accept loop, zero allocations where possible
This is the same philosophy as Go's `net/http` — simple, explicit, composable.
## License
Part of the Bux project. See root [LICENSE](../../LICENSE).
+1
View File
@@ -0,0 +1 @@
EXIT:137
+8
View File
@@ -0,0 +1,8 @@
[Package]
Name = "boko-framework"
Version = "0.2.0"
Type = "bin"
Description = "Boko — async web framework for Bux, inspired by FastAPI"
[Build]
Output = "Bin"
+1
View File
@@ -0,0 +1 @@
EXIT:137
+1
View File
@@ -0,0 +1 @@
EXIT:143
+1
View File
@@ -0,0 +1 @@
EXIT:137
+2
View File
@@ -0,0 +1,2 @@
bash: ред 1: ../../buxc_lir: Няма такъв файл или директория
EXIT:127
+1
View File
@@ -0,0 +1 @@
EXIT:143
+1
View File
@@ -0,0 +1 @@
EXIT:143
+2
View File
@@ -0,0 +1,2 @@
bux 0.1.0 (bootstrap)
EXIT:0
+40
View File
@@ -0,0 +1,40 @@
nim c -o:buxc -d:release --opt:size bootstrap/main.nim
Hint: used config file '/etc/nim/nim.cfg' [Conf]
Hint: used config file '/etc/nim/config.nims' [Conf]
...........................................................................................................................................
/home/ziko/z-git/bux/bux/bootstrap/lexer.nim(340, 12) Hint: 'check3' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lexer.nim(350, 12) Hint: 'checkEq' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lexer.nim(64, 6) Hint: 'match' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lexer.nim(84, 6) Hint: 'emitWarning' is declared but not used [XDeclaredButNotUsed]
..
/home/ziko/z-git/bux/bux/bootstrap/parser.nim(663, 7) Hint: 'loc' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/parser.nim(1001, 9) Hint: 'isVar' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/parser.nim(59, 6) Hint: 'match' is declared but not used [XDeclaredButNotUsed]
.......
/home/ziko/z-git/bux/bux/bootstrap/sema.nim(1150, 9) Hint: 'operandType' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/sema.nim(1155, 9) Hint: 'operandType' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/sema.nim(1165, 9) Hint: 'subjectType' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/sema.nim(1200, 9) Hint: 'operand' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/sema.nim(94, 6) Hint: 'emitWarning' is declared but not used [XDeclaredButNotUsed]
.
/home/ziko/z-git/bux/bux/bootstrap/manifest.nim(93, 11) Hint: 'val' is declared but not used [XDeclaredButNotUsed]
..
/home/ziko/z-git/bux/bux/bootstrap/hir_lower.nim(1057, 9) Hint: 'loweredIter' is declared but not used [XDeclaredButNotUsed]
.
/home/ziko/z-git/bux/bux/bootstrap/lir.nim(5, 8) Warning: imported and not used: 'types' [UnusedImport]
.
/home/ziko/z-git/bux/bux/bootstrap/lir_lower.nim(419, 9) Hint: 't' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lir_lower.nim(595, 9) Hint: 'bodyLbl' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lir_lower.nim(734, 11) Hint: 'exprVal' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lir_lower.nim(746, 9) Hint: 'exprVal' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lir_lower.nim(6, 8) Warning: imported and not used: 'ast' [UnusedImport]
.
/home/ziko/z-git/bux/bux/bootstrap/lir_c_backend.nim(20, 6) Hint: 'emit' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lir_c_backend.nim(46, 6) Hint: 'typeFromValue' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/lir_c_backend.nim(58, 6) Hint: 'setTempType' is declared but not used [XDeclaredButNotUsed]
/home/ziko/z-git/bux/bux/bootstrap/cli.nim(2, 55) Warning: imported and not used: 'lir' [UnusedImport]
Hint: [Link]
Hint: mm: orc; threads: on; opt: size; options: -d:release
76982 lines; 1.552s; 174.824MiB peakmem; proj: /home/ziko/z-git/bux/bux/bootstrap/main.nim; out: /home/ziko/z-git/bux/bux/buxc [SuccessX]
# strip buxc
EXIT:0
+19
View File
@@ -0,0 +1,19 @@
#!/bin/bash
cd /home/ziko/z-git/bux/bux/apps/boko-framework
echo "=== CHECK ==="
../../buxc check 2>&1
echo "CHECK_EXIT=$?"
echo ""
echo "=== BUILD ==="
../../buxc build 2>&1
echo "BUILD_EXIT=$?"
echo ""
echo "=== FILES ==="
find . -type f -newer bux.toml 2>/dev/null | head -30
echo ""
echo "=== BINARY SEARCH ==="
file boko-framework 2>/dev/null || echo "no boko-framework"
ls -la build/ 2>/dev/null || echo "no build dir"
+467
View File
@@ -0,0 +1,467 @@
// =============================================================================
// Boko — Async Web Framework for Bux (inspired by FastAPI)
// Rewritten with modern Bux: methods on structs, generic StringMap, for-in
// loops, algebraic enums, StringBuilder, and string interpolation.
// =============================================================================
module Boko {
import Std::Io::{PrintLine, Print, PrintInt};
import Std::Net::{Net_Create, Net_SetReuse, Net_Bind, Net_Listen, Net_Accept, Net_Send, Net_Recv, Net_Close, Net_LastError};
import Std::String::{
String_Len, String_Eq, String_IsNull,
String_StartsWith, String_Contains,
String_Find, String_Offset, String_Slice,
String_SplitCount, String_SplitPart,
StringBuilder, StringBuilder_New, StringBuilder_Append,
StringBuilder_AppendInt, StringBuilder_Build, StringBuilder_Free
};
import Std::Map::{StringMap, StringMap_New, StringMap_Set, StringMap_Get, StringMap_Has};
// =============================================================================
// HTTP Methods
// =============================================================================
enum HttpVerb {
GET,
POST,
PUT,
DELETE,
PATCH,
HEAD,
OPTIONS,
}
func HttpVerb_MethodName(verb: HttpVerb) -> String {
if verb.tag == HttpVerb_GET { return "GET"; }
if verb.tag == HttpVerb_POST { return "POST"; }
if verb.tag == HttpVerb_PUT { return "PUT"; }
if verb.tag == HttpVerb_DELETE { return "DELETE"; }
if verb.tag == HttpVerb_PATCH { return "PATCH"; }
if verb.tag == HttpVerb_HEAD { return "HEAD"; }
if verb.tag == HttpVerb_OPTIONS { return "OPTIONS"; }
return "?";
}
func HttpVerb_Parse(methodStr: String) -> HttpVerb {
if String_Eq(methodStr, "GET") { return HttpVerb { tag: HttpVerb_GET }; }
if String_Eq(methodStr, "POST") { return HttpVerb { tag: HttpVerb_POST }; }
if String_Eq(methodStr, "PUT") { return HttpVerb { tag: HttpVerb_PUT }; }
if String_Eq(methodStr, "DELETE") { return HttpVerb { tag: HttpVerb_DELETE }; }
if String_Eq(methodStr, "PATCH") { return HttpVerb { tag: HttpVerb_PATCH }; }
if String_Eq(methodStr, "HEAD") { return HttpVerb { tag: HttpVerb_HEAD }; }
if String_Eq(methodStr, "OPTIONS") { return HttpVerb { tag: HttpVerb_OPTIONS }; }
return HttpVerb { tag: HttpVerb_GET };
}
// =============================================================================
// Request — parsed incoming HTTP request
// =============================================================================
struct Request {
method: HttpVerb,
path: String,
body: String,
headers: StringMap<String>,
query: StringMap<String>,
pathParams: StringMap<String>,
}
extend Request {
func GetHeader(self: Request, name: String) -> String {
if StringMap_Has<String>(&self.headers, name) {
return StringMap_Get<String>(&self.headers, name);
}
return "";
}
func GetQuery(self: Request, name: String) -> String {
if StringMap_Has<String>(&self.query, name) {
return StringMap_Get<String>(&self.query, name);
}
return "";
}
func HasQuery(self: Request, name: String) -> bool {
return StringMap_Has<String>(&self.query, name);
}
func GetPathParam(self: Request, name: String) -> String {
if StringMap_Has<String>(&self.pathParams, name) {
return StringMap_Get<String>(&self.pathParams, name);
}
return "";
}
}
// =============================================================================
// Response — outgoing HTTP response
// =============================================================================
struct Response {
statusCode: int,
contentType: String,
body: String,
extraHeaders: String,
}
// --- Constructors ---
func Response_New(status: int, contentType: String, body: String) -> Response {
return Response { statusCode: status, contentType: contentType, body: body, extraHeaders: "" };
}
func Response_Ok(body: String) -> Response {
return Response_New(200, "text/html; charset=utf-8", body);
}
func Response_Html(html: String) -> Response {
return Response_New(200, "text/html; charset=utf-8", html);
}
func Response_Json(json: String) -> Response {
return Response_New(200, "application/json; charset=utf-8", json);
}
func Response_Text(text: String) -> Response {
return Response_New(200, "text/plain; charset=utf-8", text);
}
func Response_Redirect(url: String) -> Response {
let sb: StringBuilder = StringBuilder_New();
StringBuilder_Append(&sb, "Location: ");
StringBuilder_Append(&sb, url);
StringBuilder_Append(&sb, "\r\n");
let headers: String = StringBuilder_Build(&sb);
StringBuilder_Free(&sb);
return Response { statusCode: 302, contentType: "", body: "", extraHeaders: headers };
}
func Response_NotFound() -> Response {
return Response_New(404, "application/json; charset=utf-8", "{\"error\":\"not_found\"}");
}
func Response_Error(status: int, message: String) -> Response {
let sb: StringBuilder = StringBuilder_New();
StringBuilder_Append(&sb, "{\"error\":\"");
StringBuilder_Append(&sb, message);
StringBuilder_Append(&sb, "\"}");
let body: String = StringBuilder_Build(&sb);
StringBuilder_Free(&sb);
return Response_New(status, "application/json; charset=utf-8", body);
}
func Response_NoContent() -> Response {
return Response_New(204, "", "");
}
// --- Instance helpers ---
extend Response {
func WithHeader(self: Response, key: String, value: String) -> Response {
let sb: StringBuilder = StringBuilder_New();
StringBuilder_Append(&sb, self.extraHeaders);
StringBuilder_Append(&sb, key);
StringBuilder_Append(&sb, ": ");
StringBuilder_Append(&sb, value);
StringBuilder_Append(&sb, "\r\n");
self.extraHeaders = StringBuilder_Build(&sb);
StringBuilder_Free(&sb);
return self;
}
}
// =============================================================================
// Status phrase lookup
// =============================================================================
func StatusPhrase(code: int) -> String {
if code == 200 { return "OK"; }
if code == 201 { return "Created"; }
if code == 204 { return "No Content"; }
if code == 301 { return "Moved Permanently"; }
if code == 302 { return "Found"; }
if code == 400 { return "Bad Request"; }
if code == 401 { return "Unauthorized"; }
if code == 403 { return "Forbidden"; }
if code == 404 { return "Not Found"; }
if code == 500 { return "Internal Server Error"; }
if code == 502 { return "Bad Gateway"; }
if code == 503 { return "Service Unavailable"; }
return "OK";
}
// =============================================================================
// Build HTTP response string from Response struct
// =============================================================================
func Response_Build(resp: Response) -> String {
let sb: StringBuilder = StringBuilder_New();
// Status line
StringBuilder_Append(&sb, "HTTP/1.1 ");
StringBuilder_AppendInt(&sb, resp.statusCode as int64);
StringBuilder_Append(&sb, " ");
StringBuilder_Append(&sb, StatusPhrase(resp.statusCode));
StringBuilder_Append(&sb, "\r\n");
// Server
StringBuilder_Append(&sb, "Server: Boko/0.2.0 (Bux)\r\n");
// Extra headers
if String_Len(resp.extraHeaders) > 0 {
StringBuilder_Append(&sb, resp.extraHeaders);
}
// Content-Type
if String_Len(resp.contentType) > 0 {
StringBuilder_Append(&sb, "Content-Type: ");
StringBuilder_Append(&sb, resp.contentType);
StringBuilder_Append(&sb, "\r\n");
}
// Content-Length
let bodyLen: uint = String_Len(resp.body);
StringBuilder_Append(&sb, "Content-Length: ");
StringBuilder_AppendInt(&sb, bodyLen as int64);
StringBuilder_Append(&sb, "\r\n");
// Connection close
StringBuilder_Append(&sb, "Connection: close\r\n");
StringBuilder_Append(&sb, "\r\n");
if bodyLen > 0 {
StringBuilder_Append(&sb, resp.body);
}
let result: String = StringBuilder_Build(&sb);
StringBuilder_Free(&sb);
return result;
}
// =============================================================================
// Parse query string: ?key=val&key2=val2 → generic StringMap
// =============================================================================
func Query_Parse(queryString: String) -> StringMap<String> {
let map: StringMap<String> = StringMap_New<String>(8);
if String_Len(queryString) == 0 {
return map;
}
let pairCount: uint = String_SplitCount(queryString, "&");
for i in 0..pairCount {
let pair: String = String_SplitPart(queryString, "&", i);
let eqPos: String = String_Find(pair, "=");
var key: String = pair;
var value: String = "";
if !String_IsNull(eqPos) {
let keyLen: uint = String_Offset(eqPos, pair);
key = String_Slice(pair, 0, keyLen);
let valStart: uint = keyLen + 1;
let pairLen: uint = String_Len(pair);
if valStart < pairLen {
value = String_Slice(pair, valStart, pairLen - valStart);
}
}
StringMap_Set<String>(&map, key, value);
}
return map;
}
// =============================================================================
// Parse incoming HTTP request from raw bytes
// =============================================================================
func Request_Parse(raw: String) -> Request {
var req: Request;
req.method = HttpVerb { tag: HttpVerb_GET };
req.path = "/";
req.body = "";
req.headers = StringMap_New<String>(8);
req.query = StringMap_New<String>(4);
req.pathParams = StringMap_New<String>(4);
if String_Len(raw) == 0 { return req; }
// Find header/body boundary
let boundary: String = String_Find(raw, "\r\n\r\n");
let rawLen: uint = String_Len(raw);
var headerLen: uint = rawLen;
if !String_IsNull(boundary) {
headerLen = String_Offset(boundary, raw);
let bodyStart: uint = headerLen + 4;
if bodyStart < rawLen {
req.body = String_Slice(raw, bodyStart, rawLen - bodyStart);
}
}
if headerLen == 0 { return req; }
let headerBlock: String = String_Slice(raw, 0, headerLen);
let lineCount: uint = String_SplitCount(headerBlock, "\r\n");
if lineCount == 0 { return req; }
// Parse request line
let requestLine: String = String_SplitPart(headerBlock, "\r\n", 0);
if String_Len(requestLine) > 0 {
let methodStr: String = String_SplitPart(requestLine, " ", 0);
req.method = HttpVerb_Parse(methodStr);
let fullPath: String = String_SplitPart(requestLine, " ", 1);
let qmark: String = String_Find(fullPath, "?");
if !String_IsNull(qmark) {
let pathLen: uint = String_Offset(qmark, fullPath);
req.path = String_Slice(fullPath, 0, pathLen);
let qsStart: uint = pathLen + 1;
let fullLen: uint = String_Len(fullPath);
if qsStart < fullLen {
let qs: String = String_Slice(fullPath, qsStart, fullLen - qsStart);
req.query = Query_Parse(qs);
}
} else {
req.path = fullPath;
}
}
// Parse headers
for i in 1..lineCount {
let line: String = String_SplitPart(headerBlock, "\r\n", i);
let colonPos: String = String_Find(line, ": ");
if !String_IsNull(colonPos) {
let keyLen: uint = String_Offset(colonPos, line);
let key: String = String_Slice(line, 0, keyLen);
let valStart: uint = keyLen + 2;
let lineLen: uint = String_Len(line);
var value: String = "";
if valStart < lineLen {
value = String_Slice(line, valStart, lineLen - valStart);
}
StringMap_Set<String>(&req.headers, key, value);
}
}
return req;
}
// =============================================================================
// Path pattern matching: /users/{id} against /users/42 → extracts id=42
// =============================================================================
func Path_Match(pattern: String, path: String, req: *Request) -> bool {
let patParts: uint = String_SplitCount(pattern, "/");
let pathParts: uint = String_SplitCount(path, "/");
if patParts != pathParts { return false; }
req.pathParams = StringMap_New<String>(8);
for i in 0..patParts {
let patPart: String = String_SplitPart(pattern, "/", i);
let pathPart: String = String_SplitPart(path, "/", i);
if String_StartsWith(patPart, "{") && String_Contains(patPart, "}") {
let nameLen: uint = String_Len(patPart) - 2;
let name: String = String_Slice(patPart, 1, nameLen);
StringMap_Set<String>(&req.pathParams, name, pathPart);
} else {
if !String_Eq(patPart, pathPart) { return false; }
}
}
return true;
}
// =============================================================================
// App — server instance
// =============================================================================
struct App {
port: int,
threadCount: int,
serverName: String,
}
func App_New(port: int, threadCount: int) -> App {
return App { port: port, threadCount: threadCount, serverName: "Boko/0.2.0 (Bux)" };
}
// =============================================================================
// Forward declaration — user must implement this in their module
// =============================================================================
func Boko_Router(req: Request) -> Response;
// =============================================================================
// Handle a single connection: parse → dispatch → respond
// =============================================================================
func App_HandleConnection(clientFd: int) {
let raw: String = Net_Recv(clientFd, 8192);
if String_Len(raw) == 0 { return; }
let req: Request = Request_Parse(raw);
let resp: Response = Boko_Router(req);
// Log
Print(HttpVerb_MethodName(req.method));
Print(" ");
Print(req.path);
Print(" → ");
PrintInt(resp.statusCode);
PrintLine("");
// Send
let respStr: String = Response_Build(resp);
Net_Send(clientFd, respStr);
}
// =============================================================================
// Worker thread: accept loop
// =============================================================================
func App_Worker(serverFd: int) {
while true {
let clientFd: int = Net_Accept(serverFd);
if clientFd < 0 { continue; }
App_HandleConnection(clientFd);
Net_Close(clientFd);
}
}
// =============================================================================
// App_Run: start the server (blocking)
// =============================================================================
func App_Run(app: *App) {
PrintLine("╔══════════════════════════════════════╗");
PrintLine("║ Boko Framework v0.2.0 ║");
PrintLine("║ Async web framework for Bux ║");
PrintLine("╚══════════════════════════════════════════════════════╝");
PrintLine("");
let fd: int = Net_Create();
if fd < 0 {
PrintLine("FATAL: socket() failed");
return;
}
Net_SetReuse(fd);
if !Net_Bind(fd, "0.0.0.0", app.port) {
Print("FATAL: bind(:");
PrintInt(app.port);
Print(") failed: ");
PrintLine(Net_LastError());
Net_Close(fd);
return;
}
if !Net_Listen(fd, 128) {
PrintLine("FATAL: listen() failed");
Net_Close(fd);
return;
}
Print("✓ Boko running on http://0.0.0.0:");
PrintInt(app.port);
PrintLine("");
Print("✓ Workers: ");
PrintInt(app.threadCount);
PrintLine("");
PrintLine("");
// Spawn workers
for i in 0..(app.threadCount - 1) {
spawn App_Worker(fd);
}
// Main thread is the last worker
App_Worker(fd);
}
} // module Boko
+152
View File
@@ -0,0 +1,152 @@
// =============================================================================
// Boko Example App — demonstrates the modern framework API
// =============================================================================
module Main {
import Std::Io::{PrintLine, Print};
import Std::String::{
String_Eq, String_Len,
StringBuilder, StringBuilder_New, StringBuilder_Append,
StringBuilder_Build, StringBuilder_Free
};
import Boko::{
App, App_New, App_Run,
Request, Response,
Response_Html, Response_Json, Response_NotFound, Response_Redirect,
Path_Match,
HttpVerb
};
// =============================================================================
// Boko_Router — user-defined dispatch (called by the framework)
// =============================================================================
func Boko_Router(req: Request) -> Response {
// --- GET / ---
if String_Eq(req.path, "/") && req.method.tag == HttpVerb_GET {
return Response_Html(PageHome());
}
// --- GET /api/health ---
if String_Eq(req.path, "/api/health") {
return Response_Json("{\"status\":\"ok\",\"framework\":\"Boko\",\"version\":\"0.2.0\"}");
}
// --- GET /api/info ---
if String_Eq(req.path, "/api/info") {
return Response_Json("{\"name\":\"Boko\",\"language\":\"Bux\",\"inspiration\":\"FastAPI\",\"features\":[\"routing\",\"path-params\",\"query-params\",\"json\"]}");
}
// --- GET /hello?name=World ---
if String_Eq(req.path, "/hello") {
var name: String = req.GetQuery("name");
if String_Len(name) == 0 { name = "World"; }
let html: String = f"<h1>Hello, {name}!</h1>";
return Response_Html(html);
}
// --- GET /users/{id} ---
if Path_Match("/users/{id}", req.path, &req) {
let id: String = req.GetPathParam("id");
let sb: StringBuilder = StringBuilder_New();
StringBuilder_Append(&sb, "{\"id\":");
StringBuilder_Append(&sb, id);
StringBuilder_Append(&sb, ",\"name\":\"User ");
StringBuilder_Append(&sb, id);
StringBuilder_Append(&sb, "\"}");
let json: String = StringBuilder_Build(&sb);
StringBuilder_Free(&sb);
return Response_Json(json);
}
// --- GET /posts/{postId}/comments/{commentId} ---
if Path_Match("/posts/{postId}/comments/{commentId}", req.path, &req) {
let pid: String = req.GetPathParam("postId");
let cid: String = req.GetPathParam("commentId");
let sb: StringBuilder = StringBuilder_New();
StringBuilder_Append(&sb, "{\"postId\":");
StringBuilder_Append(&sb, pid);
StringBuilder_Append(&sb, ",\"commentId\":");
StringBuilder_Append(&sb, cid);
StringBuilder_Append(&sb, ",\"text\":\"Great post!\"}");
let json: String = StringBuilder_Build(&sb);
StringBuilder_Free(&sb);
return Response_Json(json);
}
// --- GET /redirect → 302 to / ---
if String_Eq(req.path, "/redirect") {
return Response_Redirect("/");
}
// --- POST /api/echo ---
if String_Eq(req.path, "/api/echo") && req.method.tag == HttpVerb_POST {
let sb: StringBuilder = StringBuilder_New();
StringBuilder_Append(&sb, "{\"echo\":");
StringBuilder_Append(&sb, req.body);
StringBuilder_Append(&sb, "}");
let json: String = StringBuilder_Build(&sb);
StringBuilder_Free(&sb);
return Response_Json(json);
}
// --- 404 ---
return Response_NotFound();
}
// =============================================================================
// HTML landing page (raw multi-line string via backticks)
// =============================================================================
func PageHome() -> String {
return `<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Boko Framework</title>
<style>
*{margin:0;padding:0;box-sizing:border-box}
body{font-family:system-ui,sans-serif;background:#0d1117;color:#c9d1d9;min-height:100vh;display:flex;align-items:center;justify-content:center}
.card{background:#161b22;border:1px solid #30363d;border-radius:12px;padding:3rem;max-width:560px;width:100%}
h1{color:#58a6ff;font-size:2rem;margin-bottom:.5rem}
.tag{color:#8b949e;font-size:.9rem;margin-bottom:2rem}
h3{color:#d2a8ff;margin:1.5rem 0 .75rem}
.ep{display:flex;gap:1rem;padding:.35rem 0;font-family:monospace;font-size:.9rem}
.ep .m{color:#3fb950;font-weight:bold;min-width:52px}
.ep .p{color:#c9d1d9}
.ep .d{color:#484f58;margin-left:auto}
a{color:#58a6ff}
</style>
</head>
<body>
<div class="card">
<h1>⚡ Boko</h1>
<p class="tag">Async web framework for Bux — inspired by FastAPI</p>
<h3>Try it</h3>
<div class="ep"><span class="m">GET</span><span class="p"><a href="/hello?name=Bux">/hello?name=Bux</a></span><span class="d">query param</span></div>
<div class="ep"><span class="m">GET</span><span class="p"><a href="/users/42">/users/42</a></span><span class="d">path param</span></div>
<div class="ep"><span class="m">GET</span><span class="p"><a href="/posts/7/comments/3">/posts/7/comments/3</a></span><span class="d">multi params</span></div>
<div class="ep"><span class="m">GET</span><span class="p"><a href="/redirect">/redirect</a></span><span class="d">302 → /</span></div>
<div class="ep"><span class="m">GET</span><span class="p"><a href="/api/health">/api/health</a></span><span class="d">JSON</span></div>
<div class="ep"><span class="m">GET</span><span class="p"><a href="/api/info">/api/info</a></span><span class="d">JSON</span></div>
<h3>Features</h3>
<div class="ep"><span class="m">✓</span><span class="p">Path routing with {params}</span></div>
<div class="ep"><span class="m">✓</span><span class="p">Query parameter extraction</span></div>
<div class="ep"><span class="m">✓</span><span class="p">JSON / HTML / Text responses</span></div>
<div class="ep"><span class="m">✓</span><span class="p">Multi-threaded (configurable)</span></div>
<div class="ep"><span class="m">✓</span><span class="p">Redirects (302)</span></div>
<div class="ep"><span class="m">✓</span><span class="p">POST body access</span></div>
</div>
</body>
</html>`;
}
// =============================================================================
// Main — start the server
// =============================================================================
func Main() -> int {
let app: App = App_New(8080, 4);
App_Run(&app);
return 0;
}
} // module Main
+177
View File
@@ -0,0 +1,177 @@
# jwt-pitbul
**JWT CLI tool for the Bux ecosystem — sign, verify, decode, and key generation.**
A standalone command-line utility for working with JSON Web Tokens (RFC 7519). Built on `Std::Crypto`, backed by OpenSSL.
This version (0.2.0) is rewritten with modern Bux: algebraic enums, generic `Array<String>`, methods on `JwtAlg`, string interpolation, `for` loops, and structured `Result`/`Option`-style error handling.
## Quick Start
```bash
cd apps/jwt-pitbul
../../buxc build
./build/jwt-pitbul
```
## Commands
### `sign` — Create a JWT
```
jwt-pitbul sign <alg> <key> <claims_json>
```
| Argument | Description |
|----------|-------------|
| `<alg>` | Algorithm: `HS256`, `HS384`, `HS512`, `RS256`, `RS384`, `RS512`, `ES256`, `ES384`, `EdDSA` |
| `<key>` | HMAC secret string, or path to PEM file (RSA/ECDSA), or base64 raw key (Ed25519) |
| `<claims_json>` | JSON payload string (e.g. `{"sub":"123","exp":1735689600}`) |
```bash
# HMAC
jwt-pitbul sign HS256 'my-secret-key' '{"sub":"123","role":"admin"}'
# RSA (private key from file)
jwt-pitbul sign RS256 private.pem '{"sub":"456"}'
# ECDSA
jwt-pitbul sign ES256 ec_private.pem '{"sub":"789"}'
# Ed25519 (base64-encoded 32-byte private key)
jwt-pitbul sign EdDSA 'MC4CAQAwBQYDK2VwBCIEIJ...' '{"sub":"000"}'
```
### `verify` — Verify and decode
```
jwt-pitbul verify <token> <alg> <key>
```
```bash
jwt-pitbul verify eyJhbGciOiJ... HS256 'my-secret-key'
# ✓ Signature valid
#
# Header:
# {"alg":"HS256","typ":"JWT"}
#
# Payload:
# {"sub":"123","role":"admin"}
```
### `decode` — Decode without verification
```
jwt-pitbul decode <token>
```
```bash
jwt-pitbul decode eyJhbGciOiJ...
# Decoded (no verification):
#
# Header:
# {"alg":"HS256","typ":"JWT"}
#
# Payload:
# {"sub":"123"}
#
# Signature (base64url): xxxxxxxxx...
```
### `keygen` — Generate keys
```
jwt-pitbul keygen <type>
```
| Type | Output |
|------|--------|
| `ed25519` | Generates an Ed25519 keypair (prints base64 public/private keys) |
| `rsa` | Prints OpenSSL CLI commands to generate RSA 2048-bit keys |
| `ecdsa` | Prints OpenSSL CLI commands to generate ECDSA P-256 keys |
```bash
jwt-pitbul keygen ed25519
# Ed25519 keypair (base64):
# Public: MCowBQYDK2VwAyEA...
# Private: MC4CAQAwBQYDK2VwBCIEIJ...
jwt-pitbul keygen rsa
# RSA key generation requires OpenSSL CLI:
# openssl genpkey -algorithm RSA -out private.pem -pkeyopt rsa_keygen_bits:2048
# openssl rsa -in private.pem -pubout -out public.pem
```
## Supported Algorithms
| Algorithm | Type | Key Format |
|-----------|------|------------|
| HS256 | HMAC-SHA256 | Raw secret string |
| HS384 | HMAC-SHA384 | Raw secret string |
| HS512 | HMAC-SHA512 | Raw secret string |
| RS256 | RSA PKCS#1 v1.5 SHA-256 | PEM file path |
| RS384 | RSA PKCS#1 v1.5 SHA-384 | PEM file path |
| RS512 | RSA PKCS#1 v1.5 SHA-512 | PEM file path |
| ES256 | ECDSA P-256 | PEM file path |
| ES384 | ECDSA P-384 | PEM file path |
| EdDSA | Ed25519 | Base64 32-byte raw key |
## Generating Keys with OpenSSL
### RSA 2048-bit
```bash
openssl genpkey -algorithm RSA -out private.pem -pkeyopt rsa_keygen_bits:2048
openssl rsa -in private.pem -pubout -out public.pem
```
### ECDSA P-256
```bash
openssl ecparam -genkey -name prime256v1 -noout -out ec_private.pem
openssl ec -in ec_private.pem -pubout -out ec_public.pem
```
### ECDSA P-384
```bash
openssl ecparam -genkey -name secp384r1 -noout -out ec_private.pem
openssl ec -in ec_private.pem -pubout -out ec_public.pem
```
### Ed25519
```bash
# In-app generation:
jwt-pitbul keygen ed25519
# Or via OpenSSL:
openssl genpkey -algorithm Ed25519 -out ed25519_private.pem
openssl pkey -in ed25519_private.pem -pubout -out ed25519_public.pem
```
## Error Codes
| Exit | Meaning |
|------|---------|
| 0 | Success |
| 1 | Error (invalid arguments, bad signature, missing file, etc.) |
## Building
```bash
cd apps/jwt-pitbul
../../buxc build # Bootstrap compiler (Nim)
# or
../../buxc_lir build # Self-hosted compiler
```
## Dependencies
- Bux standard library (`Std::Crypto`, `Std::Io`, `Std::String`)
- OpenSSL 1.1.1+ (linked via runtime)
- Bux compiler (`buxc` or `buxc_lir`)
## License
Part of the Bux project. See root [LICENSE](../../LICENSE).
+8
View File
@@ -0,0 +1,8 @@
[Package]
Name = "jwt-pitbul"
Version = "0.2.0"
Type = "bin"
Description = "JWT CLI tool — encode, decode, verify, and key generation"
[Build]
Output = "Bin"
+468
View File
@@ -0,0 +1,468 @@
// =============================================================================
// jwt-pitbul — JWT CLI Tool (encode, decode, verify, keygen)
// Rewritten with modern Bux: algebraic enums, methods, string interpolation,
// for-in loops, generic arrays, and structured error handling.
// =============================================================================
module Main {
import Std::Io::{PrintLine, Print, PrintInt};
import Std::String::{
String_Len, String_Eq,
String_SplitCount, String_SplitPart,
StringBuilder, StringBuilder_New, StringBuilder_Append,
StringBuilder_AppendInt, StringBuilder_Build, StringBuilder_Free
};
import Std::Crypto::Jwt::{
JwtAlg,
Jwt_MakeHeader,
Jwt_Encode,
Jwt_Decode,
Jwt_EncodeHS256, Jwt_EncodeHS384, Jwt_EncodeHS512,
Jwt_EncodeRS256, Jwt_EncodeES256, Jwt_EncodeEdDSA
};
import Std::Crypto::Base64::{Base64URL_Decode, Base64_Encode};
import Std::Crypto::Ed25519::{Ed25519_Keypair};
extern func bux_argc() -> int;
extern func bux_argv(index: int) -> String;
extern func bux_alloc(size: uint) -> *void;
extern func bux_read_file(path: String) -> String;
extern func bux_file_exists(path: String) -> int;
extern func bux_base64_encode(data: String, len: int) -> String;
// =============================================================================
// Constants
// =============================================================================
const AppName: String = "jwt-pitbul";
const Version: String = "0.2.0";
// =============================================================================
// Algebraic enums for optionals and results
// =============================================================================
enum AlgOption {
Some(JwtAlg),
None,
}
func AlgOption_MakeSome(value: JwtAlg) -> AlgOption {
let o: AlgOption = AlgOption { tag: AlgOption_Some };
o.data.Some_0 = value;
return o;
}
func AlgOption_MakeNone() -> AlgOption {
return AlgOption { tag: AlgOption_None };
}
enum CmdOption {
Some(Cmd),
None,
}
func CmdOption_MakeSome(value: Cmd) -> CmdOption {
let o: CmdOption = CmdOption { tag: CmdOption_Some };
o.data.Some_0 = value;
return o;
}
func CmdOption_MakeNone() -> CmdOption {
return CmdOption { tag: CmdOption_None };
}
enum KeyTypeOption {
Some(KeyType),
None,
}
func KeyTypeOption_MakeSome(value: KeyType) -> KeyTypeOption {
let o: KeyTypeOption = KeyTypeOption { tag: KeyTypeOption_Some };
o.data.Some_0 = value;
return o;
}
func KeyTypeOption_MakeNone() -> KeyTypeOption {
return KeyTypeOption { tag: KeyTypeOption_None };
}
enum KeyResult {
Ok(String),
Err(String),
}
func KeyResult_MakeOk(value: String) -> KeyResult {
let r: KeyResult = KeyResult { tag: KeyResult_Ok };
r.data.Ok_0 = value;
return r;
}
func KeyResult_MakeErr(msg: String) -> KeyResult {
let r: KeyResult = KeyResult { tag: KeyResult_Err };
r.data.Err_0 = msg;
return r;
}
enum ExitResult {
Ok(int),
Err(String),
}
func ExitResult_MakeOk(value: int) -> ExitResult {
let r: ExitResult = ExitResult { tag: ExitResult_Ok };
r.data.Ok_0 = value;
return r;
}
func ExitResult_MakeErr(msg: String) -> ExitResult {
let r: ExitResult = ExitResult { tag: ExitResult_Err };
r.data.Err_0 = msg;
return r;
}
// =============================================================================
// Commands
// =============================================================================
enum Cmd {
Sign,
Verify,
Decode,
Keygen,
Help,
}
func ParseCmd(name: String) -> CmdOption {
if String_Eq(name, "sign") { return CmdOption_MakeSome(Cmd { tag: Cmd_Sign }); }
if String_Eq(name, "verify") { return CmdOption_MakeSome(Cmd { tag: Cmd_Verify }); }
if String_Eq(name, "decode") { return CmdOption_MakeSome(Cmd { tag: Cmd_Decode }); }
if String_Eq(name, "keygen") { return CmdOption_MakeSome(Cmd { tag: Cmd_Keygen }); }
if String_Eq(name, "help") || String_Eq(name, "--help") || String_Eq(name, "-h") {
return CmdOption_MakeSome(Cmd { tag: Cmd_Help });
}
return CmdOption_MakeNone();
}
// =============================================================================
// Key type generation
// =============================================================================
enum KeyType {
Rsa,
Ecdsa,
Ed25519,
}
func ParseKeyType(name: String) -> KeyTypeOption {
if String_Eq(name, "rsa") { return KeyTypeOption_MakeSome(KeyType { tag: KeyType_Rsa }); }
if String_Eq(name, "ecdsa") { return KeyTypeOption_MakeSome(KeyType { tag: KeyType_Ecdsa }); }
if String_Eq(name, "ed25519") { return KeyTypeOption_MakeSome(KeyType { tag: KeyType_Ed25519 }); }
return KeyTypeOption_MakeNone();
}
// =============================================================================
// Extend JwtAlg with parsing / introspection methods
// =============================================================================
extend JwtAlg {
func Name(self: JwtAlg) -> String {
match self {
JwtAlg::HS256 => "HS256",
JwtAlg::HS384 => "HS384",
JwtAlg::HS512 => "HS512",
JwtAlg::RS256 => "RS256",
JwtAlg::RS384 => "RS384",
JwtAlg::RS512 => "RS512",
JwtAlg::ES256 => "ES256",
JwtAlg::ES384 => "ES384",
JwtAlg::EdDSA => "EdDSA",
_ => "HS256",
}
}
func IsHmac(self: JwtAlg) -> bool {
return self.tag == JwtAlg_HS256 || self.tag == JwtAlg_HS384 || self.tag == JwtAlg_HS512;
}
func NeedsPemFile(self: JwtAlg) -> bool {
return self.tag == JwtAlg_RS256 || self.tag == JwtAlg_RS384 || self.tag == JwtAlg_RS512 ||
self.tag == JwtAlg_ES256 || self.tag == JwtAlg_ES384;
}
}
func ParseAlg(name: String) -> AlgOption {
if String_Eq(name, "HS256") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_HS256 }); }
if String_Eq(name, "HS384") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_HS384 }); }
if String_Eq(name, "HS512") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_HS512 }); }
if String_Eq(name, "RS256") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_RS256 }); }
if String_Eq(name, "RS384") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_RS384 }); }
if String_Eq(name, "RS512") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_RS512 }); }
if String_Eq(name, "ES256") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_ES256 }); }
if String_Eq(name, "ES384") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_ES384 }); }
if String_Eq(name, "EdDSA") { return AlgOption_MakeSome(JwtAlg { tag: JwtAlg_EdDSA }); }
return AlgOption_MakeNone();
}
// =============================================================================
// Help / Usage
// =============================================================================
func PrintUsage() {
PrintLine("╔══════════════════════════════════════════════════════╗");
PrintLine(f"║ {AppName} — JWT CLI Tool v{Version} ║");
PrintLine("║ Sign, verify, decode JSON Web Tokens ║");
PrintLine("╚══════════════════════════════════════════════════════╝");
PrintLine("");
PrintLine("Usage:");
PrintLine(f" {AppName} sign <alg> <key> <claims>");
PrintLine(f" {AppName} verify <token> <alg> <key>");
PrintLine(f" {AppName} decode <token>");
PrintLine(f" {AppName} keygen <type>");
PrintLine("");
PrintLine("Commands:");
PrintLine(" sign Create a signed JWT from claims JSON");
PrintLine(" verify Verify a JWT signature and print payload");
PrintLine(" decode Decode a JWT without verification");
PrintLine(" keygen Generate cryptographic keys");
PrintLine("");
PrintLine("Algorithms:");
PrintLine(" HS256, HS384, HS512 — HMAC (symmetric)");
PrintLine(" RS256, RS384, RS512 — RSA PKCS#1 v1.5");
PrintLine(" ES256, ES384 — ECDSA P-256 / P-384");
PrintLine(" EdDSA — Ed25519");
PrintLine("");
PrintLine("Key formats:");
PrintLine(" HMAC: raw secret string");
PrintLine(" RSA: path to PEM private/public key file");
PrintLine(" ECDSA: path to PEM private/public key file");
PrintLine(" EdDSA: base64-encoded 32-byte raw key");
PrintLine("");
PrintLine("Key generation:");
PrintLine(f" {AppName} keygen rsa # RSA 2048-bit (PEM)");
PrintLine(f" {AppName} keygen ecdsa # ECDSA P-256 (PEM)");
PrintLine(f" {AppName} keygen ed25519 # Ed25519 (raw base64)");
PrintLine("");
PrintLine("Examples:");
Print(f" {AppName} sign HS256 'my-secret' ");
PrintLine("'{\"sub\":\"123\"}'");
PrintLine(f" {AppName} verify eyJh... HS256 'my-secret'");
PrintLine(f" {AppName} decode eyJh...");
PrintLine(f" {AppName} keygen ed25519");
}
// =============================================================================
// Key resolution — for RSA/ECDSA, read PEM file; for HMAC/EdDSA, pass through
// =============================================================================
func ResolveKey(alg: JwtAlg, keyArg: String) -> KeyResult {
if !alg.NeedsPemFile() {
return KeyResult_MakeOk(keyArg);
}
if bux_file_exists(keyArg) == 0 {
return KeyResult_MakeErr(f"ERROR: PEM file not found: {keyArg}");
}
let pem: String = bux_read_file(keyArg);
if String_Len(pem) == 0 {
return KeyResult_MakeErr(f"ERROR: could not read PEM file: {keyArg}");
}
return KeyResult_MakeOk(pem);
}
// =============================================================================
// Command: sign
// =============================================================================
func CmdSign(algName: String, keyArg: String, claimsJson: String) -> ExitResult {
let algOpt: AlgOption = ParseAlg(algName);
if algOpt.tag != AlgOption_Some {
return ExitResult_MakeErr(f"ERROR: unknown algorithm '{algName}'");
}
let alg: JwtAlg = algOpt.data.Some_0;
let keyRes: KeyResult = ResolveKey(alg, keyArg);
if keyRes.tag != KeyResult_Ok {
return ExitResult_MakeErr(keyRes.data.Err_0);
}
let key: String = keyRes.data.Ok_0;
let header: String = Jwt_MakeHeader(alg);
let token: String = Jwt_Encode(header, claimsJson, alg, key);
if String_Len(token) == 0 {
return ExitResult_MakeErr("ERROR: signing failed");
}
PrintLine(token);
return ExitResult_MakeOk(0);
}
// =============================================================================
// Command: verify
// =============================================================================
func CmdVerify(token: String, algName: String, keyArg: String) -> ExitResult {
let algOpt: AlgOption = ParseAlg(algName);
if algOpt.tag != AlgOption_Some {
return ExitResult_MakeErr(f"ERROR: unknown algorithm '{algName}'");
}
let alg: JwtAlg = algOpt.data.Some_0;
let keyRes: KeyResult = ResolveKey(alg, keyArg);
if keyRes.tag != KeyResult_Ok {
return ExitResult_MakeErr(keyRes.data.Err_0);
}
let key: String = keyRes.data.Ok_0;
var header: String = "";
var payload: String = "";
if !Jwt_Decode(token, alg, key, &header, &payload) {
return ExitResult_MakeErr("✗ Signature INVALID (or malformed token)");
}
PrintLine("✓ Signature valid");
PrintLine("");
PrintLine("Header:");
PrintLine(header);
PrintLine("");
PrintLine("Payload:");
PrintLine(payload);
return ExitResult_MakeOk(0);
}
// =============================================================================
// Command: decode (no verification)
// =============================================================================
func CmdDecode(token: String) -> ExitResult {
let partCount: uint = String_SplitCount(token, ".");
if partCount != 3 {
return ExitResult_MakeErr("ERROR: not a valid JWT (expected 3 parts)");
}
let headerB64: String = String_SplitPart(token, ".", 0);
let payloadB64: String = String_SplitPart(token, ".", 1);
let sigB64: String = String_SplitPart(token, ".", 2);
let headerJson: String = Base64URL_Decode(headerB64);
let payloadJson: String = Base64URL_Decode(payloadB64);
PrintLine("Decoded (no verification):");
PrintLine("");
PrintLine("Header:");
PrintLine(headerJson);
PrintLine("");
PrintLine("Payload:");
PrintLine(payloadJson);
PrintLine("");
Print("Signature (base64url): ");
PrintLine(sigB64);
return ExitResult_MakeOk(0);
}
// =============================================================================
// Command: keygen
// =============================================================================
func CmdKeygen(keyType: String) -> ExitResult {
let ktOpt: KeyTypeOption = ParseKeyType(keyType);
if ktOpt.tag != KeyTypeOption_Some {
return ExitResult_MakeErr(f"ERROR: unknown key type '{keyType}'. Use: rsa, ecdsa, ed25519");
}
let kt: KeyType = ktOpt.data.Some_0;
if kt.tag == KeyType_Ed25519 {
let pubBuf: *void = bux_alloc(32);
let priv: *void = bux_alloc(32);
if !Ed25519_Keypair(pubBuf, priv) {
return ExitResult_MakeErr("ERROR: Ed25519 key generation failed (OpenSSL 1.1.1+ required)");
}
let pubB64: String = bux_base64_encode(pubBuf as String, 32);
let privB64: String = bux_base64_encode(priv as String, 32);
PrintLine("Ed25519 keypair (base64):");
PrintLine(f" Public: {pubB64}");
PrintLine(f" Private: {privB64}");
return ExitResult_MakeOk(0);
}
if kt.tag == KeyType_Rsa {
PrintLine("RSA key generation requires OpenSSL CLI:");
PrintLine(" openssl genpkey -algorithm RSA -out private.pem -pkeyopt rsa_keygen_bits:2048");
PrintLine(" openssl rsa -in private.pem -pubout -out public.pem");
PrintLine("");
PrintLine("(Key generation from within Bux requires PEM write support — coming soon)");
return ExitResult_MakeOk(0);
}
if kt.tag == KeyType_Ecdsa {
PrintLine("ECDSA key generation requires OpenSSL CLI:");
PrintLine(" openssl ecparam -genkey -name prime256v1 -noout -out ec_private.pem");
PrintLine(" openssl ec -in ec_private.pem -pubout -out ec_public.pem");
PrintLine("");
PrintLine("(Key generation from within Bux requires PEM write support — coming soon)");
return ExitResult_MakeOk(0);
}
return ExitResult_MakeErr(f"ERROR: unknown key type '{keyType}'");
}
// =============================================================================
// Dispatch a parsed command to its handler
// =============================================================================
func RunCommand(cmd: Cmd, args: Array<String>) -> ExitResult {
if cmd.tag == Cmd_Help {
PrintUsage();
return ExitResult_MakeOk(0);
}
if cmd.tag == Cmd_Sign {
if args.len < 5 {
return ExitResult_MakeErr("ERROR: 'sign' requires: <alg> <key> <claims_json>");
}
return CmdSign(args[2], args[3], args[4]);
}
if cmd.tag == Cmd_Verify {
if args.len < 5 {
return ExitResult_MakeErr("ERROR: 'verify' requires: <token> <alg> <key>");
}
return CmdVerify(args[2], args[3], args[4]);
}
if cmd.tag == Cmd_Decode {
if args.len < 3 {
return ExitResult_MakeErr("ERROR: 'decode' requires: <token>");
}
return CmdDecode(args[2]);
}
if cmd.tag == Cmd_Keygen {
if args.len < 3 {
return ExitResult_MakeErr("ERROR: 'keygen' requires: <type>");
}
return CmdKeygen(args[2]);
}
return ExitResult_MakeErr("ERROR: unhandled command");
}
// =============================================================================
// Main Entry Point
// =============================================================================
func Main() -> int {
let argc: int = bux_argc();
// Collect CLI arguments into a generic Array<String>
let args: Array<String> = Array_New<String>(argc as uint);
for i in 0..argc {
Array_Push<String>(&args, bux_argv(i));
}
if args.len < 2 {
PrintUsage();
return 0;
}
let command: String = args[1];
let cmdOpt: CmdOption = ParseCmd(command);
if cmdOpt.tag != CmdOption_Some {
PrintLine(f"ERROR: unknown command '{command}'");
PrintLine(f"Run '{AppName} help' for usage.");
return 1;
}
let cmd: Cmd = cmdOpt.data.Some_0;
let result: ExitResult = RunCommand(cmd, args);
if result.tag == ExitResult_Err {
PrintLine(result.data.Err_0);
return 1;
}
return result.data.Ok_0;
}
} // module Main
+218
View File
@@ -0,0 +1,218 @@
# Nexus
**High-performance, multi-threaded HTTP/1.1, HTTP/2 & WebSocket server — built with the [Bux](https://github.com/bux-lang/bux) programming language.**
Nexus is a from-scratch web server that demonstrates Bux's systems-programming capabilities: raw TCP sockets, pthread-based concurrency, manual memory management, and zero-dependency C ABI interop — all from a clean, modern syntax.
---
## Features
| Area | What's Implemented |
|------|-------------------|
| **HTTP/1.1** | Full request parsing, **keep-alive** (reuse TCP), status codes, content negotiation |
| **Multi-threaded** | Worker pool + channel task queue; configurable via `NEXUS_WORKERS` |
| **HTTP/2** | Connection preface detection (`PRI * HTTP/2.0`), upgrade-aware routing |
| **WebSocket** | RFC 6455 upgrade handshake detection, `Sec-WebSocket-Key` extraction |
| **Static files** | Serves from `public/` with MIME-type detection for 20+ file types, directory-traversal protection |
| **JSON API** | Built-in `/api/health` and `/api/info` endpoints |
| **Logging** | Access log: `METHOD path status duration_ms` (`NEXUS_ACCESS_LOG=0` to disable) |
| **Limits** | `NEXUS_MAX_BODY` (default 1 MiB) → HTTP 413 when exceeded |
| **Graceful stop** | SIGINT/SIGTERM: close listen fd, poison workers, exit 0 |
| **TLS / HTTPS** | OpenSSL server mode via `NEXUS_TLS=1` + PEM cert/key |
| **mTLS** | `NEXUS_TLS_CLIENT_CA` PEM → require client certificates |
## Quick Start
```bash
# From the project root
cd apps/nexus
# Build with the bootstrap compiler (Nim)
../../buxc build
# Or with the self-hosted compiler
../../buxc_lir build
# Run
./nexus
# Optional env (also used by `make bench-nexus`)
# NEXUS_PORT=18080 NEXUS_BIND=127.0.0.1 NEXUS_WORKERS=4 \
# NEXUS_MAX_BODY=1048576 NEXUS_ACCESS_LOG=1 ./build/nexus
# HTTPS (self-signed example)
# openssl req -x509 -newkey rsa:2048 -nodes -keyout key.pem -out cert.pem -days 365 -subj /CN=localhost
# NEXUS_TLS=1 NEXUS_TLS_CERT=cert.pem NEXUS_TLS_KEY=key.pem NEXUS_PORT=8443 ./build/nexus
# curl -k https://127.0.0.1:8443/api/health
# mTLS (require client cert signed by CA)
# NEXUS_TLS_CLIENT_CA=ca.pem NEXUS_TLS=1 NEXUS_TLS_CERT=server.pem NEXUS_TLS_KEY=server.key …
# curl --cert client.pem --key client.key --cacert ca.pem https://…
```
Server starts on `http://0.0.0.0:8080` (or `https://` when TLS is enabled).
Stop with **Ctrl+C** or `kill -TERM` (graceful: workers drained via poison pills).
Smoke: `make test-nexus-tls` (self-signed cert + curl -k).
### Docker
```bash
# Full Nexus (needs libssl3)
../../buxc --release build # from apps/nexus
docker build -f ../../examples/docker/Dockerfile.nexus -t bux-nexus ../..
docker run --rm -p 8080:8080 bux-nexus
```
```
╔══════════════════════════════════════════════╗
║ Nexus HTTP Server v0.1.0 ║
║ High-performance multi-threaded HTTP/1.1 ║
║ HTTP/2 & WebSocket detection included ║
║ Built with Bux ║
╚══════════════════════════════════════════════╝
✓ Server listening on http://0.0.0.0:8080
✓ Worker threads: 4
✓ Static files: ./public/
Endpoints:
GET / — Static files (public/)
GET /api/health — Health check (JSON)
GET /api/info — Server info (JSON)
GET /ws — WebSocket upgrade
ANY /* — Static file serving
```
## Testing It
```bash
# Home page (HTML)
curl http://localhost:8080/
# Health check
curl http://localhost:8080/api/health
# → {"status":"ok","server":"Nexus","version":"0.1.0"}
# Server info
curl http://localhost:8080/api/info
# → {"name":"Nexus","language":"Bux","features":[...]}
# Static file (404 page)
curl http://localhost:8080/404.html
# Nonexistent file
curl http://localhost:8080/nope
# → 404 Not Found
# WebSocket upgrade attempt
curl -H "Upgrade: websocket" \
-H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
http://localhost:8080/ws
# → 101 Switching Protocols
```
## Architecture
### Thread Model — Multi-Accept
Nexus uses the **multi-accept** pattern rather than a traditional thread pool with a work queue:
```
Main Thread Worker 1 Worker 2 Worker 3
│ │ │ │
├─ socket()/bind()/listen() │ │ │
├─ spawn Worker() ──────────► │ │
├─ spawn Worker() ─────────────────────────► │
├─ spawn Worker() ────────────────────────────────────────►
│ │ │ │
▼ Worker() ▼ accept() ▼ accept() ▼ accept()
accept() loop │ │ │
```
Each worker thread calls `accept()` on the **same** listening socket. The Linux kernel distributes incoming connections across the blocked accept calls — the same mechanism used by nginx and Apache prefork. No mutex, no queue, no context switching between a dispatcher and workers.
### Request Lifecycle
```
Client connects
Net_Accept() → client fd
Net_Recv(fd, 8192) → raw bytes
ParseRequest()
├─ Split headers on \r\n\r\n
├─ Parse request line → method, path, version
└─ Parse header lines → key-value array
Router_Dispatch()
├─ /api/* → JSON handlers
├─ /ws → WebSocket upgrade
├─ Upgrade header → HTTP/2 or WS detection
└─ /* → Static file serving
BuildResponse() → HTTP/1.1 status line + headers + body
Net_Send(fd, response) → bytes to client
Net_Close(fd)
```
### Design Decisions
- **No keep-alive by default.** Each connection is closed after one response. This avoids blocking worker threads on idle clients (Bux doesn't yet expose `SO_RCVTIMEO`).
- **Linear header array instead of hash map.** HTTP requests typically carry 515 headers. A linear scan over a key-value array is faster than hashing for this N, and avoids the complexity of iterating over Bux's generic `StringMap`.
- **Raw extern calls for the string builder.** The stdlib `StringBuilder` wrapper adds a struct indirection. Calling `bux_sb_new` / `bux_sb_append` / `bux_sb_build` directly is simpler and equally safe.
- **WebSocket accept key is a placeholder.** A full implementation needs SHA-1 hashing and Base64 encoding. These aren't in Bux's stdlib yet; they can be added as extern C functions when needed.
## Project Structure
```
apps/nexus/
├── bux.toml # Package manifest
├── README.md # This file
├── src/
│ └── Main.bux # The entire server (~640 lines)
└── public/
├── index.html # Landing page
└── 404.html # Error page
```
Everything lives in one file (`src/Main.bux`) by design — it keeps the module graph flat and the build fast. As the server grows, the HTTP parser, router, and handlers can be split into separate modules.
## Configuration
Edit the constants at the top of `src/Main.bux`:
```bux
const SERVER_PORT: int = 8080; // Listen port
const THREAD_COUNT: int = 4; // Number of worker threads
const RECV_BUF_SIZE: int = 8192; // Receive buffer per request
const SERVER_NAME: String = "Nexus/0.1.0 (Bux)";
const PUBLIC_DIR: String = "public"; // Static files directory
```
## Roadmap
- [ ] Keep-alive with configurable socket timeout
- [ ] Full WebSocket frame read/write (requires SHA-1 + Base64)
- [ ] HTTP/2 binary framing layer (HPACK, stream multiplexing)
- [ ] SSL/TLS via OpenSSL extern bindings
- [ ] Middleware / filter chain
- [ ] Request body parsing (JSON, form-encoded, multipart)
- [ ] Virtual hosts
- [ ] Access logging to file
- [ ] Rate limiting
## License
Nexus is part of the Bux project. See the root [LICENSE](../../LICENSE) for terms.
+1
View File
@@ -0,0 +1 @@
EXIT:137
+1
View File
@@ -0,0 +1 @@
EXIT:124
+1
View File
@@ -0,0 +1 @@
EXIT:124
+1
View File
@@ -0,0 +1 @@
EXIT:124
+1
View File
@@ -0,0 +1 @@
EXIT:124
+2
View File
@@ -0,0 +1,2 @@
timeout: командата „„../../buxc““ не може да бъде изпълнена: Няма такъв файл или директория
EXIT:127
+1
View File
@@ -0,0 +1 @@
EXIT:137
+8
View File
@@ -0,0 +1,8 @@
[Package]
Name = "nexus"
Version = "0.1.0"
Type = "bin"
Description = "High-performance multi-threaded HTTP/1.1, HTTP/2 and WebSocket server"
[Build]
Output = "Bin"
+31
View File
@@ -0,0 +1,31 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>404 — Not Found | Nexus</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background: #0d1117;
color: #c9d1d9;
min-height: 100vh;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
}
h1 { font-size: 4rem; color: #f85149; margin-bottom: 0.5rem; }
p { color: #8b949e; font-size: 1.2rem; }
.back { margin-top: 2rem; }
.back a { color: #58a6ff; text-decoration: none; }
.back a:hover { text-decoration: underline; }
</style>
</head>
<body>
<h1>404</h1>
<p>The requested resource was not found on this server.</p>
<p class="back"><a href="/">← Back to Home</a></p>
</body>
</html>
+1
View File
@@ -0,0 +1 @@
<html><body>Hello Nexus</body></html>
+37
View File
@@ -0,0 +1,37 @@
module Config {
pub struct ServerConfig {
bindAddr: String;
port: int;
workerCount: int;
publicDir: String;
backlog: int;
/// Max raw request bytes (recv buffer / body safety). Default 1 MiB.
maxBodyBytes: int;
/// Access log to stdout (method path status ms). Default true.
accessLog: bool;
/// Enable HTTPS (OpenSSL). Requires tlsCertPath + tlsKeyPath.
tlsEnabled: bool;
tlsCertPath: String;
tlsKeyPath: String;
/// Optional client CA PEM → mTLS (require client certificate).
tlsClientCaPath: String;
}
pub const func DefaultConfig() -> ServerConfig {
return ServerConfig {
bindAddr: "0.0.0.0",
port: 8080,
workerCount: 4,
publicDir: "public",
backlog: 128,
maxBodyBytes: 1048576,
accessLog: true,
tlsEnabled: false,
tlsCertPath: "",
tlsKeyPath: "",
tlsClientCaPath: "",
};
}
}
+65
View File
@@ -0,0 +1,65 @@
module Errors {
import Http::{HttpRequest};
pub enum HttpError {
BadRequest,
NotFound,
MethodNotAllowed,
InternalError(String),
}
pub enum ParseResult {
Ok(HttpRequest),
Err(HttpError),
}
pub enum FileResult {
Ok(String),
Err(HttpError),
}
pub func ParseResult_NewOk(req: HttpRequest) -> ParseResult {
let r: ParseResult = ParseResult { tag: ParseResult_Ok };
r.data.Ok_0 = req;
return r;
}
pub func ParseResult_NewErr(err: HttpError) -> ParseResult {
let r: ParseResult = ParseResult { tag: ParseResult_Err };
r.data.Err_0 = err;
return r;
}
pub func FileResult_NewOk(content: String) -> FileResult {
let r: FileResult = FileResult { tag: FileResult_Ok };
r.data.Ok_0 = content;
return r;
}
pub func FileResult_NewErr(err: HttpError) -> FileResult {
let r: FileResult = FileResult { tag: FileResult_Err };
r.data.Err_0 = err;
return r;
}
pub func FileResult_IsOk(r: FileResult) -> bool {
return r.tag == FileResult_Ok;
}
pub func FileResult_Unwrap(r: FileResult) -> String {
return r.data.Ok_0;
}
pub func FileResult_UnwrapErr(r: FileResult) -> HttpError {
return r.data.Err_0;
}
pub func HttpError_ToString(err: HttpError) -> String {
if err.tag == HttpError_BadRequest { return "Bad Request"; }
if err.tag == HttpError_NotFound { return "Not Found"; }
if err.tag == HttpError_MethodNotAllowed { return "Method Not Allowed"; }
return err.data.InternalError_0;
}
}
+95
View File
@@ -0,0 +1,95 @@
module Handlers {
import Std::String::{String_Eq, String_Contains};
import Http::{HttpMethod, HttpRequest, HttpResponse, Http_NewResponse, Http_MimeType, RequestHeader_Get};
import Errors::{HttpError, FileResult, FileResult_NewOk, FileResult_NewErr, FileResult_IsOk, FileResult_Unwrap, FileResult_UnwrapErr, HttpError_ToString};
extern func bux_strlen(s: String) -> uint;
extern func bux_file_exists(path: String) -> int;
extern func bux_read_file(path: String) -> String;
extern func bux_path_join(a: String, b: String) -> String;
extern func bux_sb_new(initial_cap: uint) -> *void;
extern func bux_sb_append(sb: *void, s: String);
extern func bux_sb_append_int(sb: *void, n: int64);
extern func bux_sb_build(sb: *void) -> String;
extern func bux_sb_free(sb: *void);
pub func NotFoundResponse() -> HttpResponse {
return Http_NewResponse(404, "application/json; charset=utf-8", "{\"error\":\"not_found\"}");
}
pub func MethodNotAllowedResponse() -> HttpResponse {
return Http_NewResponse(405, "text/plain; charset=utf-8", "Method Not Allowed");
}
pub func ReadStaticFile(requestPath: String) -> FileResult {
if String_Contains(requestPath, "..") {
return FileResult_NewErr(HttpError { tag: HttpError_NotFound });
}
var filePath: String = requestPath;
if String_Eq(filePath, "/") {
filePath = "/index.html";
}
let fullPath: String = bux_path_join("public", filePath);
if bux_file_exists(fullPath) == 0 {
return FileResult_NewErr(HttpError { tag: HttpError_NotFound });
}
let content: String = bux_read_file(fullPath);
return FileResult_NewOk(content);
}
func FileErrorResponse(err: HttpError) -> HttpResponse {
match err {
HttpError::NotFound => NotFoundResponse(),
_ => Http_NewResponse(500, "text/plain; charset=utf-8", HttpError_ToString(err)),
}
}
pub func ServeStaticFile(req: HttpRequest) -> HttpResponse {
if req.method != HttpMethod_GET && req.method != HttpMethod_HEAD {
return MethodNotAllowedResponse();
}
let result: FileResult = ReadStaticFile(req.path);
if FileResult_IsOk(result) {
let content: String = FileResult_Unwrap(result);
return Http_NewResponse(200, Http_MimeType(req.path), content);
}
return FileErrorResponse(FileResult_UnwrapErr(result));
}
pub func HandleApiHealth() -> HttpResponse {
return Http_NewResponse(200, "application/json; charset=utf-8",
"{\"status\":\"ok\",\"server\":\"Nexus\",\"version\":\"0.6.0\"}");
}
pub func HandleApiInfo() -> HttpResponse {
return Http_NewResponse(200, "application/json; charset=utf-8",
"{\"name\":\"Nexus\",\"language\":\"Bux\",\"features\":[\"HTTP/1.1\",\"TLS\",\"mTLS\",\"keep-alive\",\"thread-pool\",\"graceful-stop\",\"access-log\",\"max-body\"]}");
}
pub func HandleWebSocketUpgrade(req: HttpRequest) -> HttpResponse {
var reqMut: HttpRequest = req;
let wsKey: String = RequestHeader_Get(&reqMut, "Sec-WebSocket-Key");
var resp: HttpResponse;
resp.statusCode = 101;
resp.contentType = "";
resp.body = "";
let sb: *void = bux_sb_new(256);
bux_sb_append(sb, "Upgrade: websocket\r\n");
bux_sb_append(sb, "Connection: Upgrade\r\n");
bux_sb_append(sb, "Sec-WebSocket-Accept: ");
bux_sb_append(sb, wsKey);
bux_sb_append(sb, "\r\n");
resp.extraHeaders = bux_sb_build(sb);
bux_sb_free(sb);
return resp;
}
}
+170
View File
@@ -0,0 +1,170 @@
module Http {
import Std::Array::{Array, Array_Len, Array_Get};
import Std::String::{String_Eq, String_EndsWith, String_Contains, String_Len, String_StartsWith};
pub enum HttpMethod {
GET,
POST,
PUT,
DELETE,
PATCH,
HEAD,
OPTIONS,
UNKNOWN,
}
pub struct HeaderEntry {
key: String;
value: String;
}
pub struct HttpRequest {
method: HttpMethod;
path: String;
version: String;
body: String;
headers: Array<HeaderEntry>;
}
pub struct HttpResponse {
statusCode: int;
contentType: String;
body: String;
extraHeaders: String;
}
pub func Http_StatusText(code: int) -> String {
if code == 200 { return "OK"; }
if code == 201 { return "Created"; }
if code == 204 { return "No Content"; }
if code == 301 { return "Moved Permanently"; }
if code == 302 { return "Found"; }
if code == 304 { return "Not Modified"; }
if code == 400 { return "Bad Request"; }
if code == 401 { return "Unauthorized"; }
if code == 403 { return "Forbidden"; }
if code == 404 { return "Not Found"; }
if code == 405 { return "Method Not Allowed"; }
if code == 413 { return "Payload Too Large"; }
if code == 414 { return "URI Too Long"; }
if code == 500 { return "Internal Server Error"; }
if code == 501 { return "Not Implemented"; }
if code == 503 { return "Service Unavailable"; }
return "Unknown";
}
pub func Http_MimeType(path: String) -> String {
if String_EndsWith(path, ".html") || String_EndsWith(path, ".htm") { return "text/html; charset=utf-8"; }
if String_EndsWith(path, ".css") { return "text/css; charset=utf-8"; }
if String_EndsWith(path, ".js") { return "application/javascript; charset=utf-8"; }
if String_EndsWith(path, ".json") { return "application/json; charset=utf-8"; }
if String_EndsWith(path, ".xml") { return "application/xml; charset=utf-8"; }
if String_EndsWith(path, ".txt") { return "text/plain; charset=utf-8"; }
if String_EndsWith(path, ".png") { return "image/png"; }
if String_EndsWith(path, ".jpg") || String_EndsWith(path, ".jpeg") { return "image/jpeg"; }
if String_EndsWith(path, ".gif") { return "image/gif"; }
if String_EndsWith(path, ".svg") { return "image/svg+xml"; }
if String_EndsWith(path, ".ico") { return "image/x-icon"; }
if String_EndsWith(path, ".webp") { return "image/webp"; }
if String_EndsWith(path, ".woff2") { return "font/woff2"; }
if String_EndsWith(path, ".woff") { return "font/woff"; }
if String_EndsWith(path, ".wasm") { return "application/wasm"; }
return "application/octet-stream";
}
pub func Http_MethodName(m: HttpMethod) -> String {
match m {
HttpMethod::GET => "GET",
HttpMethod::POST => "POST",
HttpMethod::PUT => "PUT",
HttpMethod::DELETE => "DELETE",
HttpMethod::PATCH => "PATCH",
HttpMethod::HEAD => "HEAD",
HttpMethod::OPTIONS => "OPTIONS",
HttpMethod::UNKNOWN => "UNKNOWN",
}
}
pub func Http_NewResponse(code: int, contentType: String, body: String) -> HttpResponse {
var resp: HttpResponse;
resp.statusCode = code;
resp.contentType = contentType;
resp.body = body;
resp.extraHeaders = "";
return resp;
}
func CharLower(c: int) -> int {
if c >= 65 && c <= 90 {
return c + 32;
}
return c;
}
/// Case-insensitive string equality (HTTP header names/values).
pub func String_EqIgnoreCase(a: String, b: String) -> bool {
let la: uint = String_Len(a);
let lb: uint = String_Len(b);
if la != lb {
return false;
}
var i: uint = 0;
while i < la {
if CharLower(a[i] as int) != CharLower(b[i] as int) {
return false;
}
i = i + 1;
}
return true;
}
pub func RequestHeader_Get(req: *HttpRequest, key: String) -> String {
// Index-based walk — for-in over Array<struct{String}> can corrupt string fields.
let n: uint = Array_Len<HeaderEntry>(&req.headers);
var i: uint = 0;
while i < n {
let entry: HeaderEntry = Array_Get<HeaderEntry>(&req.headers, i);
if String_EqIgnoreCase(entry.key, key) {
return entry.value;
}
i = i + 1;
}
return "";
}
/// HTTP/1.1 defaults to keep-alive; Connection: close forces close;
/// HTTP/1.0 needs explicit keep-alive. Uses parsed headers (safe after
/// Parser field-move ownership handoff).
pub func Request_WantsKeepAlive(req: *HttpRequest) -> bool {
let conn: String = RequestHeader_Get(req, "Connection");
if String_EqIgnoreCase(conn, "close") {
return false;
}
if String_EqIgnoreCase(conn, "keep-alive") {
return true;
}
if String_StartsWith(req.version, "HTTP/1.0") {
return false;
}
return true;
}
/// Fallback when only raw bytes are available.
pub func RawRequest_WantsKeepAlive(raw: String) -> bool {
if String_Contains(raw, "Connection: close") || String_Contains(raw, "connection: close") ||
String_Contains(raw, "CONNECTION: CLOSE") {
return false;
}
if String_Contains(raw, "HTTP/1.0") {
if String_Contains(raw, "Connection: keep-alive") ||
String_Contains(raw, "Connection: Keep-Alive") ||
String_Contains(raw, "connection: keep-alive") {
return true;
}
return false;
}
return true;
}
}
+119
View File
@@ -0,0 +1,119 @@
module Main {
import Config::{ServerConfig, DefaultConfig};
import Http::{HttpMethod};
import Router::{Handler, Route, Router};
import Server::{RunServer};
import Std::Array::{Array, Array_New, Array_Push};
import Std::Os::{Os_GetEnv};
import Std::String::{String_Len, String_ToInt, String_Eq};
func BuildRouter() -> Router {
var routes: Array<Route> = Array_New<Route>(8);
Array_Push<Route>(&routes, Route {
method: HttpMethod { tag: HttpMethod_GET },
path: "/api/health",
handler: Handler { tag: Handler_ApiHealth },
});
Array_Push<Route>(&routes, Route {
method: HttpMethod { tag: HttpMethod_GET },
path: "/api/info",
handler: Handler { tag: Handler_ApiInfo },
});
Array_Push<Route>(&routes, Route {
method: HttpMethod { tag: HttpMethod_GET },
path: "/ws",
handler: Handler { tag: Handler_WsUpgrade },
});
Array_Push<Route>(&routes, Route {
method: HttpMethod { tag: HttpMethod_GET },
path: "/",
handler: Handler { tag: Handler_StaticFile },
});
return Router {
routes: routes,
notFound: Handler { tag: Handler_NotFound },
};
}
/// Apply optional env overrides for benches / ops:
/// NEXUS_PORT, NEXUS_WORKERS, NEXUS_BIND, NEXUS_PUBLIC,
/// NEXUS_MAX_BODY (bytes), NEXUS_ACCESS_LOG (0/1/false/true),
/// NEXUS_TLS=1 + NEXUS_TLS_CERT + NEXUS_TLS_KEY (PEM paths)
/// NEXUS_TLS_CLIENT_CA (optional PEM → mTLS)
func ApplyEnvConfig(config: *ServerConfig) {
let portEnv: String = Os_GetEnv("NEXUS_PORT");
if String_Len(portEnv) > 0 {
let p: int64 = String_ToInt(portEnv);
if p > 0 && p < 65536 {
config.port = p as int;
}
}
let workersEnv: String = Os_GetEnv("NEXUS_WORKERS");
if String_Len(workersEnv) > 0 {
let w: int64 = String_ToInt(workersEnv);
if w > 0 && w <= 256 {
config.workerCount = w as int;
}
}
let bindEnv: String = Os_GetEnv("NEXUS_BIND");
if String_Len(bindEnv) > 0 {
config.bindAddr = bindEnv;
}
let pubEnv: String = Os_GetEnv("NEXUS_PUBLIC");
if String_Len(pubEnv) > 0 {
config.publicDir = pubEnv;
}
let bodyEnv: String = Os_GetEnv("NEXUS_MAX_BODY");
if String_Len(bodyEnv) > 0 {
let b: int64 = String_ToInt(bodyEnv);
if b >= 1024 && b <= 67108864 {
config.maxBodyBytes = b as int;
}
}
let logEnv: String = Os_GetEnv("NEXUS_ACCESS_LOG");
if String_Len(logEnv) > 0 {
if String_Eq(logEnv, "0") || String_Eq(logEnv, "false") || String_Eq(logEnv, "off") {
config.accessLog = false;
} else {
config.accessLog = true;
}
}
let tlsEnv: String = Os_GetEnv("NEXUS_TLS");
if String_Len(tlsEnv) > 0 {
if String_Eq(tlsEnv, "1") || String_Eq(tlsEnv, "true") || String_Eq(tlsEnv, "on") ||
String_Eq(tlsEnv, "https") {
config.tlsEnabled = true;
}
}
let certEnv: String = Os_GetEnv("NEXUS_TLS_CERT");
if String_Len(certEnv) > 0 {
config.tlsCertPath = certEnv;
config.tlsEnabled = true;
}
let keyEnv: String = Os_GetEnv("NEXUS_TLS_KEY");
if String_Len(keyEnv) > 0 {
config.tlsKeyPath = keyEnv;
config.tlsEnabled = true;
}
let caEnv: String = Os_GetEnv("NEXUS_TLS_CLIENT_CA");
if String_Len(caEnv) > 0 {
config.tlsClientCaPath = caEnv;
config.tlsEnabled = true;
}
}
func Main() -> int {
var config: ServerConfig = DefaultConfig();
ApplyEnvConfig(&config);
let router: Router = BuildRouter();
return RunServer(config, router);
}
}
+133
View File
@@ -0,0 +1,133 @@
module Parser {
import Std::String::{String_Len, String_Eq, String_Trim};
import Std::Array::{Array, Array_New, Array_Push};
import Http::{HttpMethod, HttpRequest, HeaderEntry};
import Errors::{HttpError, ParseResult, ParseResult_NewOk, ParseResult_NewErr};
extern func bux_strlen(s: String) -> uint;
extern func bux_strstr(haystack: String, needle: String) -> String;
extern func bux_str_slice(s: String, start: uint, len: uint) -> String;
extern func bux_str_offset(pos: String, base: String) -> uint;
pub func ParseMethod(s: String) -> HttpMethod {
if String_Eq(s, "GET") { return HttpMethod { tag: HttpMethod_GET }; }
if String_Eq(s, "POST") { return HttpMethod { tag: HttpMethod_POST }; }
if String_Eq(s, "PUT") { return HttpMethod { tag: HttpMethod_PUT }; }
if String_Eq(s, "DELETE") { return HttpMethod { tag: HttpMethod_DELETE }; }
if String_Eq(s, "PATCH") { return HttpMethod { tag: HttpMethod_PATCH }; }
if String_Eq(s, "HEAD") { return HttpMethod { tag: HttpMethod_HEAD }; }
if String_Eq(s, "OPTIONS") { return HttpMethod { tag: HttpMethod_OPTIONS }; }
return HttpMethod { tag: HttpMethod_UNKNOWN };
}
func Slice(raw: String, start: int, len: int) -> String {
if start < 0 || len <= 0 { return ""; }
return bux_str_slice(raw, start as uint, len as uint);
}
func FindCrlf(raw: String, start: int) -> int {
let rawLen: uint = bux_strlen(raw);
if start as uint >= rawLen { return -1; }
let tail: String = bux_str_slice(raw, start as uint, rawLen - start as uint);
let hit: String = bux_strstr(tail, "\r\n");
if String_Len(hit) == 0 { return -1; }
let offset: uint = bux_str_offset(hit, tail);
return start + offset as int;
}
func ParseHeaders(raw: String, start: int, end: int) -> Array<HeaderEntry> {
var headers: Array<HeaderEntry> = Array_New<HeaderEntry>(16);
var pos: int = start;
while pos < end {
let lineEnd: int = FindCrlf(raw, pos);
if lineEnd < 0 || lineEnd >= end { break; }
let lineLen: int = lineEnd - pos;
if lineLen > 0 {
let line: String = Slice(raw, pos, lineLen);
let colon: String = bux_strstr(line, ":");
if String_Len(colon) > 0 {
let keyLen: uint = bux_str_offset(colon, line);
let key: String = String_Trim(Slice(line, 0, keyLen as int));
let valStart: int = keyLen as int + 1;
let val: String = String_Trim(Slice(line, valStart, lineLen - valStart));
let entry: HeaderEntry = HeaderEntry { key: key, value: val };
Array_Push<HeaderEntry>(&headers, entry);
}
}
pos = lineEnd + 2;
}
return headers;
}
pub func ParseRequest(raw: String) -> ParseResult {
let rawLen: uint = bux_strlen(raw);
if rawLen == 0 {
return ParseResult_NewErr(HttpError { tag: HttpError_BadRequest });
}
// Find end of request line
let lineEnd: int = FindCrlf(raw, 0);
if lineEnd < 0 {
return ParseResult_NewErr(HttpError { tag: HttpError_BadRequest });
}
// Split request line: METHOD PATH VERSION
var methodEnd: int = -1;
var pathStart: int = -1;
var pathEnd: int = -1;
var i: int = 0;
while i < lineEnd {
if raw[i] as int == 32 { // space
if methodEnd < 0 {
methodEnd = i;
pathStart = i + 1;
} else if pathEnd < 0 {
pathEnd = i;
break;
}
}
i = i + 1;
}
if methodEnd < 0 || pathStart < 0 || pathEnd < 0 {
return ParseResult_NewErr(HttpError { tag: HttpError_BadRequest });
}
let methodStr: String = Slice(raw, 0, methodEnd);
let path: String = Slice(raw, pathStart, pathEnd - pathStart);
let version: String = Slice(raw, pathEnd + 1, lineEnd - pathEnd - 1);
// Find header/body boundary
let boundary: String = bux_strstr(raw, "\r\n\r\n");
// Single assignment path (no Array_New then overwrite). Field-move of
// `headers` into HttpRequest skips auto-Drop (compiler movedOutLocals).
var headers: Array<HeaderEntry>;
var body: String = "";
if String_Len(boundary) > 0 {
let headerEnd: uint = bux_str_offset(boundary, raw);
headers = ParseHeaders(raw, lineEnd + 2, headerEnd as int);
let bodyStart: uint = headerEnd + 4;
if bodyStart < rawLen {
body = bux_str_slice(raw, bodyStart, rawLen - bodyStart);
}
} else {
headers = ParseHeaders(raw, lineEnd + 2, rawLen as int);
}
if String_Eq(path, "") {
// auto-drop of `headers` runs on error return
return ParseResult_NewErr(HttpError { tag: HttpError_BadRequest });
}
let req: HttpRequest = HttpRequest {
method: ParseMethod(methodStr),
path: path,
version: version,
body: body,
headers: headers,
};
return ParseResult_NewOk(req);
}
}
+46
View File
@@ -0,0 +1,46 @@
module Router {
import Std::Array::{Array};
import Std::String::{String_Eq};
import Http::{HttpMethod, HttpRequest, HttpResponse, Http_NewResponse};
import Handlers::{ServeStaticFile, HandleApiHealth, HandleApiInfo, HandleWebSocketUpgrade, NotFoundResponse};
pub enum Handler {
StaticFile,
ApiHealth,
ApiInfo,
WsUpgrade,
NotFound,
}
pub struct Route {
method: HttpMethod;
path: String;
handler: Handler;
}
pub struct Router {
routes: Array<Route>;
notFound: Handler;
}
pub func Handler_Handle(h: Handler, req: HttpRequest) -> HttpResponse {
match h {
Handler::StaticFile => ServeStaticFile(req),
Handler::ApiHealth => HandleApiHealth(),
Handler::ApiInfo => HandleApiInfo(),
Handler::WsUpgrade => HandleWebSocketUpgrade(req),
Handler::NotFound => NotFoundResponse(),
}
}
pub func Router_Dispatch(r: Router, req: HttpRequest) -> HttpResponse {
for route in r.routes {
if route.method == req.method && String_Eq(route.path, req.path) {
return Handler_Handle(route.handler, req);
}
}
return Handler_Handle(r.notFound, req);
}
}
+348
View File
@@ -0,0 +1,348 @@
module Server {
import Std::Io::{Print, PrintLine, PrintInt};
import Std::Net::{
Net_Create, Net_SetReuse, Net_Bind, Net_Listen, Net_Accept, Net_Send, Net_Recv, Net_Close, Net_LastError,
Tls_ServerCtx, Tls_ServerCtxMtls, Tls_CtxFree, Tls_Accept, Tls_Send, Tls_Recv, Tls_Close, Tls_LastError
};
import Std::String::{String_Len, String_StartsWith, String_Eq};
import Std::Channel::{Channel, Channel_New, Channel_Send, Channel_Recv};
import Std::Array::{Array_Drop};
import Std::Os::{Os_InstallStopHandlers, Os_ShouldStop, Os_SetStopListenFd};
import Std::Time::{Time_NowMs};
import Config::{ServerConfig};
import Http::{HttpRequest, HttpResponse, Http_StatusText, Http_NewResponse, Http_MethodName, Request_WantsKeepAlive, HeaderEntry};
import Errors::{ParseResult};
import Parser::{ParseRequest};
import Router::{Router, Router_Dispatch};
extern func bux_strlen(s: String) -> uint;
extern func bux_sb_new(initial_cap: uint) -> *void;
extern func bux_sb_append(sb: *void, s: String);
extern func bux_sb_append_int(sb: *void, n: int64);
extern func bux_sb_build(sb: *void) -> String;
extern func bux_sb_free(sb: *void);
/// Cap requests per TCP connection (safety + fair scheduling).
const MAX_KEEPALIVE_REQUESTS: int = 1000;
/// fd < 0 is a poison pill: worker exits cleanly.
/// tls is null for plain HTTP; non-null SSL* for HTTPS (session 78).
pub struct ConnectionTask {
fd: int;
tls: *void;
}
pub func BuildResponse(resp: HttpResponse, keepAlive: bool) -> String {
let sb: *void = bux_sb_new(4096);
bux_sb_append(sb, "HTTP/1.1 ");
bux_sb_append_int(sb, resp.statusCode as int64);
bux_sb_append(sb, " ");
bux_sb_append(sb, Http_StatusText(resp.statusCode));
bux_sb_append(sb, "\r\n");
bux_sb_append(sb, "Server: Nexus/0.6.0 (Bux)\r\n");
if bux_strlen(resp.extraHeaders) > 0 {
bux_sb_append(sb, resp.extraHeaders);
}
if bux_strlen(resp.contentType) > 0 {
bux_sb_append(sb, "Content-Type: ");
bux_sb_append(sb, resp.contentType);
bux_sb_append(sb, "\r\n");
}
let bodyLen: uint = bux_strlen(resp.body);
bux_sb_append(sb, "Content-Length: ");
bux_sb_append_int(sb, bodyLen as int64);
bux_sb_append(sb, "\r\n");
if keepAlive {
bux_sb_append(sb, "Connection: keep-alive\r\n");
bux_sb_append(sb, "Keep-Alive: timeout=5, max=1000\r\n");
} else {
bux_sb_append(sb, "Connection: close\r\n");
}
bux_sb_append(sb, "\r\n");
if bodyLen > 0 {
bux_sb_append(sb, resp.body);
}
let result: String = bux_sb_build(sb);
bux_sb_free(sb);
return result;
}
func ConnRecv(fd: int, tls: *void, maxLen: int) -> String {
if tls != null as *void {
return Tls_Recv(tls, maxLen);
}
return Net_Recv(fd, maxLen);
}
func ConnSend(fd: int, tls: *void, data: String) -> int {
if tls != null as *void {
return Tls_Send(tls, data);
}
return Net_Send(fd, data);
}
/// Structured access log: method path status duration_ms
func AccessLog(method: String, path: String, status: int, ms: int64) {
Print(method);
Print(" ");
Print(path);
Print(" ");
PrintInt(status);
Print(" ");
PrintInt(ms as int);
PrintLine("ms");
}
/// Serve one TCP (or TLS) client: zero or more HTTP/1.1 keep-alive requests.
pub func HandleConnection(fd: int, tls: *void, router: Router, config: ServerConfig) {
var reqCount: int = 0;
let maxRecv: int = config.maxBodyBytes;
while reqCount < MAX_KEEPALIVE_REQUESTS {
if Os_ShouldStop() {
return;
}
let raw: String = ConnRecv(fd, tls, maxRecv);
if String_Len(raw) == 0 {
return;
}
if String_Len(raw) as int >= maxRecv {
let resp: HttpResponse = Http_NewResponse(413, "text/plain; charset=utf-8", "Payload Too Large");
ConnSend(fd, tls, BuildResponse(resp, false));
return;
}
if String_StartsWith(raw, "PRI * HTTP/2.0") {
let resp: HttpResponse = Http_NewResponse(200, "text/plain; charset=utf-8",
"HTTP/2 detected — full support planned for future release.\r\n");
ConnSend(fd, tls, BuildResponse(resp, false));
return;
}
let t0: int64 = Time_NowMs();
let parsed: ParseResult = ParseRequest(raw);
var keepAlive: bool = false;
var statusOut: int = 400;
var methodName: String = "?";
var pathOut: String = "-";
if parsed.tag == ParseResult_Ok {
var req: HttpRequest = parsed.data.Ok_0;
methodName = Http_MethodName(req.method);
pathOut = req.path;
if String_Len(req.body) as int > maxRecv {
let resp: HttpResponse = Http_NewResponse(413, "text/plain; charset=utf-8", "Payload Too Large");
ConnSend(fd, tls, BuildResponse(resp, false));
Array_Drop<HeaderEntry>(&req.headers);
return;
}
keepAlive = Request_WantsKeepAlive(&req);
if reqCount + 1 >= MAX_KEEPALIVE_REQUESTS {
keepAlive = false;
}
let resp: HttpResponse = Router_Dispatch(router, req);
statusOut = resp.statusCode;
ConnSend(fd, tls, BuildResponse(resp, keepAlive));
Array_Drop<HeaderEntry>(&req.headers);
} else {
let resp: HttpResponse = Http_NewResponse(400, "text/plain; charset=utf-8", "Bad Request");
statusOut = 400;
ConnSend(fd, tls, BuildResponse(resp, false));
if config.accessLog {
let dt: int64 = Time_NowMs() - t0;
AccessLog("?", "-", 400, dt);
}
return;
}
if config.accessLog {
let dt: int64 = Time_NowMs() - t0;
AccessLog(methodName, pathOut, statusOut, dt);
}
reqCount = reqCount + 1;
if !keepAlive {
return;
}
}
}
pub struct WorkerCtx {
taskQueue: *Channel<ConnectionTask>;
router: Router;
config: ServerConfig;
}
pub func Worker(ctx: *WorkerCtx) {
while true {
let task: ConnectionTask = Channel_Recv<ConnectionTask>(ctx.taskQueue);
if task.fd < 0 {
return;
}
HandleConnection(task.fd, task.tls, ctx.router, ctx.config);
if task.tls != null as *void {
Tls_Close(task.tls);
}
Net_Close(task.fd);
}
}
pub struct AcceptorCtx {
serverFd: int;
taskQueue: *Channel<ConnectionTask>;
workerCount: int;
tlsCtx: *void;
}
pub func Acceptor(ctx: *AcceptorCtx) {
while !Os_ShouldStop() {
let fd: int = Net_Accept(ctx.serverFd);
if fd >= 0 {
var tls: *void = null as *void;
if ctx.tlsCtx != null as *void {
tls = Tls_Accept(ctx.tlsCtx, fd);
if tls == null as *void {
Print("WARN: TLS handshake failed: ");
PrintLine(Tls_LastError());
Net_Close(fd);
continue;
}
}
let task: ConnectionTask = ConnectionTask { fd: fd, tls: tls };
Channel_Send<ConnectionTask>(ctx.taskQueue, task);
} else {
if Os_ShouldStop() {
break;
}
}
}
var i: int = 0;
while i < ctx.workerCount {
let poison: ConnectionTask = ConnectionTask { fd: -1, tls: null as *void };
Channel_Send<ConnectionTask>(ctx.taskQueue, poison);
i = i + 1;
}
PrintLine("acceptor: stop — workers poisoned");
}
pub func RunServer(config: ServerConfig, router: Router) -> int {
PrintLine("================================================");
PrintLine(" Nexus HTTP Server v0.6.0");
PrintLine(" HTTP/1.1 + TLS/mTLS + keep-alive + graceful stop");
PrintLine(" Built with Bux");
PrintLine("================================================");
PrintLine("");
Os_InstallStopHandlers();
var tlsCtx: *void = null as *void;
if config.tlsEnabled {
if String_Eq(config.tlsCertPath, "") || String_Eq(config.tlsKeyPath, "") {
PrintLine("FATAL: TLS enabled but NEXUS_TLS_CERT / NEXUS_TLS_KEY not set");
return 1;
}
if !String_Eq(config.tlsClientCaPath, "") {
tlsCtx = Tls_ServerCtxMtls(config.tlsCertPath, config.tlsKeyPath, config.tlsClientCaPath);
PrintLine("mTLS: client certificates required");
Print("Client CA: ");
PrintLine(config.tlsClientCaPath);
} else {
tlsCtx = Tls_ServerCtx(config.tlsCertPath, config.tlsKeyPath);
}
if tlsCtx == null as *void {
Print("FATAL: TLS context failed: ");
PrintLine(Tls_LastError());
return 1;
}
Print("TLS cert: ");
PrintLine(config.tlsCertPath);
}
let serverFd: int = Net_Create();
if serverFd < 0 {
PrintLine("FATAL: socket() failed");
if tlsCtx != null as *void { Tls_CtxFree(tlsCtx); }
return 1;
}
if !Net_SetReuse(serverFd) {
PrintLine("WARN: SO_REUSEADDR failed");
}
if !Net_Bind(serverFd, config.bindAddr, config.port) {
Print("FATAL: bind failed: ");
PrintLine(Net_LastError());
Net_Close(serverFd);
if tlsCtx != null as *void { Tls_CtxFree(tlsCtx); }
return 1;
}
if !Net_Listen(serverFd, config.backlog) {
PrintLine("FATAL: listen() failed");
Net_Close(serverFd);
if tlsCtx != null as *void { Tls_CtxFree(tlsCtx); }
return 1;
}
Os_SetStopListenFd(serverFd);
if config.tlsEnabled {
Print("Listening on https://");
} else {
Print("Listening on http://");
}
Print(config.bindAddr);
Print(":");
PrintInt(config.port);
PrintLine("");
PrintInt(config.workerCount);
Print(" worker threads | keep-alive: on | max-body: ");
PrintInt(config.maxBodyBytes);
PrintLine(" B");
PrintLine("Endpoints: / /api/health /api/info /ws");
PrintLine("Stop: Ctrl+C / SIGTERM (graceful).");
PrintLine("");
let taskQueue: Channel<ConnectionTask> = Channel_New<ConnectionTask>(config.backlog as int64);
let workerCtx: WorkerCtx = WorkerCtx {
taskQueue: &taskQueue,
router: router,
config: config
};
let acceptorCtx: AcceptorCtx = AcceptorCtx {
serverFd: serverFd,
taskQueue: &taskQueue,
workerCount: config.workerCount,
tlsCtx: tlsCtx
};
var i: int = 0;
while i < config.workerCount {
spawn Worker(&workerCtx);
i = i + 1;
}
Acceptor(&acceptorCtx);
Os_SetStopListenFd(-1);
if serverFd >= 0 {
Net_Close(serverFd);
}
if tlsCtx != null as *void {
Tls_CtxFree(tlsCtx);
}
PrintLine("nexus: acceptor stopped — exit");
return 0;
}
}
+2
View File
@@ -0,0 +1,2 @@
bux 0.1.0 (bootstrap)
EXIT:0
+76
View File
@@ -0,0 +1,76 @@
# SimpleDB
File-backed key-value database for Bux.
## Usage
```
simpledb <dbfile> set <key> <value>
simpledb <dbfile> get <key>
simpledb <dbfile> del <key>
simpledb <dbfile> has <key>
simpledb <dbfile> keys
simpledb <dbfile> count
```
## Examples
```sh
$ simpledb data.db set name BuxLang
OK name = BuxLang
$ simpledb data.db set version 0.1.0
OK version = 0.1.0
$ simpledb data.db get name
BuxLang
$ simpledb data.db has name
true name
$ simpledb data.db keys
keys: 2
name
version
$ simpledb data.db count
count: 2
$ simpledb data.db del version
DEL version
$ cat data.db
name=BuxLang
```
## Storage
Data is stored as plain text, one `key=value` per line. The file is created automatically on first write.
## Build
```sh
cd apps/simpledb
../../buxc build
./build/simpledb data.db set hello world
```
## API
The `Database` struct and its functions are defined in `src/Main.bux` and can be imported into other Bux projects:
```bux
import Simpledb::{Database, DB_New, DB_Load, DB_Save, DB_Get, DB_Set, DB_Del, DB_Has, DB_Count, DB_Keys};
```
| Function | Signature |
|----------|-----------|
| `DB_New` | `(path: String) -> Database` |
| `DB_Load` | `(db: *Database) -> bool` |
| `DB_Save` | `(db: *Database) -> bool` |
| `DB_Get` | `(db: *Database, key: String) -> String` |
| `DB_Set` | `(db: *Database, key: String, value: String)` |
| `DB_Del` | `(db: *Database, key: String) -> bool` |
| `DB_Has` | `(db: *Database, key: String) -> bool` |
| `DB_Count` | `(db: *Database) -> uint` |
| `DB_Keys` | `(db: *Database) -> *String` |
+8
View File
@@ -0,0 +1,8 @@
[Package]
Name = "simpledb"
Version = "0.1.0"
Type = "bin"
Description = "SimpleDB — file-backed JSON key-value database for Bux"
[Build]
Output = "Bin"
+339
View File
@@ -0,0 +1,339 @@
module Main {
import Std::Io::{PrintLine, Print, PrintInt, ReadFile, WriteFile, FileExists};
import Std::Os::{Os_ArgsCount, Os_Args};
import Std::String::{String_Len, String_Eq, String_StartsWith, String_Find, String_Slice, String_Offset};
extern func bux_strlen(s: String) -> uint;
extern func bux_strstr(haystack: String, needle: String) -> String;
extern func bux_str_slice(s: String, start: uint, len: uint) -> String;
extern func bux_str_split_count(s: String, delim: String) -> uint;
extern func bux_str_split_part(s: String, delim: String, index: uint) -> String;
extern func bux_sb_new(initial_cap: uint) -> *void;
extern func bux_sb_append(sb: *void, s: String);
extern func bux_sb_append_char(sb: *void, c: char8);
func SB_AppendChar(sb: *void, c: int) {
bux_sb_append_char(sb, c as char8);
}
extern func bux_sb_build(sb: *void) -> String;
extern func bux_sb_free(sb: *void);
extern func bux_alloc(size: uint) -> *void;
extern func bux_free(ptr: *void);
struct Database {
path: String,
sb: *void,
loaded: bool,
count: uint,
}
func DB_New(filePath: String) -> Database {
var db: Database;
db.path = filePath;
db.sb = bux_sb_new(4096);
db.loaded = false;
db.count = 0;
return db;
}
func DB_Load(self: *Database) -> bool {
if self.loaded { return true; }
if !FileExists(self.path) {
self.loaded = true;
return true;
}
let raw: String = ReadFile(self.path);
if String_Len(raw) == 0 || bux_strlen(raw) == 0 {
self.loaded = true;
return true;
}
bux_sb_append(self.sb, raw);
let lineCount: uint = bux_str_split_count(raw, "\n");
var i: uint = 0;
var n: uint = 0;
while i < lineCount {
let line: String = bux_str_split_part(raw, "\n", i);
if bux_strlen(line) > 0 {
n = n + 1;
}
i = i + 1;
}
self.count = n;
self.loaded = true;
return true;
}
func DB_Get(self: *Database, key: String) -> String {
let raw: String = bux_sb_build(self.sb);
let lineCount: uint = bux_str_split_count(raw, "\n");
var i: uint = 0;
while i < lineCount {
let line: String = bux_str_split_part(raw, "\n", i);
if String_StartsWith(line, key) {
let eqPos: String = String_Find(line, "=");
if String_Len(eqPos) > 0 {
let keyLen: uint = bux_strlen(key);
let valStart: uint = keyLen + 1;
let lineLen: uint = bux_strlen(line);
if valStart < lineLen {
return bux_str_slice(line, valStart, lineLen - valStart);
}
}
}
i = i + 1;
}
return "";
}
func DB_Set(self: *Database, key: String, value: String) {
let raw: String = bux_sb_build(self.sb);
let lineCount: uint = bux_str_split_count(raw, "\n");
var newSb: *void = bux_sb_new(4096);
var found: bool = false;
var i: uint = 0;
while i < lineCount {
let line: String = bux_str_split_part(raw, "\n", i);
if bux_strlen(line) > 0 {
if String_StartsWith(line, key) {
let eqPos: String = String_Find(line, "=");
if String_Len(eqPos) > 0 {
bux_sb_append(newSb, key);
SB_AppendChar(newSb, 61);
bux_sb_append(newSb, value);
SB_AppendChar(newSb, 10);
found = true;
} else {
bux_sb_append(newSb, line);
SB_AppendChar(newSb, 10);
}
} else {
bux_sb_append(newSb, line);
SB_AppendChar(newSb, 10);
}
}
i = i + 1;
}
if !found {
bux_sb_append(newSb, key);
SB_AppendChar(newSb, 61);
bux_sb_append(newSb, value);
SB_AppendChar(newSb, 10);
self.count = self.count + 1;
}
bux_sb_free(self.sb);
self.sb = newSb;
}
func DB_Del(self: *Database, key: String) {
let raw: String = bux_sb_build(self.sb);
let lineCount: uint = bux_str_split_count(raw, "\n");
var newSb: *void = bux_sb_new(4096);
var i: uint = 0;
var delCount: uint = 0;
while i < lineCount {
let line: String = bux_str_split_part(raw, "\n", i);
if bux_strlen(line) > 0 {
if String_StartsWith(line, key) {
let eqPos: String = String_Find(line, "=");
if String_Len(eqPos) > 0 {
delCount = delCount + 1;
} else {
bux_sb_append(newSb, line);
SB_AppendChar(newSb, 10);
}
} else {
bux_sb_append(newSb, line);
SB_AppendChar(newSb, 10);
}
}
i = i + 1;
}
if delCount > 0 {
self.count = self.count - delCount;
}
bux_sb_free(self.sb);
self.sb = newSb;
}
func DB_Has(self: *Database, key: String) -> bool {
let val: String = DB_Get(self, key);
return bux_strlen(val) > 0;
}
func DB_Count(self: *Database) -> uint {
return self.count;
}
func DB_Keys(self: *Database) -> *String {
if self.count == 0 { return null as *String; }
let raw: String = bux_sb_build(self.sb);
let lineCount: uint = bux_str_split_count(raw, "\n");
let keys: *String = bux_alloc(self.count * sizeof(String)) as *String;
var found: uint = 0;
var i: uint = 0;
while i < lineCount && found < self.count {
let line: String = bux_str_split_part(raw, "\n", i);
if bux_strlen(line) > 0 {
let eqPos: String = String_Find(line, "=");
if String_Len(eqPos) > 0 {
let keyLen: uint = String_Offset(eqPos, line);
keys[found] = bux_str_slice(line, 0, keyLen);
found = found + 1;
}
}
i = i + 1;
}
return keys;
}
func DB_Save(self: *Database) -> bool {
let content: String = bux_sb_build(self.sb);
let ok: bool = WriteFile(self.path, content);
return ok;
}
func PrintUsage() {
PrintLine("SimpleDB — file-backed key-value database");
PrintLine("Usage:");
PrintLine(" simpledb <dbfile> set <key> <value>");
PrintLine(" simpledb <dbfile> get <key>");
PrintLine(" simpledb <dbfile> del <key>");
PrintLine(" simpledb <dbfile> has <key>");
PrintLine(" simpledb <dbfile> keys");
PrintLine(" simpledb <dbfile> count");
}
func Main() -> int {
let argc: int = Os_ArgsCount();
if argc < 3 {
PrintUsage();
return 1;
}
let dbFile: String = Os_Args(1);
let cmd: String = Os_Args(2);
var db: Database = DB_New(dbFile);
if String_Eq(cmd, "get") {
DB_Load(&db);
if argc < 4 {
PrintLine("get: missing key");
return 1;
}
let key: String = Os_Args(3);
let val: String = DB_Get(&db, key);
if bux_strlen(val) > 0 {
PrintLine(val);
} else {
Print("(not found) ");
PrintLine(key);
}
return 0;
}
if String_Eq(cmd, "set") {
DB_Load(&db);
if argc < 5 {
PrintLine("set: missing key or value");
return 1;
}
let key: String = Os_Args(3);
let val: String = Os_Args(4);
DB_Set(&db, key, val);
let saved: bool = DB_Save(&db);
if saved {
Print("OK ");
Print(key);
Print(" = ");
PrintLine(val);
} else {
PrintLine("set: save failed");
}
return 0;
}
if String_Eq(cmd, "del") {
DB_Load(&db);
if argc < 4 {
PrintLine("del: missing key");
return 1;
}
let key: String = Os_Args(3);
DB_Del(&db, key);
DB_Save(&db);
Print("DEL ");
PrintLine(key);
return 0;
}
if String_Eq(cmd, "has") {
DB_Load(&db);
if argc < 4 {
PrintLine("has: missing key");
return 1;
}
let key: String = Os_Args(3);
let found: bool = DB_Has(&db, key);
if found {
Print("true ");
PrintLine(key);
} else {
Print("false ");
PrintLine(key);
}
return 0;
}
if String_Eq(cmd, "keys") {
DB_Load(&db);
let n: uint = DB_Count(&db);
Print("keys: ");
PrintInt(n as int);
PrintLine("");
if n == 0 {
return 0;
}
let list: *String = DB_Keys(&db);
var i: uint = 0;
while i < n {
Print(" ");
PrintLine(list[i]);
i = i + 1;
}
bux_free(list as *void);
return 0;
}
if String_Eq(cmd, "count") {
DB_Load(&db);
let n: uint = DB_Count(&db);
Print("count: ");
PrintInt(n as int);
PrintLine("");
return 0;
}
Print("unknown command: ");
PrintLine(cmd);
return 1;
}
}
+58
View File
@@ -0,0 +1,58 @@
# Bux benchmarks (E.5)
Micro-kernels + optional Nexus HTTP throughput for regression and language comparison.
## Quick run
```bash
# From repo root
make bench # Bux micro + C + Nim (+ Zig if installed)
make bench-nexus # wrk vs apps/nexus /api/health
BENCH_NEXUS=1 make bench
```
## Micro suites
| Suite | Kernels | Build |
|-------|---------|-------|
| `micro/` | Bux: `int_loop`, `fib30`, `string_concat`, `array_push` | `buxc build` |
| `c/` | C: `fib30`, `int_loop` | `gcc -O2` |
| `nim/` | Nim twins | `nim c -d:release --opt:speed` |
| `zig/` | Zig twins (optional) | `zig build-exe -OReleaseFast` |
Output lines:
```
BENCH <name> iters=N total_us=T us_per_op=P
```
## Nexus throughput
`tools/bench_nexus.sh` / `make bench-nexus`:
1. Builds `apps/nexus`
2. Starts on `NEXUS_PORT` (default **18080**), bind `127.0.0.1`
3. `wrk -t4 -c64 -d5s` against `/api/health`
4. Prints `BENCH nexus_health rps=…`
Env knobs:
| Variable | Default | Meaning |
|----------|---------|---------|
| `NEXUS_PORT` | `18080` | Listen port (also used by the server binary) |
| `NEXUS_WORKERS` | `4` | Worker threads |
| `NEXUS_BENCH_DURATION` | `5s` | wrk `-d` |
| `NEXUS_BENCH_C` | `64` | wrk connections |
| `NEXUS_BENCH_T` | `4` | wrk threads |
Server-side (any nexus run):
- `NEXUS_PORT`, `NEXUS_WORKERS`, `NEXUS_BIND`, `NEXUS_PUBLIC`
## Notes
- Numbers vary by machine; use them relatively (same host, same day).
- Nexus **v0.3** uses HTTP/1.1 keep-alive (default). Sample RPS on one machine:
- close-only era: ~45k req/s
- keep-alive: ~80k+ req/s (`wrk -t4 -c64 -d5s /api/health`)
- Requires `wrk` for `bench-nexus` (`apt install wrk` on Debian/Ubuntu).
+26
View File
@@ -0,0 +1,26 @@
/* C reference: recursive fib(30) — compare with benches/micro fib30 */
#include <stdio.h>
#include <stdint.h>
#include <time.h>
static int fib(int n) {
if (n <= 1) return n;
return fib(n - 1) + fib(n - 2);
}
static int64_t now_us(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (int64_t)ts.tv_sec * 1000000LL + (int64_t)ts.tv_nsec / 1000LL;
}
int main(void) {
(void)fib(20); /* warmup */
int64_t t0 = now_us();
int r = fib(30);
int64_t t1 = now_us();
if (r < 0) return 1;
printf("BENCH fib30 iters=1 total_us=%lld us_per_op=%lld\n",
(long long)(t1 - t0), (long long)(t1 - t0));
return 0;
}
+24
View File
@@ -0,0 +1,24 @@
/* C reference: 1e6 integer arithmetic loop */
#include <stdio.h>
#include <stdint.h>
#include <time.h>
static int64_t now_us(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (int64_t)ts.tv_sec * 1000000LL + (int64_t)ts.tv_nsec / 1000LL;
}
int main(void) {
const int iters = 1000000;
int acc = 0;
int64_t t0 = now_us();
for (int i = 0; i < iters; i++) {
acc = acc + i * 3 - 1;
}
int64_t t1 = now_us();
if (acc == 0 && iters > 0) return 1;
printf("BENCH int_loop iters=%d total_us=%lld us_per_op=%lld\n",
iters, (long long)(t1 - t0), (long long)((t1 - t0) / iters));
return 0;
}
+8
View File
@@ -0,0 +1,8 @@
[Package]
Name = "micro"
Version = "0.1.0"
Type = "bin"
Description = "Micro-benchmarks for Bux (E.5)"
[Build]
Output = "Bin"
+104
View File
@@ -0,0 +1,104 @@
// Micro-benchmarks (E.5) — recursive fib, string concat, array push.
// Prints lines: BENCH name iters total_us us_per_op
import Std::Io::{PrintLine, Print};
import Std::Time::{Time_NowUs};
import Std::String::{String_FromInt, String_Concat, String_Len};
import Std::Array::{Array, Array_New, Array_Push, Array_Len, Array_Get};
func Fib(n: int) -> int {
if n <= 1 {
return n;
}
return Fib(n - 1) + Fib(n - 2);
}
func Report(name: String, iters: int, usTotal: int64) {
var perOpUs: int64 = 0;
if iters > 0 {
perOpUs = usTotal / (iters as int64);
}
Print("BENCH ");
Print(name);
Print(" iters=");
Print(String_FromInt(iters as int64));
Print(" total_us=");
Print(String_FromInt(usTotal));
Print(" us_per_op=");
PrintLine(String_FromInt(perOpUs));
}
func BenchIntLoop() {
let iters: int = 1000000;
var acc: int = 0;
var i: int = 0;
let t0: int64 = Time_NowUs();
while i < iters {
acc = acc + i * 3 - 1;
i = i + 1;
}
let t1: int64 = Time_NowUs();
if acc == 0 && iters > 0 {
PrintLine("unreachable");
}
Report("int_loop", iters, t1 - t0);
}
func BenchFib() {
discard Fib(20);
let n: int = 30;
let t0: int64 = Time_NowUs();
let r: int = Fib(n);
let t1: int64 = Time_NowUs();
if r < 0 {
PrintLine("unreachable");
}
Report("fib30", 1, t1 - t0);
}
func BenchStringConcat() {
let iters: int = 50000;
var i: int = 0;
let t0: int64 = Time_NowUs();
var s: String = "";
while i < iters {
s = String_Concat(s, "x");
if String_Len(s) > 256 {
s = "x";
}
i = i + 1;
}
let t1: int64 = Time_NowUs();
if String_Len(s) < 0 {
PrintLine("unreachable");
}
Report("string_concat", iters, t1 - t0);
}
func BenchArrayPush() {
let iters: int = 100000;
var arr: Array<int> = Array_New<int>(16);
var i: int = 0;
let t0: int64 = Time_NowUs();
while i < iters {
Array_Push<int>(&arr, i);
i = i + 1;
}
let t1: int64 = Time_NowUs();
let n: uint = Array_Len<int>(&arr);
if n as int != iters {
Print("array len mismatch: ");
PrintLine(String_FromInt(n as int64));
}
discard Array_Get<int>(&arr, (iters - 1) as uint);
Report("array_push", iters, t1 - t0);
}
func Main() -> int {
PrintLine("=== Bux micro-benchmarks (E.5) ===");
BenchIntLoop();
BenchFib();
BenchStringConcat();
BenchArrayPush();
PrintLine("=== done ===");
return 0;
}
+16
View File
@@ -0,0 +1,16 @@
## Nim twin of benches/c/fib.c — recursive fib(30)
import std/[times, strformat]
proc fib(n: int): int =
if n <= 1: return n
fib(n - 1) + fib(n - 2)
proc nowUs(): int64 =
int64(epochTime() * 1_000_000.0)
discard fib(20)
let t0 = nowUs()
let r = fib(30)
let t1 = nowUs()
if r < 0: quit(1)
echo &"BENCH fib30 iters=1 total_us={t1 - t0} us_per_op={t1 - t0}"
+15
View File
@@ -0,0 +1,15 @@
## Nim twin of benches/c/int_loop.c — 1e6 integer arithmetic loop
import std/[times, strformat]
proc nowUs(): int64 =
int64(epochTime() * 1_000_000.0)
const iters = 1_000_000
var acc = 0
let t0 = nowUs()
for i in 0 ..< iters:
acc = acc + i * 3 - 1
let t1 = nowUs()
if acc == 0 and iters > 0: quit(1)
let total = t1 - t0
echo &"BENCH int_loop iters={iters} total_us={total} us_per_op={total div iters}"
+22
View File
@@ -0,0 +1,22 @@
// Zig twin of benches/c/fib.c — recursive fib(30)
// Build: zig build-exe -OReleaseFast -femit-bin=build/fib fib.zig
const std = @import("std");
fn fib(n: i32) i32 {
if (n <= 1) return n;
return fib(n - 1) + fib(n - 2);
}
fn nowUs() i64 {
return @divTrunc(@as(i64, @intCast(std.time.nanoTimestamp())), 1000);
}
pub fn main() !void {
_ = fib(20);
const t0 = nowUs();
const r = fib(30);
const t1 = nowUs();
if (r < 0) return error.Unreachable;
const out = std.io.getStdOut().writer();
try out.print("BENCH fib30 iters=1 total_us={d} us_per_op={d}\n", .{ t1 - t0, t1 - t0 });
}
+22
View File
@@ -0,0 +1,22 @@
// Zig twin of benches/c/int_loop.c — 1e6 integer arithmetic loop
// Build: zig build-exe -OReleaseFast -femit-bin=build/int_loop int_loop.zig
const std = @import("std");
fn nowUs() i64 {
return @divTrunc(@as(i64, @intCast(std.time.nanoTimestamp())), 1000);
}
pub fn main() !void {
const iters: i32 = 1_000_000;
var acc: i32 = 0;
const t0 = nowUs();
var i: i32 = 0;
while (i < iters) : (i += 1) {
acc = acc + i * 3 - 1;
}
const t1 = nowUs();
if (acc == 0 and iters > 0) return error.Unreachable;
const total = t1 - t0;
const out = std.io.getStdOut().writer();
try out.print("BENCH int_loop iters={d} total_us={d} us_per_op={d}\n", .{ iters, total, @divTrunc(total, iters) });
}
+98
View File
@@ -33,6 +33,7 @@ type
tekDynRef ## &dyn Trait — trait object (fat pointer)
tekTuple
tekSelf
tekFunc
TypeExpr* = ref object
loc*: SourceLocation
@@ -54,6 +55,9 @@ type
tupleElements*: seq[TypeExpr]
of tekSelf:
discard
of tekFunc:
funcParams*: seq[TypeExpr]
funcRet*: TypeExpr
# ---------------------------------------------------------------------------
# Patterns
@@ -124,8 +128,17 @@ type
ekUnwrap ## expr! — unwrap or panic
ekSpawn ## spawn expr — create a new task
ekAwait ## expr.await — suspend until future resolves
ekBorrow ## borrow &mut expr — explicit borrow expression
ekBlock
ekMatch
ekStringInterp
ekClosure
ekMacroCall ## name!(args) — expanded before sema
ekMacroStmt ## `$s:stmt` arg wrapper (expand only)
ekMacroPat ## `$p:pat` arg wrapper (expand only)
ekMacroTt ## `$x:tt` bound fragment (expand only; may flatten groups)
ekMacroRep ## `$( expr ),*` expression-level rep in templates (expand only)
ekMacroType ## `$t:type` bound type (expand only)
MatchArm* = object
loc*: SourceLocation
@@ -172,6 +185,7 @@ type
of ekCall:
exprCallCallee*: Expr
exprCallArgs*: seq[Expr]
exprCallArgNames*: seq[string] ## empty = positional, non-empty = named arg
exprCallInferredTypeArgs*: seq[TypeExpr] ## filled by sema for inferred generic calls
of ekGenericCall:
exprGenericCallee*: string
@@ -210,11 +224,48 @@ type
exprSpawnAsync*: bool
of ekAwait:
exprAwaitOperand*: Expr
of ekBorrow:
exprBorrowOperand*: Expr
exprBorrowMutable*: bool
of ekBlock:
exprBlock*: Block
of ekMatch:
exprMatchSubject*: Expr
exprMatchArms*: seq[MatchArm]
of ekStringInterp:
exprInterpTexts*: seq[string]
exprInterpExprs*: seq[Expr]
of ekClosure:
exprClosureParams*: seq[Param]
exprClosureBody*: Block
exprClosureReturnType*: TypeExpr
captureCount*: int
captureNames*: seq[string]
captureTypeKinds*: seq[int]
of ekMacroCall:
exprMacroName*: string
exprMacroArgs*: seq[Expr]
## Group lengths for multi-rep: `m!(1,2; 3,4)` → @[2, 2].
## Empty means a single group of all args.
exprMacroGroupLens*: seq[int]
of ekMacroStmt:
## Statement fragment argument (`$s:stmt`) — only during expand
exprMacroStmt*: Stmt
of ekMacroPat:
## Pattern fragment argument (`$p:pat`) — only during expand
exprMacroPat*: Pattern
of ekMacroTt:
## Bound `:tt` fragment. `exprMacroTtGroup` is true for parenthesized
## multi-element groups (parsed as tuples) — flattened when spliced as
## the sole argument of a call (`$f($args)` → `f(a, b)`).
exprMacroTtInner*: Expr
exprMacroTtGroup*: bool
of ekMacroRep:
## Expression-level `$( body ),*` / `$( body )*` in macro templates.
exprMacroRepBody*: Expr
of ekMacroType:
## Bound `:type` fragment.
exprMacroType*: TypeExpr
# ---------------------------------------------------------------------------
# Statements
@@ -234,13 +285,21 @@ type
skStaticAssert
skComptime
skEmit
skDefer
skSwitch
skDecl
skMacroRep ## $( … )* template repetition (macro body only)
ElseIf* = object
loc*: SourceLocation
cond*: Expr
blk*: Block
SwitchCase* = object
loc*: SourceLocation
caseValue*: Expr
caseBody*: Block
Block* = ref object
loc*: SourceLocation
stmts*: seq[Stmt]
@@ -294,8 +353,16 @@ type
of skEmit:
stmtEmitExpr*: Expr
stmtEmitEvaluated*: string ## filled by sema CTFE
of skDefer:
stmtDeferBody*: Expr
of skSwitch:
stmtSwitchExpr*: Expr
stmtSwitchCases*: seq[SwitchCase]
stmtSwitchDefault*: Block
of skDecl:
stmtDecl*: Decl
of skMacroRep: ## $( stmts… )* in macro templates
stmtMacroRepBody*: Block
# ---------------------------------------------------------------------------
# Type Parameters (for generics with trait bounds)
@@ -322,6 +389,31 @@ type
dkExternFunc
dkExternVar
dkExternBlock
dkMacro ## macro! name { ($x:expr) => { … } }
## One declarative macro arm: ($a:expr, $($x:expr),*) => { template }
MacroFragKind* = enum
mfkExpr ## any expression
mfkIdent ## bare identifier (after expand must be ekIdent)
mfkTt ## token-tree (MVP: same as expr)
mfkLiteral ## int/float/string/char/bool literal only
mfkBlock ## block expression `{ … }`
mfkStmt ## one statement (let/if/… or expression-stmt)
mfkPat ## match/let pattern
mfkType ## type expression (`int`, `*int`, `String`, …)
MacroFragment* = object
name*: string ## primary / first name (compat)
kind*: MacroFragKind ## primary kind (compat)
names*: seq[string] ## one or more $names (compound rep: $a,$b)
kinds*: seq[MacroFragKind] ## parallel to names
isRep*: bool ## true for $( … ),* or $( … )*
repSep*: string ## "," if separator was present before *, else ""
MacroRule* = object
loc*: SourceLocation
frags*: seq[MacroFragment]
body*: Block ## template (substituted, then used as ekBlock)
Param* = object
loc*: SourceLocation
@@ -414,6 +506,9 @@ type
declExtBlockDll*: string
declExtBlockCallConv*: CallingConvention
declExtBlockItems*: seq[Decl]
of dkMacro:
declMacroName*: string
declMacroRules*: seq[MacroRule]
# ---------------------------------------------------------------------------
# Module (AST root)
@@ -441,3 +536,6 @@ proc newBinaryExpr*(op: TokenKind, left, right: Expr, loc: SourceLocation): Expr
proc newUnaryExpr*(op: TokenKind, operand: Expr, loc: SourceLocation): Expr =
result = Expr(kind: ekUnary, loc: loc, exprUnaryOp: op, exprUnaryOperand: operand)
proc newStringInterpExpr*(texts: seq[string], exprs: seq[Expr], loc: SourceLocation): Expr =
result = Expr(kind: ekStringInterp, loc: loc, exprInterpTexts: texts, exprInterpExprs: exprs)
+112 -22
View File
@@ -8,6 +8,7 @@ type
varCounter*: int
declaredVars*: seq[string]
sliceTypeDefs*: seq[tuple[name: string, elem: string]] ## Generated Slice_<T> typedefs
deferStack*: seq[seq[string]] ## Function-level deferred statements (LIFO)
proc cEscape(s: string): string =
## Escape a string for use as a C string literal.
@@ -45,6 +46,28 @@ proc freshVar(be: var CBackend): string =
inc be.varCounter
result = &"__tmp_{be.varCounter}"
# ---------------------------------------------------------------------------
# Defer helpers
# ---------------------------------------------------------------------------
proc pushDeferScope(be: var CBackend) =
be.deferStack.add(@[])
proc popDeferScope(be: var CBackend) =
if be.deferStack.len > 0:
be.deferStack.setLen(be.deferStack.len - 1)
proc emitCurrentDefers(be: var CBackend) =
## Emit all deferred statements in current scope, LIFO order
if be.deferStack.len == 0: return
let idx = be.deferStack.len - 1
for i in countdown(be.deferStack[idx].len - 1, 0):
be.emitLine(be.deferStack[idx][i])
proc clearCurrentDefers(be: var CBackend) =
if be.deferStack.len > 0:
let idx = be.deferStack.len - 1
be.deferStack[idx].setLen(0)
# Type conversion: Bux Type → C type string
proc typeToC*(be: var CBackend, typ: Type): string =
if typ == nil: return "void"
@@ -105,8 +128,31 @@ proc typeToC*(be: var CBackend, typ: Type): string =
of "float64": return "double"
of "bool": return "bool"
else: return typ.name
of tkTuple: return "void*" # TODO: proper tuple struct
of tkFunc: return "void*" # TODO: function pointer
of tkTuple:
if typ.inner.len == 0:
return "Tuple_Empty"
var parts: seq[string] = @[]
for e in typ.inner:
var p = typeToC(be, e)
p = p.replace("const char*", "cstr").replace("unsigned int", "uint")
p = p.replace(" ", "_").replace("*", "Ptr").replace("(", "").replace(")", "").replace(",", "_")
parts.add(p)
let tname = "Tuple_" & parts.join("_")
# Ensure typedef is collected alongside slices
var already = false
for d in be.sliceTypeDefs:
if d.name == tname:
already = true
break
if not already:
# Reuse sliceTypeDefs as a generic "extra typedef" bag: elem holds field list markup
be.sliceTypeDefs.add((name: tname, elem: "/*tuple*/"))
return tname
of tkFunc:
if typ.inner.len == 0: return "void (*)(void)"
let params = typ.inner[0..^2].mapIt(typeToC(be, it)).join(", ")
let ret = typeToC(be, typ.inner[^1])
return ret & " (*)(" & params & ")"
else:
when defined(release):
return "void*"
@@ -249,6 +295,10 @@ proc emitExpr(be: var CBackend, node: HirNode): string =
let base = be.emitExpr(node.fieldPtrBase)
return &"(&({base}.{node.fieldName}))"
of hFieldAccess:
let base = be.emitExpr(node.fieldAccessBase)
return &"({base}.{node.fieldAccessName})"
of hArrowField:
let base = be.emitExpr(node.arrowFieldBase)
return &"(&({base}->{node.arrowFieldName}))"
@@ -284,10 +334,11 @@ proc emitExpr(be: var CBackend, node: HirNode): string =
return &"({base}).data[{idx}]"
of hTupleInit:
var elems: seq[string] = @[]
for e in node.tupleInitElements:
elems.add(be.emitExpr(e))
return &"{{{elems.join(\", \")}}}"
let typeName = typeToC(be, node.typ)
var fields: seq[string] = @[]
for i, e in node.tupleInitElements:
fields.add(&"._{i} = {be.emitExpr(e)}")
return &"(({typeName}){{{fields.join(\", \")}}})"
of hCast:
let operand = be.emitExpr(node.castOperand)
@@ -350,7 +401,8 @@ proc emitExpr(be: var CBackend, node: HirNode): string =
return "0"
of hMatch:
return "0" # TODO: match expression lowering
# Match should be desugared in hir_lower to if-else. Fallback: 0.
return "0"
else:
return "0"
@@ -358,7 +410,14 @@ proc emitExpr(be: var CBackend, node: HirNode): string =
proc emitStmt(be: var CBackend, node: HirNode) =
if node == nil: return
case node.kind
of hDefer:
let expr = be.emitExpr(node.deferBody)
if be.deferStack.len > 0:
let idx = be.deferStack.len - 1
be.deferStack[idx].add(expr & ";")
of hReturn:
be.emitCurrentDefers()
if node.returnValue != nil:
let val = be.emitExpr(node.returnValue)
be.emitLine(&"return {val};")
@@ -394,9 +453,11 @@ proc emitStmt(be: var CBackend, node: HirNode) =
be.emitLine("}")
of hBreak:
be.emitCurrentDefers()
be.emitLine("break;")
of hContinue:
be.emitCurrentDefers()
be.emitLine("continue;")
of hEmit:
@@ -444,6 +505,18 @@ proc emitStmt(be: var CBackend, node: HirNode) =
be.emitLine(&"{expr};")
proc emitFunc*(be: var CBackend, hfunc: HirFunc) =
# Emit env struct and global instance for closures with captures
if hfunc.captureNames.len > 0 and hfunc.envStructName != "":
be.emitLine(&"struct {hfunc.envStructName} {{")
inc be.indent
for i in 0 ..< hfunc.captureNames.len:
let capName = hfunc.captureNames[i]
let capType = if i < hfunc.captureTypes.len: typeToC(be, hfunc.captureTypes[i]) else: "int"
be.emitLine(&"{capType} {capName};")
dec be.indent
be.emitLine("};")
be.emitLine(&"static struct {hfunc.envStructName} {hfunc.envInstanceName};")
be.emitLine("")
let retType = typeToC(be, hfunc.retType)
var params: seq[string] = @[]
for p in hfunc.params:
@@ -453,15 +526,23 @@ proc emitFunc*(be: var CBackend, hfunc: HirFunc) =
let paramsStr = params.join(", ")
be.emitLine(&"{retType} {hfunc.name}({paramsStr}) {{")
inc be.indent
be.pushDeferScope()
if hfunc.body != nil:
if hfunc.body.kind == hBlock:
for stmt in hfunc.body.blockStmts:
be.emitStmt(stmt)
if hfunc.body.blockExpr != nil and hfunc.retType.kind != tkVoid:
be.emitCurrentDefers()
let val = be.emitExpr(hfunc.body.blockExpr)
be.emitLine(&"return {val};")
else:
be.emitCurrentDefers()
else:
be.emitStmt(hfunc.body)
be.emitCurrentDefers()
else:
be.emitCurrentDefers()
be.popDeferScope()
dec be.indent
be.emitLine("}")
be.emitLine("")
@@ -514,20 +595,17 @@ proc emitEnum*(be: var CBackend, name: string, variants: seq[HirEnumVariant]) =
be.emitLine(&"typedef union {{")
inc be.indent
for v in variants:
if v.fields.len > 0:
# Positional fields
for i, f in v.fields:
let typ = typeToC(be, f)
be.emitLine(&"{typ} {v.name}_{i};")
if v.fields.len == 1:
# Single positional field — flat union member (compat: data.Variant_0)
let typ = typeToC(be, v.fields[0])
be.emitLine(&"{typ} {v.name}_0;")
elif v.fields.len > 1:
# Multi positional — named nested typedef Enum_Variant_Payload
let nestedName = name & "_" & v.name & "_Payload"
be.emitLine(&"{nestedName} {v.name};")
elif v.namedFields.len > 0:
# Named fields - generate as struct
be.emitLine(&"struct {{")
inc be.indent
for nf in v.namedFields:
let typ = typeToC(be, nf.typ)
be.emitLine(&"{typ} {nf.name};")
dec be.indent
be.emitLine(&"}} {v.name};")
let nestedName = name & "_" & v.name & "_Payload"
be.emitLine(&"{nestedName} {v.name};")
dec be.indent
be.emitLine(&"}} {name}_Data;")
be.emitLine("")
@@ -627,14 +705,26 @@ proc emitModule*(be: var CBackend, module: HirModule): string =
if module.structs.len > 0:
be.emitLine("")
# Enum definitions (must come before structs that reference them)
# Nested multi-field enum payloads (Enum_Variant_Payload) must be fully
# defined before the algebraic enum union that embeds them by value.
var payloadStructNames: seq[string] = @[]
for e in module.enums:
for v in e.variants:
if v.fields.len > 1 or v.namedFields.len > 0:
payloadStructNames.add(e.name & "_" & v.name & "_Payload")
for s in module.structs:
if s.name in payloadStructNames:
be.emitStruct(s.name, s.fields)
# Enum definitions (after payload structs; before user structs that may use them)
for e in module.enums:
be.emitEnum(e.name, e.variants)
if module.enums.len > 0:
be.emitLine("")
# Struct definitions
# Remaining struct definitions (skip payloads already emitted)
for s in module.structs:
if s.name notin payloadStructNames:
be.emitStruct(s.name, s.fields)
# Slice fat-pointer typedefs
+820 -39
View File
File diff suppressed because it is too large Load Diff
+202
View File
@@ -0,0 +1,202 @@
## docgen.nim — Extract `///` (and adjacent `/* */`) docs into Markdown.
## Used by `bux doc [path...]`.
import std/[os, strutils, strformat, algorithm]
type
DocItem* = object
kind*: string ## module | func | struct | enum | interface | extern
name*: string
signature*: string ## first declaration line (trimmed)
docs*: string
file*: string
line*: int
proc isDeclStart(line: string): bool =
let s = line.strip()
if s.len == 0: return false
# Leading attributes @[Checked] etc. — not a decl by themselves
if s.startsWith("@["): return false
if s.startsWith("func ") or s.startsWith("pub func ") or
s.startsWith("extern func ") or s.startsWith("const func ") or
s.startsWith("async func "):
return true
if s.startsWith("struct ") or s.startsWith("pub struct ") or
s.startsWith("enum ") or s.startsWith("pub enum ") or
s.startsWith("union ") or s.startsWith("interface ") or
s.startsWith("module ") or s.startsWith("type "):
return true
return false
proc declKindAndName(line: string): tuple[kind, name: string] =
var s = line.strip()
# Strip leading pub/extern/const/async
for prefix in ["pub ", "extern ", "const ", "async "]:
if s.startsWith(prefix):
s = s[prefix.len .. ^1].strip()
var kind = "item"
if s.startsWith("func "):
kind = "func"
s = s["func ".len .. ^1]
elif s.startsWith("struct "):
kind = "struct"
s = s["struct ".len .. ^1]
elif s.startsWith("enum "):
kind = "enum"
s = s["enum ".len .. ^1]
elif s.startsWith("union "):
kind = "union"
s = s["union ".len .. ^1]
elif s.startsWith("interface "):
kind = "interface"
s = s["interface ".len .. ^1]
elif s.startsWith("module "):
kind = "module"
s = s["module ".len .. ^1]
elif s.startsWith("type "):
kind = "type"
s = s["type ".len .. ^1]
# Name: until `<` `(` `{` `:` space
var name = ""
for ch in s:
if ch in {' ', '<', '(', '{', ':', ';'}:
break
name.add(ch)
if name.len == 0:
name = s
# extern funcs already stripped "extern "
if kind == "func" and line.strip().startsWith("extern"):
kind = "extern"
return (kind, name)
proc extractDocsFromSource*(source, path: string): seq[DocItem] =
result = @[]
var pending: seq[string] = @[]
var inBlockComment = false
var blockDoc: seq[string] = @[]
var lineNo = 0
for rawLine in source.splitLines():
inc lineNo
var line = rawLine
let stripped = line.strip()
# Block comment handling (/* ... */ used in stdlib today)
if inBlockComment:
let endIdx = stripped.find("*/")
if endIdx >= 0:
let before = stripped[0 ..< endIdx].strip()
if before.len > 0:
blockDoc.add(before)
inBlockComment = false
# Treat completed block as pending doc if non-empty
if blockDoc.len > 0:
pending = blockDoc
blockDoc = @[]
continue
else:
blockDoc.add(stripped)
continue
if stripped.startsWith("/*") and not stripped.startsWith("/***"):
let rest = stripped["/*".len .. ^1]
let endIdx = rest.find("*/")
if endIdx >= 0:
let body = rest[0 ..< endIdx].strip()
if body.len > 0:
pending = @[body]
else:
inBlockComment = true
blockDoc = @[]
let body = rest.strip()
if body.len > 0:
blockDoc.add(body)
continue
# Triple-slash doc comments
if stripped.startsWith("///"):
var body = stripped["///".len .. ^1]
if body.startsWith(" "):
body = body[1 .. ^1]
pending.add(body)
continue
# Empty line: keep pending docs (allow blank lines inside doc blocks)
if stripped.len == 0:
continue
# Attributes immediately before decl: keep pending
if stripped.startsWith("@["):
continue
if isDeclStart(line):
if pending.len > 0:
let (kind, name) = declKindAndName(line)
result.add(DocItem(
kind: kind,
name: name,
signature: stripped,
docs: pending.join("\n"),
file: path,
line: lineNo
))
pending = @[]
continue
# Other code clears pending (except plain // comments)
if stripped.startsWith("//"):
continue
pending = @[]
proc collectBuxFilesForDoc*(root: string): seq[string] =
result = @[]
if fileExists(root) and root.endsWith(".bux"):
result.add(root)
return
if not dirExists(root):
return
for path in walkDirRec(root):
if path.endsWith(".bux"):
result.add(path)
result.sort(system.cmp)
proc renderMarkdown*(items: seq[DocItem], title: string = "API Reference"): string =
var sb: string
sb.add(&"# {title}\n\n")
sb.add("Generated by `bux doc` from `///` and `/* */` documentation comments.\n\n")
if items.len == 0:
sb.add("_No documented items found._\n")
return sb
# Group by file
var byFile: seq[string] = @[]
for it in items:
if it.file notin byFile:
byFile.add(it.file)
byFile.sort(system.cmp)
for f in byFile:
let base = splitFile(f).name
sb.add(&"## `{base}`\n\n")
sb.add(&"_Source: `{f}`_\n\n")
for it in items:
if it.file != f:
continue
sb.add(&"### `{it.name}` _{it.kind}_\n\n")
sb.add("```bux\n")
sb.add(it.signature)
sb.add("\n```\n\n")
if it.docs.len > 0:
sb.add(it.docs)
sb.add("\n\n")
return sb
proc generateDocs*(paths: seq[string]): seq[DocItem] =
result = @[]
var files: seq[string] = @[]
for p in paths:
for f in collectBuxFilesForDoc(p):
if f notin files:
files.add(f)
files.sort(system.cmp)
for f in files:
let src = readFile(f)
result.add(extractDocsFromSource(src, f))
+106
View File
@@ -0,0 +1,106 @@
## fmt.nim — Indentation-based Bux source formatter (bootstrap).
## Mirrors selfhost `src/fmt.bux`: re-indent by brace depth, preserve content.
import std/[strutils, os, algorithm]
proc isInStringOrComment(line: string, pos: int): bool =
## Simplified: track `//`, `"..."`, and `'...'` up to `pos`.
var inString = false
var inChar = false
var inComment = false
var i = 0
while i < pos and i < line.len:
let c = line[i]
let n = if i + 1 < line.len: line[i + 1] else: '\0'
if inComment:
inc i
continue
if c == '/' and n == '/':
inComment = true
inc i
continue
if c == '"' and not inChar:
inString = not inString
if c == '\'' and not inString:
inChar = not inChar
inc i
return inString or inChar or inComment
proc countBraceDelta(line: string): int =
var delta = 0
for i in 0 ..< line.len:
if isInStringOrComment(line, i):
continue
let c = line[i]
if c == '{':
inc delta
elif c == '}':
dec delta
return delta
proc formatSource*(source: string): string =
## Re-indent each non-empty line to 4 spaces × brace depth.
## Idempotent: formatting a clean file is a no-op.
var sb: string
var indent = 0
# Nim's splitLines leaves a trailing "" when the source ends with '\n'.
# Drop that artifact so we don't accumulate blank lines on re-format.
var lines = source.splitLines(keepEol = false)
if source.len > 0 and source.endsWith('\n') and lines.len > 0 and lines[^1].len == 0:
lines.setLen(lines.len - 1)
for line in lines:
let trimmed = line.strip(leading = true, trailing = false)
if trimmed.len == 0:
sb.add('\n')
continue
let delta = countBraceDelta(trimmed)
let firstChar = trimmed[0]
if firstChar == '}':
dec indent
if indent < 0:
indent = 0
for _ in 0 ..< indent:
sb.add(" ")
sb.add(trimmed)
sb.add('\n')
if firstChar != '}':
indent = indent + delta
else:
# Net delta after the initial decrease for a leading `}`
indent = indent + delta + 1
if indent < 0:
indent = 0
return sb
proc formatFile*(path: string, checkOnly: bool): tuple[ok: bool, changed: bool, msg: string] =
## Format `path` in place, or only check if reformatting would change it.
if not fileExists(path):
return (false, false, "file not found: " & path)
let source = readFile(path)
let formatted = formatSource(source)
if formatted == source:
return (true, false, "")
if checkOnly:
return (true, true, "would reformat")
try:
writeFile(path, formatted)
return (true, true, "formatted")
except CatchableError as e:
return (false, false, e.msg)
proc collectBuxFiles*(root: string): seq[string] =
## Collect `.bux` files: single file, or recursive directory walk.
result = @[]
if fileExists(root) and root.endsWith(".bux"):
result.add(root)
return
if not dirExists(root):
return
for path in walkDirRec(root):
if path.endsWith(".bux"):
result.add(path)
result.sort(system.cmp)
+23
View File
@@ -17,11 +17,13 @@ type
hBreak
hContinue
hReturn
hDefer
# Memory
hAlloca
hLoad
hStore
hFieldPtr
hFieldAccess
hArrowField
hIndexPtr
hSliceIndex
@@ -84,6 +86,8 @@ type
continueLabel*: string
of hReturn:
returnValue*: HirNode
of hDefer:
deferBody*: HirNode
of hAlloca:
allocaType*: Type
allocaName*: string
@@ -95,6 +99,9 @@ type
of hFieldPtr:
fieldPtrBase*: HirNode
fieldName*: string
of hFieldAccess:
fieldAccessBase*: HirNode
fieldAccessName*: string
of hArrowField:
arrowFieldBase*: HirNode
arrowFieldName*: string
@@ -156,6 +163,7 @@ type
HirMatchArm* = object
pattern*: Pattern
body*: HirNode
guard*: HirNode ## optional: lowered `if cond` from `p if cond => body`
HirFunc* = object
name*: string
@@ -163,6 +171,11 @@ type
retType*: Type
body*: HirNode
isPublic*: bool
# Closure capture metadata
captureNames*: seq[string]
captureTypes*: seq[Type]
envStructName*: string
envInstanceName*: string
HirEnumVariant* = object
name*: string
@@ -177,6 +190,10 @@ type
consts*: seq[tuple[name: string, typ: Type, value: HirNode]]
interfaces*: seq[tuple[name: string, hasAssocTypes: bool, methods: seq[tuple[name: string, params: seq[Type], ret: Type]]]]
vtables*: seq[tuple[interfaceName: string, concreteType: string, methodNames: seq[string], hasAssocTypes: bool]]
## Named functions used as values → need __adapt_ wrappers for fat-func ABI
funcAdapters*: seq[tuple[name: string, typ: Type]]
## Extra func types seen in locals/closures that need BuxFn_* typedefs
seenFatTypes*: seq[Type]
# Constructor helpers
proc hirLit*(tok: Token, typ: Type, loc: SourceLocation): HirNode =
@@ -200,6 +217,9 @@ proc hirCall*(callee: string, args: seq[HirNode], typ: Type, loc: SourceLocation
proc hirReturn*(value: HirNode, loc: SourceLocation): HirNode =
HirNode(kind: hReturn, returnValue: value, typ: makeVoid(), loc: loc)
proc hirDefer*(body: HirNode, loc: SourceLocation): HirNode =
HirNode(kind: hDefer, deferBody: body, typ: makeVoid(), loc: loc)
proc hirBlock*(stmts: seq[HirNode], expr: HirNode, typ: Type, loc: SourceLocation, isScope: bool = false): HirNode =
HirNode(kind: hBlock, blockStmts: stmts, blockExpr: expr, typ: typ, loc: loc, isScope: isScope)
@@ -213,6 +233,9 @@ proc hirAlloca*(name: string, typ: Type, loc: SourceLocation): HirNode =
proc hirStore*(ptrNode, value: HirNode, loc: SourceLocation): HirNode =
HirNode(kind: hStore, storePtr: ptrNode, storeValue: value, typ: makeVoid(), loc: loc)
proc hirAssign*(target, value: HirNode, loc: SourceLocation): HirNode =
HirNode(kind: hAssign, assignOp: tkAssign, assignTarget: target, assignValue: value, typ: makeVoid(), loc: loc)
proc hirLoad*(ptrNode: HirNode, typ: Type, loc: SourceLocation): HirNode =
HirNode(kind: hLoad, loadPtr: ptrNode, typ: typ, loc: loc)
+1719 -134
View File
File diff suppressed because it is too large Load Diff
+20 -24
View File
@@ -61,12 +61,6 @@ proc advance(lex: var Lexer): char =
else:
inc lex.col
proc match(lex: var Lexer, expected: char): bool =
if lex.isAtEnd(): return false
if lex.peek() != expected: return false
discard lex.advance()
return true
proc matchStr(lex: var Lexer, s: string): bool =
for i, c in s:
if lex.peek(i) != c:
@@ -76,14 +70,12 @@ proc matchStr(lex: var Lexer, s: string): bool =
return true
proc currentLocation(lex: Lexer): SourceLocation =
result = SourceLocation(line: lex.line, column: lex.col, offset: uint32(lex.pos))
result = SourceLocation(line: lex.line, column: lex.col, offset: uint32(lex.pos),
file: lex.sourceName)
proc emitError(lex: var Lexer, loc: SourceLocation, message: string) =
lex.diagnostics.add(LexerDiagnostic(severity: ldsError, loc: loc, message: message))
proc emitWarning(lex: var Lexer, loc: SourceLocation, message: string) =
lex.diagnostics.add(LexerDiagnostic(severity: ldsWarning, loc: loc, message: message))
proc makeToken(lex: Lexer, kind: TokenKind, startLoc: SourceLocation, startPos: int): Token =
let text = lex.source[startPos ..< lex.pos]
result = Token(kind: kind, text: text, loc: startLoc)
@@ -236,6 +228,9 @@ proc scanEscapeSequence(lex: var Lexer): string =
of 'r': result = "\r"
of 't': result = "\t"
of '0': result = "\0"
of '{', '}':
# Preserve \{ and \} so f"..." interpolation can treat them as literal braces
result = "\\" & $c
of 'x':
var hexVal = ""
for _ in 0..<2:
@@ -337,19 +332,6 @@ proc scanSymbol(lex: var Lexer, startLoc: SourceLocation): Token =
else:
return lex.makeToken(kind1, startLoc, startPos)
template check3(c2: char, kind2: TokenKind, c3: char, kind3: TokenKind, kind1: TokenKind) =
if lex.peek() == c2:
discard lex.advance()
if lex.peek() == c3:
discard lex.advance()
return lex.makeToken(kind3, startLoc, startPos)
return lex.makeToken(kind2, startLoc, startPos)
else:
return lex.makeToken(kind1, startLoc, startPos)
template checkEq(c2: char, kind2: TokenKind, kind1: TokenKind) =
check2(c2, kind2, kind1)
case c1
of '(': return lex.makeToken(tkLParen, startLoc, startPos)
of ')': return lex.makeToken(tkRParen, startLoc, startPos)
@@ -482,6 +464,17 @@ proc scanSymbol(lex: var Lexer, startLoc: SourceLocation): Token =
return lex.makeToken(tkCaretAssign, startLoc, startPos)
else:
return lex.makeToken(tkCaret, startLoc, startPos)
of '$':
# $name fragment, or bare $ for macro repetition $( ... )*
if isIdentStart(lex.peek()):
discard lex.advance() # first ident char ( $ already consumed as c1)
while not lex.isAtEnd() and isIdentChar(lex.peek()):
discard lex.advance()
# text includes leading '$'
return lex.makeToken(tkIdent, startLoc, startPos)
else:
# bare $ (c1 already consumed)
return lex.makeToken(tkDollar, startLoc, startPos)
of '#':
# Check for intrinsics: #line, #column, #file, #function, #date, #time, #module
let afterHash = lex.peek()
@@ -527,7 +520,10 @@ proc nextToken(lex: var Lexer): Token =
discard lex.advance()
return Token(kind: tkNewLine, text: "\n", loc: startLoc)
# String prefixes: c8" c16" c32" — must come before ident check
# String prefixes: f" c8" c16" c32" — must come before ident check
if c == 'f' and lex.peek(1) == '"':
discard lex.advance() # f
return lex.scanString(startLoc, 1)
if c == 'c' and lex.peek(1) in {'8', '1', '3'}:
let d = lex.peek(1)
if d == '8' and lex.peek(2) == '"':
-2
View File
@@ -2,8 +2,6 @@
## Linear 3-address code IR, designed for straightforward C emission.
## Each HIR construct lowers to 5-30 LIR instructions.
import types
type
LirKind* = enum
# ── Data movement ──
+455 -62
View File
@@ -2,7 +2,7 @@
## Emits clean, well-structured C code from LIR instructions.
## Since LIR is already linear and low-level, C emission is straightforward.
import std/[strutils, strformat, tables, sequtils]
import std/[strutils, strformat, tables, sequtils, sets]
import lir, hir, types, token
type
@@ -10,16 +10,19 @@ type
output*: string
indent*: int
tempTypes*: Table[string, string] ## Track C types of temp variables
emitDebugLines*: bool ## Emit #line → .bux for DWARF (E.4)
lastDebugLine*: int
lastDebugFile*: string
proc initLirCBackend*(): LirCBackend =
proc initLirCBackend*(emitDebugLines: bool = true): LirCBackend =
result = LirCBackend(
indent: 0,
tempTypes: initTable[string, string](),
emitDebugLines: emitDebugLines,
lastDebugLine: 0,
lastDebugFile: "",
)
proc emit(be: var LirCBackend, s: string) =
be.output.add(s)
proc emitIndent(be: var LirCBackend) =
for i in 0 ..< be.indent:
be.output.add(" ")
@@ -29,6 +32,22 @@ proc emitLine(be: var LirCBackend, s: string) =
be.output.add(s)
be.output.add("\n")
proc emitDebugLine(be: var LirCBackend, instr: LirInstr) =
## Map generated C back to Bux source for gdb/DWARF via #line.
if not be.emitDebugLines: return
if instr.locLine <= 0: return
if instr.locLine == be.lastDebugLine and instr.locFile == be.lastDebugFile:
return
be.lastDebugLine = instr.locLine
be.lastDebugFile = instr.locFile
var path = instr.locFile
if path.len == 0:
path = "<bux>"
# Escape for C string literal
path = path.replace("\\", "\\\\").replace("\"", "\\\"")
# #line must start at column 0
be.output.add(&"#line {instr.locLine} \"{path}\"\n")
proc valToC(be: var LirCBackend, v: LirValue): string =
## Convert a LirValue to its C representation.
case v.kind
@@ -43,24 +62,17 @@ proc valToC(be: var LirCBackend, v: LirValue): string =
of lvkField: v.strVal
of lvkType: v.strVal
proc typeFromValue(be: var LirCBackend, v: LirValue): string =
## Infer a C type for a value. Temps are tracked; named vars use lookup.
case v.kind
of lvkTemp:
if be.tempTypes.hasKey(v.strVal):
return be.tempTypes[v.strVal]
return "int" # Default
of lvkString: return "const char*"
of lvkInt: return "int"
of lvkFloat: return "double"
else: return ""
proc setTempType(be: var LirCBackend, temp: string, cType: string) =
be.tempTypes[temp] = cType
proc cParamDecl(cType, name: string): string =
## Emit a C parameter declaration, handling function-pointer syntax.
if cType.contains("(*)"):
return cType.replace("(*)", "(*" & name & ")")
else:
return cType & " " & name
# ── Per-instruction emission ──
proc emitInstr(be: var LirCBackend, instr: LirInstr) =
be.emitDebugLine(instr)
template v(x: LirValue): string = valToC(be, x)
case instr.kind
@@ -108,14 +120,15 @@ proc emitInstr(be: var LirCBackend, instr: LirInstr) =
of lirShl: "<<"
of lirShr: ">>"
else: "?"
be.emitLine(&"{v(instr.dst)} = {v(instr.src)} {op} {v(instr.src2)};")
# Parenthesize so future non-temp operands cannot be rewritten by C precedence
be.emitLine(&"{v(instr.dst)} = ({v(instr.src)} {op} {v(instr.src2)});")
of lirNeg:
be.emitLine(&"{v(instr.dst)} = -{v(instr.src)};")
be.emitLine(&"{v(instr.dst)} = -({v(instr.src)});")
of lirNot:
be.emitLine(&"{v(instr.dst)} = !{v(instr.src)};")
be.emitLine(&"{v(instr.dst)} = !({v(instr.src)});")
of lirBNot:
be.emitLine(&"{v(instr.dst)} = ~{v(instr.src)};")
be.emitLine(&"{v(instr.dst)} = ~({v(instr.src)});")
# ── Comparison ──
of lirCmpEq, lirCmpNe, lirCmpLt, lirCmpLe, lirCmpGt, lirCmpGe:
@@ -160,14 +173,22 @@ proc emitInstr(be: var LirCBackend, instr: LirInstr) =
be.emitLine(&"{v(instr.src)}({argsStr});")
of lirCallIndirect:
## Fat function pointer call: f.code(f.env, args...)
var argsStr = ""
for i, arg in instr.extra:
if i > 0: argsStr.add(", ")
argsStr.add(v(arg))
let callee = v(instr.src)
if instr.dst.kind != lvkVoid:
be.emitLine(&"{v(instr.dst)} = ({v(instr.src)})({argsStr});")
if argsStr.len > 0:
be.emitLine(&"{v(instr.dst)} = ({callee}.code)({callee}.env, {argsStr});")
else:
be.emitLine(&"({v(instr.src)})({argsStr});")
be.emitLine(&"{v(instr.dst)} = ({callee}.code)({callee}.env);")
else:
if argsStr.len > 0:
be.emitLine(&"({callee}.code)({callee}.env, {argsStr});")
else:
be.emitLine(&"({callee}.code)({callee}.env);")
# ── Return ──
of lirRet:
@@ -183,7 +204,7 @@ proc emitInstr(be: var LirCBackend, instr: LirInstr) =
let inferred = be.tempTypes[instr.dst.strVal]
if inferred != "" and inferred != ct:
ct = inferred
be.emitLine(&"{ct} {v(instr.dst)};")
be.emitLine(cParamDecl(ct, v(instr.dst)) & ";")
# ── Pointers ──
of lirAddrOf:
@@ -241,14 +262,24 @@ proc emitInstr(be: var LirCBackend, instr: LirInstr) =
# ── Function emission ──
proc emitFunc(be: var LirCBackend, f: LirFunc, funcRetTypes: Table[string, string]) =
proc emitFunc(be: var LirCBackend, f: LirFunc, funcRetTypes: Table[string, string], funcPtrTypes: Table[string, string]) =
var paramsStr = ""
for i, p in f.params:
if i > 0: paramsStr.add(", ")
paramsStr.add(&"{p.cType} {p.name}")
paramsStr.add(cParamDecl(p.cType, p.name))
if f.params.len == 0:
paramsStr = "void"
# Point the function entry at the first Bux location so gdb `list Main` works
# (otherwise leftover #line from the previous function pollutes the prologue).
if be.emitDebugLines:
for instr in f.instrs:
if instr.locLine > 0:
be.lastDebugLine = 0
be.lastDebugFile = ""
be.emitDebugLine(instr)
break
be.emitLine(&"{f.retType} {f.name}({paramsStr}) {{")
be.indent += 1
@@ -318,7 +349,12 @@ proc emitFunc(be: var LirCBackend, f: LirFunc, funcRetTypes: Table[string, strin
newType = be.tempTypes[instr.extra[0].strVal]
elif instr.src2.kind == lvkVar and varTypes.hasKey(instr.src2.strVal):
newType = varTypes[instr.src2.strVal]
of lirAddrOf, lirFieldPtr, lirArrowFieldPtr, lirIndexPtr, lirPtrAdd:
of lirAddrOf:
if funcPtrTypes.hasKey(instr.src.strVal):
newType = funcPtrTypes[instr.src.strVal]
else:
newType = "void*";
of lirFieldPtr, lirArrowFieldPtr, lirIndexPtr, lirPtrAdd:
newType = "void*"
of lirAdd, lirSub, lirMul, lirDiv, lirMod, lirNeg,
lirCmpEq, lirCmpNe, lirCmpLt, lirCmpLe, lirCmpGt, lirCmpGe,
@@ -342,7 +378,7 @@ proc emitFunc(be: var LirCBackend, f: LirFunc, funcRetTypes: Table[string, strin
let ct = be.tempTypes[instr.dst.strVal]
if ct != "":
declared.add(instr.dst.strVal)
be.emitLine(&"{ct} {instr.dst.strVal};")
be.emitLine(cParamDecl(ct, instr.dst.strVal) & ";")
# ── Pass 4: emit instructions ──
for instr in f.instrs:
@@ -354,6 +390,21 @@ proc emitFunc(be: var LirCBackend, f: LirFunc, funcRetTypes: Table[string, strin
# ── Struct/Enum emission (from HIR module) ──
proc sanitizeCTypeNamePart(s: string): string =
result = s
result = result.replace("const char*", "cstr")
result = result.replace("unsigned int", "uint")
result = result.replace(" ", "_")
result = result.replace("*", "Ptr")
result = result.replace("(", "")
result = result.replace(")", "")
result = result.replace(",", "_")
result = result.replace(".", "_")
proc typeToCStr(typ: Type): string
proc funcFatTypeName(typ: Type): string
proc funcCodePtrType(typ: Type): string
proc typeToCStr(typ: Type): string =
## Duplicate from lir_lower for self-containedness
if typ == nil: return "int"
@@ -402,8 +453,39 @@ proc typeToCStr(typ: Type): string =
of "float64": return "double"
of "bool": return "bool"
else: return typ.name
of tkTuple:
if typ.inner.len == 0:
return "Tuple_Empty"
var parts: seq[string] = @[]
for e in typ.inner:
parts.add(sanitizeCTypeNamePart(typeToCStr(e)))
return "Tuple_" & parts.join("_")
of tkFunc:
return funcFatTypeName(typ)
else: return "int"
proc funcFatTypeName(typ: Type): string =
if typ == nil or typ.kind != tkFunc:
return "BuxFn_void"
let ret = if typ.inner.len > 0: typeToCStr(typ.inner[^1]) else: "void"
var parts: seq[string] = @[sanitizeCTypeNamePart(ret)]
if typ.inner.len > 1:
for p in typ.inner[0 ..^ 2]:
parts.add(sanitizeCTypeNamePart(typeToCStr(p)))
else:
parts.add("void")
return "BuxFn_" & parts.join("_")
proc funcCodePtrType(typ: Type): string =
if typ == nil or typ.kind != tkFunc:
return "void (*)(void*)"
let ret = if typ.inner.len > 0: typeToCStr(typ.inner[^1]) else: "void"
var params: seq[string] = @["void* env"]
if typ.inner.len > 1:
for p in typ.inner[0 ..^ 2]:
params.add(typeToCStr(p))
return ret & " (*)(" & params.join(", ") & ")"
proc emitStructDef(be: var LirCBackend, name: string, fields: seq[tuple[name: string, typ: Type]]) =
be.emitLine(&"typedef struct {name} {{")
be.indent += 1
@@ -448,16 +530,16 @@ proc emitEnumDef(be: var LirCBackend, name: string, variants: seq[HirEnumVariant
be.emitLine(&"typedef union {{")
be.indent += 1
for v in variants:
if v.fields.len > 0:
for i, f in v.fields:
be.emitLine(&"{typeToCStr(f)} {v.name}_{i};")
if v.fields.len == 1:
# Single positional field — flat (compat: data.Variant_0)
be.emitLine(&"{typeToCStr(v.fields[0])} {v.name}_0;")
elif v.fields.len > 1:
# Multi positional — named nested struct Enum_Variant_Payload
let nestedName = name & "_" & v.name & "_Payload"
be.emitLine(&"{nestedName} {v.name};")
elif v.namedFields.len > 0:
be.emitLine(&"struct {{")
be.indent += 1
for nf in v.namedFields:
be.emitLine(&"{typeToCStr(nf.typ)} {nf.name};")
be.indent -= 1
be.emitLine(&"}} {v.name};")
let nestedName = name & "_" & v.name & "_Payload"
be.emitLine(&"{nestedName} {v.name};")
be.indent -= 1
be.emitLine(&"}} {name}_Data;")
be.emitLine("")
@@ -470,6 +552,51 @@ proc emitEnumDef(be: var LirCBackend, name: string, variants: seq[HirEnumVariant
be.emitLine(&"}} {name};")
be.emitLine("")
# ── Type dependency ordering ──
proc collectValueDeps(typ: Type): seq[string] =
## Return type names that must be fully defined before a value of `typ`
## can be declared. Pointers/refs only need a forward declaration, so
## they do not introduce a dependency.
if typ == nil: return @[]
case typ.kind
of tkNamed:
return @[typ.name]
of tkSlice:
return @[typeToCStr(typ)]
of tkTuple:
var deps: seq[string] = @[]
for e in typ.inner:
for d in collectValueDeps(e):
if d notin deps:
deps.add(d)
if e != nil and e.kind == tkTuple:
let tn = typeToCStr(e)
if tn notin deps:
deps.add(tn)
return deps
of tkPointer, tkRef, tkMutRef, tkFunc:
return @[]
else:
return @[]
proc emitTupleDef(be: var LirCBackend, typ: Type) =
## typedef struct { T0 _0; T1 _1; ... } Tuple_...;
let name = typeToCStr(typ)
be.emitLine(&"typedef struct {name} {{")
be.indent += 1
if typ.inner.len == 0:
be.emitLine("char _pad;")
else:
for i, e in typ.inner:
be.emitLine(&"{typeToCStr(e)} _{i};")
be.indent -= 1
be.emitLine(&"}} {name};")
be.emitLine("")
proc emitSliceTypeDef(be: var LirCBackend, name: string, elem: string) =
be.emitLine(&"typedef struct {{ {elem}* data; size_t len; }} {name};")
# ── Module emission ──
proc emitModule*(be: var LirCBackend, builder: LirBuilder, module: HirModule): string =
@@ -483,6 +610,17 @@ proc emitModule*(be: var LirCBackend, builder: LirBuilder, module: HirModule): s
for f in module.externFuncs:
funcRetTypes[f.name] = typeToCStr(f.retType)
# Build function-pointer type lookup table (for address-of)
var funcPtrTypes = initTable[string, string]()
for f in module.funcs:
let params = f.params.mapIt(typeToCStr(it.typ)).join(", ")
let ret = typeToCStr(f.retType)
funcPtrTypes[f.name] = ret & " (*)(" & params & ")"
for f in module.externFuncs:
let params = f.params.mapIt(typeToCStr(it.typ)).join(", ")
let ret = typeToCStr(f.retType)
funcPtrTypes[f.name] = ret & " (*)(" & params & ")"
# Header
be.emitLine("/* Generated by Bux Compiler (LIR backend) */")
be.emitLine("#include <stdio.h>")
@@ -512,7 +650,7 @@ proc emitModule*(be: var LirCBackend, builder: LirBuilder, module: HirModule): s
let rt = typeToCStr(ef.retType)
var params: seq[string] = @[]
for p in ef.params:
params.add(&"{typeToCStr(p.typ)} {p.name}")
params.add(cParamDecl(typeToCStr(p.typ), p.name))
if params.len == 0: params.add("void")
be.emitLine(&"extern {rt} {ef.name}({params.join(\", \")});")
be.emitLine("")
@@ -529,29 +667,260 @@ proc emitModule*(be: var LirCBackend, builder: LirBuilder, module: HirModule): s
else: discard
be.emitLine("")
# Enum definitions
# Collect local type names (structs and enums defined in this module).
var localTypeNames: HashSet[string]
for s in module.structs:
localTypeNames.incl(s.name)
for e in module.enums:
be.emitEnumDef(e.name, e.variants)
if module.enums.len > 0:
localTypeNames.incl(e.name)
# Emit tuple typedefs early — enums/structs may embed them by value
# (e.g. Box::Val((int,int)) → Tuple_int_int Val_0 in the union).
var tupleTypes: seq[Type] = @[]
var tupleNames: HashSet[string]
proc registerTuple(t: Type) =
if t == nil: return
case t.kind
of tkTuple:
for e in t.inner:
registerTuple(e)
let name = typeToCStr(t)
if not tupleNames.contains(name):
tupleNames.incl(name)
tupleTypes.add(t)
of tkPointer, tkRef, tkMutRef, tkSlice:
if t.inner.len > 0:
registerTuple(t.inner[0])
of tkFunc:
for e in t.inner:
registerTuple(e)
else:
discard
proc walkHirForTuples(n: HirNode) =
if n == nil: return
registerTuple(n.typ)
case n.kind
of hAlloca:
registerTuple(n.allocaType)
of hBlock:
for s in n.blockStmts: walkHirForTuples(s)
walkHirForTuples(n.blockExpr)
of hIf:
walkHirForTuples(n.ifCond)
walkHirForTuples(n.ifThen)
walkHirForTuples(n.ifElse)
of hWhile:
walkHirForTuples(n.whileCond)
walkHirForTuples(n.whileBody)
of hLoop:
walkHirForTuples(n.loopBody)
of hReturn:
walkHirForTuples(n.returnValue)
of hStore:
walkHirForTuples(n.storePtr)
walkHirForTuples(n.storeValue)
of hAssign:
walkHirForTuples(n.assignTarget)
walkHirForTuples(n.assignValue)
of hBinary:
walkHirForTuples(n.binaryLeft)
walkHirForTuples(n.binaryRight)
of hUnary:
walkHirForTuples(n.unaryOperand)
of hCall:
for a in n.callArgs: walkHirForTuples(a)
of hCallIndirect:
walkHirForTuples(n.callIndirectCallee)
for a in n.callIndirectArgs: walkHirForTuples(a)
of hLoad:
walkHirForTuples(n.loadPtr)
of hFieldPtr:
walkHirForTuples(n.fieldPtrBase)
of hFieldAccess:
walkHirForTuples(n.fieldAccessBase)
of hStructInit:
for f in n.structInitFields: walkHirForTuples(f.value)
of hTupleInit:
for e in n.tupleInitElements: walkHirForTuples(e)
else:
discard
for f in module.funcs:
registerTuple(f.retType)
for p in f.params:
registerTuple(p.typ)
walkHirForTuples(f.body)
for ef in module.externFuncs:
registerTuple(ef.retType)
for p in ef.params:
registerTuple(p.typ)
for s in module.structs:
for f in s.fields:
registerTuple(f.typ)
for e in module.enums:
for v in e.variants:
for ft in v.fields:
registerTuple(ft)
for nf in v.namedFields:
registerTuple(nf.typ)
if tupleTypes.len > 0:
be.emitLine("/* Tuple types */")
for tt in tupleTypes:
be.emitTupleDef(tt)
# Collect slice types used in struct fields and enum payloads.
var sliceTypes: seq[tuple[name: string, elem: string]] = @[]
var sliceNames: HashSet[string]
proc registerSlice(t: Type) =
if t == nil or t.kind != tkSlice: return
let name = typeToCStr(t)
if sliceNames.contains(name): return
sliceNames.incl(name)
let elem = if t.inner.len > 0: typeToCStr(t.inner[0]) else: "void"
sliceTypes.add((name, elem))
for s in module.structs:
for f in s.fields:
registerSlice(f.typ)
for e in module.enums:
for v in e.variants:
for ft in v.fields:
registerSlice(ft)
for nf in v.namedFields:
registerSlice(nf.typ)
# Build dependency graph among structs, enums, and slice types.
# Edge A -> B means "A depends on B, so B must be emitted before A".
var deps: Table[string, seq[string]]
for s in module.structs:
deps[s.name] = @[]
for e in module.enums:
deps[e.name] = @[]
for st in sliceTypes:
deps[st.name] = @[]
proc addDeps(node: string, t: Type) =
for dep in collectValueDeps(t):
if dep == node: continue
if localTypeNames.contains(dep) or sliceNames.contains(dep):
if dep notin deps[node]:
deps[node].add(dep)
for s in module.structs:
for f in s.fields:
addDeps(s.name, f.typ)
for e in module.enums:
for v in e.variants:
for ft in v.fields:
addDeps(e.name, ft)
for nf in v.namedFields:
addDeps(e.name, nf.typ)
# Multi-field / named-field nested struct must be defined before the enum
if v.fields.len > 1 or v.namedFields.len > 0:
addDeps(e.name, makeNamed(e.name & "_" & v.name & "_Payload"))
# Topological sort (Kahn's algorithm).
var inDegree: Table[string, int]
var dependents: Table[string, seq[string]]
for node in deps.keys:
inDegree[node] = 0
for node, nodeDeps in deps:
for d in nodeDeps:
if not inDegree.hasKey(d): inDegree[d] = 0
inDegree[node] += 1
dependents.mgetOrPut(d, @[]).add(node)
var queue: seq[string] = @[]
for node, deg in inDegree:
if deg == 0:
queue.add(node)
var sorted: seq[string] = @[]
while queue.len > 0:
let node = queue.pop()
sorted.add(node)
for depNode in dependents.getOrDefault(node):
inDegree[depNode] -= 1
if inDegree[depNode] == 0:
queue.add(depNode)
if sorted.len < deps.len:
# Cycle detected; fall back to a safe deterministic order.
sorted = @[]
for s in module.structs: sorted.add(s.name)
for e in module.enums: sorted.add(e.name)
for st in sliceTypes: sorted.add(st.name)
# Map type names back to their definitions.
var structMap: Table[string, seq[tuple[name: string, typ: Type]]]
for s in module.structs: structMap[s.name] = s.fields
var enumMap: Table[string, seq[HirEnumVariant]]
for e in module.enums: enumMap[e.name] = e.variants
var sliceMap: Table[string, string]
for st in sliceTypes: sliceMap[st.name] = st.elem
# Emit type definitions in dependency order.
for name in sorted:
if structMap.hasKey(name):
be.emitStructDef(name, structMap[name])
elif enumMap.hasKey(name):
be.emitEnumDef(name, enumMap[name])
elif sliceMap.hasKey(name):
be.emitSliceTypeDef(name, sliceMap[name])
# Fat function-pointer typedefs (BuxFn_*) — before forward decls that use them
var fatTypes: seq[Type] = @[]
var fatNames: HashSet[string]
proc registerFat(t: Type) =
if t == nil: return
case t.kind
of tkFunc:
for e in t.inner: registerFat(e)
let n = funcFatTypeName(t)
if not fatNames.contains(n):
fatNames.incl(n)
fatTypes.add(t)
of tkPointer, tkRef, tkMutRef, tkSlice:
if t.inner.len > 0: registerFat(t.inner[0])
of tkTuple:
for e in t.inner: registerFat(e)
else: discard
for f in module.funcs:
registerFat(f.retType)
for p in f.params: registerFat(p.typ)
for ef in module.externFuncs:
registerFat(ef.retType)
for p in ef.params: registerFat(p.typ)
for a in module.funcAdapters:
registerFat(a.typ)
for t in module.seenFatTypes:
registerFat(t)
if fatTypes.len > 0:
be.emitLine("/* Fat function pointer types (code + env) */")
for ft in fatTypes:
let n = funcFatTypeName(ft)
let codeT = funcCodePtrType(ft)
be.emitLine(&"typedef struct {n} {{")
be.indent += 1
be.emitLine(cParamDecl(codeT, "code") & ";")
be.emitLine("void* env;")
be.indent -= 1
be.emitLine(&"}} {n};")
be.emitLine("")
# Struct definitions
for s in module.structs:
be.emitStructDef(s.name, s.fields)
# Slice types (collect from functions/structs)
# Simple: scan function params/returns for slice types
var sliceTypes: seq[tuple[name: string, elem: string]] = @[]
# Env structs for closures with captures (heap-allocated per value)
for f in module.funcs:
for p in f.params:
let ct = typeToCStr(p.typ)
if ct.startsWith("Slice_"):
let elem = ct[6 .. ^1]
if not sliceTypes.anyIt(it.name == ct):
sliceTypes.add((ct, elem))
if sliceTypes.len > 0:
for st in sliceTypes:
be.emitLine(&"typedef struct {{ {st.elem}* data; size_t len; }} {st.name};")
if f.captureNames.len > 0 and f.envStructName != "":
be.emitLine(&"typedef struct {f.envStructName} {{")
be.indent += 1
for i in 0 ..< f.captureNames.len:
let capName = f.captureNames[i]
let capType = if i < f.captureTypes.len: typeToCStr(f.captureTypes[i]) else: "int"
be.emitLine(&"{capType} {capName};")
be.indent -= 1
be.emitLine(&"}} {f.envStructName};")
be.emitLine("")
# Forward function declarations
@@ -559,7 +928,7 @@ proc emitModule*(be: var LirCBackend, builder: LirBuilder, module: HirModule): s
let rt = typeToCStr(f.retType)
var params: seq[string] = @[]
for p in f.params:
params.add(&"{typeToCStr(p.typ)} {p.name}")
params.add(cParamDecl(typeToCStr(p.typ), p.name))
if params.len == 0: params.add("void")
be.emitLine(&"{rt} {f.name}({params.join(\", \")});")
be.emitLine("")
@@ -573,7 +942,7 @@ proc emitModule*(be: var LirCBackend, builder: LirBuilder, module: HirModule): s
for m in iface.methods:
var paramCTypes: seq[string] = @["void* self"]
for i in 1 ..< m.params.len:
paramCTypes.add(typeToCStr(m.params[i]) & " param")
paramCTypes.add(cParamDecl(typeToCStr(m.params[i]), "param"))
let rt = typeToCStr(m.ret)
be.emitLine(&"{rt} (*{m.name})({paramCTypes.join(\", \")});")
be.indent -= 1
@@ -598,9 +967,33 @@ proc emitModule*(be: var LirCBackend, builder: LirBuilder, module: HirModule): s
be.emitLine("};")
be.emitLine("")
# Adapters for named functions used as fat-func values (after forward decls)
if module.funcAdapters.len > 0:
be.emitLine("/* Fat-func adapters for named functions */")
for a in module.funcAdapters:
let ret = if a.typ.inner.len > 0: typeToCStr(a.typ.inner[^1]) else: "void"
var params: seq[string] = @["void* env"]
var argNames: seq[string] = @[]
if a.typ.inner.len > 1:
for i, p in a.typ.inner[0 ..^ 2]:
let pn = "a" & $i
params.add(typeToCStr(p) & " " & pn)
argNames.add(pn)
let argsStr = argNames.join(", ")
be.emitLine(&"static {ret} __adapt_{a.name}({params.join(\", \")}) {{")
be.indent += 1
be.emitLine("(void)env;")
if ret == "void":
be.emitLine(&"{a.name}({argsStr});")
else:
be.emitLine(&"return {a.name}({argsStr});")
be.indent -= 1
be.emitLine("}")
be.emitLine("")
# Emit all LIR functions
for f in builder.funcs:
be.emitFunc(f, funcRetTypes)
be.emitFunc(f, funcRetTypes, funcPtrTypes)
# C main wrapper
var hasMain = false
+94 -9
View File
@@ -3,7 +3,7 @@
## Each HIR node kind lowers to 1-20 LIR instructions.
import std/[strutils, strformat, tables, sequtils]
import ast, types, token, hir, lir
import types, token, hir, lir
## Convert LirValue to C expression string (no % prefix)
proc lirValToC(v: LirValue): string =
@@ -57,6 +57,22 @@ proc cEscape(s: string): string =
of '\0': result.add("\\0")
else: result.add(c)
proc sanitizeCTypeNamePart(s: string): string =
## Make a C type string safe for use inside a typedef name.
result = s
result = result.replace("const char*", "cstr")
result = result.replace("unsigned int", "uint")
result = result.replace(" ", "_")
result = result.replace("*", "Ptr")
result = result.replace("(", "")
result = result.replace(")", "")
result = result.replace(",", "_")
result = result.replace(".", "_")
proc typeToCStr(typ: Type): string
proc funcFatTypeName*(typ: Type): string
proc funcCodePtrType*(typ: Type): string
proc typeToCStr(typ: Type): string =
## Convert a Bux Type to a C type string.
if typ == nil: return "int"
@@ -105,8 +121,44 @@ proc typeToCStr(typ: Type): string =
of "float64": return "double"
of "bool": return "bool"
else: return typ.name
of tkTuple:
## (T, U) → typedef struct { T _0; U _1; } Tuple_T_U;
if typ.inner.len == 0:
return "Tuple_Empty"
var parts: seq[string] = @[]
for e in typ.inner:
parts.add(sanitizeCTypeNamePart(typeToCStr(e)))
return "Tuple_" & parts.join("_")
of tkFunc:
## Fat function pointer: { code(env, args...), env }
## Enables multi-instance closures with captures.
return funcFatTypeName(typ)
else: return "int"
proc funcFatTypeName*(typ: Type): string =
## BuxFn_<ret>_<params...> (sanitized)
if typ == nil or typ.kind != tkFunc:
return "BuxFn_void"
let ret = if typ.inner.len > 0: typeToCStr(typ.inner[^1]) else: "void"
var parts: seq[string] = @[sanitizeCTypeNamePart(ret)]
if typ.inner.len > 1:
for p in typ.inner[0 ..^ 2]:
parts.add(sanitizeCTypeNamePart(typeToCStr(p)))
else:
parts.add("void")
return "BuxFn_" & parts.join("_")
proc funcCodePtrType*(typ: Type): string =
## C type of the .code field: ret (*)(void* env, params...)
if typ == nil or typ.kind != tkFunc:
return "void (*)(void*)"
let ret = if typ.inner.len > 0: typeToCStr(typ.inner[^1]) else: "void"
var params: seq[string] = @["void* env"]
if typ.inner.len > 1:
for i, p in typ.inner[0 ..^ 2]:
params.add(typeToCStr(p))
return ret & " (*)(" & params.join(", ") & ")"
proc hirTypeToC(ctx: var LowerToLirCtx, node: HirNode): string =
if node == nil: return "int"
result = typeToCStr(node.typ)
@@ -139,10 +191,20 @@ proc cmpOpToLir(op: TokenKind): LirKind =
proc lowerExpr(ctx: var LowerToLirCtx, node: HirNode): LirValue
proc lowerStmt(ctx: var LowerToLirCtx, node: HirNode)
proc setSourceLoc(ctx: var LowerToLirCtx, node: HirNode) =
## Stamp LIR instructions with HIR source locations for #line / DWARF.
if node == nil: return
if node.loc.line > 0:
ctx.builder.commentLine = int(node.loc.line)
if node.loc.file.len > 0:
ctx.builder.commentFile = node.loc.file
ctx.currentFile = node.loc.file
# ── Lowering: Expressions → LirValue ──
proc lowerExpr(ctx: var LowerToLirCtx, node: HirNode): LirValue =
if node == nil: return lirInt(0)
setSourceLoc(ctx, node)
template b: var LirBuilder = ctx.builder
case node.kind
@@ -340,6 +402,14 @@ proc lowerExpr(ctx: var LowerToLirCtx, node: HirNode): LirValue =
b.emitRawC(&"{t.strVal} = (void*)&({lirValToC(base)}.{node.fieldName});")
return t
of hFieldAccess:
let base = lowerExpr(ctx, node.fieldAccessBase)
let cType = hirTypeToC(ctx, node)
let t = b.freshTemp()
b.emitAlloca(t.strVal, cType)
b.emitRawC(&"{t.strVal} = {lirValToC(base)}.{node.fieldAccessName};")
return t
of hArrowField:
let base = lowerExpr(ctx, node.arrowFieldBase)
let t = b.freshTemp()
@@ -416,7 +486,6 @@ proc lowerExpr(ctx: var LowerToLirCtx, node: HirNode): LirValue =
# ── SizeOf ──
of hSizeOf:
let ctype = typeToCStr(node.sizeOfType)
let t = b.freshTemp()
b.emit(LirInstr(kind: lirRawC, src: lirStr(&"/* sizeof({ctype}) */")))
return lirVar(&"sizeof({ctype})")
@@ -486,7 +555,12 @@ proc lowerExpr(ctx: var LowerToLirCtx, node: HirNode): LirValue =
for e in node.tupleInitElements:
elems.add(lowerExpr(ctx, e))
let t = b.freshTemp()
b.emitRawC(&"/* tuple */ {t.strVal} = {{{elems.mapIt($it).join(\", \")}}};")
let typeName = typeToCStr(node.typ)
b.emitAlloca(t.strVal, typeName)
var fields: seq[string] = @[]
for i, e in elems:
fields.add(&"._{i} = {lirValToC(e)}")
b.emitRawC(&"{t.strVal} = ({typeName}){{{fields.join(\", \")}}};")
return t
# ── If expression (ternary) ──
@@ -544,6 +618,9 @@ proc buildLval(ctx: var LowerToLirCtx, n: HirNode): string =
of hArrowField:
let baseStr = buildLval(ctx, n.arrowFieldBase)
return baseStr & "->" & n.arrowFieldName
of hFieldAccess:
let baseStr = buildLval(ctx, n.fieldAccessBase)
return baseStr & "." & n.fieldAccessName
of hIndexPtr:
let baseStr = buildLval(ctx, n.indexPtrBase)
let idx = lowerExpr(ctx, n.indexPtrIndex)
@@ -556,6 +633,7 @@ proc buildLval(ctx: var LowerToLirCtx, n: HirNode): string =
proc lowerStmt(ctx: var LowerToLirCtx, node: HirNode) =
if node == nil: return
setSourceLoc(ctx, node)
template b: var LirBuilder = ctx.builder
case node.kind
@@ -592,7 +670,6 @@ proc lowerStmt(ctx: var LowerToLirCtx, node: HirNode) =
# ── While statement ──
of hWhile:
let startLbl = b.freshLabel("while")
let bodyLbl = b.freshLabel("wbody")
let endLbl = b.freshLabel("wend")
ctx.loopStartLabels.add(startLbl.strVal)
@@ -664,6 +741,9 @@ proc lowerStmt(ctx: var LowerToLirCtx, node: HirNode) =
of hFieldPtr:
let lval = buildLval(ctx, node.assignTarget)
b.emitRawC(&"{lval} = {lirValToC(value)};")
of hFieldAccess:
let lval = buildLval(ctx, node.assignTarget)
b.emitRawC(&"{lval} = {lirValToC(value)};")
of hArrowField:
let lval = buildLval(ctx, node.assignTarget)
b.emitRawC(&"{lval} = {lirValToC(value)};")
@@ -731,9 +811,8 @@ proc lowerStmt(ctx: var LowerToLirCtx, node: HirNode) =
for stmt in node.blockStmts:
lowerStmt(ctx, stmt)
if node.blockExpr != nil:
let exprVal = lowerExpr(ctx, node.blockExpr)
# If block is an expression, store result
discard
# If block is an expression, result is unused at statement level
discard lowerExpr(ctx, node.blockExpr)
if node.isScope:
b.emitRawC("}")
@@ -743,9 +822,8 @@ proc lowerStmt(ctx: var LowerToLirCtx, node: HirNode) =
# ── Expression statement ──
else:
let exprVal = lowerExpr(ctx, node)
# Expression evaluated for side effects; temp is unused
discard
discard lowerExpr(ctx, node)
# ── Module-level lowering ──
@@ -765,6 +843,13 @@ proc lowerModuleToLir*(hirMod: HirModule): LirBuilder =
ctx.funcRetType = retCT
ctx.builder.beginFunc(f.name, params, retCT, f.isPublic)
# Closure thunks: materialize env from fat-func env pointer (by-value copy)
if f.captureNames.len > 0 and f.envStructName.len > 0 and f.envInstanceName.len > 0:
ctx.builder.emitRawC(&"struct {f.envStructName} {f.envInstanceName} = *((struct {f.envStructName}*)__env);")
elif f.params.len > 0 and f.params[0].name == "__env":
# Capture-less closure thunk still receives env
ctx.builder.emitRawC("(void)__env;")
if f.body != nil:
if f.body.kind == hBlock:
for stmt in f.body.blockStmts:
File diff suppressed because it is too large Load Diff
-1
View File
@@ -90,7 +90,6 @@ proc parseInlineTable(s: string): OrderedTableRef[string, TomlValue] =
if content[i] == '{': inc braceCount
elif content[i] == '}': dec braceCount
inc i
let val = content[valStart ..< i]
result[key] = TomlValue(kind: tvkInlineTable, inlineVal: newOrderedTable[string, TomlValue]())
else:
while i < content.len and content[i] notin {',', ' ', '\t'}:
+510 -24
View File
@@ -1,5 +1,5 @@
import std/strutils
import token, source_location, ast
import token, source_location, ast, lexer
type
ParserDiagnosticSeverity* = enum
@@ -21,12 +21,14 @@ type
pos: int
diagnostics: seq[ParserDiagnostic]
structInitAllowed: bool ## disabled inside if/while/for/match conditions
macroTemplateMode: bool ## true while parsing macro! rule body (allows $(…)*)
proc initParser*(tokens: seq[Token], sourceName: string = "<input>"): Parser =
result.tokens = tokens
result.sourceName = sourceName
result.pos = 0
result.structInitAllowed = true
result.macroTemplateMode = false
# ---------------------------------------------------------------------------
# Token helpers
@@ -48,6 +50,12 @@ proc advance(p: var Parser): Token =
if p.pos < p.tokens.len:
inc p.pos
proc hasErrors*(p: Parser): bool =
for d in p.diagnostics:
if d.severity == pdsError:
return true
return false
proc check(p: Parser, kind: TokenKind): bool =
p.peek() == kind
@@ -56,21 +64,24 @@ proc checkAny(p: Parser, kinds: openArray[TokenKind]): bool =
if p.peek() == k: return true
return false
proc match(p: var Parser, kind: TokenKind): bool =
if p.check(kind):
discard p.advance()
return true
return false
proc isTypeArgListAhead(p: Parser): bool =
## Lookahead to determine if '<' starts a type argument list.
## Returns true if we can find a matching '>' before EOF, '{', or ';'.
## Lookahead to determine if '<' starts a type argument list (`Foo<int>`).
## Returns true if we find a matching '>' before tokens that cannot appear
## inside type arguments. Must NOT match comparison `x < 0` when a later
## `x > 0` exists in the same expression/region (e.g. match arm guards).
if not p.check(tkLt): return false
var depth = 0
var ahead = 0
while true:
let kind = p.peek(ahead)
if kind == tkEndOfFile or kind == tkLBrace or kind == tkSemicolon:
# Hard stops: cannot appear inside `<...>` type args
if kind in {tkEndOfFile, tkLBrace, tkRBrace, tkSemicolon, tkFatArrow,
tkIf, tkElse, tkWhile, tkFor, tkMatch, tkReturn, tkLet, tkVar,
tkEq, tkNe, tkLe, tkGe, tkAmpAmp, tkPipePipe, tkAssign}:
return false
# Literals and comparison/arithmetic mean this is a value expression, not types
if kind in {tkIntLiteral, tkFloatLiteral, tkStringLiteral, tkCharLiteral,
tkBoolLiteral, tkPlus, tkMinus, tkSlash, tkPercent}:
return false
if kind == tkLt:
inc depth
@@ -143,7 +154,7 @@ proc synchronize(p: var Parser) =
if p.previous.kind == tkSemicolon: return
case p.peek()
of tkFunc, tkStruct, tkEnum, tkUnion, tkInterface, tkExtend,
tkModule, tkImport, tkConst, tkType, tkExtern, tkPub:
tkModule, tkImport, tkConst, tkType, tkExtern, tkPub, tkMacro:
return
else:
discard p.advance()
@@ -169,14 +180,26 @@ type
callConv*: CallingConvention
targetOs*: string
checked*: bool ## @[Checked] — enable borrow checking
shared*: bool ## @[Shared] — mark function as thread-safe
drop*: bool ## @[Drop] — auto-call Type_Drop at scope exit
release*: bool ## @[Release] — explicit zero-cost (no borrow checks)
proc parseAttrs(p: var Parser): ParsedAttrs =
while p.check(tkAt):
while true:
p.skipNewlines()
if not p.check(tkAt):
break
discard p.advance() # @
discard p.expect(tkLBracket, "expected '[' after '@'")
let name = p.expect(tkIdent, "expected attribute name").text
if name == "Checked":
result.checked = true
elif name == "Shared":
result.shared = true
elif name == "Drop":
result.drop = true
elif name == "Release":
result.release = true
elif name == "Import":
discard p.expect(tkLParen, "expected '('")
let key = p.expect(tkIdent, "expected attribute key").text
@@ -201,6 +224,10 @@ proc parseBaseType(p: var Parser): TypeExpr =
var typeArgs: seq[TypeExpr] = @[]
discard p.advance()
while not p.check(tkGt) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkGt) or p.isAtEnd:
break
typeArgs.add(p.parseType())
if p.check(tkComma):
discard p.advance()
@@ -235,6 +262,20 @@ proc parseBaseType(p: var Parser): TypeExpr =
discard p.advance()
discard p.expect(tkRParen, "expected ')' to close tuple type")
return TypeExpr(kind: tekTuple, loc: loc, tupleElements: elems)
of tkFunc:
discard p.advance() # func
discard p.expect(tkLParen, "expected '(' after 'func'")
var params: seq[TypeExpr] = @[]
while not p.check(tkRParen) and not p.isAtEnd:
params.add(p.parseType())
if p.check(tkComma):
discard p.advance()
discard p.expect(tkRParen, "expected ')' to close func param types")
var ret: TypeExpr = nil
if p.check(tkArrow):
discard p.advance()
ret = p.parseType()
return TypeExpr(kind: tekFunc, loc: loc, funcParams: params, funcRet: ret)
else:
p.emitError(loc, "expected type expression")
return TypeExpr(kind: tekNamed, loc: loc, typeName: "")
@@ -303,13 +344,21 @@ proc parsePrimaryPattern(p: var Parser): Pattern =
return Pattern(kind: pkEnum, loc: loc, patEnumPath: path, patEnumArgs: args, patEnumNamed: named)
return Pattern(kind: pkEnum, loc: loc, patEnumPath: path, patEnumArgs: @[], patEnumNamed: @[])
elif p.check(tkLBrace):
# Struct pattern: Point { x: 0, y: 0 }
# Struct pattern: Point { x: px, y: py } or shorthand Point { x, y }
discard p.advance()
var fields: seq[tuple[name: string, pattern: Pattern]] = @[]
while not p.check(tkRBrace) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkRBrace) or p.isAtEnd:
break
let fieldName = p.expect(tkIdent, "expected field name in struct pattern").text
discard p.expect(tkColon, "expected ':' after field name in pattern")
if p.check(tkColon):
discard p.advance()
fields.add((fieldName, p.parsePattern()))
else:
# Shorthand: { x } means { x: x }
fields.add((fieldName, Pattern(kind: pkIdent, loc: loc, patIdent: fieldName)))
if p.check(tkComma):
discard p.advance()
discard p.expect(tkRBrace, "expected '}' to close struct pattern")
@@ -346,8 +395,8 @@ proc parsePattern(p: var Parser): Pattern =
let inclusive = p.check(tkDotDotEqual)
discard p.advance()
let right = p.parsePrimaryPattern()
return Pattern(kind: pkRange, loc: loc, patRangeLo: left, patRangeHi: right, patRangeInclusive: inclusive)
# Guarded pattern
left = Pattern(kind: pkRange, loc: loc, patRangeLo: left, patRangeHi: right, patRangeInclusive: inclusive)
# Guarded pattern: `p if cond` (also after range: `1..10 if x % 2 == 0`)
if p.check(tkIf):
discard p.advance()
let guard = p.parseExpr()
@@ -380,11 +429,113 @@ proc parseAssign(p: var Parser): Expr
proc parseExpr(p: var Parser): Expr =
p.parseAssign()
proc isMacroStmtStart(p: Parser): bool =
## Keywords that begin a statement (for `$s:stmt` call-site args).
p.peek() in {tkLet, tkVar, tkIf, tkWhile, tkFor, tkLoop, tkMatch, tkReturn,
tkBreak, tkContinue, tkDefer, tkSwitch, tkDo}
proc parseMacroArg(p: var Parser): Expr =
## Macro call argument:
## - statement keywords → ekMacroStmt
## - `_` / pattern-only starts → ekMacroPat (also `$p:pat` from expr via coerce)
## - else expression
let loc = p.currentLoc
if p.isMacroStmtStart():
let st = p.parseStmt()
return Expr(kind: ekMacroStmt, loc: loc, exprMacroStmt: st)
# Wildcard is not a valid expression; parse as pattern for `$p:pat`
if p.check(tkUnderscore):
let pat = p.parsePattern()
return Expr(kind: ekMacroPat, loc: loc, exprMacroPat: pat)
p.parseExpr()
proc parseMacroRepExpr(p: var Parser): Expr =
## Expression-level `$( body ),*` or `$( body )*` inside call args (templates only).
let loc = p.currentLoc
discard p.expect(tkDollar, "expected '$' to start expression macro rep")
discard p.expect(tkLParen, "expected '(' after '$' in expression macro rep")
let body = p.parseExpr()
discard p.expect(tkRParen, "expected ')' after expression macro rep body")
# Optional separator token before Kleene star: `),*` vs `)*`
if p.check(tkComma):
discard p.advance()
discard p.expect(tkStar, "expected '*' after expression macro rep")
return Expr(kind: ekMacroRep, loc: loc, exprMacroRepBody: body)
proc parseStringInterpolation(p: var Parser, tok: Token): Expr =
## Parse a string literal that contains {expr} interpolations.
let text = tok.text
# Only f"..." strings support interpolation; plain "...", backtick, and c8/c16/c32 are raw.
if text.len < 3 or text[0] != 'f' or text[1] != '"' or text[^1] != '"':
return newLiteralExpr(tok)
let inner = text[2 ..< ^1] # strip f" ... "
var texts: seq[string] = @[]
var exprs: seq[Expr] = @[]
var i = 0
var currentText = ""
while i < inner.len:
if inner[i] == '\\' and i + 1 < inner.len:
if inner[i + 1] == '{':
currentText.add('{')
i += 2
continue
elif inner[i + 1] == '}':
currentText.add('}')
i += 2
continue
elif inner[i] == '{':
texts.add(currentText)
currentText = ""
var j = i + 1
var depth = 1
while j < inner.len and depth > 0:
if inner[j] == '{': depth += 1
elif inner[j] == '}': depth -= 1
j += 1
if depth != 0:
p.diagnostics.add(ParserDiagnostic(severity: pdsError, loc: tok.loc,
message: "unmatched '{' in string interpolation"))
return newLiteralExpr(tok)
let exprStr = inner[i + 1 ..< j - 1]
if exprStr.len == 0:
p.diagnostics.add(ParserDiagnostic(severity: pdsError, loc: tok.loc,
message: "empty interpolation {}"))
return newLiteralExpr(tok)
let lexRes = tokenize(exprStr, p.sourceName & "<interp>")
if lexRes.hasErrors:
p.diagnostics.add(ParserDiagnostic(severity: pdsError, loc: tok.loc,
message: "invalid expression in string interpolation: " & exprStr))
return newLiteralExpr(tok)
var subParser = initParser(lexRes.tokens, p.sourceName & "<interp>")
let expr = subParser.parseExpr()
if subParser.hasErrors:
for d in subParser.diagnostics:
p.diagnostics.add(ParserDiagnostic(severity: pdsError, loc: tok.loc,
message: "interpolation parse error: " & d.message))
return newLiteralExpr(tok)
exprs.add(expr)
i = j
continue
currentText.add(inner[i])
i += 1
texts.add(currentText)
if exprs.len == 0:
# f"plain" / f"use \{x\}" with no real {expr} — use processed text (escapes applied)
var litTok = tok
litTok.text = "\"" & texts[0] & "\""
return newLiteralExpr(litTok)
return newStringInterpExpr(texts, exprs, tok.loc)
proc parsePrimary(p: var Parser): Expr =
while p.check(tkNewLine):
discard p.advance()
let loc = p.currentLoc
case p.peek()
of tkIntLiteral, tkFloatLiteral, tkStringLiteral, tkCharLiteral, tkBoolLiteral:
return newLiteralExpr(p.advance())
let tok = p.advance()
if tok.kind == tkStringLiteral:
return parseStringInterpolation(p, tok)
return newLiteralExpr(tok)
of tkSelf:
discard p.advance()
return Expr(kind: ekSelf, loc: loc)
@@ -433,9 +584,16 @@ proc parsePrimary(p: var Parser): Expr =
p.structInitAllowed = false
let subject = p.parseExpr()
p.structInitAllowed = true
# Same newline rules as statement-form match (needed for `let x = match ...`)
while p.check(tkNewLine):
discard p.advance()
discard p.expect(tkLBrace, "expected '{' to start match")
var arms: seq[MatchArm] = @[]
while not p.check(tkRBrace) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkRBrace) or p.isAtEnd:
break
let armLoc = p.currentLoc
let pat = p.parsePattern()
discard p.expect(tkFatArrow, "expected '=>' in match arm")
@@ -490,6 +648,37 @@ proc parsePrimary(p: var Parser): Expr =
of tkNull:
discard p.advance()
return newLiteralExpr(Token(kind: tkNull, text: "null", loc: loc))
of tkPipePipe:
# Empty-param closure written as `||` — lexer merges two '|' into tkPipePipe.
# Disambiguate from logical-or (which only appears mid-expression): as a primary,
# `|| -> T { ... }` / `|| { ... }` is always a zero-param closure.
discard p.advance() # ||
var retTypePP: TypeExpr = nil
if p.check(tkArrow):
discard p.advance() # ->
retTypePP = p.parseType()
let bodyPP = p.parseBlock()
return Expr(kind: ekClosure, loc: loc, exprClosureParams: @[], exprClosureBody: bodyPP, exprClosureReturnType: retTypePP)
of tkPipe:
# Closure: |params| -> Ret { body }
discard p.advance() # |
var params: seq[Param] = @[]
while not p.check(tkPipe) and not p.isAtEnd:
let nameTok = p.expect(tkIdent, "expected parameter name in closure")
discard p.expect(tkColon, "expected ':' in closure parameter")
let ptype = p.parseType()
params.add(Param(name: nameTok.text, ptype: ptype))
if p.check(tkComma):
discard p.advance()
else:
break
discard p.expect(tkPipe, "expected '|' to close closure params")
var retType: TypeExpr = nil
if p.check(tkArrow):
discard p.advance() # ->
retType = p.parseType()
let body = p.parseBlock()
return Expr(kind: ekClosure, loc: loc, exprClosureParams: params, exprClosureBody: body, exprClosureReturnType: retType)
else:
p.emitError(loc, "expected expression")
discard p.advance()
@@ -498,23 +687,39 @@ proc parsePrimary(p: var Parser): Expr =
proc parsePostfix(p: var Parser): Expr =
var left = p.parsePrimary()
while true:
while p.check(tkNewLine):
discard p.advance()
let loc = p.currentLoc
case p.peek()
of tkLParen:
# Call expression
discard p.advance()
var args: seq[Expr] = @[]
var argNames: seq[string] = @[]
while not p.check(tkRParen) and not p.isAtEnd:
if p.check(tkDotDotDot):
discard p.advance()
let operand = p.parseExpr()
args.add(Expr(kind: ekSpread, loc: operand.loc, exprSpreadOperand: operand))
argNames.add("")
elif p.macroTemplateMode and p.check(tkDollar) and p.peek(1) == tkLParen:
# Expression-level `$( expr ),*` / `$( expr )*` (session 84)
args.add(p.parseMacroRepExpr())
argNames.add("")
elif p.peek() == tkIdent and p.peek(1) == tkColon:
# Named argument: name: value
let nameTok = p.advance()
discard p.advance() # :
let value = p.parseExpr()
args.add(value)
argNames.add(nameTok.text)
else:
args.add(p.parseExpr())
argNames.add("")
if p.check(tkComma):
discard p.advance()
discard p.expect(tkRParen, "expected ')' to close call")
left = Expr(kind: ekCall, loc: loc, exprCallCallee: left, exprCallArgs: args)
left = Expr(kind: ekCall, loc: loc, exprCallCallee: left, exprCallArgs: args, exprCallArgNames: argNames)
of tkLt:
# Generic type arguments: Max<int>(10, 20)
# Only treat '<' as generic args if lookahead confirms a matching '>'
@@ -537,11 +742,15 @@ proc parsePostfix(p: var Parser): Expr =
discard p.expect(tkRBracket, "expected ']' to close index")
left = Expr(kind: ekIndex, loc: loc, exprIndexObj: left, exprIndexIdx: idx, exprIndexBoundsCheck: false)
of tkDot:
# Field expression or .await
# Field expression, tuple index (.0, .1), or .await
discard p.advance()
if p.check(tkAwait):
discard p.advance()
left = Expr(kind: ekAwait, loc: loc, exprAwaitOperand: left)
elif p.check(tkIntLiteral):
# Tuple element access: t.0 → field "_0"
let idxText = p.advance().text
left = Expr(kind: ekField, loc: loc, exprFieldObj: left, exprFieldName: "_" & idxText)
else:
let fieldName = p.expectIdentOrKeyword("expected field name after '.'").text
left = Expr(kind: ekField, loc: loc, exprFieldObj: left, exprFieldName: fieldName)
@@ -561,6 +770,42 @@ proc parsePostfix(p: var Parser): Expr =
left = Expr(kind: ekTry, loc: loc, exprTryOperand: left, exprTryType: nil)
of tkBang:
discard p.advance()
# name!(args) → declarative macro call (not unwrap)
if left.kind == ekIdent and p.check(tkLParen):
discard p.advance() # (
var margs: seq[Expr] = @[]
var groupLens: seq[int] = @[]
var curGroup = 0
while not p.check(tkRParen) and not p.isAtEnd:
p.skipNewlines()
if p.check(tkRParen): break
# `;` starts a new arg group for multi-rep patterns
if p.check(tkSemicolon):
discard p.advance()
groupLens.add(curGroup)
curGroup = 0
p.skipNewlines()
continue
margs.add(p.parseMacroArg())
inc curGroup
p.skipNewlines()
if p.check(tkComma):
discard p.advance()
elif p.check(tkSemicolon):
discard
# handled at loop top
else:
# allow end of args
discard
if curGroup > 0 or groupLens.len == 0:
groupLens.add(curGroup)
# single group of all args → empty groupLens means "one group" for expander
if groupLens.len == 1:
groupLens = @[]
discard p.expect(tkRParen, "expected ')' to close macro arguments")
left = Expr(kind: ekMacroCall, loc: loc, exprMacroName: left.exprIdent,
exprMacroArgs: margs, exprMacroGroupLens: groupLens)
else:
left = Expr(kind: ekUnwrap, loc: loc, exprUnwrapOperand: left)
of tkLBrace:
if p.structInitAllowed and left.kind in {ekIdent, ekPath, ekGenericCall}:
@@ -594,8 +839,18 @@ proc parsePostfix(p: var Parser): Expr =
return left
proc parseUnary(p: var Parser): Expr =
while p.check(tkNewLine):
discard p.advance()
let loc = p.currentLoc
case p.peek()
of tkBorrow:
discard p.advance() # borrow
discard p.expect(tkAmp, "expected '&' after 'borrow'")
var isMut = p.check(tkMut)
if isMut:
discard p.advance() # mut
let operand = p.parseUnary() # parse the moved value
return Expr(kind: ekBorrow, loc: loc, exprBorrowOperand: operand, exprBorrowMutable: isMut)
of tkBang, tkMinus, tkTilde, tkStar, tkAmp:
let op = p.advance().kind
if op == tkAmp and p.check(tkMut):
@@ -649,7 +904,6 @@ proc parseShift(p: var Parser): Expr =
return left
proc parseCast(p: var Parser): Expr =
let loc = p.currentLoc
var left = p.parseShift()
# 'as' and 'is' are handled in postfix for chaining
return left
@@ -776,7 +1030,26 @@ proc parseBlock(p: var Parser): Block =
# ---------------------------------------------------------------------------
proc parseStmt(p: var Parser): Stmt =
while p.check(tkNewLine):
discard p.advance()
let loc = p.currentLoc
# Macro template repetition: $( stmts… )*
if p.macroTemplateMode and p.check(tkDollar) and p.peek(1) == tkLParen:
discard p.advance() # $
discard p.advance() # (
var stmts: seq[Stmt] = @[]
while not p.check(tkRParen) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkRParen) or p.isAtEnd:
break
stmts.add(p.parseStmt())
discard p.expect(tkRParen, "expected ')' to close macro repetition")
discard p.expect(tkStar, "expected '*' after macro repetition")
if p.check(tkSemicolon):
discard p.advance()
return Stmt(kind: skMacroRep, loc: loc,
stmtMacroRepBody: Block(loc: loc, stmts: stmts))
case p.peek()
of tkLet, tkVar:
let isMut = p.peek() == tkVar
@@ -876,6 +1149,37 @@ proc parseStmt(p: var Parser): Stmt =
discard p.advance()
discard p.expect(tkRBrace, "expected '}' to close match")
return Stmt(kind: skExpr, loc: loc, stmtExpr: Expr(kind: ekMatch, loc: loc, exprMatchSubject: subject, exprMatchArms: arms))
of tkSwitch:
discard p.advance()
p.structInitAllowed = false
let subject = p.parseExpr()
p.structInitAllowed = true
while p.check(tkNewLine):
discard p.advance()
discard p.expect(tkLBrace, "expected '{' to start switch body")
var cases: seq[SwitchCase] = @[]
var defaultBlock: Block = nil
while not p.check(tkRBrace) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkRBrace) or p.isAtEnd:
break
if p.check(tkDefault):
discard p.advance()
discard p.expect(tkColon, "expected ':' after default")
defaultBlock = Block(loc: p.currentLoc, stmts: @[p.parseStmt()])
continue
if p.check(tkCase):
discard p.advance()
let caseVal = p.parseExpr()
discard p.expect(tkColon, "expected ':' after case value")
let caseBody = Block(loc: p.currentLoc, stmts: @[p.parseStmt()])
cases.add(SwitchCase(loc: caseVal.loc, caseValue: caseVal, caseBody: caseBody))
continue
# Unknown token in switch body — skip
discard p.advance()
discard p.expect(tkRBrace, "expected '}' to close switch")
return Stmt(kind: skSwitch, loc: loc, stmtSwitchExpr: subject, stmtSwitchCases: cases, stmtSwitchDefault: defaultBlock)
of tkReturn:
discard p.advance()
var val: Expr = nil
@@ -934,6 +1238,12 @@ proc parseStmt(p: var Parser): Stmt =
# No-op: emit literal 0 as expression statement
let zeroTok = Token(kind: tkIntLiteral, text: "0", loc: loc)
return Stmt(kind: skExpr, loc: loc, stmtExpr: Expr(kind: ekLiteral, loc: loc, exprLit: zeroTok))
of tkDefer:
discard p.advance()
let expr = p.parseExpr()
if p.check(tkSemicolon):
discard p.advance()
return Stmt(kind: skDefer, loc: loc, stmtDeferBody: expr)
of tkFunc, tkStruct, tkEnum, tkUnion, tkInterface, tkExtend, tkModule,
tkImport, tkConst, tkType, tkExtern, tkPub:
# Local declaration
@@ -954,6 +1264,10 @@ proc parseTypeParams(p: var Parser): seq[TypeParam] =
if p.check(tkLt):
discard p.advance()
while not p.check(tkGt) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkGt) or p.isAtEnd:
break
var name = ""
var isLifetime = false
if p.check(tkIdent):
@@ -986,8 +1300,11 @@ proc parseTypeParams(p: var Parser): seq[TypeParam] =
proc parseParamList(p: var Parser, allowVariadic: bool = false): seq[Param] =
discard p.expect(tkLParen, "expected '('")
while not p.check(tkRParen) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkRParen) or p.isAtEnd:
break
let loc = p.currentLoc
var isVar = false
if allowVariadic and p.check(tkDotDotDot):
discard p.advance()
let nameTok = p.at
@@ -1037,6 +1354,8 @@ proc parseFuncDecl(p: var Parser, isPublic: bool, isAsm: bool, attrs: ParsedAttr
discard p.advance()
var declAttrs: seq[string] = @[]
if attrs.checked: declAttrs.add("Checked")
if attrs.release: declAttrs.add("Release")
if attrs.drop: declAttrs.add("Drop")
return Decl(kind: dkFunc, loc: loc, isPublic: isPublic,
declAttrs: declAttrs,
declFuncAsm: isAsm, declFuncCallConv: attrs.callConv,
@@ -1045,7 +1364,7 @@ proc parseFuncDecl(p: var Parser, isPublic: bool, isAsm: bool, attrs: ParsedAttr
declFuncParams: params, declFuncReturnType: retType,
declFuncBody: body)
proc parseStructDecl(p: var Parser, isPublic: bool): Decl =
proc parseStructDecl(p: var Parser, isPublic: bool, attrs: ParsedAttrs = ParsedAttrs()): Decl =
let loc = p.currentLoc
discard p.expect(tkStruct, "expected 'struct'")
let name = p.expect(tkIdent, "expected struct name").text
@@ -1074,7 +1393,10 @@ proc parseStructDecl(p: var Parser, isPublic: bool): Decl =
if p.pos == startPos:
discard p.advance()
discard p.expect(tkRBrace, "expected '}' to close struct")
return Decl(kind: dkStruct, loc: loc, isPublic: isPublic,
var declAttrs: seq[string] = @[]
if attrs.drop: declAttrs.add("Drop")
if attrs.checked: declAttrs.add("Checked")
return Decl(kind: dkStruct, loc: loc, isPublic: isPublic, declAttrs: declAttrs,
declStructName: name, declStructTypeParams: typeParams,
declStructFields: fields)
@@ -1250,6 +1572,10 @@ proc parseUseDecl(p: var Parser, attrs: ParsedAttrs): Decl =
if p.check(tkLBrace):
discard p.advance()
while not p.check(tkRBrace) and not p.isAtEnd:
while p.check(tkNewLine):
discard p.advance()
if p.check(tkRBrace) or p.isAtEnd:
break
names.add(p.expect(tkIdent, "expected name in multi-import").text)
if p.check(tkComma):
discard p.advance()
@@ -1332,6 +1658,164 @@ proc parseExternDecl(p: var Parser, isPublic: bool, attrs: ParsedAttrs): Decl =
return Decl(kind: dkExternVar, loc: loc, isPublic: isPublic,
declExtVarName: vName, declExtVarType: vType)
proc parseMacroFragKind(p: var Parser, kindTok: Token): MacroFragKind =
case kindTok.text
of "expr": mfkExpr
of "ident": mfkIdent
of "tt": mfkTt
of "literal", "lit": mfkLiteral
of "block": mfkBlock
of "stmt": mfkStmt
of "pat", "pattern": mfkPat
of "type": mfkType
else:
p.emitError(kindTok.loc,
"unsupported macro fragment kind '" & kindTok.text &
"' (expr|ident|tt|literal|block|stmt|pat|type)")
mfkExpr
proc parseMacroFragment(p: var Parser): MacroFragment =
## $name:kind (single non-rep fragment)
let fragTok = p.expect(tkIdent, "expected $name fragment in macro pattern")
if not fragTok.text.startsWith("$"):
p.emitError(fragTok.loc, "macro fragment must start with '$' (e.g. $x:expr)")
discard p.expect(tkColon, "expected ':' after macro fragment name")
# `type` is a keyword (tkType); other kinds are bare idents
var kindTok: Token
if p.check(tkType):
kindTok = p.advance()
kindTok.text = "type"
else:
kindTok = p.expect(tkIdent, "expected fragment kind (expr|ident|tt|literal|block|stmt|pat|type)")
let k = p.parseMacroFragKind(kindTok)
result = MacroFragment(
name: fragTok.text,
kind: k,
names: @[fragTok.text],
kinds: @[k],
isRep: false,
repSep: "")
proc parseMacroRepGroup(p: var Parser): MacroFragment =
## $( $a:kind , $b:kind , … ) ,* or … )*
## Compound: multiple frags inside one rep → parallel lists (zipped).
discard p.expect(tkDollar, "expected '$'")
discard p.expect(tkLParen, "expected '(' after '$'")
p.skipNewlines()
var names: seq[string] = @[]
var kinds: seq[MacroFragKind] = @[]
while not p.check(tkRParen) and not p.isAtEnd:
let fragTok = p.expect(tkIdent, "expected $name inside repetition")
if not fragTok.text.startsWith("$"):
p.emitError(fragTok.loc, "macro fragment must start with '$'")
discard p.expect(tkColon, "expected ':' after fragment name")
var kindTok: Token
if p.check(tkType):
kindTok = p.advance()
kindTok.text = "type"
else:
kindTok = p.expect(tkIdent, "expected fragment kind")
names.add(fragTok.text)
kinds.add(p.parseMacroFragKind(kindTok))
p.skipNewlines()
if p.check(tkComma):
discard p.advance()
p.skipNewlines()
else:
break
if names.len == 0:
p.emitError(p.currentLoc, "empty macro repetition group")
names.add("$x")
kinds.add(mfkExpr)
discard p.expect(tkRParen, "expected ')' after repeated fragment(s)")
var sep = ""
if p.check(tkComma):
discard p.advance()
sep = ","
discard p.expect(tkStar, "expected '*' after macro repetition")
result = MacroFragment(
name: names[0],
kind: kinds[0],
names: names,
kinds: kinds,
isRep: true,
repSep: sep)
proc parseMacroDecl(p: var Parser, isPublic: bool): Decl =
## macro! name {
## ( $a:ident, $($x:expr),* ) => { … }
## ( $($a:expr, $b:expr),* ) => { … } # compound / zipped
## ( $($x:expr),* ; $($y:expr),* ) => { … } # multi-rep groups
## }
let loc = p.currentLoc
discard p.expect(tkMacro, "expected 'macro'")
discard p.expect(tkBang, "expected '!' after macro")
let name = p.expect(tkIdent, "expected macro name").text
p.skipNewlines()
discard p.expect(tkLBrace, "expected '{' to start macro body")
var rules: seq[MacroRule] = @[]
while not p.check(tkRBrace) and not p.isAtEnd:
p.skipNewlines()
if p.check(tkRBrace) or p.isAtEnd:
break
let rloc = p.currentLoc
discard p.expect(tkLParen, "expected '(' to start macro pattern")
var frags: seq[MacroFragment] = @[]
while not p.check(tkRParen) and not p.isAtEnd:
p.skipNewlines()
if p.check(tkRParen): break
# Group separator for multi-rep: `;` between pattern elements
if p.check(tkSemicolon):
discard p.advance()
p.skipNewlines()
continue
# $( … ),* compound or single rep
if p.check(tkDollar) and p.peek(1) == tkLParen:
frags.add(p.parseMacroRepGroup())
p.skipNewlines()
# optional `;` after rep continues with more elements
if p.check(tkSemicolon):
discard p.advance()
p.skipNewlines()
continue
if p.check(tkComma):
discard p.advance()
p.skipNewlines()
continue
# no more separators → only rparen expected next
break
else:
frags.add(p.parseMacroFragment())
p.skipNewlines()
if p.check(tkComma):
discard p.advance()
elif p.check(tkSemicolon):
discard p.advance()
elif p.check(tkDollar):
# Juxtaposition (session 86): `$f:ident $args:tt` without comma
continue
elif p.check(tkIdent) and p.at.text.startsWith("$"):
continue
else:
break
discard p.expect(tkRParen, "expected ')' to close macro pattern")
p.skipNewlines()
discard p.expect(tkFatArrow, "expected '=>' after macro pattern")
p.skipNewlines()
let savedTpl = p.macroTemplateMode
p.macroTemplateMode = true
let body = p.parseBlock()
p.macroTemplateMode = savedTpl
rules.add(MacroRule(loc: rloc, frags: frags, body: body))
p.skipNewlines()
if p.check(tkComma) or p.check(tkSemicolon):
discard p.advance()
discard p.expect(tkRBrace, "expected '}' to close macro")
if rules.len == 0:
p.emitError(loc, "macro '" & name & "' has no rules")
return Decl(kind: dkMacro, loc: loc, isPublic: isPublic,
declMacroName: name, declMacroRules: rules)
proc parseDecl(p: var Parser): Decl =
let loc = p.currentLoc
var isPublic = false
@@ -1358,7 +1842,7 @@ proc parseDecl(p: var Parser): Decl =
of tkFunc:
return p.parseFuncDecl(isPublic, false, attrs, isConst, isAsync)
of tkStruct:
return p.parseStructDecl(isPublic)
return p.parseStructDecl(isPublic, attrs)
of tkEnum:
return p.parseEnumDecl(isPublic)
of tkUnion:
@@ -1377,6 +1861,8 @@ proc parseDecl(p: var Parser): Decl =
return p.parseTypeAliasDecl(isPublic)
of tkExtern:
return p.parseExternDecl(isPublic, attrs)
of tkMacro:
return p.parseMacroDecl(isPublic)
else:
p.emitError(loc, "expected declaration")
p.synchronize()
+244
View File
@@ -0,0 +1,244 @@
## registry.nim — Bux package registry index (E.1 + HTTP URL)
##
## Index format (TOML-ish, one package per [[package]] table):
##
## [[package]]
## name = "greet"
## version = "0.1.0"
## source = "file:packages/greet" # relative to the index file
## description = "Hello helpers"
##
## [[package]]
## name = "net"
## version = "1.2.0"
## source = "https://github.com/bux-lang/net.git"
##
## Lookup order for the index:
## 1. $BUX_REGISTRY — local file path **or** http(s):// URL
## 2. ~/.bux/registry.toml
## 3. <repo>/config/registry.toml next to the compiler / cwd
##
## HTTP indices are downloaded to ~/.bux/cache/registry_http.toml.
## Relative file:/path: sources in a remote index are resolved against that
## cache directory — prefer absolute paths or git URLs for remote registries.
import std/[os, strutils, strformat, algorithm, osproc]
type
RegistryPackage* = object
name*: string
version*: string
source*: string ## raw source as written in the index
description*: string
resolvedPath*: string ## absolute path for file: sources (filled on load)
Registry* = object
path*: string ## index file path (local, or cached path for HTTP)
sourceUrl*: string ## non-empty when loaded from an HTTP URL
packages*: seq[RegistryPackage]
proc resolvePackageSource(pkg: var RegistryPackage, indexDir: string) =
if pkg.source.startsWith("file:"):
var p = pkg.source["file:".len .. ^1]
if p.startsWith("//"):
p = p[2 .. ^1]
if not p.isAbsolute:
p = indexDir / p
# Collapse ../ segments for cleaner lockfiles (session 82)
pkg.resolvedPath = expandFilename(p)
elif pkg.source.startsWith("path:"):
var p = pkg.source["path:".len .. ^1]
if not p.isAbsolute:
p = indexDir / p
pkg.resolvedPath = expandFilename(p)
pkg.source = "file:" & pkg.resolvedPath
proc parseRegistryToml(content, indexPath: string): seq[RegistryPackage] =
## Minimal parser for repeated [[package]] blocks with string keys.
result = @[]
var cur: RegistryPackage
var inPkg = false
let indexDir = indexPath.parentDir
for raw in content.splitLines():
let line = raw.strip()
if line.len == 0 or line.startsWith("#"):
continue
if line == "[[package]]" or line == "[[Package]]":
if inPkg and cur.name.len > 0:
resolvePackageSource(cur, indexDir)
result.add(cur)
cur = RegistryPackage()
inPkg = true
continue
if not inPkg:
continue
let eq = line.find('=')
if eq < 0: continue
let key = line[0 ..< eq].strip().toLowerAscii()
var val = line[eq + 1 .. ^1].strip()
if val.len >= 2 and val[0] == '"' and val[^1] == '"':
val = val[1 ..< ^1]
case key
of "name": cur.name = val
of "version": cur.version = val
of "source": cur.source = val
of "description": cur.description = val
else: discard
if inPkg and cur.name.len > 0:
resolvePackageSource(cur, indexDir)
result.add(cur)
proc isHttpUrl*(s: string): bool =
s.startsWith("http://") or s.startsWith("https://")
proc registryCacheDir*(): string =
getHomeDir() / ".bux" / "cache"
proc fetchRegistryUrl*(url: string): string =
## Download a remote registry.toml into ~/.bux/cache/.
## Returns the local cache path on success, or "" on failure.
## Re-fetches when the URL changes or when $BUX_REGISTRY_REFRESH is set.
let cacheDir = registryCacheDir()
try:
createDir(cacheDir)
except OSError, IOError:
return ""
let cachePath = cacheDir / "registry_http.toml"
let metaPath = cacheDir / "registry_http.url"
let force = getEnv("BUX_REGISTRY_REFRESH").len > 0
var cachedUrl = ""
if fileExists(metaPath):
try:
cachedUrl = readFile(metaPath).strip()
except CatchableError:
cachedUrl = ""
if not force and fileExists(cachePath) and cachedUrl == url:
return cachePath.absolutePath
# Prefer curl; fall back to wget.
# BUX_REGISTRY_INSECURE=1 → allow self-signed HTTPS (dev / smoke only).
let insecure = getEnv("BUX_REGISTRY_INSECURE").len > 0
var ok = false
if findExe("curl").len > 0:
let kflag = if insecure: " -k" else: ""
let cmd = &"curl -fsSL{kflag} --max-time 30 -o {quoteShell(cachePath)} {quoteShell(url)}"
let (_, code) = execCmdEx(cmd)
ok = code == 0 and fileExists(cachePath) and getFileSize(cachePath) > 0
elif findExe("wget").len > 0:
let nflag = if insecure: " --no-check-certificate" else: ""
let cmd = &"wget -q{nflag} -T 30 -O {quoteShell(cachePath)} {quoteShell(url)}"
let (_, code) = execCmdEx(cmd)
ok = code == 0 and fileExists(cachePath) and getFileSize(cachePath) > 0
else:
return ""
if not ok:
return ""
try:
writeFile(metaPath, url & "\n")
except CatchableError:
discard
return cachePath.absolutePath
proc findRegistryIndex*(): tuple[path: string, url: string] =
## Locate the registry index. Returns (localPath, sourceUrl).
## sourceUrl is non-empty only when the index was (or should be) fetched via HTTP.
result = ("", "")
let env = getEnv("BUX_REGISTRY")
if env.len > 0:
if isHttpUrl(env):
let local = fetchRegistryUrl(env)
if local.len > 0:
return (local, env)
return ("", env) # URL set but fetch failed — caller can report
if fileExists(env):
return (env.absolutePath, "")
let homeIdx = getHomeDir() / ".bux" / "registry.toml"
if fileExists(homeIdx):
return (homeIdx, "")
let candidates = @[
getAppDir() / ".." / "config" / "registry.toml",
getAppDir() / "config" / "registry.toml",
getCurrentDir() / "config" / "registry.toml",
getCurrentDir() / ".." / "config" / "registry.toml",
]
for c in candidates:
if fileExists(c):
return (c.absolutePath, "")
return ("", "")
proc loadRegistry*(path: string = ""): Registry =
result.path = ""
result.sourceUrl = ""
result.packages = @[]
if path.len > 0:
if isHttpUrl(path):
result.sourceUrl = path
result.path = fetchRegistryUrl(path)
else:
result.path = path
else:
let (p, u) = findRegistryIndex()
result.path = p
result.sourceUrl = u
if result.path.len == 0 or not fileExists(result.path):
return
try:
let content = readFile(result.path)
result.packages = parseRegistryToml(content, result.path)
except CatchableError:
result.packages = @[]
proc registryLookup*(reg: Registry, name: string, versionReq: string = "*"): RegistryPackage =
## Find a package by name. versionReq `*` picks the last matching entry
## (index order; put newest last). Exact version matches preferred.
result = RegistryPackage()
var candidates: seq[RegistryPackage] = @[]
for p in reg.packages:
if p.name.toLowerAscii() == name.toLowerAscii():
candidates.add(p)
if candidates.len == 0:
return
if versionReq.len == 0 or versionReq == "*":
return candidates[^1]
for p in candidates:
if p.version == versionReq:
return p
# Semver prefix match: "1" matches "1.0.0"
for p in candidates:
if p.version.startsWith(versionReq):
return p
return candidates[^1]
proc registrySearch*(reg: Registry, query: string): seq[RegistryPackage] =
result = @[]
let q = query.toLowerAscii()
for p in reg.packages:
if q.len == 0 or
q in p.name.toLowerAscii() or
q in p.description.toLowerAscii():
result.add(p)
result.sort(proc (a, b: RegistryPackage): int =
cmp(a.name.toLowerAscii(), b.name.toLowerAscii()))
proc isGitSource*(source: string): bool =
source.startsWith("http://") or source.startsWith("https://") or
source.startsWith("git@") or source.startsWith("git://") or
source.startsWith("ssh://")
proc isFileSource*(source: string): bool =
source.startsWith("file:") or source.startsWith("path:")
proc formatRegistryDepLine*(name: string, pkg: RegistryPackage): string =
## Produce a bux.toml Dependencies line for a resolved registry package.
if pkg.resolvedPath.len > 0 and dirExists(pkg.resolvedPath):
return &"{name} = {{ Path = \"{pkg.resolvedPath}\" }}"
if isGitSource(pkg.source):
let ver = if pkg.version.len > 0: pkg.version else: "*"
return &"{name} = {{ Version = \"{ver}\", Source = \"{pkg.source}\" }}"
if isFileSource(pkg.source) and pkg.resolvedPath.len > 0:
return &"{name} = {{ Path = \"{pkg.resolvedPath}\" }}"
# Fallback: version-only (install will re-resolve)
let ver = if pkg.version.len > 0: pkg.version else: "*"
return &"{name} = \"{ver}\""
+10
View File
@@ -44,3 +44,13 @@ proc lookupLocal*(scope: Scope, name: string): Symbol =
if scope.table.hasKey(name):
return scope.table[name]
return nil
proc lookupUpTo*(scope: Scope, name: string, limit: Scope): Symbol =
var cur = scope
while cur != nil:
if cur.table.hasKey(name):
return cur.table[name]
if cur == limit:
break
cur = cur.parent
return nil
+956 -117
View File
File diff suppressed because it is too large Load Diff
+4
View File
@@ -3,6 +3,10 @@ type
line*: uint32 ## 1-based
column*: uint32 ## 1-based (UTF-8 byte offset in line)
offset*: uint32 ## byte offset from start of file
file*: string ## source file path (empty if unknown)
proc `$`*(loc: SourceLocation): string =
if loc.file.len > 0:
loc.file & ":" & $loc.line & ":" & $loc.column
else:
$loc.line & ":" & $loc.column
+22 -2
View File
@@ -25,6 +25,9 @@ type
tkContinue # continue
tkReturn # return
tkMatch # match
tkSwitch # switch
tkCase # case
tkDefault # default
##Declaration keywords
tkFunc # func
@@ -51,6 +54,7 @@ type
tkSizeOf # sizeof
tkOwn # own (gradual ownership transfer)
tkMut # mut (mutable reference)
tkBorrow # borrow (explicit borrow expression)
tkDiscard # discard (evaluate and throw away)
tkAsync # async
tkAwait # await
@@ -58,7 +62,10 @@ type
tkStaticAssert # static_assert
tkComptime # comptime
tkDyn # dyn
tkDefer # defer
tkLifetime # 'a (lifetime parameter)
tkMacro # macro (declarative macro! definitions)
tkDollar # bare $ for macro rep $( ... )*
##Punctuation
tkLParen # (
@@ -148,10 +155,10 @@ type
proc isKeyword*(kind: TokenKind): bool =
case kind
of tkIf, tkElse, tkWhile, tkDo, tkLoop, tkFor, tkIn,
tkBreak, tkContinue, tkReturn, tkMatch,
tkBreak, tkContinue, tkReturn, tkMatch, tkSwitch, tkCase, tkDefault,
tkFunc, tkLet, tkVar, tkConst, tkType, tkStruct, tkEnum,
tkUnion, tkInterface, tkExtend, tkModule, tkImport,
tkPub, tkExtern, tkAs, tkIs, tkNull, tkSelf, tkSuper, tkOwn, tkMut, tkDiscard, tkAsync, tkAwait, tkSpawn, tkStaticAssert, tkComptime, tkDyn:
tkPub, tkExtern, tkAs, tkIs, tkNull, tkSelf, tkSuper, tkOwn, tkMut, tkBorrow, tkDiscard, tkAsync, tkAwait, tkSpawn, tkStaticAssert, tkComptime, tkDyn, tkDefer:
true
else:
false
@@ -199,6 +206,9 @@ proc keywordKind*(text: string): TokenKind =
of "continue": tkContinue
of "return": tkReturn
of "match": tkMatch
of "switch": tkSwitch
of "case": tkCase
of "default": tkDefault
of "as": tkAs
of "is": tkIs
of "null": tkNull
@@ -207,6 +217,7 @@ proc keywordKind*(text: string): TokenKind =
of "sizeof": tkSizeOf
of "own": tkOwn
of "mut": tkMut
of "borrow": tkBorrow
of "discard": tkDiscard
of "async": tkAsync
of "await": tkAwait
@@ -214,6 +225,8 @@ proc keywordKind*(text: string): TokenKind =
of "static_assert": tkStaticAssert
of "comptime": tkComptime
of "dyn": tkDyn
of "defer": tkDefer
of "macro": tkMacro
of "true", "false": tkBoolLiteral
else: tkIdent
@@ -238,6 +251,9 @@ proc tokenKindName*(kind: TokenKind): string =
of tkContinue: "'continue'"
of tkReturn: "'return'"
of tkMatch: "'match'"
of tkSwitch: "'switch'"
of tkCase: "'case'"
of tkDefault: "'default'"
of tkFunc: "'func'"
of tkLet: "'let'"
of tkVar: "'var'"
@@ -259,6 +275,7 @@ proc tokenKindName*(kind: TokenKind): string =
of tkSuper: "'super'"
of tkOwn: "'own'"
of tkMut: "'mut'"
of tkBorrow: "'borrow'"
of tkDiscard: "'discard'"
of tkAsync: "'async'"
of tkAwait: "'await'"
@@ -266,6 +283,9 @@ proc tokenKindName*(kind: TokenKind): string =
of tkStaticAssert: "'static_assert'"
of tkComptime: "'comptime'"
of tkDyn: "'dyn'"
of tkDefer: "'defer'"
of tkMacro: "'macro'"
of tkDollar: "'$'"
of tkLifetime: "lifetime"
of tkLParen: "'('"
of tkRParen: "')'"
+4
View File
@@ -183,6 +183,10 @@ proc isAssignableTo*(a, b: Type): bool =
return true
if b.isDynRef and a.isMutRef:
return true
# &mut func(...) -> func(...) (function pointer decay)
if a.isMutRef and b.kind == tkFunc:
if a.inner.len > 0 and a.inner[0].kind == tkFunc:
return true
return false
# String representation
+1 -1
View File
@@ -1,6 +1,6 @@
[Package]
Name = "buxc"
Version = "0.3.0"
Version = "1.0.0"
Type = "bin"
[Build]
+24
View File
@@ -0,0 +1,24 @@
# Bux package registry index (E.1 + HTTP URL)
# Used by `bux add <name>` and `bux install` when no --path/--git is given.
#
# Override with: export BUX_REGISTRY=/path/to/registry.toml
# or: export BUX_REGISTRY=https://example.com/registry.toml
# Or copy to: ~/.bux/registry.toml
# Force HTTP re-fetch: BUX_REGISTRY_REFRESH=1
#
# source forms:
# file:relative/or/absolute — local package (relative to this file)
# path:relative/or/absolute — same as file:
# https://...git — git clone into ~/.bux/packages/<name>
[[package]]
name = "greet"
version = "0.1.0"
source = "file:../registry/packages/greet"
description = "Tiny Hello helper library (demo registry package)"
[[package]]
name = "greet"
version = "0.1.1"
source = "file:../registry/packages/greet"
description = "Tiny Hello helper library (demo registry package, patch)"
+234 -5
View File
@@ -20,7 +20,7 @@ On macOS:
brew install nim gcc make openssl
```
> **Note:** The `Std::Crypto` module requires OpenSSL (`-lcrypto`). The build system links it automatically.
> **Note:** Crypto + TLS require OpenSSL (`-lssl -lcrypto`). The build system links both automatically on the full (POSIX) runtime.
---
@@ -50,6 +50,10 @@ This compiles `buxc2` using the bootstrap compiler. The self-hosted compiler gen
# Create a new package in a new directory
./buxc new myproject
cd myproject
# Or initialize in the current directory
mkdir myproject && cd myproject
./buxc init
```
This generates:
@@ -89,8 +93,13 @@ Output = "Bin"
./buxc run
./buxc run ./myproject
# Run tests
# Run tests (builds and runs the binary, reports pass/fail)
./buxc test
./buxc test ./myproject
# Format code
./buxc fmt src/Main.bux # single file
./buxc fmt src/ # all .bux files in directory
# Clean build artifacts
./buxc clean
@@ -98,21 +107,238 @@ Output = "Bin"
Build output goes to `build/` by default.
### Cross-Compilation, static, and thin runtime (Linux-first)
Bux targets **Linux** (primary), **cloud/containers**, and **embedded/cross**. Windows is not a product focus.
```bash
# Thin runtime (no pthread / OpenSSL / sockets) — good for CLI & embed
BUX_RUNTIME=minimal ./buxc build
# Fully-static binary (implies minimal runtime; container / distroless friendly)
./buxc --static --release build
# same: BUX_STATIC=1 ./buxc --release build
# Cross-compile for ARM64 Linux (prefers aarch64-linux-gnu-gcc, else clang -target)
./buxc --static --release --target aarch64-linux-gnu build
# Cross-compile for RISC-V 64 (session 85; needs riscv64-linux-gnu-gcc)
./buxc --static --release --target riscv64-linux-gnu build
# Override C compiler
BUX_CC=aarch64-linux-gnu-gcc ./buxc --static --target aarch64-linux-gnu build
# musl fully-static (Alpine-friendly; needs musl-tools or zig)
BUX_CC=musl-gcc BUX_RUNTIME=minimal ./buxc --static --release build
# or: BUX_CC='zig cc -target x86_64-linux-musl' … (use a wrapper script)
make test-musl-static # SKIP if no musl-gcc/zig
```
| Switch / env | Effect |
|--------------|--------|
| `BUX_RUNTIME=full` | `rt/runtime.c` — POSIX + OpenSSL (default on Unix) |
| `BUX_RUNTIME=minimal` / `thin` / `embed` | `rt/runtime_minimal.c` — thin single-threaded |
| `BUX_RUNTIME=win` | `rt/runtime_win.c` — historical MinGW smoke only |
| `--static` / `BUX_STATIC=1` | `-static` link; defaults to minimal runtime |
| `--target <triple>` | Cross compile; defaults to minimal runtime |
| `BUX_CC` | Force C compiler binary |
| `BUX_CFLAGS` | Extra flags appended to the C line |
```bash
# Smoke all of the above (+ CTFE CRC + optional aarch64/riscv64 cross)
make test-linux-targets
# Build static hello for Docker scratch/distroless
./tools/build_static_hello.sh
docker build -f examples/docker/Dockerfile.static \
--build-arg BIN=build/hello_static -t bux-hello-static .
```
> **Note:** Full runtime + fully-static OpenSSL is intentionally not the default (painful). Use minimal for static containers; keep full runtime for servers that need net/crypto (`nexus`).
### Cross toolchains
| Triple | Typical package | Smoke |
|--------|-----------------|-------|
| `aarch64-linux-gnu` | `gcc-aarch64-linux-gnu` | `make test-linux-targets` (SKIP if missing) |
| `riscv64-linux-gnu` | `gcc-riscv64-linux-gnu` | same (session 85) |
`clang -target <triple>` alone is **not** enough: you still need target headers and libc
(sysroot). Prefer `*-gcc` from a cross package, or set `BUX_CC` to a wrapper that
already knows the sysroot (e.g. Zig `zig cc -target …`).
### Freestanding / bare-metal (research spike, not v1.0)
`BUX_RUNTIME=minimal` / `--static` is the **Linux userspace / container / CTFE** path:
it still links against a libc (`malloc`, `printf`, `strlen`, …). It is *not* true
no-libc freestanding firmware.
| Layer | Status | Notes |
|-------|--------|-------|
| Thin runtime (no pthread/OpenSSL) | ✅ | `rt/runtime_minimal.c` |
| Static musl / distroless | ✅ | `make test-musl-static`, Dockerfiles |
| Linux multi-arch cross | ✅ | aarch64 + riscv64 smokes (SKIP without gcc) |
| True freestanding (`-ffreestanding`, no libc) | 🔬 spike | Needs custom alloc, panic, and I/O stubs |
| Cortex-M / qemu-system | 🔬 spike | Same; plus linker scripts and startup |
**Practical path today:** build with `BUX_RUNTIME=minimal --static --target …` for
Linux userspace on foreign ISAs; treat bare-metal as a research project that
starts from a custom `runtime_freestanding.c` (not shipped) and does **not**
import `Std::Net` / `Std::Task` / OpenSSL.
---
## Running Tests
### Example suite
```bash
make test-examples # all examples/ programs (40+)
make test-errors # golden Rust-style diagnostic output
make test-stdlib # stdlib golden packages
make test-registry # package registry (local + HTTP index)
make test-apps # showcase apps build + simpledb/jwt CLI smoke (in `make test`)
make test-dwarf # #line maps + .debug_info + --release (in `make test`)
make test-linux-targets # minimal + static + aarch64/riscv64 cross (SKIP) + CTFE CRC
make test-registry # package registry local + HTTP (in `make test`)
make test-selfhost-smoke # buxc2: move_field + multi-file #line (in `make test`)
make test-lsp # hover + references/rename + call hierarchy
make bench # micro-benchmarks (Bux + C/Nim/Zig twins)
make bench-nexus # wrk throughput vs apps/nexus /api/health
```
### Debug builds (E.4)
```bash
./buxc build # -O0 -g, #line → .bux (gdb-friendly; bootstrap)
./buxc build --release # -O2 -DNDEBUG, no #line / -g
# Selfhost (buxc2): default -O0 -g; multi-file #line from Decl.sourceFile
# (stdlib + each src/*.bux get their own path — no env needed)
export BUX_NO_LINE=1 # disable selfhost #line maps
export BUX_DEBUG_FILE=/abs/path.bux # optional: force all #line to one path
export BUX_CFLAGS="-fno-omit-frame-pointer"
gdb --args ./build/myapp
# (gdb) break Main
# (gdb) list # Bux source via #line (correct file per function)
```
### Compiler Tests
```bash
make test
```
This runs:
- Example suite (`test-examples`)
- Error diagnostic goldens (`test-errors`)
- Lexer unit tests
- Parser unit tests
- Semantic analysis unit tests
- HIR lowering unit tests
- Integration tests (`buxc new`, `buxc --version`)
- Golden C-codegen tests (8 examples)
### Project Tests (`bux test`)
```bash
./buxc test # run all tests/*.bux in the current package
./buxc test --filter first # only tests whose name contains "first"
./buxc test --filter=first _test_runner
```
Discovers `tests/*.bux`, builds each as a temp package, and runs it. Prints a
summary table and exits:
- `0` — all selected tests passed
- `1` — at least one failure, or no tests matched the filter
Use `Std::Test` module for assertions inside test code.
### Continuous integration
```bash
make test # full sequential suite (local)
```
| Workflow | When | What runs |
|----------|------|-----------|
| **`.github/workflows/ci.yml`** | every PR + push to `main` | **split jobs** (see below) + macOS smoke |
| **`.github/workflows/selfhost-loop.yml`** | weekly / manual / path-filtered main | `make selfhost-loop` |
**`ci.yml` layout (faster PR feedback):**
| Job | OS | Targets |
|-----|-----|---------|
| `build` | ubuntu | `make build` → upload `buxc` artifact |
| `unit` | ubuntu | `fmt-check` + `test-unit` (reuse artifact) |
| `examples` | ubuntu | `test-examples` (full list) |
| `goldens` | ubuntu | `test-errors` + `test-stdlib` + `test-registry` + `test-dwarf` + `test-drop-move` + `test-linux-targets` |
| `apps` | ubuntu | `test-apps` |
| `selfhost` | ubuntu | `test-selfhost-smoke` |
| `macos` | macos-14 | rebuild + `test-unit` + `test-examples-smoke` (subset) |
| `windows` | windows-latest | `buxc.exe` + Nim unit tests + CLI + **MinGW `hello`** |
| `ci-gate` | ubuntu | fails if any required job failed (branch protection) |
**CI speed helpers:**
- Pin Nim **2.0.8**; cache `.nim_runtime` (big win on macOS — Nim is built from source there;
Windows uses a prebuilt Nim zip)
- Project-local `nimcache/` via `NIMFLAGS=--nimcache:nimcache`, cached per job by source hash
- macOS skips full EXAMPLES (Linux already runs them) and skips `fmt-check` (Linux unit job)
- **Windows** runs `tools/smoke_windows_hello.sh` with **MinGW gcc** + `rt/runtime_win.c`
(no pthread/OpenSSL/ucontext). Full POSIX runtime (`rt/runtime.c`) remains Unix-only.
Locally on Linux/macOS: `BUX_RUNTIME=win ./tools/smoke_windows_hello.sh`.
Parallel Linux jobs set `BUX_SKIP_BUILD=1` after downloading the `buxc` artifact.
Locally, `make test` still runs the full suite sequentially and builds once.
`make test-examples-smoke` runs the macOS-sized subset locally.
`make test` includes examples, goldens, registry, apps, DWARF, unit tests, and selfhost smoke
(not the slow gen2↔gen3 fixed-point).
### Selfhost loop (optional CI)
```bash
make selfhost-loop # bootstrap builds src/ twice; C+ELF match
BUX_SELFHOST_FIXED_POINT=1 make selfhost-loop # buxc2→buxc3→buxc4 fixed-point
```
Fixed-point compares **gen2 vs gen3** (same selfhost C backend), not bootstrap
vs selfhost. Not part of default `make test`.
### Format (`bux fmt`)
```bash
./buxc fmt examples/hello.bux # reformat one file
./buxc fmt lib/ # reformat a directory tree
make fmt # reformat lib/ examples/ src/ tests/ apps/
./buxc fmt --check path/ # exit 1 if any file would change
make fmt-check # CI: full-tree clean + dirty smoke
```
Indentation is 4 spaces by brace depth. The formatter is idempotent (safe to re-run).
`make fmt-check` enforces a clean tree under `lib/`, `examples/`, `src/`, `tests/`, and `apps/`.
### Stdlib golden tests
```bash
make test-stdlib
# or: tests/stdlib_golden/run.sh ./buxc
```
Behavioral packages under `tests/stdlib_golden/` (`array`, `string`, `collections`)
assert core Array/String/Map/Set/Result/Option APIs and match expected `PASS` lines.
### API docs (`bux doc`)
```bash
./buxc doc lib/ # Markdown to stdout
./buxc doc --out docs/api/stdlib.md lib/
make docs # writes docs/api/stdlib.md
```
Scans `///` line comments (and bootstrap also accepts adjacent `/* */`) immediately
before `func` / `struct` / `enum` / `interface` / `module` declarations.
### Language Server (`bux-lsp` 0.4.0)
```bash
make lsp # → tools/bux-lsp
nim r --path:bootstrap tools/test_lsp_locals.nim
./tools/smoke_lsp_hover.sh
```
Features: diagnostics (`buxc check`), hover, go-to-def, outline, completion.
**Locals are position-sensitive** (nested scopes / shadowing). **Inferred `let` types**
appear on hover (`let x: int · inferred`).
### Example Programs
```bash
@@ -137,17 +363,18 @@ cd examples_pkg/hello && ../../buxc run
bux/
├── src/ # Self-hosted compiler source (Bux)
│ ├── Main.bux # Entry point
│ ├── Cli.bux # CLI commands
│ ├── Cli.bux # CLI commands (build, run, test, fmt, new, init)
│ ├── Lexer.bux # Tokenizer
│ ├── Parser.bux # Parser
│ ├── Ast.bux # AST definitions
│ ├── Sema.bux # Semantic analysis
│ ├── Sema.bux # Semantic analysis (borrow checker)
│ ├── Types.bux # Type system
│ ├── Scope.bux # Symbol table
│ ├── Hir.bux # High-level IR
│ ├── HirLower.bux # AST → HIR lowering
│ ├── CBackend.bux # HIR → C code generation
│ ├── Manifest.bux # bux.toml parser
│ ├── Fmt.bux # Code formatter
│ └── Token.bux # Token definitions
├── bootstrap/ # Bootstrap compiler (Nim) — compiles src/ → buxc
│ ├── main.nim
@@ -166,7 +393,9 @@ bux/
│ ├── Task.bux
│ └── Channel.bux
├── rt/ # C runtime
│ ├── runtime.c
│ ├── runtime.c # full POSIX + OpenSSL (Unix default)
│ ├── runtime_minimal.c # thin: no pthread/net/crypto (static/embed)
│ ├── runtime_win.c # MinGW historical (BUX_RUNTIME=win)
│ └── io.c
├── examples/ # Example programs
├── tests/ # Unit tests (Nim)
+785 -29
View File
@@ -16,10 +16,13 @@ This document describes the Bux programming language as implemented by the boots
8. [Pattern Matching](#pattern-matching)
9. [Methods and Interfaces](#methods-and-interfaces)
10. [Generics](#generics)
11. [Error Handling](#error-handling)
12. [Modules and Imports](#modules-and-imports)
13. [Async/Await](#asyncawait)
14. [Operators](#operators)
11. [Gradual Ownership](#gradual-ownership-phase-82--implemented) — Checked / Release / [Drop & RAII](#drop-and-raii)
12. [Error Handling](#error-handling)
13. [Modules and Imports](#modules-and-imports)
14. [Async/Await](#asyncawait)
15. [Operator Overloading](#operator-overloading)
16. [Operators](#operators)
17. [Macros](#macros)
---
@@ -43,7 +46,7 @@ Identifiers start with a letter or underscore, followed by letters, digits, or u
func, let, var, const, type, struct, enum, union, interface, extend
module, import, pub, extern, if, else, while, do, loop, for, in
break, continue, return, match, as, is, null, self, super, sizeof
async, await, spawn
async, await, spawn, defer, switch, case, default, checked
```
### String Literals
@@ -56,6 +59,7 @@ c32"Hello" // *char32
`line 1
line 2
line 3` // Newlines preserved as-is
f"Hello, {name}" // Interpolated string — expressions inside {}
```
**Backtick raw strings** (`` `...` ``) treat all characters literally:
@@ -63,6 +67,11 @@ line 3` // Newlines preserved as-is
- Actual newlines in source are preserved in the string
- No way to escape the backtick character itself (use regular strings if needed)
**Interpolated strings** (`f"..."`):
- Expressions inside `{}` are evaluated and converted to `String`
- Supported types: `int`, `uint`, `float`, `bool`, `String`
- Escaped braces: `\{` and `\}`
### Number Literals
```bux
42 // int
@@ -93,12 +102,55 @@ line 3` // Newlines preserved as-is
```bux
*T // Pointer to T
&T // Shared reference (read-only in checked functions)
&mut T // Mutable reference (exclusive borrow)
own T // Owned value (move semantics)
T[] // Slice (unsized)
T[N] // Fixed-size array
(T1, T2, T3) // Tuple
func(T1) -> T2 // Function type
(T1, T2, T3) // Tuple — access fields with .0, .1, .2
func(T1) -> T2 // Function pointer type
```
### Tuples
```bux
func Pair(a: int, b: int) -> (int, int) {
return (a, b);
}
func Main() -> int {
let t: (int, int) = Pair(10, 20);
PrintInt(t.0); // 10
PrintInt(t.1); // 20
return 0;
}
```
### Function pointers and closures
```bux
func Apply(f: func(int) -> int, x: int) -> int {
return f(x);
}
func Double(n: int) -> int { return n * 2; }
func MakeAdder(base: int) -> func(int) -> int {
// Each call allocates its own capture environment
return |a: int| -> int { return a + base; };
}
func Main() -> int {
let g: func(int) -> int = Double; // named func → fat pointer
let a10 = MakeAdder(10);
let a20 = MakeAdder(20);
// a10 and a20 are independent instances
return Apply(g, 21) + a10(1) + a20(1); // 42 + 11 + 21
}
```
`func(T) -> R` values are **fat pointers** `{ code, env }`:
- capturing closures store captures in a heap env
- capture-less closures and named functions use `env = null`
### Structs
```bux
struct Point {
@@ -161,6 +213,27 @@ func Min<T>(a: T, b: T) -> T {
}
return b;
}
// Named and default parameters
func HttpResponse(code: int = 200, body: String = "") -> Response { ... }
let r: Response = HttpResponse(body: "hello"); // code defaults to 200
let s: Response = HttpResponse(404, body: "err"); // positional + named mixed
// Operator overloading (bootstrap only)
func Vec2_operator_add(self: *Vec2, other: Vec2) -> Vec2 { ... }
func Vec2_operator_sub(self: *Vec2, other: Vec2) -> Vec2 { ... }
func Vec2_operator_eq(self: *Vec2, other: Vec2) -> bool { ... }
func Vec2_operator_lt(self: *Vec2, other: Vec2) -> bool { ... }
func MyArray_operator_index_get(self: *MyArray, idx: int) -> int { ... }
func MyArray_operator_index_set(self: *MyArray, idx: int, value: int) { ... }
// Closures (capture-less for now)
let add: func(int, int) -> int = |a: int, b: int| -> int { return a + b; };
let sum: int = add(3, 4); // 7
// Closure passed to higher-order function
func Apply(x: int, op: func(int) -> int) -> int { return op(x); }
let doubled: int = Apply(5, |x: int| -> int { return x * 2; }); // 10
```
---
@@ -211,6 +284,31 @@ outer: loop {
}
```
### `defer`
Runs an expression when the current scope exits (LIFO order).
```bux
func ReadFile(path: String) -> String {
let fd: int = Open(path);
defer Close(fd);
defer PrintLine("done");
let data: String = ReadAll(fd);
return data; // both defers run before return
}
```
### `switch` / `case`
Desugars to an if-else chain. Supports a `default` case.
```bux
switch statusCode {
case 200: PrintLine("OK");
case 404: PrintLine("Not Found");
case 500: PrintLine("Server Error");
default: PrintLine("Unknown");
}
```
---
## Structs
@@ -249,10 +347,21 @@ enum Result {
Err(String)
}
enum Pair {
Two(int, int), // multi-field → nested payload
One(int), // single-field → flat data.One_0
None
}
func Main() -> int {
let r: Result = Result { tag: Result_Ok };
r.data.Ok_0 = 42;
// Multi-field construction: data.Variant.Variant_i
var p: Pair = Pair { tag: Pair_Two };
p.data.Two.Two_0 = 3;
p.data.Two.Two_1 = 4;
if r.tag == Result_Ok {
PrintInt(r.data.Ok_0);
}
@@ -260,26 +369,75 @@ func Main() -> int {
}
```
Layout notes:
- Single positional field: flat union member `data.Variant_0`
- Multi-field: nested payload `data.Variant.Variant_0` / `data.Variant.Variant_1` (C type `Enum_Variant_Payload`)
---
## Pattern Matching
```bux
match opt {
Option::Some(value) => PrintInt(value),
Option::None => PrintLine("none")
// Payload bindings: names in Variant(args) are bound in the arm body
func GetValue(opt: Option) -> int {
match opt {
Option::Some(value) => value,
Option::None => 0
}
}
match n {
0 => 100,
1..5 => 200,
6..=10 => 300,
_ => -1
}
```
Supported patterns:
- Wildcard: `_`
- Literal: `42`, `"hello"`, `true`
- Identifier: `name`
- Identifier catch-all: `name` (binds whole subject)
- Range: `1..9`, `1..=9`
- Enum destructuring: `Shape::Circle(r)`
- Struct destructuring: `Point { x: 0, y: 0 }`
- Tuple: `(a, b, c)`
- Guard: `t if t < 0`
- Enum tags + **payload bindings**: `Option::Some(value)`, `Pair::Two(a, b)`
- **Nested**: `Box::Val((a, b))`, `Shape::Dot(Point { x, y })`
- **Tuple patterns**: `(a, b)` → binds `subject._0`, `subject._1`
- **Struct patterns**: `Point { x: px, y: py }` or shorthand `Point { x, y }`
- Guard patterns: parsed; full lowering still evolving
```bux
match pair {
(a, b) => a + b,
_ => 0
}
match p {
Point { x, y } => x * 10 + y,
_ => -1
}
match bx {
Box::Val((a, c)) => a + c,
Box::Empty => 0
}
match sh {
Shape::Dot(Point { x, y }) => x * 10 + y,
Shape::Empty => -1
}
// Multi-statement arm bodies (block expression; last expr is the value)
match n {
1 => {
let a: int = 10;
a + 1
},
_ => 0
}
// Block as expression
let r: int = {
let x: int = 5;
x + 6
};
```
---
@@ -366,32 +524,44 @@ func Main() -> int {
## Gradual Ownership (Phase 8.2) ✅ Implemented
Bux introduces **gradual ownership** — opt-in borrow checking. By default, Bux is permissive like C. With `@[Checked]`, the borrow checker enforces memory safety rules.
Bux has **gradual ownership** — opt-in borrow checking. Default is permissive
(C-like). Turn safety on where it matters; turn it off on hot paths with zero cost.
### Syntax
### Three tiers
| Mode | Attribute | Checks | Cost |
|------|-----------|--------|------|
| **Default** | (none) | None | Zero — raw `*T`, free aliasing |
| **Checked** | `@[Checked]` | Moves, exclusive `&mut`, shared/`&mut` conflicts, dangling returns, elision | Compile-time only |
| **Release** | `@[Release]` | **Forced off** (even if also `@[Checked]`) | Zero — same codegen as default |
**Story:** write most code unchecked for speed of iteration; mark critical APIs
`@[Checked]`; mark micro-hotspots `@[Release]` (or both) when you need C-level
performance without false positives.
```bux
// Default: permissive mode (like C/Nim) — raw pointers, no checks
// Tier 1 — default: C-like, no borrow checker
func QuickSort(arr: *int, len: int) {
for i in 0..len {
arr[i] = arr[i] * 2;
}
// free to alias, no move tracking
}
// Opt-in: @[Checked] enables borrow checking
// Tier 2 — opt-in safety
@[Checked]
func Scale(val: &mut int) {
*val = *val * 2; // OK: &mut T allows mutation
*val = *val * 2;
}
@[Checked]
func Read(val: &int) -> int {
return *val; // OK: &T allows reading
// Tier 3 — zero-cost escape (e.g. hot loop helper)
@[Release]
func HotInc(p: *int) {
*p = *p + 1; // no checks; same as default, documents intent
}
// Release wins over Checked when both are present
@[Checked]
func BadWrite(val: &int) {
*val = 42; // ERROR: cannot write through shared reference '&T'
@[Release]
func HotButDocumented(p: &mut int) {
*p = *p + 1; // no borrow checks
}
```
@@ -430,12 +600,327 @@ Moves happen in three contexts:
- **Assignment**: `b = a` moves `a` into `b`
- **Return**: `return x` moves `x` out of the function
### Rules in @[Checked] functions
### Rules in `@[Checked]` functions (not `@[Release]`)
- `&T` cannot be used to mutate data (compile-time error)
- `&mut T` allows mutation
- `*T` pointers are unrestricted (escape hatch)
- `&mut T` coerces to `&T` and `*T`
- **Double mutable borrow**: two live `&mut` of the same var (call args or let-bound)
```bux
Swap(&mut x, &mut x); // ERROR
let a: &mut int = &mut x;
let b: &mut int = &mut x; // ERROR: exclusive mut already live
```
- **Use while mutably borrowed**: assign/use of `x` while a let-bound `&mut x` is live
- **Shared while mut**: cannot form `&x` while `&mut x` is live
- **Use after move**: using a moved `own T` until reassigned
- **No dangling returns**: cannot return a reference to a local
```bux
@[Checked]
func Bad(p: &int) -> &int {
var x: int = 1;
return &x; // ERROR
}
```
### `@[Release]` (C.4 zero-cost path)
Use when a function must stay check-free:
1. **Documented hot path** — same IR as unchecked, but the attribute states intent.
2. **Override Checked**`@[Checked] @[Release]` on a method that would otherwise inherit team-wide Checked defaults.
There is **no runtime cost**: the attribute only disables the checker for that function body. Prefer `@[Release]` on the smallest possible surface; keep call boundaries `@[Checked]` when you still want API-level safety.
```bux
@[Checked]
func SafeApi(buf: &mut int) {
// checked here
HotPath(buf);
}
@[Release]
func HotPath(p: &mut int) {
// no move / borrow tracking — write like C
*p = *p + 1;
}
```
### Lifetime elision (C.1)
In `@[Checked]` functions (and not `@[Release]`), most reference signatures need
**no** lifetime annotations. Elision applies the usual single-input rules:
1. Each elided input `&T` / `&mut T` parameter gets a distinct lifetime.
2. If there is **exactly one** input lifetime, it is assigned to all elided outputs.
3. If the first parameter is named `self` / `Self`, that input lifetime is preferred for outputs.
4. Multiple input references + elided return → **error** (write an explicit lifetime).
```bux
// Elided — one input ref, return shares its lifetime
@[Checked]
func Identity(p: &int) -> &int {
return p; // OK
}
// Explicit — required when several inputs could be returned
@[Checked]
func Pick<'a>(a: &'a int, b: &'a int) -> &'a int {
return a;
}
// Syntax: &'a T and &mut / &'a mut T (lifetime before `mut`)
// Type parameters: func F<'a, T>(...)
```
Default and `@[Release]` functions ignore lifetime rules (C-like). Explicit `'a`
is optional documentation when a single input would already elide correctly.
### Drop and RAII
Bux uses **static destructors** (no GC): when a value goes out of scope, the
compiler may emit `TypeName_Drop(&local)`. That is the RAII story — resources
are released at every exit path without manual `defer` on every return.
#### Declaring cleanup
Two equivalent ways to opt a type into auto-drop:
```bux
// 1) Attribute — compiler looks up TypeName_Drop
@[Drop]
struct Token {
id: int,
counter: *int,
}
func Token_Drop(self: *Token) {
// free / close / decrement …
}
// 2) Interface (stdlib `lib/Drop.bux`) — same static call, no vtable
import Drop;
extend Buffer for Drop {
func Drop(self: *Buffer) {
Mem_Free(self.data);
}
}
```
Stdlib collections implement Drop (`Array_Drop`, `Map_Drop`, …). Calling
`Array_Drop` is the same cleanup as `Array_Free` for `Array<T>`.
#### When auto-drop runs
Auto-drop is **not** gated on `@[Checked]`. Any function can receive injected
`Type_Drop` at:
| Exit | Behavior |
|------|----------|
| End of block / function | Drop locals still owned |
| Early `return` | Drop all live locals **after** materializing the return value |
| Branch scope end | Only locals from the taken branch |
| Nested scopes | Drop in reverse order of declaration |
```bux
@[Drop]
struct Token { id: int, counter: *int }
func Token_Drop(self: *Token) { /* … */ }
func Early(flag: int, counter: *int) -> int {
let t: Token = Token { id: 1, counter: counter };
if flag == 0 {
return 0; // still runs Token_Drop(&t)
}
return 1; // Token_Drop(&t) here too
}
```
See `examples/drop_early_return.bux` for branch-local vs fallthrough counts.
#### Field-move: skip Drop of the source (critical)
**Problem:** a local is moved **by value** into a struct field (or another local).
If the compiler still auto-dropped the source, you get a **double free** — the
field and the original local would both run `Array_Drop` on the same buffer.
**Rule:** after a **value move** out of a local, that local is **not** dropped.
```bux
struct Box {
items: Array<int>;
}
func MakeBox() -> Box {
var items: Array<int> = Array_New<int>(4);
Array_Push<int>(&items, 10);
Array_Push<int>(&items, 20);
// Move `items` into the field — compiler skips Drop of `items`
let b: Box = Box { items: items };
return b; // also: return-by-value skips Drop of `b` (caller owns it)
}
```
What the C backend does for `MakeBox` (simplified):
```c
Box MakeBox(void) {
Array_int items = Array_New_int(4);
Array_Push_int(&items, 10);
Array_Push_int(&items, 20);
Box b = (Box){ .items = items };
return b;
/* no Array_Drop_int(&items); — moved into b.items */
/* no Array_Drop on b; — moved to caller via return */
}
```
Ownership after `MakeBox`:
1. Heap buffer lives inside `b.items` (and later the caller's `Box`).
2. `items` is **moved-out** → skip auto-Drop.
3. `b` is **returned by value** → skip auto-Drop at the return site; the caller
(or the next owner) is responsible.
The same skip applies to:
- **Struct field init**`S { field: local }` (field-move)
- **Assignment**`a = b` when `b` is moved (value types with Drop)
- **Call argument** by value into a consuming parameter
- **`return x`** — move-on-return
Live, unmoved Drop locals still clean up on error paths (e.g. early `return`
before the move). That is intentional: only the **successful transfer** path
skips Drop.
Runnable check: `examples/move_field.bux` (also covered by
`make test-selfhost-smoke` on buxc2).
#### Partial field moves
Moving a **droppable field** out of a local (return or `let`) also skips Drop
of the **parent** local:
```bux
@[Drop]
struct Bag {
items: Array<int>,
tag: int,
}
func Bag_Drop(self: *Bag) {
Array_Drop<int>(&self.items);
}
func TakeItems() -> Array<int> {
var items: Array<int> = Array_New<int>(4);
Array_Push<int>(&items, 42);
let bag: Bag = Bag { items: items, tag: 7 };
return bag.items; // Bag_Drop skipped — items ownership transferred
}
```
Rules:
- Applies only when the **field type** is droppable (`Array_*`, `@[Drop]` types,
etc.). Reading `bag.tag` (`int`) does **not** mark `bag` moved.
- After `let moved = bag.items`, `Bag_Drop(&bag)` is skipped; `moved` owns the
array and is auto-dropped at scope end.
- **Remaining fields:** if the parent has other droppable fields that were *not*
moved out, those still run their `Type_Drop` / collection Drop (session 70).
Example: move `pair.left` → skip `PairBag_Drop`, still `Tracked_Drop(&pair.right)`.
```bux
@[Drop]
struct PairBag {
left: Array<int>,
right: Tracked, // also @[Drop]
}
func TakeLeft() -> Array<int> {
let pair: PairBag = …;
return pair.left; // Tracked_Drop(&pair.right) still runs
}
```
#### Nested path moves (`a.b.c`)
Moving a **deep** droppable field also works. The full dotted path is recorded
so remaining fields at every level still Drop:
```bux
@[Drop]
struct Outer {
inner: Inner, // Inner has items: Array + note: Tracked
tag: Tracked,
}
func TakeNested() -> Array<int> {
let outer: Outer = …;
return outer.inner.items;
// skips Outer_Drop
// still: Tracked_Drop(&outer.inner.note) + Tracked_Drop(&outer.tag)
}
```
#### Field moves through pointers
When a local pointer aliases a local owner (`let p = &bag`), moving a field
through the pointer marks the **owner**, not the pointer:
```bux
let bag: Bag = …;
let p: *Bag = &bag;
return p.items; // same as (*p).items
// skips Bag_Drop; still Tracked_Drop(&bag.tag)
```
Nested paths work the same: `p.inner.items` resolves `p → outer` then path
`inner.items`.
#### Cross-function pointer transfer (session 76)
When the **caller** passes `&bag` (or a pointer alias) into a function whose
parameter is `*Bag`, and the **callee** moves fields of that param
(`return p.items` / `let x = p.items`), the call site marks `bag` the same way
as a local partial move — parent `Bag_Drop` is skipped; remaining fields Drop.
```bux
func TakeItems(p: *Bag) -> Array<int> {
return p.items;
}
func Caller() {
let bag: Bag = …;
let items: Array<int> = TakeItems(&bag);
// bag.items transferred; Tracked_Drop(&bag.tag) still runs
}
```
Analysis is **same-module / known callee body** only (bootstrap HIR today).
Golden smoke: `make test-drop-move` / `examples/move_field_partial.bux` /
`examples/move_field_remaining.bux` / `examples/move_field_nested.bux` /
`examples/move_field_ptr.bux` / `examples/move_cross_fn.bux`.
#### Manual Drop and non-Drop types
- Types **without** `@[Drop]` / `Drop` impl are never auto-dropped (plain C layout).
- You can still call `Type_Drop(&x)` or use `defer` for explicit cleanup.
- `@[Release]` / default functions still get auto-drop for Drop types — Release
only turns off the **borrow checker**, not RAII.
#### Limits (honest)
- Partial field moves skip the **parent** `Type_Drop` and drop **remaining**
droppable fields individually, including nested paths `a.b.c`, local pointer
aliases `p = &owner`, and **cross-function** `&owner` args when the callee
body is visible (sessions 70/73/74/76).
- Local pointer aliases (`p = &local`) are tracked in-function; cross-function
uses callee-body scan of pointer params (not full borrow checking).
- Interface Drop uses a static `TypeName_Drop` symbol (zero cost), not dynamic
dispatch through a vtable.
- Double-free bugs in **unchecked** code that manually free *and* auto-drop are
still possible if you free without invalidating the value — prefer one owner.
---
@@ -632,6 +1117,34 @@ func Main() -> int {
---
## Operator Overloading
> **Status:** ✅ Implemented in bootstrap. Selfhost reserves syntax but has no method-table yet.
Overloadable operators use the naming convention `TypeName_operator_<op>`:
| Operator | Function Name | Signature Example |
|----------|--------------|-------------------|
| `+` | `operator_add` | `func T_operator_add(self: *T, other: T) -> T` |
| `-` | `operator_sub` | `func T_operator_sub(self: *T, other: T) -> T` |
| `*` | `operator_mul` | `func T_operator_mul(self: *T, other: T) -> T` |
| `/` | `operator_div` | `func T_operator_div(self: *T, other: T) -> T` |
| `%` | `operator_mod` | `func T_operator_mod(self: *T, other: T) -> T` |
| `==` | `operator_eq` | `func T_operator_eq(self: *T, other: T) -> bool` |
| `!=` | `operator_ne` | `func T_operator_ne(self: *T, other: T) -> bool` |
| `<` | `operator_lt` | `func T_operator_lt(self: *T, other: T) -> bool` |
| `<=` | `operator_le` | `func T_operator_le(self: *T, other: T) -> bool` |
| `>` | `operator_gt` | `func T_operator_gt(self: *T, other: T) -> bool` |
| `>=` | `operator_ge` | `func T_operator_ge(self: *T, other: T) -> bool` |
| `[]` (get) | `operator_index_get` | `func T_operator_index_get(self: *T, idx: int) -> U` |
| `[]` (set) | `operator_index_set` | `func T_operator_index_set(self: *T, idx: int, value: U)` |
**Notes:**
- Short-circuit operators (`&&`, `||`) cannot be overloaded.
- Generic method instantiation is supported.
---
## Operators
### Arithmetic
@@ -659,3 +1172,246 @@ func Main() -> int {
- `..` — Range (exclusive): `0..10`
- `..=` — Range (inclusive): `0..=10`
- `sizeof` — Size of type: `sizeof(Type)`
---
## Macros
Bux supports **declarative macros**. Expansion runs after parse and before
type-checking. Expanded AST uses **call-site** source locations (quote hygiene).
Both bootstrap and selfhost (`buxc2`) expand macros.
### Definition
```bux
macro! twice {
($x:expr) => {
($x) + ($x)
}
}
// Trailing repetition
macro! sum_n {
( $($x:expr),* ) => {
var acc: int = 0;
$( acc = acc + $x; )*
acc
}
}
// Compound / zip: parallel lists from interleaved args
macro! add_pairs {
( $($a:expr, $b:expr),* ) => {
var acc: int = 0;
$( acc = acc + ($a + $b); )*
acc
}
}
// Multi-rep groups: `;` separates arg groups at the call site
macro! sum_groups {
( $($x:expr),* ; $($y:expr),* ) => {
var s: int = 0;
$( s = s + $x; )*
$( s = s + $y; )*
s
}
}
// Nested template repetition (outer list → inner expands once per item)
macro! double_each_sum {
( $($x:expr),* ) => {
var t: int = 0;
$(
$( t = t + $x; )*
$( t = t + $x; )*
)*
t
}
}
// ident fragment: bare identifier at the call site
macro! call0 {
( $f:ident ) => {
$f()
}
}
// literal (alias: lit) — only int/float/string/char/bool literals
macro! only_lit {
( $x:literal ) => { $x }
}
// block — only `{ … }` block expressions
macro! wrap_block {
( $b:block ) => { $b }
}
// stmt — one statement (let/if/… or expression-statement)
macro! with_setup {
( $s:stmt, $body:expr ) => {
{
$s
$body
}
}
}
// call: with_setup!(let x: int = 10, x + 1)
// pat — match/let pattern (literals, `_`, enum variants, …)
macro! matches {
( $p:pat, $e:expr ) => {
match $e { $p => 1, _ => 0 }
}
}
// call: matches!(1, 1) · matches!(_, 99) · matches!(Opt::Some(v), opt)
// gensym: template locals renamed per expansion
macro! with_acc {
( $start:literal ) => {
var n: int = $start;
n = n + 1;
n
}
}
```
- Introduced with the `macro!` keyword.
- Each **rule** is `( pattern ) => { template }`.
- **Fragment kinds:**
| Kind | Matches |
|------|---------|
| `expr` | any expression |
| `ident` | bare identifier (`ekIdent`) |
| `tt` | token-tree: any single call-site AST fragment; **delimiter-balanced multi-element groups** `(a, b)` and `[a, b]` flatten when spliced as the sole call argument (`$f($args)``f(a, b)`, not `f((a, b))`). Non-group `tt` unwraps to the value. Broader than `expr`. |
| `literal` / `lit` | int/float/string/char/bool literal only |
| `block` | block expression `{ … }` |
| `stmt` | one statement (`let`/`if`/… or expression-stmt) |
| `pat` / `pattern` | match pattern (`_`, literals, `Enum::Var(…)`, …) |
| `type` | type expression from call-site shape: named (`int`), pointer (`*int`); spliced into `sizeof($t)`, `as $t`, `let x: $t` |
- Fragment names start with `$` (lexer `$ident`).
- **Repetition:** `$( $x:expr ),*` / `$( $x:expr )*` — one or more rep fragments per pattern.
- **Compound rep:** `$( $a:expr, $b:expr ),*` — interleaved args zip into parallel lists.
- **Multi-rep:** two (or more) `$(…)*` in one pattern; call site uses `;` between groups:
`sum_groups!(1, 2; 10, 20, 30)`.
- Template `$( stmt; … )*` expands once per list item (zip when multiple lists used).
- Nested `$( $(…)* )*`: after outer binds list items as singles, inner expands once.
- **Expression-level rep in templates:** `$f( $($a),* )``$( expr ),*` / `$( expr )*`
inside call arguments expands to N positional args (session 84).
- **Delimiter-balanced `:tt` groups:** `apply_tt!(Add, (3, 4))` or
`apply_tt!(Add, [3, 4])` with `($f:ident, $args:tt) => { $f($args) }`
expands to `Add(3, 4)`. Contrast `:expr`, which keeps the group as one value.
- **Free-form juxta (session 86):** pattern `$f:ident $args:tt` (comma optional
between fragments) matches a **single call-site argument** that is a call
expression: `apply_juxta!(Add(2, 5))` → binds `$f=Add`, `$args` = arg-list
group, then `$f($args)` flattens to `Add(2, 5)`.
- **Type fragments (session 87):**
```bux
macro! size_of {
( $t:type ) => { sizeof($t) as int }
}
let n: int = size_of!(int);
let p: int = size_of!(*int);
```
### Invocation
```bux
let n = twice!(21);
let s = sum_n!(1, 2, 3); // 6
let z = sum_n!(); // 0
let p = add_pairs!(1, 10, 2, 20); // (1+10)+(2+20) = 33
let g = sum_groups!(1, 2; 10, 20, 30); // 63
let d = double_each_sum!(3, 4); // 14
call0!(SomeFunc);
let a = with_acc!(10); // 11
let b = with_acc!(20); // 21 — different gensym'd `n`
let c = only_lit!(7);
// only_lit!(1 + 2); // ERROR: no matching rule
let w = wrap_block!({ 1 + 2 }); // 3
```
- Syntax: `name!( arg, … )` (not unwrap: unwrap is `expr!` without `(`).
- Matching: fixed-arity by count; kind constraints; rep by groups / remaining args / chunk.
### Built-in `quote!`
```bux
let x = quote!(1 + 2); // identity expand; locations grafted to call site
```
### Hygiene
Two layers (both bootstrap + selfhost):
1. **Call-site graft** — expanded AST uses the call sites line/col/`sourceFile`
(so diagnostics and `#line` point at the user call, not the macro definition).
2. **Gensym of template binders** — each expansion renames:
- `let` / `var` locals introduced by the template
- `for` loop binders in the template
- Nested scopes (if/while/for bodies, MacroRep bodies)
so two expansions of the same macro in one function do not collide under the
C backends **function-scoped** locals (e.g. `__m1_n` and `__m2_n`).
Spliced `$frags` in expression positions are **not** gensymd — they keep
call-site names/values.
#### Unhygienic binders (`var $name`)
To **introduce a binder whose name comes from the call site**, use a `$frag`
as the binder itself. That name is **not** gensymd:
```bux
macro! let_mut {
( $name:ident, $init:literal ) => {
var $name: int = $init; // unhygienic: becomes `counter`, not __m1_…
$name = $name + 1;
$name
}
}
// expands with local `counter` (and hygienic locals still unique)
let a = let_mut!(counter, 10); // 11
let b = let_mut!(other, 20); // 21
macro! double_acc {
( $start:literal ) => {
var acc: int = $start; // hygienic → __m1_acc / __m2_acc
acc = acc + acc;
acc
}
}
```
| Binder form | After expand | Gensym? |
|-------------|--------------|---------|
| `var acc = …` (plain name in template) | `__mN_acc` | yes |
| `var $name = …` with `$name:ident` | call-site ident | **no** |
| `for $i in …` with `$i:ident` | call-site ident | **no** |
The binder must be a **`:ident` fragment** bound to a bare identifier. A plain
template name is always hygienic.
Examples: `examples/macro_hygiene.bux`, `examples/macro_unhygienic.bux`.
### Limits
- Up to two named rep lists per rule on selfhost (enough for zip + multi-rep).
- Compound chunk size currently 1 or 2.
- Nested macro *calls* expanded recursively (depth limit 32).
- Unhygienic binders only rename `let`/`var`/`for` binders — not full
Scheme/Rust colored identifiers or `stmt`/`pat` token trees.
- Macro expansion still yields a **block expression**; unhygienic names are
scoped to that block (not automatically injected into the caller scope).
- Expression-level `$(…)*` is only parsed inside **call argument lists** in
templates (not as a free-standing primary expression).
- Raw delimiter-balanced `tt` covers **tuple** `(a, b)` and **slice lit**
`[a, b]` groups, plus **juxta call-split** for `$f:ident $args:tt` matching
`F(a, b)`. Arbitrary free-form token pastes (operators-only, type-only
without AST) remain out of scope.
Examples: `examples/macro_tt.bux`, `examples/macro_tt_raw.bux`,
`examples/macro_repeat.bux`, `examples/macro_nested.bux`.

Some files were not shown because too many files have changed in this diff Show More