Includes two raft fixes the e2e exposed:
- heartbeatLoop drops cached peer sockets on leadership acquisition
(half-dead sockets to a restarted peer never errored, so no redial
ever happened and the cluster livelocked without heartbeats)
- restoreSnapshot repoints tcpServer ctx/db at the reopened database
(queries kept reading the closed pre-restore LSM, serving 0 rows)
- lastSnapshotIndex/Term bound the compacted prefix; lastLogIndex/Term
and AppendEntries prevLog checks respect the snapshot base
- compactLog drops entries only through min(matchIndex, lastApplied) on
the leader so lagging peers can still catch up via AppendEntries
- Persist snapshot fields in raft_state.bin; BARADB_RAFT_LOG_MAX_ENTRIES
- Fix commit-index scan to use findLogEntryByIndex (works after compact)
- BARADB_RAFT_CLIENT_PEERS maps node id → SQL client host:port
- Followers proxy DML/DDL to the known leader over the wire protocol
(falls back to "not leader" when the map is missing)
- E2E: follower CREATE/INSERT succeed via forward; docs updated
- isRaftDdl + leader-only gate for CREATE/DROP/ALTER (not DATABASE)
- appendDdlToRaft ships original SQL; applyCommand re-executes via
applyReplicatedDdl (idempotent on leader double-apply)
- Mixed DDL+DML batches use the DDL path so order is preserved
- Fix secondary-index point lookup to use entry.lsmKey (not filter col)
- E2E: CREATE only on leader, schema + index SELECT on follower
- applyReplicatedPut/Delete keep in-memory graphs in sync with node/edge
table rows (idempotent edges via addEdgeWithIdIfAbsent).
- When Raft is enabled, DML is refused on any database other than
'default' (the only DB the state machine is wired to).
- Docs updated (en/bg); unit test for graph apply.
- Reject follower writes if any statement in the batch is DML/COMMIT
(not only stmts[0]).
- COMMIT always emits empty-valued kvPairs for isDelete entries.
- applyCommand updates LSM plus secondary B-tree/FTS/HNSW indexes
(applyReplicatedPut/Delete) so follower index scans see replicated rows.
- Tests: not-leader append, commit timeout, index apply unit, E2E
index-backed SELECT on follower.
- Fix runTcpServer to run the already-wired Server (raftNode was assigned
on a different instance that never accepted clients).
- Cap raft peer connect at 200ms and fan out heartbeats in parallel so a
dead peer cannot stall AppendEntries to the live majority.
- Add raft_writes_e2e_test: 3-node write replication, follower rejection,
and post-failover writes; wire into nimble test + gitignore.
- parser: clause keywords (header, format, status, user, csv, ...) now work
as identifiers everywhere; IMPORT/EXPORT accept FORMAT csv/HEADER true
- nimble test + CI run all 13 test suites (650 checks green)
- ExecutionContext.registry is now {.cursor.} (breaks registry<->ctx cycle)
- ORC crash reproduced and bisected (tests/orc_repro.py); ARC stays the MM
Replace bare except: with CatchableError across storage, query, Raft,
backup, and protocol code so Defects are not swallowed. Break uncollectable
ARC cycles in server shard/gossip callbacks via Server-owned refs and
cursor locals. Align package license with LICENSE (BSD-3-Clause), sync
README version, and point test_all at the canonical clients/nim baradb
client (parseConnectionString + aliases).
Core storage: hash MemTable, WAL group commit, L0 compaction rebuild,
reader-writer lock, and a global StorageGate so HTTP workers and TCP
share the LSM safely under multi-thread access.
Schema: durable CREATE/ALTER/DROP under _schema:tables:* with full LSM
restore on open. Executor types/values/schema split into query/exec/.
Wire protocol: switch default MM to ARC — ORC cycle collector segfaulted
after ~20 async INSERTs. Fair multi-tier benchmarks (SQLite/HTTP/wire/PG)
and honesty docs for mixed-tier comparisons.
The test was comparing a Value to a string literal which always failed.
Changed to check s.kind == vkNull on both calls.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause: seq.del(i) does swap-with-last (destroys sorted order)
while seq.delete(i) preserves order. This caused massive data loss
during B-tree removes.
Changes:
- btree.nim: Replace seq.del with seq.delete (critical data corruption fix)
- btree.nim: search() traverses leaf linked list for duplicate boundary keys
- btree.nim: splitChild() consolidates duplicate boundary key values before split
- btree.nim: len() counts unique keys via leaf traversal
- btree.nim: remove() traverses leaf linked list to remove ALL occurrences
- btree.nim: Fix separator update in internal node (node.keys[i-1] not [i])
- prop_test.nim: Use set equality for interleaved test (rebalancing reorders)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Refactored Row from Table[string, string] to Table[string, Value]
- Added Value operators: ==, !=, $, in for string interop
- Fixed valueToString for vkNull to return '\\N' instead of ''
- Fixed evalExpr irekField to return vkNull when field not found
- Added IN (val1, val2, ...) parser support
- Fixed nkPath column names in multi-table joins
- Fixed LATERAL JOIN null padding when no matching rows
- Added CREATE/DROP/USE/SHOW DATABASE parser support
- Adapted all tests for new Value type
- Add isDelete flag to VersionedRecord to fix COMMIT treating empty values
as DELETE. Previously version.xmax != TxnId(0) was never true for active
transactions, so deletes inside transactions never removed rows.
- Fix MIN/MAX aggregates skipping NULL: was checking for "\\N" instead
of correct "\N" sentinel, so NULL values were never skipped.
- Add duplicate alias deduplication in lowerSelect() and getSelectColumns().
Identical aliases now get _1, _2 suffixes (e.g. count(*), count(*)_1).
- Add nil guard for expr.subqueryPlan in irekSubquery evalExpr handler.
- Initialize txnManager in newExecutionContext to prevent segfault on BEGIN.
- Add regression tests: DELETE in transaction, PK-only row survival,
MIN/MAX NULL skipping.
evalExpr was missing irekSubquery case — scalar subqueries in SELECT/WHERE
fell through to else:return "" producing empty strings. Add:
- irekSubquery handler in evalExpr that executes the subquery plan
- outerRow + subqueryPlan fields on ExecutionContext for correlation
- qualified column injection in execScan for correlated filter refs
- collectCorrelatedTablesFromPlan to discover outer table references
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
dimgigov