# Changelog

All notable changes to BaraDB are documented in this file.

## [1.2.0] — Unreleased

### Raft cluster (C3a / C3b / ops)

Production-ready path from config → election → SQL/DDL replication → ops.

- **C3a — Networked bootstrap** — `BARADB_RAFT_PEERS=id@host:port`, election timer in production, heartbeat timer reset on AppendEntries, `raft_state.bin` persistence, partial-read-safe frames; E2E `tests/raft_e2e_test.nim` (3-node election + failover)
- **C3b — SQL writes through Raft** — leader appends DML KV pairs and waits for majority commit; followers reject or **forward** via `BARADB_RAFT_CLIENT_PEERS`; apply path updates LSM + B-tree/FTS/HNSW + graphs; multi-statement write gate; writes only on `default` database
- **DDL replication** — schema DDL (`CREATE`/`DROP`/`ALTER` table, index, view, graph, …) as `ddl` log entries; re-executed on apply; `CREATE`/`DROP DATABASE` excluded
- **Leader write forwarding** — followers proxy DML/DDL to the leader SQL port when client peers are configured
- **Safe log compaction** — soft cap `BARADB_RAFT_LOG_MAX_ENTRIES` (default 256); leader never discards past peer `matchIndex`; snapshot base (`lastSnapshotIndex`/`Term`) persisted
- **Secondary-index point lookup fix** — index scans use `entry.lsmKey` (not the filter column as PK)
- **Metrics** — Prometheus raft series on `GET /metrics` (HTTP = TCP port + 440); `GET /health` includes `raft` role/term/leader/lag/log size
- **E2E writes** — `tests/raft_writes_e2e_test.nim` (schema, forward, index SELECT, failover)
- **Docs** — `docs/en|bg/distributed.md`, `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`

### Core Storage Hardening

Foundational LSM improvements for write performance, durability, and compaction correctness.

- **Hash-table MemTable** (`storage/lsm.nim`) — O(1) put/get instead of O(n) sorted-seq insert; sort only on flush to SSTable
- **Timestamp-aware MemTable overwrite** — WAL recovery keeps newest version per key
- **WAL rewrite after flush** (`storage/wal.nim`) — `truncate` / `rewriteLive` so recovery is O(unflushed) not O(history); fsync on truncate/rewrite/close
- **WAL group commit** — `WalSyncMode`: `none` | `group` (default) | `every`; fsync every N entries and/or every interval ms
- **Config knobs** — `wal_sync_mode`, `wal_group_every`, `wal_sync_interval_ms` / env `BARADB_WAL_*`; registry opens DBs with config
- **L0 file-count compaction trigger** — RocksDB-style `L0CompactionTrigger` (default 4) instead of size-only for overlapping L0
- **`rebuildFromLSM` for compaction** (`storage/compaction.nim`) — strategy always syncs from live SSTable catalog (no drift after flush)
- **Safe mmap close after compaction** — release handles for compacted inputs after unlink
- **Recovery flag** — flush during WAL replay does not rewrite the open WAL file mid-read
- **Fair WAL micro-bench** — `benchWalDurabilityModes` compares none/group/every on the same workload
- **RwLock for LSM** (`storage/rwlock.nim`) — writer-preferring reader-writer lock; default API uses exclusive mode
- **Deep-copy on get** — returned values are copied so callers never share seq buffers with the store
- **`-d:baraConcurrentReads`** — opt-in shared read locks (unsafe with default ORC across OS threads)
- **`scanRange(start, end)`** — inclusive multi-level range scan (memtable + SSTables)
- **Focused tests** — `tests/test_storage_hardening.nim` (RwLock, WAL group, scanRange, stress)
- **Note:** Nim ORC must not share GC'd `LSMTree` refs across OS threads without isolation
- **StorageGate** (`storage/gate.nim`) — global exclusive lock serializing HTTP (Hunos workers), TCP, compaction, Raft apply
- **HTTP stop no longer closes shared registry** — ownership stays with main; `stop(closeStorage=true)` for standalone HTTP
- **Schema persistence** — stable keys `_schema:tables:<name>`; restore from full LSM (`scanAll`), not only memtable; DROP/ALTER update durable catalog; secondary index rebuild on open
- **Schema tests** — `tests/test_schema_persist.nim` (create/flush/reopen, drop, alter, multi-table)
- **Executor split** — `query/exec/{types,values,schema}.nim`; `executor.nim` re-exports for API stability; see `query/exec/README.md`
- **Fair benchmarks** — `benchmarks/fair_bench.py` multi-tier (embedded SQLite↔LSM; client-server HTTP/wire↔PG); batch multi-row INSERT; wire protocol via Python client; `generate_report.py --fair`; `nimble bench_fair`
- **Fix wire INSERT SIGSEGV** — ORC cycle collector crash under async TCP load; switch default MM to `--mm:arc` in `nim.cfg`; regression `tests/test_wire_insert_stress.nim`
- **Honest bench docs** — tier warnings in `bench_all`, `pg_bench`, `compare.nim`; `benchmarks/README.md`
- **Regression suite** — `Core Storage Hardening` tests in `tests/test_all.nim`

### Search Module (new)

A unified search module combining vector similarity, full-text, and structured
search into a single high-performance engine.

- **Heap-optimized HNSW search** — priority-queue-based candidate selection, 2.4x faster than baseline (`search/hnsw_opt.nim`)
- **Segment-based inverted indexing** — partitioned posting lists for concurrent indexing and reduced lock contention (`search/inverted.nim`)
- **Phrase and proximity search** — ordered phrase matching with configurable slop distance (`search/phrase.nim`)
- **Boolean query parser** — full boolean algebra with AND, OR, NOT, and range expressions (e.g. `price:[10 TO 100]`) (`search/boolean.nim`)
- **N-gram fuzzy search** — character n-gram index for typo-tolerant retrieval (`search/ngram.nim`)
- **Faceted search** — filter results and aggregate counts by arbitrary field values (`search/facet.nim`)
- **Porter2 stemmers** — morphological stemming for English, Bulgarian, German, French, and Russian (`search/stemmer.nim`)
- **UnifiedSearchEngine API** — single entry point combining all search modes with consistent scoring (`search/engine.nim`)
- **Search benchmarks** — reproducible performance measurement suite (`benchmarks/bench_search.nim`)

---

## [1.1.7] — 2026-05-29

### Security (5 critical + 5 high)

- **Fix REP/DISTTXN protocol auth bypass** (`server.nim`) — unauthenticated TCP clients could write data or manipulate distributed transactions
- **Fix HTTP backup/restore path traversal** (`httpserver.nim`) — `..` and absolute paths rejected
- **Fix empty JWT secret when auth enabled** (`server.nim`) — server now refuses to start with `authEnabled: true` and no `jwtSecret`
- **Fix HTTP admin panel served without auth** (`httpserver.nim`) — admin UI now requires authentication when `authEnabled`
- **Fix timing attacks on HMAC/SCRAM comparison** (`auth.nim`, `scram.nim`) — constant-time comparison
- **Fix WebSocket JWT expiration not validated** (`websocket.nim`) — `exp` claim now checked
- **Fix sync replication returning success on partial ack** (`replication.nim`) — returns 0 when not all replicas acknowledge
- **Fix SSL verifyPeer not applied** (`ssl.nim`) — `verifyMode` now passed to `newContext()`
- **Fix JWT JSON parser missing escape handling** (`auth.nim`) — backslash escapes now parsed correctly

### Data Integrity (3 critical + 3 high + 2 medium)

- **Fix WAL write race with flush** (`lsm.nim`) — WAL write now under `db.lock`, preventing data loss after crash
- **Fix 2PC marking uncontacted participants as prepared/committed** (`disttxn.nim`) — only contacted nodes are marked
- **Fix Raft commit index for even-sized clusters** (`raft.nim`) — correct majority calculation
- **Fix MVCC savepoint/rollback no-op** (`mvcc.nim`) — deep copy writeSet at savepoint time
- **Fix table mutation during iteration** (`mvcc.nim`) — collect stale txns before deleting
- **Fix B-tree leaf merge phantom separator key** (`btree.nim`) — no longer inserts empty-valued separator at leaf level
- **Fix writeSSTable partial file on crash** (`lsm.nim`) — write to `.tmp` then atomic rename
- **Fix compaction mmap leak** (`compaction.nim`) — close SSTables after reading

### Query Correctness (1 high + 2 medium)

- **Fix LIMIT 0 returning all rows** (`executor.nim`) — now returns empty result
- **Fix COUNT(col) counting NULL values** (`executor.nim`) — 3 locations fixed to check `v.kind != vkNull`
- **Fix EXISTS subquery always false** (`executor.nim`) — lowering now sets `existsSubquery` plan
- **Fix multi-CTE queries losing earlier CTE tables** (`executor.nim`) — save/restore `cteTables` around inner execution
- **Fix JSON injection in hybrid_search_filtered** (`executor.nim`) — escape quotes/backslashes in ID

### Raft Consensus (3 high + 1 low)

- **Fix Raft appendEntries using array index instead of log-index** (`raft.nim`) — uses `findLogEntryByIndex`
- **Fix Raft applyCommitted using logical index as array position** (`raft.nim`) — uses `findLogEntryByIndex`
- **Fix Raft loadState silently swallowing errors** (`raft.nim`) — now logs warning

### Storage Engine (2 medium)

- **Fix loadSSTable missing minimum file-size check** (`lsm.nim`) — rejects files < 40 bytes
- **Fix substr(s, start) returning single char** (`udf.nim`) — now returns rest-of-string

### Distributed Systems (2 high + 1 medium)

- **Fix sharding connectWithTimeout missing SO_ERROR check** (`sharding.nim`) — verifies connection actually succeeded
- **Fix replication healthCheck double-close socket** (`replication.nim`) — safe close with try/except

### Resource Management (3 medium)

- **Fix unbounded plan cache** (`adaptive.nim`) — max 10000 entries, auto-evict
- **Fix MVCC unbounded committedTxns/abortedTxns** (`mvcc.nim`) — prune entries older than oldest active snapshot
- **Fix connection pool not checking maxLifetime** (`pool.nim`) — lifetime check added to `acquire`

### Operations (1 medium)

- **Fix migration lock persisting after crash** (`executor.nim`) — stores timestamp, auto-releases after 1 hour

### Other

- **Fix nl_to_sql DML validation** (`executor.nim`) — requires `is_superuser` session variable for DML
- **Fix lexer readIdent double column counting** (`lexer.nim`) — removed manual `inc l.col`
- **Fix WebSocket frame 32-bit overflow** (`websocket.nim`) — guard against `len > high(int)`
- **Fix admin panel auth** (`httpserver.nim`) — check auth when `authEnabled`
- **Fix unused imports** (`backup.nim`, `repair.nim`, `raft.nim`) — moved `parseopt` into `when isMainModule`, removed unused `algorithm`

### Build

- **Fix hunos 1.3.1 compatibility with Nim 2.2.x** — patched `getRandomBytes` → `urandom` in `hunos/sessions.nim` and `hunos/csrf.nim` (see `HUNOS_ISSUE.md`)
- Updated `baradadb.nimble` version to `1.1.7`

### Tests

- All 448 tests passing, 0 failures

---

## [1.1.6] — previous

See git log for changes prior to this release.