baraba/Baradb
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: raft cluster status, plans closed, CHANGELOG/README/monitoring
CI / test (push) Has been cancelled
Details
CI / verify (push) Has been cancelled
Details

Browse Source 
- Add raft-cluster-status overview (C3a/C3b/post-C3b shipped on main)
- Mark C3a/C3b design+plans done; refresh operator docs en/bg
- CHANGELOG 1.2.0 Raft section; README cluster example and status line
- monitoring.md health/metrics match real HTTP port+440 and raft series
This commit is contained in:
dimgigov
2026-07-30 21:41:15 +03:00
parent 1b3c26123a
commit 16ec8b5dc4
12 changed files with 263 additions and 220 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/bg/distributed.md
+3 -1
View File
@@ -5,9 +5,11 @@ BaraDB поддържа разпределено внедряване с Raft к
> ⚠️ **Ограничение при множество бази данни**
> Разпределените модули (Raft, шардиране и репликация) в момента работят само с **`default`** базата данни. Ако използвате множество бази (`CREATE DATABASE`, `USE DATABASE`), разпределените функции още не ги обхващат. Всяка база данни се нуждае от отделна кластър конфигурация.
> **Статус (2026-07-30):** Raft C3a (мрежова election), C3b (SQL записи), DDL репликация, leader forwarding, log compaction и metrics са **на `main`**. Преглед: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
## Raft Консенсус
Leader election и log репликация през TCP. Включване:
Leader election и log репликация през TCP; SQL DML/DDL за **default** минават през raft log. Включване:
| Env | Значение |
|-----|----------|
docs/bg/monitoring.md
+16 -11
View File
@@ -4,29 +4,28 @@
### HTTP Health Endpoint
HTTP слуша на **TCP порт + 440** (напр. `BARADB_PORT=9472` → health на `9912`).
```bash
curl http://localhost:9470/health
curl http://localhost:9912/health
```
Отговор:
Без raft:
```json
{
"status": "healthy",
"status": "ok",
"version": "1.1.6",
"uptime_seconds": 86400,
"checks": {
"storage": "ok",
"memory": "ok",
"connections": "ok"
}
"raft": { "enabled": false }
}
```
С `BARADB_RAFT_ENABLED=true` — обект `raft` (`role`, `term`, `leader_id`, `commit_index`, `apply_lag`, `log_entries`, `snapshot_index`).
### Readiness Probe
```bash
curl http://localhost:9470/ready
curl http://localhost:9912/ready
```
Връща `200 OK` когато сървърът е готов да приема трафик, `503` по време на стартиране.
@@ -35,10 +34,16 @@ curl http://localhost:9470/ready
### Prometheus-Съвместими Метрики
Същият HTTP порт като health (`BARADB_PORT + 440`).
```bash
curl http://localhost:9470/metrics
curl http://localhost:9912/metrics
```
Базови: `baradb_queries_total`, `baradb_query_errors_total`, `baradb_inserts_total`, `baradb_selects_total`, `baradb_connections_active`.
С raft: `baradb_raft_is_leader`, `baradb_raft_term`, `baradb_raft_log_entries`, `baradb_raft_apply_lag`, `baradb_raft_commit_wait_ms_total`, `baradb_raft_elections_total`, `baradb_raft_forwards_total`, `baradb_raft_compactions_total` и др. Пълен списък: [distributed.md](distributed.md) / [en/monitoring.md](../en/monitoring.md).
Примерен изход:
```
docs/en/distributed.md
+30 -1
View File
@@ -5,9 +5,11 @@ BaraDB supports distributed deployment with Raft consensus, sharding, and replic
> ⚠️ **Multi-Database Limitation**
> The distributed modules (Raft, sharding, and replication) are currently wired to the **`default`** database only. If you use multiple databases (`CREATE DATABASE`, `USE DATABASE`), distributed features do not yet span across them. Each database would need its own cluster setup.
> **Status (2026-07-30):** Raft C3a (network election), C3b (SQL writes), DDL replication, leader forwarding, log compaction, and metrics are **shipped on `main`**. Design/history: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
## Raft Consensus
Leader election and log replication over TCP. Enable with:
Leader election and log replication over TCP; SQL DML/DDL on the default DB go through the raft log. Enable with:
| Env | Meaning |
|-----|---------|
@@ -25,6 +27,26 @@ When Raft is enabled, SQL DML (`INSERT`/`UPDATE`/`DELETE`/`MERGE` and transactio
**Metrics:** with raft enabled, `GET /metrics` (HTTP port = `BARADB_PORT + 440`) includes Prometheus lines such as `baradb_raft_is_leader`, `baradb_raft_term`, `baradb_raft_log_entries`, `baradb_raft_apply_lag`, `baradb_raft_commit_wait_ms_total`, `baradb_raft_elections_total`, `baradb_raft_forwards_total`, and `baradb_raft_compactions_total`. `GET /health` embeds a `raft` object (`role`, `term`, `leader_id`, `commit_index`, `apply_lag`, …).
### Minimal 3-node example
```bash
# Shared peers (raft ports) and client peers (SQL ports)
export BARADB_RAFT_ENABLED=true
export BARADB_RAFT_PEERS=n1@127.0.0.1:46101,n2@127.0.0.1:46102,n3@127.0.0.1:46103
export BARADB_RAFT_CLIENT_PEERS=n1@127.0.0.1:46010,n2@127.0.0.1:46020,n3@127.0.0.1:46030
# Terminal 1
BARADB_PORT=46010 BARADB_RAFT_PORT=46101 BARADB_RAFT_NODE_ID=n1 \
BARADB_DATA_DIR=./data/n1 ./build/baradadb
# Terminal 2 / 3 — n2@46020/46102, n3@46030/46103 similarly
# After a leader appears (check logs for "became leader"):
# curl http://127.0.0.1:46450/health # n1 HTTP = 46010+440
```
### In-process API (tests / embedding)
```nim
import barabadb/core/raft
@@ -39,6 +61,13 @@ n1.becomeLeader()
let entry = n1.appendLog("SET key1 value1")
```
### E2E tests
| Test | What it proves |
|------|----------------|
| `tests/raft_e2e_test.nim` | 3 real processes; election + kill-leader failover |
| `tests/raft_writes_e2e_test.nim` | DDL/DML via raft, follower forward, index SELECT, failover writes |
## Sharding
Distribute data across nodes:
docs/en/monitoring.md
+56 -10
View File
@@ -4,21 +4,39 @@
### HTTP Health Endpoint
HTTP listens on **TCP port + 440** (e.g. `BARADB_PORT=9472` → health on `9912`).
```bash
curl http://localhost:9470/health
curl http://localhost:9912/health
```
Response:
Response (raft disabled):
```json
{
"status": "healthy",
"version": "0.1.0",
"uptime_seconds": 86400,
"checks": {
"storage": "ok",
"memory": "ok",
"connections": "ok"
"status": "ok",
"version": "1.1.6",
"raft": { "enabled": false }
}
```
With `BARADB_RAFT_ENABLED=true`, a `raft` object is included:
```json
{
"status": "ok",
"version": "1.1.6",
"raft": {
"enabled": true,
"node_id": "n1",
"role": "leader",
"term": 2,
"leader_id": "n1",
"commit_index": 42,
"last_applied": 42,
"apply_lag": 0,
"log_entries": 12,
"snapshot_index": 30
}
}
```
@@ -35,10 +53,38 @@ Returns `200 OK` when the server is ready to accept traffic, `503` during startu
### Prometheus-Compatible Metrics
Same HTTP base port as health (`BARADB_PORT + 440`). When auth is enabled, send a Bearer token.
```bash
curl http://localhost:9470/metrics
curl http://localhost:9912/metrics
```
Always present:
| Metric | Meaning |
|--------|---------|
| `baradb_queries_total` | HTTP queries handled |
| `baradb_query_errors_total` | Failed HTTP queries |
| `baradb_inserts_total` / `baradb_selects_total` | Statement class counts |
| `baradb_connections_active` | Active connections |
With raft enabled, additional series (labels include `node="…"`):
| Metric | Meaning |
|--------|---------|
| `baradb_raft_is_leader` | 1 if this process is leader |
| `baradb_raft_term` | Current term |
| `baradb_raft_log_entries` | In-memory log length |
| `baradb_raft_commit_index` / `baradb_raft_last_applied` | Raft indices |
| `baradb_raft_apply_lag` | commit − applied |
| `baradb_raft_snapshot_index` | Compacted log base |
| `baradb_raft_elections_total` | Times this node became leader |
| `baradb_raft_commit_wait_ms_total` / `_avg` | Wait-for-commit latency |
| `baradb_raft_forwards_total` | Follower→leader SQL forwards |
| `baradb_raft_compactions_total` | Log prefix compactions |
See also [distributed.md](distributed.md) for cluster env vars and ops notes.
Example output:
```
docs/superpowers/plans/2026-07-30-raft-network-bootstrap.md
+2 -1
View File
@@ -1,6 +1,7 @@
# Networked Raft Bootstrap (C3a) Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
> **Status: DONE** — shipped on `main`. Historical plan; do not re-run tasks.
> Overview: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
**Goal:** A 3-node BaraDB cluster started with ordinary config elects a leader over TCP, maintains it with heartbeats, and re-elects after the leader is killed.
docs/superpowers/plans/2026-07-30-raft-sql-writes.md
+3 -1
View File
@@ -1,6 +1,8 @@
# SQL Writes Through Raft (C3b) Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
> **Status: DONE** — shipped on `main` (plus post-C3b DDL/forward/compact/metrics).
> Historical plan; do not re-run tasks.
> Overview: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
**Goal:** When Raft is enabled, SQL writes commit through the Raft log before the client sees success; followers reject writes naming the leader and apply committed entries via the existing applyCommand loop.
docs/superpowers/specs/2026-07-30-raft-cluster-status.md
+62
View File
@@ -0,0 +1,62 @@
# Raft Cluster Status — C3a / C3b / post-C3b
Date: 2026-07-30
Status: **Shipped on `main`** (tip includes metrics).
Branch: all work merged to `main` only (feature branch removed).
## Phase map
| Phase | Spec / plan | Status | What landed |
|-------|-------------|--------|-------------|
| **C3a** Network bootstrap | `raft-network-bootstrap-design.md` + plan | **Done** | `id@host:port` peers, election timer in production, AppendEntries resets timer, `dataDir` persistence, partial-read frames, `tests/raft_e2e_test.nim` |
| **C3b** SQL writes | `raft-sql-writes-design.md` + plan | **Done** | `isWrite`, follower reject / forward, leader append+wait-commit, rich apply (LSM+index+graph), multi-stmt gate, default-DB only, `tests/raft_writes_e2e_test.nim` |
| **C3c-lite** DDL + ops | (this status doc) | **Done** | DDL via `ddl` log entries; leader forwarding (`BARADB_RAFT_CLIENT_PEERS`); safe log compact + snapshot metadata; Prometheus + `/health` raft |
## Key commits (main)
| Commit (short) | Summary |
|----------------|---------|
| C3a series | peers, timer, frames, e2e election |
| `38c1c01`…`a462d21` | C3b classification → append → e2e → docs |
| `0d51497` | multi-stmt gate, rich apply, delete kv |
| `50f827f` | graph apply, non-default DB reject |
| `095698b` | DDL through raft |
| `9df8316` | leader write/DDL forwarding |
| `53704e1` | safe log compaction + snapshot base |
| `1b3c261` | raft metrics on `/metrics` + `/health` |
## Production behavior (summary)
1. Enable with `BARADB_RAFT_*` env (see `docs/en/distributed.md`).
2. Cluster elects a leader over TCP; state in `dataDir/raft/raft_state.bin`.
3. DML/DDL on **default** DB only: leader appends, waits for majority, returns.
4. Followers forward to leader if `BARADB_RAFT_CLIENT_PEERS` is set; else `not leader`.
5. Apply updates LSM + secondary engines; DDL re-executes SQL on each node.
6. Log soft-cap via safe prefix compact; metrics on HTTP port `BARADB_PORT+440`.
## Explicit non-goals still open
- Multi-database raft (only `default`)
- `CREATE`/`DROP DATABASE` replication
- Membership change (join/leave) protocol
- InstallSnapshot with full SM / LSM payload (v1 uses safe-prefix compact only)
- Raft port TLS / mutual auth
- Read consistency levels (read-your-writes, linearizable reads on followers)
- Automatic client redirect without `CLIENT_PEERS`
## Tests
| Suite | Covers |
|-------|--------|
| `tests/raft_e2e_test.nim` | 3-process election + failover |
| `tests/raft_writes_e2e_test.nim` | DDL + DML replicate, forward, index SELECT, failover writes |
| `tests/test_all.nim` | in-process raft, append/wait, compact, metrics |
| `tests/tla_faithfulness.nim` | ElectionSafety, LogMatching, … |
| `tests/bugfix_test.nim` | peers / client peers / isWrite / isRaftDdl |
## Docs to keep in sync
- `docs/en/distributed.md` / `docs/bg/distributed.md` — operator guide
- `docs/en/monitoring.md` — health/metrics (raft section)
- `CHANGELOG.md` — `[1.2.0] Unreleased` Raft section
- README raft status line
docs/superpowers/specs/2026-07-30-raft-network-bootstrap-design.md
+22 -104
View File
@@ -1,116 +1,34 @@
# Networked Raft Bootstrap (C3a) — Design
Date: 2026-07-30
Status: Approved direction (user: "продължи"); implementation follows.
Date: 2026-07-30
Status: **Done** (merged to `main`). See also `2026-07-30-raft-cluster-status.md`.
## Problem
Raft in BaraDB is half-wired for real networking. `core/raft.nim` already has
a TCP transport (`RaftNetwork`, binary framing, serialization, election-over-
TCP proven by a test), but in production:
1. `node.peerAddrs` is **never populated** (`baradadb.nim` creates the node
from `config.raftPeers` but never parses addresses) — all sends silently
no-op (`raft.nim:560-561`).
2. **No election timer runs** — `tick` is only called from tests; a deployed
node never starts an election.
3. `handleAppendEntries` on the wire path does **not reset the election
timer** — even if timers ran, followers would start elections despite a
healthy leader.
4. Raft **state persistence is disabled** in server startup (`dataDir` not
passed to `newRaftNode`, `baradadb.nim:333`).
5. Framing reads assume full TCP reads (`recv(4)`/`recv(payloadLen)`,
`raft.nim:604-611`) — partial reads corrupt the stream.
Result: `BARADB_RAFT_ENABLED=true` today starts a listening socket that can
never elect anyone. This phase wires what exists; it does NOT change the SQL
write path (that is C3b) or add membership/snapshots (C3c).
Raft in BaraDB was half-wired for real networking. `core/raft.nim` already had
a TCP transport, but production never populated `peerAddrs`, never ran an
election timer, did not reset timers on AppendEntries, skipped state
persistence, and used unsafe partial frame reads.
## Goal
A 3-node BaraDB cluster started with ordinary config elects a leader over
TCP, maintains it with heartbeats, and re-elects after the leader is killed —
verified end-to-end with real server processes.
A 3-node BaraDB cluster elects a leader over TCP, maintains it with heartbeats,
and re-elects after the leader is killed — verified with real processes
(`tests/raft_e2e_test.nim`).
Non-goals: SQL writes through Raft (C3b), membership changes, snapshots,
reconnect/backoff hardening, TLS/auth on the raft port (C3c).
## Delivered
## Design
| Item | Implementation |
|------|----------------|
| Peer addresses | `BARADB_RAFT_PEERS=id@host:port` → `raftPeerAddrs` |
| Election timer | `timerLoop` inside `RaftNetwork.run` |
| Heartbeat reset | `processMessage` resets timer on AppendEntries with valid term |
| State persistence | `dataDir` → `raft_state.bin` |
| Partial reads | `recvExact` frame reassembly |
| E2E | `tests/raft_e2e_test.nim` (election + failover) |
### 1. Peer address configuration
## Non-goals (handled later)
- Env (existing mechanism): `BARADB_RAFT_PEERS` entries extended from bare
`nodeId` to `nodeId@host:port`. Comma-separated, e.g.
`BARADB_RAFT_PEERS="n1@127.0.0.1:9473,n2@127.0.0.1:9474,n3@127.0.0.1:9475"`.
Bare entries (no `@`) keep current meaning (peer id, no address).
- Parsing lives in `core/config.nim` next to the existing `BARADB_RAFT_*`
env handling (config.nim:174-179), producing `raftPeerAddrs: Table[string,
(string, int)]` on BaraConfig. Malformed entries fail startup with a clear
error (config-time, not runtime).
- `baradadb.nim` startup copies `config.raftPeerAddrs` into
`node.peerAddrs` and passes the data dir (see §3).
### 2. Election timer in production
- A `timerLoop` async proc in `core/raft.nim` (next to `heartbeatLoop`,
raft.nim:625): every 50ms calls `tick(node)`; on election timeout calls
the existing `startElection` path (raft.nim:659-685), which sends
RequestVote over `RaftNetwork.send`.
- `RaftNetwork.run` (raft.nim:633) starts `timerLoop` alongside
`heartbeatLoop` via `asyncCheck` — single place, no baradadb.nim changes
beyond startup wiring.
- Wire-path timer reset: in the message-receive handling of `RaftNetwork`,
after a valid AppendEntries (or heartbeat) is processed, reset the
follower's election timer (`ElectionTimer.lastHeartbeat = now`) — wherever
the existing handler processes inbound AppendEntries, matching what the
in-process tests do manually.
### 3. State persistence on
- `baradadb.nim:333`: pass `config.dataDir` (or a raft subdirectory of it —
check what `newRaftNode` expects; `saveState`/`loadState` write
`raft_state.bin`) so currentTerm/votedFor/log survive restarts, per the
Raft spec (and the TLA models).
### 4. Framing robustness
- Replace the `recv(4)` / `recv(payloadLen)` assumptions with the existing
`recvExact`-style helper pattern used by the main server
(`core/server.nim:312-329` has `recvExact`/`recvExactWithTimeout` — mirror
the approach inside raft.nim; do NOT import server.nim into raft.nim).
### 5. What does NOT change
- SQL write path (ReplicationManager, server.nim) — untouched.
- Raft state machine, message format, serialization — untouched (the TLA-
faithfulness tests pin them).
- `applyCommand` hook in baradadb.nim — stays as-is; committed entries
(from tests / future C3b) still apply to the default DB.
## Testing
TDD where feasible; the headline test is end-to-end:
1. **Config parsing** (unit, `tests/test_all.nim` or bugfix_test): peers
with and without `@host:port`, malformed entries error clearly.
2. **Framing** (unit): partial-read feed of a serialized message through the
new read path (socketpair or chunked strings) — message reassembles.
3. **E2E cluster** (new file `tests/raft_e2e_test.nim`): start 3 real
`build/baradadb` processes with raft enabled on distinct ports
(client/raft ports offset per node), wait for a leader (queryable how?
— simplest: each node logs its role; or a `RAFT STATUS` text command on
the client port if one exists cheaply — decide in the plan), kill the
leader, assert a new leader is elected within ~5s. Clean teardown.
4. Existing suites stay green (`nimble test`, 659+ `[OK]`) — especially the
in-process raft suites and the 3-node election TCP test.
## Risks
- Election-timer/async interactions with the main server's async loop —
`RaftNetwork.run` already runs under `asyncCheck`; timerLoop follows the
same pattern. Watch for CPU spin (50ms sleep, not busy loop).
- Port allocation in the E2E test (parallel CI) — use the same
time-derived port offset pattern as `tests/nimforum_smoke_test.nim`.
- Timer reset wiring point: the inbound message path must reach the node
the timer ticks — same `RaftNode` instance, verified by the E2E test
(no spurious elections under a healthy leader).
SQL writes (C3b), DDL/forward/compact/metrics (post-C3b / C3c-lite) — all
shipped; see cluster status doc. Still open: membership, InstallSnapshot SM
payload, multi-DB raft.
docs/superpowers/specs/2026-07-30-raft-sql-writes-design.md
+35 -89
View File
@@ -1,109 +1,55 @@
# SQL Writes Through Raft (C3b) — Design
Date: 2026-07-30
Status: Done (implemented on `feat/raft-sql-writes`).
Date: 2026-07-30
Status: **Done** (merged to `main`). Extended by post-C3b work (DDL, forward,
compact, metrics) — see `2026-07-30-raft-cluster-status.md`.
## Problem
With C3a a real Raft cluster elects a leader over TCP, but SQL writes still
bypass Raft entirely: they go straight to the local LSM and (optionally) to
the legacy `ReplicationManager` (unconnected in production). Committed Raft
entries can be applied (`applyCommand` is wired and invoked on commit), yet
nothing ever calls `appendLog` from the write path.
bypassed Raft: they went straight to the local LSM. Committed Raft entries
could be applied (`applyCommand`), yet nothing called `appendLog` from the
write path.
## Goal
## Goal (delivered)
When Raft is enabled, SQL writes are committed through the Raft log before
the client sees success:
When Raft is enabled, SQL writes commit through the Raft log before the client
sees success:
- Leader: execute locally (constraints validated as today), append the
resulting KV pairs to the Raft log, wait for majority commit, then return.
- Followers: reject write statements with a clear "not leader" error naming
the known leader; apply committed entries via the existing `applyCommand`
loop (raft.nim:195-215).
- Non-raft deployments: byte-identical behavior to today.
- **Leader:** execute locally, append KV pairs (`put` / `delete`), wait for
majority `commitIndex`, return.
- **Followers:** originally reject with `not leader; leader is '…'`; with
`BARADB_RAFT_CLIENT_PEERS` they **forward** to the leader (post-C3b).
- **Apply:** LSM + secondary B-tree/FTS/HNSW + in-memory graphs
(`applyReplicatedPut` / `Delete`).
- **Non-raft:** behavior gated on `raftNode == nil`.
Non-goals (C3c+): leader forwarding/proxy, read consistency levels,
schema/DDL replication (DDL produces no keyValuePairs today — out of scope),
membership, snapshots.
## Design (as shipped)
## Design
### Write interception
### Write interception point
`core/server.nim` `executeQuery`:
`core/server.nim:206-227` — the server-level `executeQuery` parses the AST
(server.nim:213) and, after successful execution, already ships
`res.keyValuePairs` to the ReplicationManager (server.nim:219-227). The Raft
path slots into the same place with the same data source:
1. Classify the statement right after parse: write = `stmt.kind in {nkInsert,
nkUpdate, nkDelete, nkMerge}` (DDL is out of scope; SELECT unaffected).
New helper `isWrite(stmt)` in `exec/params.nim` next to `isDDL`.
2. If Raft is active for this server (`server.raftNode != nil`):
- Not leader (`node.state != rsLeader`): return error
`not leader; leader is '<node.leaderId>'` (or `no leader elected`).
The statement is NOT executed.
- Leader: execute as today. If `res.success` and
`res.keyValuePairs.len > 0`: for each `(key, value)` append one log
entry — `cmd = "put"` with data `key \x00 value`, or `cmd = "delete"`
with data `key` when value is empty (this is EXACTLY the format the
existing `applyCommand` in baradadb.nim:336-343 already consumes — no
format changes, frozen). Then wait until `node.commitIndex` reaches the
last appended index (see below) before returning `res`.
- The legacy `replication.writeLsn` hook is skipped when the Raft write
path handled the statement (no double shipping).
3. Single-node raft (enabled, no peers): majority is self — appends commit
immediately; behavior is correct with negligible overhead.
1. Classify every statement: `isWrite` / `isRaftDdl` (not only `stmts[0]`).
2. If raft active and write/DDL: require **default** database + leader (or
forward).
3. After success: pure DML → `appendWriteToRaft`; any DDL in batch →
`appendDdlToRaft` (full SQL re-exec on apply).
### Wait-for-commit
No raft.nim changes (the state machine stays TLA-frozen). Poll from
server.nim: after appending, `while node.commitIndex < lastIdx`: sleep 10ms,
up to `config.raftWriteTimeoutMs` (new env `BARADB_RAFT_WRITE_TIMEOUT_MS`,
default 5000). Commit advances on the 50ms heartbeat cadence via the
existing `handleAppendReply`/`applyCommitted` path, so typical latency is
one heartbeat. On timeout: return an error (`raft commit timeout`) — the
entry may still commit later; documented limitation. Local execution already
happened (it validates constraints and produces kvPairs) — leader double-
applies via applyCommand on commit, which is idempotent for put/delete KV.
Poll `commitIndex` outside the storage gate (`appendWriteToRaft` /
`waitRaftCommit`); timeout → `raft commit timeout`.
### Server wiring
### Known v1 limitations (still true)
`Server` gains `raftNode*: RaftNode` (nil by default) in core/server.nim;
baradadb.nim assigns it when raft is enabled (the node already exists there).
server.nim imports core/raft (no cycle: raft.nim does not import server).
- Multi-DB raft not supported (`raft writes only supported on the 'default' database`).
- `CREATE`/`DROP DATABASE` not raft-replicated.
- Leader local execute before majority (timeout leaves local write; documented).
- No InstallSnapshot full SM dump (safe log compact only).
- No membership changes.
### Known v1 limitations (documented in code)
## Tests
- Leader losing leadership between local execution and appendLog (narrow
race): appendLog returns the empty entry (index 0) — surfaced as a
`lost leadership` error; the local write already applied (uncommitted).
- Transactional writes: kvPairs are emitted at COMMIT (executor.nim:970-979)
— the COMMIT statement is the raft-replicated unit (classify nkCommitTxn?
— resolve in planning: COMMIT's kvPairs flow through the same hook since
the existing replication hook already catches them).
- DDL is not replicated (no kvPairs today).
## Testing
1. Unit (`tests/bugfix_test.nim`): `isWrite` classification for
INSERT/UPDATE/DELETE/MERGE vs SELECT/DDL.
2. Follower rejection (in-process or E2E): write on a follower returns the
"not leader" error naming the leader; the statement is not executed.
3. E2E (extend `tests/raft_e2e_test.nim` or sibling): 3 real nodes; write a
row on the leader via the Nim client (adaptors/nim/baradb_sqlite pattern
from nimforum_smoke_test); poll a SELECT on a follower until the row
appears (deadline ~5s — follower applies on commit via applyCommand);
write on a follower → rejected; kill leader → re-election → writes work
on the new leader.
4. Full `nimble test` green (673+ `[OK]`).
## Risks
- Double application on the leader (executor + applyCommand): idempotent
KV semantics — verify put/delete idempotency holds for the value encoding
(same key → same value; delete of existing key).
- `node.log[idx-1]` positional indexing in the leader commit loop
(raft.nim:407) — pre-existing, out of scope unless the E2E trips it.
- Apply lag on followers (heartbeat cadence): tests poll with deadlines,
never fixed sleeps.
- Unit: classification, append/timeout, apply indexes/graphs, compact, metrics
- E2E: `tests/raft_writes_e2e_test.nim` (schema, forward, index SELECT, failover)