docs: raft cluster status, plans closed, CHANGELOG/README/monitoring

- Add raft-cluster-status overview (C3a/C3b/post-C3b shipped on main)
- Mark C3a/C3b design+plans done; refresh operator docs en/bg
- CHANGELOG 1.2.0 Raft section; README cluster example and status line
- monitoring.md health/metrics match real HTTP port+440 and raft series

CHANGELOG.md

@@ -4,6 +4,20 @@ All notable changes to BaraDB are documented in this file.
 
 ## [1.2.0] — Unreleased
 
+### Raft cluster (C3a / C3b / ops)
+
+Production-ready path from config → election → SQL/DDL replication → ops.
+
+- **C3a — Networked bootstrap** — `BARADB_RAFT_PEERS=id@host:port`, election timer in production, heartbeat timer reset on AppendEntries, `raft_state.bin` persistence, partial-read-safe frames; E2E `tests/raft_e2e_test.nim` (3-node election + failover)
+- **C3b — SQL writes through Raft** — leader appends DML KV pairs and waits for majority commit; followers reject or **forward** via `BARADB_RAFT_CLIENT_PEERS`; apply path updates LSM + B-tree/FTS/HNSW + graphs; multi-statement write gate; writes only on `default` database
+- **DDL replication** — schema DDL (`CREATE`/`DROP`/`ALTER` table, index, view, graph, …) as `ddl` log entries; re-executed on apply; `CREATE`/`DROP DATABASE` excluded
+- **Leader write forwarding** — followers proxy DML/DDL to the leader SQL port when client peers are configured
+- **Safe log compaction** — soft cap `BARADB_RAFT_LOG_MAX_ENTRIES` (default 256); leader never discards past peer `matchIndex`; snapshot base (`lastSnapshotIndex`/`Term`) persisted
+- **Secondary-index point lookup fix** — index scans use `entry.lsmKey` (not the filter column as PK)
+- **Metrics** — Prometheus raft series on `GET /metrics` (HTTP = TCP port + 440); `GET /health` includes `raft` role/term/leader/lag/log size
+- **E2E writes** — `tests/raft_writes_e2e_test.nim` (schema, forward, index SELECT, failover)
+- **Docs** — `docs/en|bg/distributed.md`, `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`
+
 ### Core Storage Hardening
 
 Foundational LSM improvements for write performance, durability, and compaction correctness.

PLAN.md

@@ -155,6 +155,7 @@
 | `PLAN_SQL_ADVANCED.md` — Window Functions, MERGE, etc. | ✅ Завършен |
 | `PLAN_ID_GENERATORS.md` — AUTO_INCREMENT, Sequences, FK | ✅ Завършен |
 | **Този план** — Сесии 10, 11, 12 | ✅ Завършен |
+| Raft C3a/C3b + DDL/forward/compact/metrics (2026-07-30) | ✅ Завършен на `main` — `docs/superpowers/specs/2026-07-30-raft-cluster-status.md` |
 
 ---
 

README.md

@@ -733,6 +733,23 @@ let diff = s.diff(oldSchema, newSchema)
 
 ### Raft Consensus
 
+3-node cluster over TCP (env-driven). SQL DML and schema DDL go through the
+raft log on the **default** database. See **[docs/en/distributed.md](docs/en/distributed.md)**
+for full env vars, forwarding, compaction, and metrics.
+
+```bash
+# Node n1 example
+export BARADB_PORT=46010
+export BARADB_RAFT_ENABLED=true
+export BARADB_RAFT_NODE_ID=n1
+export BARADB_RAFT_PORT=46101
+export BARADB_RAFT_PEERS=n1@127.0.0.1:46101,n2@127.0.0.1:46102,n3@127.0.0.1:46103
+export BARADB_RAFT_CLIENT_PEERS=n1@127.0.0.1:46010,n2@127.0.0.1:46020,n3@127.0.0.1:46030
+export BARADB_DATA_DIR=./data/n1
+./build/baradadb
+# Health / metrics: HTTP on BARADB_PORT+440 → curl localhost:46450/health
+```
+
 ```nim
 import barabadb/core/raft
 
@@ -1559,7 +1576,7 @@ features are still being refined:
 | LSM-Tree SSTable reads | ✅ Implemented | Full disk I/O with compaction, WAL, and bloom filters. |
 | HNSW vector search | ✅ Implemented | Hierarchical graph navigation with SIMD-optimized distance metrics. |
 | TCP server execution | ✅ Implemented | Full binary wire protocol parsing and BaraQL query execution. |
-| Raft consensus | ✅ Core logic | Leader election + log replication over TCP; SQL DML commits through the raft log when `BARADB_RAFT_ENABLED=true` (followers reject writes). |
+| Raft consensus | ✅ Cluster path | TCP election + failover; SQL DML/DDL via raft log; leader forwarding; safe log compact; `/metrics` + `/health` raft gauges. See `docs/en/distributed.md`. |
 | Graph / FTS / Columnar | ✅ Implemented | In-memory engines with serialization; FTS/vector/graph indexes persist across restarts. |
 | Query codegen | ✅ Implemented | IR plans compile to storage engine operations with optimization passes. |
 
@@ -1568,7 +1585,7 @@ reflects 100% completion across all major phases.
 
 ## Changelog
 
-See [CHANGELOG.md](CHANGELOG.md) for full release history. Package version is **v1.1.8**. The **v1.2.0** line (Unreleased) adds core storage hardening (hash MemTable, WAL group commit, schema persistence, ARC/wire stability) and the Unified Search Engine (heap-optimized HNSW, segment inverted index, boolean/phrase/n-gram/facets, multi-language stemmers).
+See [CHANGELOG.md](CHANGELOG.md) for full release history. Package version is **v1.1.8**. The **v1.2.0** line (Unreleased) adds core storage hardening, Unified Search Engine, **engine persistence** (FTS/HNSW/graphs/B-tree indexes across restart), **executor split**, and a full **Raft cluster path** (election, SQL/DDL replication, forwarding, log compact, metrics — see [docs/en/distributed.md](docs/en/distributed.md)).
 
 ## License
 

docs/bg/distributed.md

@@ -5,9 +5,11 @@ BaraDB поддържа разпределено внедряване с Raft к
 > ⚠️ **Ограничение при множество бази данни**
 > Разпределените модули (Raft, шардиране и репликация) в момента работят само с **`default`** базата данни. Ако използвате множество бази (`CREATE DATABASE`, `USE DATABASE`), разпределените функции още не ги обхващат. Всяка база данни се нуждае от отделна кластър конфигурация.
 
+> **Статус (2026-07-30):** Raft C3a (мрежова election), C3b (SQL записи), DDL репликация, leader forwarding, log compaction и metrics са **на `main`**. Преглед: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
+
 ## Raft Консенсус
 
-Leader election и log репликация през TCP. Включване:
+Leader election и log репликация през TCP; SQL DML/DDL за **default** минават през raft log. Включване:
 
 | Env | Значение |
 |-----|----------|

docs/bg/monitoring.md

@@ -4,29 +4,28 @@
 
 ### HTTP Health Endpoint
 
+HTTP слуша на **TCP порт + 440** (напр. `BARADB_PORT=9472` → health на `9912`).
+
 ```bash
-curl http://localhost:9470/health
+curl http://localhost:9912/health
 ```
 
-Отговор:
+Без raft:
 
 ```json
 {
-  "status": "healthy",
+  "status": "ok",
   "version": "1.1.6",
-  "uptime_seconds": 86400,
+  "raft": { "enabled": false }
-  "checks": {
-    "storage": "ok",
-    "memory": "ok",
-    "connections": "ok"
-  }
 }
 ```
 
+С `BARADB_RAFT_ENABLED=true` — обект `raft` (`role`, `term`, `leader_id`, `commit_index`, `apply_lag`, `log_entries`, `snapshot_index`).
+
 ### Readiness Probe
 
 ```bash
-curl http://localhost:9470/ready
+curl http://localhost:9912/ready
 ```
 
 Връща `200 OK` когато сървърът е готов да приема трафик, `503` по време на стартиране.
@@ -35,10 +34,16 @@ curl http://localhost:9470/ready
 
 ### Prometheus-Съвместими Метрики
 
+Същият HTTP порт като health (`BARADB_PORT + 440`).
+
 ```bash
-curl http://localhost:9470/metrics
+curl http://localhost:9912/metrics
 ```
 
+Базови: `baradb_queries_total`, `baradb_query_errors_total`, `baradb_inserts_total`, `baradb_selects_total`, `baradb_connections_active`.
+
+С raft: `baradb_raft_is_leader`, `baradb_raft_term`, `baradb_raft_log_entries`, `baradb_raft_apply_lag`, `baradb_raft_commit_wait_ms_total`, `baradb_raft_elections_total`, `baradb_raft_forwards_total`, `baradb_raft_compactions_total` и др. Пълен списък: [distributed.md](distributed.md) / [en/monitoring.md](../en/monitoring.md).
+
 Примерен изход:
 
 ```

docs/en/distributed.md

@@ -5,9 +5,11 @@ BaraDB supports distributed deployment with Raft consensus, sharding, and replic
 > ⚠️ **Multi-Database Limitation**
 > The distributed modules (Raft, sharding, and replication) are currently wired to the **`default`** database only. If you use multiple databases (`CREATE DATABASE`, `USE DATABASE`), distributed features do not yet span across them. Each database would need its own cluster setup.
 
+> **Status (2026-07-30):** Raft C3a (network election), C3b (SQL writes), DDL replication, leader forwarding, log compaction, and metrics are **shipped on `main`**. Design/history: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
+
 ## Raft Consensus
 
-Leader election and log replication over TCP. Enable with:
+Leader election and log replication over TCP; SQL DML/DDL on the default DB go through the raft log. Enable with:
 
 | Env | Meaning |
 |-----|---------|
@@ -25,6 +27,26 @@ When Raft is enabled, SQL DML (`INSERT`/`UPDATE`/`DELETE`/`MERGE` and transactio
 
 **Metrics:** with raft enabled, `GET /metrics` (HTTP port = `BARADB_PORT + 440`) includes Prometheus lines such as `baradb_raft_is_leader`, `baradb_raft_term`, `baradb_raft_log_entries`, `baradb_raft_apply_lag`, `baradb_raft_commit_wait_ms_total`, `baradb_raft_elections_total`, `baradb_raft_forwards_total`, and `baradb_raft_compactions_total`. `GET /health` embeds a `raft` object (`role`, `term`, `leader_id`, `commit_index`, `apply_lag`, …).
 
+### Minimal 3-node example
+
+```bash
+# Shared peers (raft ports) and client peers (SQL ports)
+export BARADB_RAFT_ENABLED=true
+export BARADB_RAFT_PEERS=n1@127.0.0.1:46101,n2@127.0.0.1:46102,n3@127.0.0.1:46103
+export BARADB_RAFT_CLIENT_PEERS=n1@127.0.0.1:46010,n2@127.0.0.1:46020,n3@127.0.0.1:46030
+
+# Terminal 1
+BARADB_PORT=46010 BARADB_RAFT_PORT=46101 BARADB_RAFT_NODE_ID=n1 \
+  BARADB_DATA_DIR=./data/n1 ./build/baradadb
+
+# Terminal 2 / 3 — n2@46020/46102, n3@46030/46103 similarly
+
+# After a leader appears (check logs for "became leader"):
+# curl http://127.0.0.1:46450/health   # n1 HTTP = 46010+440
+```
+
+### In-process API (tests / embedding)
+
 ```nim
 import barabadb/core/raft
 
@@ -39,6 +61,13 @@ n1.becomeLeader()
 let entry = n1.appendLog("SET key1 value1")
 ```
 
+### E2E tests
+
+| Test | What it proves |
+|------|----------------|
+| `tests/raft_e2e_test.nim` | 3 real processes; election + kill-leader failover |
+| `tests/raft_writes_e2e_test.nim` | DDL/DML via raft, follower forward, index SELECT, failover writes |
+
 ## Sharding
 
 Distribute data across nodes:

docs/en/monitoring.md

@@ -4,21 +4,39 @@
 
 ### HTTP Health Endpoint
 
+HTTP listens on **TCP port + 440** (e.g. `BARADB_PORT=9472` → health on `9912`).
+
 ```bash
-curl http://localhost:9470/health
+curl http://localhost:9912/health
 ```
 
-Response:
+Response (raft disabled):
 
 ```json
 {
-  "status": "healthy",
+  "status": "ok",
-  "version": "0.1.0",
+  "version": "1.1.6",
-  "uptime_seconds": 86400,
+  "raft": { "enabled": false }
-  "checks": {
+}
-    "storage": "ok",
+```
-    "memory": "ok",
+
-    "connections": "ok"
+With `BARADB_RAFT_ENABLED=true`, a `raft` object is included:
+
+```json
+{
+  "status": "ok",
+  "version": "1.1.6",
+  "raft": {
+    "enabled": true,
+    "node_id": "n1",
+    "role": "leader",
+    "term": 2,
+    "leader_id": "n1",
+    "commit_index": 42,
+    "last_applied": 42,
+    "apply_lag": 0,
+    "log_entries": 12,
+    "snapshot_index": 30
   }
 }
 ```
@@ -35,10 +53,38 @@ Returns `200 OK` when the server is ready to accept traffic, `503` during startu
 
 ### Prometheus-Compatible Metrics
 
+Same HTTP base port as health (`BARADB_PORT + 440`). When auth is enabled, send a Bearer token.
+
 ```bash
-curl http://localhost:9470/metrics
+curl http://localhost:9912/metrics
 ```
 
+Always present:
+
+| Metric | Meaning |
+|--------|---------|
+| `baradb_queries_total` | HTTP queries handled |
+| `baradb_query_errors_total` | Failed HTTP queries |
+| `baradb_inserts_total` / `baradb_selects_total` | Statement class counts |
+| `baradb_connections_active` | Active connections |
+
+With raft enabled, additional series (labels include `node="…"`):
+
+| Metric | Meaning |
+|--------|---------|
+| `baradb_raft_is_leader` | 1 if this process is leader |
+| `baradb_raft_term` | Current term |
+| `baradb_raft_log_entries` | In-memory log length |
+| `baradb_raft_commit_index` / `baradb_raft_last_applied` | Raft indices |
+| `baradb_raft_apply_lag` | commit − applied |
+| `baradb_raft_snapshot_index` | Compacted log base |
+| `baradb_raft_elections_total` | Times this node became leader |
+| `baradb_raft_commit_wait_ms_total` / `_avg` | Wait-for-commit latency |
+| `baradb_raft_forwards_total` | Follower→leader SQL forwards |
+| `baradb_raft_compactions_total` | Log prefix compactions |
+
+See also [distributed.md](distributed.md) for cluster env vars and ops notes.
+
 Example output:
 
 ```

docs/superpowers/plans/2026-07-30-raft-network-bootstrap.md

@@ -1,6 +1,7 @@
 # Networked Raft Bootstrap (C3a) Implementation Plan
 
-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+> **Status: DONE** — shipped on `main`. Historical plan; do not re-run tasks.
+> Overview: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
 
 **Goal:** A 3-node BaraDB cluster started with ordinary config elects a leader over TCP, maintains it with heartbeats, and re-elects after the leader is killed.
 

docs/superpowers/plans/2026-07-30-raft-sql-writes.md

@@ -1,6 +1,8 @@
 # SQL Writes Through Raft (C3b) Implementation Plan
 
-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+> **Status: DONE** — shipped on `main` (plus post-C3b DDL/forward/compact/metrics).
+> Historical plan; do not re-run tasks.
+> Overview: `docs/superpowers/specs/2026-07-30-raft-cluster-status.md`.
 
 **Goal:** When Raft is enabled, SQL writes commit through the Raft log before the client sees success; followers reject writes naming the leader and apply committed entries via the existing applyCommand loop.
 

docs/superpowers/specs/2026-07-30-raft-cluster-status.md

@@ -0,0 +1,62 @@
+# Raft Cluster Status — C3a / C3b / post-C3b
+
+Date: 2026-07-30  
+Status: **Shipped on `main`** (tip includes metrics).  
+Branch: all work merged to `main` only (feature branch removed).
+
+## Phase map
+
+| Phase | Spec / plan | Status | What landed |
+|-------|-------------|--------|-------------|
+| **C3a** Network bootstrap | `raft-network-bootstrap-design.md` + plan | **Done** | `id@host:port` peers, election timer in production, AppendEntries resets timer, `dataDir` persistence, partial-read frames, `tests/raft_e2e_test.nim` |
+| **C3b** SQL writes | `raft-sql-writes-design.md` + plan | **Done** | `isWrite`, follower reject / forward, leader append+wait-commit, rich apply (LSM+index+graph), multi-stmt gate, default-DB only, `tests/raft_writes_e2e_test.nim` |
+| **C3c-lite** DDL + ops | (this status doc) | **Done** | DDL via `ddl` log entries; leader forwarding (`BARADB_RAFT_CLIENT_PEERS`); safe log compact + snapshot metadata; Prometheus + `/health` raft |
+
+## Key commits (main)
+
+| Commit (short) | Summary |
+|----------------|---------|
+| C3a series | peers, timer, frames, e2e election |
+| `38c1c01`…`a462d21` | C3b classification → append → e2e → docs |
+| `0d51497` | multi-stmt gate, rich apply, delete kv |
+| `50f827f` | graph apply, non-default DB reject |
+| `095698b` | DDL through raft |
+| `9df8316` | leader write/DDL forwarding |
+| `53704e1` | safe log compaction + snapshot base |
+| `1b3c261` | raft metrics on `/metrics` + `/health` |
+
+## Production behavior (summary)
+
+1. Enable with `BARADB_RAFT_*` env (see `docs/en/distributed.md`).
+2. Cluster elects a leader over TCP; state in `dataDir/raft/raft_state.bin`.
+3. DML/DDL on **default** DB only: leader appends, waits for majority, returns.
+4. Followers forward to leader if `BARADB_RAFT_CLIENT_PEERS` is set; else `not leader`.
+5. Apply updates LSM + secondary engines; DDL re-executes SQL on each node.
+6. Log soft-cap via safe prefix compact; metrics on HTTP port `BARADB_PORT+440`.
+
+## Explicit non-goals still open
+
+- Multi-database raft (only `default`)
+- `CREATE`/`DROP DATABASE` replication
+- Membership change (join/leave) protocol
+- InstallSnapshot with full SM / LSM payload (v1 uses safe-prefix compact only)
+- Raft port TLS / mutual auth
+- Read consistency levels (read-your-writes, linearizable reads on followers)
+- Automatic client redirect without `CLIENT_PEERS`
+
+## Tests
+
+| Suite | Covers |
+|-------|--------|
+| `tests/raft_e2e_test.nim` | 3-process election + failover |
+| `tests/raft_writes_e2e_test.nim` | DDL + DML replicate, forward, index SELECT, failover writes |
+| `tests/test_all.nim` | in-process raft, append/wait, compact, metrics |
+| `tests/tla_faithfulness.nim` | ElectionSafety, LogMatching, … |
+| `tests/bugfix_test.nim` | peers / client peers / isWrite / isRaftDdl |
+
+## Docs to keep in sync
+
+- `docs/en/distributed.md` / `docs/bg/distributed.md` — operator guide
+- `docs/en/monitoring.md` — health/metrics (raft section)
+- `CHANGELOG.md` — `[1.2.0] Unreleased` Raft section
+- README raft status line

docs/superpowers/specs/2026-07-30-raft-network-bootstrap-design.md

@@ -1,116 +1,34 @@
 # Networked Raft Bootstrap (C3a) — Design
 
 Date: 2026-07-30  
-Status: Approved direction (user: "продължи"); implementation follows.
+Status: **Done** (merged to `main`). See also `2026-07-30-raft-cluster-status.md`.
 
 ## Problem
 
-Raft in BaraDB is half-wired for real networking. `core/raft.nim` already has
+Raft in BaraDB was half-wired for real networking. `core/raft.nim` already had
-a TCP transport (`RaftNetwork`, binary framing, serialization, election-over-
+a TCP transport, but production never populated `peerAddrs`, never ran an
-TCP proven by a test), but in production:
+election timer, did not reset timers on AppendEntries, skipped state
-
+persistence, and used unsafe partial frame reads.
-1. `node.peerAddrs` is **never populated** (`baradadb.nim` creates the node
-   from `config.raftPeers` but never parses addresses) — all sends silently
-   no-op (`raft.nim:560-561`).
-2. **No election timer runs** — `tick` is only called from tests; a deployed
-   node never starts an election.
-3. `handleAppendEntries` on the wire path does **not reset the election
-   timer** — even if timers ran, followers would start elections despite a
-   healthy leader.
-4. Raft **state persistence is disabled** in server startup (`dataDir` not
-   passed to `newRaftNode`, `baradadb.nim:333`).
-5. Framing reads assume full TCP reads (`recv(4)`/`recv(payloadLen)`,
-   `raft.nim:604-611`) — partial reads corrupt the stream.
-
-Result: `BARADB_RAFT_ENABLED=true` today starts a listening socket that can
-never elect anyone. This phase wires what exists; it does NOT change the SQL
-write path (that is C3b) or add membership/snapshots (C3c).
 
 ## Goal
 
-A 3-node BaraDB cluster started with ordinary config elects a leader over
+A 3-node BaraDB cluster elects a leader over TCP, maintains it with heartbeats,
-TCP, maintains it with heartbeats, and re-elects after the leader is killed —
+and re-elects after the leader is killed — verified with real processes
-verified end-to-end with real server processes.
+(`tests/raft_e2e_test.nim`).
 
-Non-goals: SQL writes through Raft (C3b), membership changes, snapshots,
+## Delivered
-reconnect/backoff hardening, TLS/auth on the raft port (C3c).
 
-## Design
+| Item | Implementation |
+|------|----------------|
+| Peer addresses | `BARADB_RAFT_PEERS=id@host:port` → `raftPeerAddrs` |
+| Election timer | `timerLoop` inside `RaftNetwork.run` |
+| Heartbeat reset | `processMessage` resets timer on AppendEntries with valid term |
+| State persistence | `dataDir` → `raft_state.bin` |
+| Partial reads | `recvExact` frame reassembly |
+| E2E | `tests/raft_e2e_test.nim` (election + failover) |
 
-### 1. Peer address configuration
+## Non-goals (handled later)
 
-- Env (existing mechanism): `BARADB_RAFT_PEERS` entries extended from bare
+SQL writes (C3b), DDL/forward/compact/metrics (post-C3b / C3c-lite) — all
-  `nodeId` to `nodeId@host:port`. Comma-separated, e.g.
+shipped; see cluster status doc. Still open: membership, InstallSnapshot SM
-  `BARADB_RAFT_PEERS="n1@127.0.0.1:9473,n2@127.0.0.1:9474,n3@127.0.0.1:9475"`.
+payload, multi-DB raft.
-  Bare entries (no `@`) keep current meaning (peer id, no address).
-- Parsing lives in `core/config.nim` next to the existing `BARADB_RAFT_*`
-  env handling (config.nim:174-179), producing `raftPeerAddrs: Table[string,
-  (string, int)]` on BaraConfig. Malformed entries fail startup with a clear
-  error (config-time, not runtime).
-- `baradadb.nim` startup copies `config.raftPeerAddrs` into
-  `node.peerAddrs` and passes the data dir (see §3).
-
-### 2. Election timer in production
-
-- A `timerLoop` async proc in `core/raft.nim` (next to `heartbeatLoop`,
-  raft.nim:625): every 50ms calls `tick(node)`; on election timeout calls
-  the existing `startElection` path (raft.nim:659-685), which sends
-  RequestVote over `RaftNetwork.send`.
-- `RaftNetwork.run` (raft.nim:633) starts `timerLoop` alongside
-  `heartbeatLoop` via `asyncCheck` — single place, no baradadb.nim changes
-  beyond startup wiring.
-- Wire-path timer reset: in the message-receive handling of `RaftNetwork`,
-  after a valid AppendEntries (or heartbeat) is processed, reset the
-  follower's election timer (`ElectionTimer.lastHeartbeat = now`) — wherever
-  the existing handler processes inbound AppendEntries, matching what the
-  in-process tests do manually.
-
-### 3. State persistence on
-
-- `baradadb.nim:333`: pass `config.dataDir` (or a raft subdirectory of it —
-  check what `newRaftNode` expects; `saveState`/`loadState` write
-  `raft_state.bin`) so currentTerm/votedFor/log survive restarts, per the
-  Raft spec (and the TLA models).
-
-### 4. Framing robustness
-
-- Replace the `recv(4)` / `recv(payloadLen)` assumptions with the existing
-  `recvExact`-style helper pattern used by the main server
-  (`core/server.nim:312-329` has `recvExact`/`recvExactWithTimeout` — mirror
-  the approach inside raft.nim; do NOT import server.nim into raft.nim).
-
-### 5. What does NOT change
-
-- SQL write path (ReplicationManager, server.nim) — untouched.
-- Raft state machine, message format, serialization — untouched (the TLA-
-  faithfulness tests pin them).
-- `applyCommand` hook in baradadb.nim — stays as-is; committed entries
-  (from tests / future C3b) still apply to the default DB.
-
-## Testing
-
-TDD where feasible; the headline test is end-to-end:
-
-1. **Config parsing** (unit, `tests/test_all.nim` or bugfix_test): peers
-   with and without `@host:port`, malformed entries error clearly.
-2. **Framing** (unit): partial-read feed of a serialized message through the
-   new read path (socketpair or chunked strings) — message reassembles.
-3. **E2E cluster** (new file `tests/raft_e2e_test.nim`): start 3 real
-   `build/baradadb` processes with raft enabled on distinct ports
-   (client/raft ports offset per node), wait for a leader (queryable how?
-   — simplest: each node logs its role; or a `RAFT STATUS` text command on
-   the client port if one exists cheaply — decide in the plan), kill the
-   leader, assert a new leader is elected within ~5s. Clean teardown.
-4. Existing suites stay green (`nimble test`, 659+ `[OK]`) — especially the
-   in-process raft suites and the 3-node election TCP test.
-
-## Risks
-
-- Election-timer/async interactions with the main server's async loop —
-  `RaftNetwork.run` already runs under `asyncCheck`; timerLoop follows the
-  same pattern. Watch for CPU spin (50ms sleep, not busy loop).
-- Port allocation in the E2E test (parallel CI) — use the same
-  time-derived port offset pattern as `tests/nimforum_smoke_test.nim`.
-- Timer reset wiring point: the inbound message path must reach the node
-  the timer ticks — same `RaftNode` instance, verified by the E2E test
-  (no spurious elections under a healthy leader).

docs/superpowers/specs/2026-07-30-raft-sql-writes-design.md

@@ -1,109 +1,55 @@
 # SQL Writes Through Raft (C3b) — Design
 
 Date: 2026-07-30  
-Status: Done (implemented on `feat/raft-sql-writes`).
+Status: **Done** (merged to `main`). Extended by post-C3b work (DDL, forward,
+compact, metrics) — see `2026-07-30-raft-cluster-status.md`.
 
 ## Problem
 
 With C3a a real Raft cluster elects a leader over TCP, but SQL writes still
-bypass Raft entirely: they go straight to the local LSM and (optionally) to
+bypassed Raft: they went straight to the local LSM. Committed Raft entries
-the legacy `ReplicationManager` (unconnected in production). Committed Raft
+could be applied (`applyCommand`), yet nothing called `appendLog` from the
-entries can be applied (`applyCommand` is wired and invoked on commit), yet
+write path.
-nothing ever calls `appendLog` from the write path.
 
-## Goal
+## Goal (delivered)
 
-When Raft is enabled, SQL writes are committed through the Raft log before
+When Raft is enabled, SQL writes commit through the Raft log before the client
-the client sees success:
+sees success:
 
-- Leader: execute locally (constraints validated as today), append the
+- **Leader:** execute locally, append KV pairs (`put` / `delete`), wait for
-  resulting KV pairs to the Raft log, wait for majority commit, then return.
+  majority `commitIndex`, return.
-- Followers: reject write statements with a clear "not leader" error naming
+- **Followers:** originally reject with `not leader; leader is '…'`; with
-  the known leader; apply committed entries via the existing `applyCommand`
+  `BARADB_RAFT_CLIENT_PEERS` they **forward** to the leader (post-C3b).
-  loop (raft.nim:195-215).
+- **Apply:** LSM + secondary B-tree/FTS/HNSW + in-memory graphs
-- Non-raft deployments: byte-identical behavior to today.
+  (`applyReplicatedPut` / `Delete`).
+- **Non-raft:** behavior gated on `raftNode == nil`.
 
-Non-goals (C3c+): leader forwarding/proxy, read consistency levels,
+## Design (as shipped)
-schema/DDL replication (DDL produces no keyValuePairs today — out of scope),
-membership, snapshots.
 
-## Design
+### Write interception
 
-### Write interception point
+`core/server.nim` `executeQuery`:
 
-`core/server.nim:206-227` — the server-level `executeQuery` parses the AST
+1. Classify every statement: `isWrite` / `isRaftDdl` (not only `stmts[0]`).
-(server.nim:213) and, after successful execution, already ships
+2. If raft active and write/DDL: require **default** database + leader (or
-`res.keyValuePairs` to the ReplicationManager (server.nim:219-227). The Raft
+   forward).
-path slots into the same place with the same data source:
+3. After success: pure DML → `appendWriteToRaft`; any DDL in batch →
-
+   `appendDdlToRaft` (full SQL re-exec on apply).
-1. Classify the statement right after parse: write = `stmt.kind in {nkInsert,
-   nkUpdate, nkDelete, nkMerge}` (DDL is out of scope; SELECT unaffected).
-   New helper `isWrite(stmt)` in `exec/params.nim` next to `isDDL`.
-2. If Raft is active for this server (`server.raftNode != nil`):
-   - Not leader (`node.state != rsLeader`): return error
-     `not leader; leader is '<node.leaderId>'` (or `no leader elected`).
-     The statement is NOT executed.
-   - Leader: execute as today. If `res.success` and
-     `res.keyValuePairs.len > 0`: for each `(key, value)` append one log
-     entry — `cmd = "put"` with data `key \x00 value`, or `cmd = "delete"`
-     with data `key` when value is empty (this is EXACTLY the format the
-     existing `applyCommand` in baradadb.nim:336-343 already consumes — no
-     format changes, frozen). Then wait until `node.commitIndex` reaches the
-     last appended index (see below) before returning `res`.
-   - The legacy `replication.writeLsn` hook is skipped when the Raft write
-     path handled the statement (no double shipping).
-3. Single-node raft (enabled, no peers): majority is self — appends commit
-   immediately; behavior is correct with negligible overhead.
 
 ### Wait-for-commit
 
-No raft.nim changes (the state machine stays TLA-frozen). Poll from
+Poll `commitIndex` outside the storage gate (`appendWriteToRaft` /
-server.nim: after appending, `while node.commitIndex < lastIdx`: sleep 10ms,
+`waitRaftCommit`); timeout → `raft commit timeout`.
-up to `config.raftWriteTimeoutMs` (new env `BARADB_RAFT_WRITE_TIMEOUT_MS`,
-default 5000). Commit advances on the 50ms heartbeat cadence via the
-existing `handleAppendReply`/`applyCommitted` path, so typical latency is
-one heartbeat. On timeout: return an error (`raft commit timeout`) — the
-entry may still commit later; documented limitation. Local execution already
-happened (it validates constraints and produces kvPairs) — leader double-
-applies via applyCommand on commit, which is idempotent for put/delete KV.
 
-### Server wiring
+### Known v1 limitations (still true)
 
-`Server` gains `raftNode*: RaftNode` (nil by default) in core/server.nim;
+- Multi-DB raft not supported (`raft writes only supported on the 'default' database`).
-baradadb.nim assigns it when raft is enabled (the node already exists there).
+- `CREATE`/`DROP DATABASE` not raft-replicated.
-server.nim imports core/raft (no cycle: raft.nim does not import server).
+- Leader local execute before majority (timeout leaves local write; documented).
+- No InstallSnapshot full SM dump (safe log compact only).
+- No membership changes.
 
-### Known v1 limitations (documented in code)
+## Tests
 
-- Leader losing leadership between local execution and appendLog (narrow
+- Unit: classification, append/timeout, apply indexes/graphs, compact, metrics
-  race): appendLog returns the empty entry (index 0) — surfaced as a
+- E2E: `tests/raft_writes_e2e_test.nim` (schema, forward, index SELECT, failover)
-  `lost leadership` error; the local write already applied (uncommitted).
-- Transactional writes: kvPairs are emitted at COMMIT (executor.nim:970-979)
-  — the COMMIT statement is the raft-replicated unit (classify nkCommitTxn?
-  — resolve in planning: COMMIT's kvPairs flow through the same hook since
-  the existing replication hook already catches them).
-- DDL is not replicated (no kvPairs today).
-
-## Testing
-
-1. Unit (`tests/bugfix_test.nim`): `isWrite` classification for
-   INSERT/UPDATE/DELETE/MERGE vs SELECT/DDL.
-2. Follower rejection (in-process or E2E): write on a follower returns the
-   "not leader" error naming the leader; the statement is not executed.
-3. E2E (extend `tests/raft_e2e_test.nim` or sibling): 3 real nodes; write a
-   row on the leader via the Nim client (adaptors/nim/baradb_sqlite pattern
-   from nimforum_smoke_test); poll a SELECT on a follower until the row
-   appears (deadline ~5s — follower applies on commit via applyCommand);
-   write on a follower → rejected; kill leader → re-election → writes work
-   on the new leader.
-4. Full `nimble test` green (673+ `[OK]`).
-
-## Risks
-
-- Double application on the leader (executor + applyCommand): idempotent
-  KV semantics — verify put/delete idempotency holds for the value encoding
-  (same key → same value; delete of existing key).
-- `node.log[idx-1]` positional indexing in the leader commit loop
-  (raft.nim:407) — pre-existing, out of scope unless the E2E trips it.
-- Apply lag on followers (heartbeat cadence): tests poll with deadlines,
-  never fixed sleeps.