fix: audit batches 3–4 — TLS verify, WS, OFFSET, B-tree, NULL equality
CI / test (push) Has been cancelled
CI / raft-e2e (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled

Close the remaining 2026-08 findings: peer TLS on leader forward, disttxn
SO_ERROR, compaction catalog order, OFFSET without LIMIT, window aggregates,
WebSocket mask/size/auth, SCRAM timing and cbind, B-tree leaf left-max
separators, and SQL three-valued NULL comparisons.
This commit is contained in:
2026-08-28 13:53:02 +03:00
parent e44341e47c
commit 1ed97fb075
21 changed files with 718 additions and 135 deletions
+25 -28
View File
@@ -3,7 +3,7 @@
> Дата: 2026-08-02
> Метод: 4 паралелни одит-агента по слоеве (Storage / Query / Core / Protocol), всеки чете всички файлове в обхвата си и проверява находките срещу реалния код.
> Обхват: **само нови дефекти** — 80-те вече оправени в `BUGS.md` / `BUG_AUDIT.md` / `BARADB_CLIENT_BUGS.md` са изключени.
> **Общо: ~28 находки | Поправени: 17 (батч 1: 5 + батч 2: 12, вкл. hygiene) | Остават: 12**
> **Общо: ~28 находки | Поправени: 28 (батч 1: 5 + батч 2: 12 + батч 3: 10 + батч 4: 2) | Остават: 0**
---
@@ -36,36 +36,33 @@
**Верификация (батч 2):** `baradadb` build чист; `tests/bugfix_test.nim` (вкл. batch-2 suite) и `tests/test_all.nim` (501 OK) минават без `[FAILED]`. `tests/prop_test.nim` B-Tree suite OK (H10 *не* е в този батч — naive left-max fix чупи interleaved remove).
## Поправени — батч 3 (10)
| # | Severity | Проблем | Файл | Fix |
|---|----------|---------|------|-----|
| H2 | 🟠 HIGH | **TLS client връзките не верифицираха сертификата** | `core/server.nim`, `core/config.nim` | Отделен `tlsClient` контекст; CA auto-enable verify; production fail-closed |
| M2 | 🟡 MEDIUM | **disttxn refused-connect wedge** | `core/disttxn.nim` | `getsockopt(SO_ERROR)` + try/except около RPC |
| M7 | 🟡 MEDIUM | **Compaction unlink преди catalog load** | `storage/compaction.nim`, `baradadb.nim` | Unlink след load + MANIFEST |
| M8 | 🟡 MEDIUM | **`OFFSET` без `LIMIT` → 0 реда** | `query/exec/lower.nim`, `plan_exec.nim` | `limitCount = -1` unlimited; clamp negative |
| M9 | 🟡 MEDIUM | **Window SUM/AVG/COUNT/MIN/MAX → NULL** | `query/exec/window.nim` | Frame aggregates |
| M10 | 🟡 MEDIUM | **WebSocket unmasked client frames** | `core/websocket.nim` | Protocol error / close |
| M11 | 🟡 MEDIUM | **WebSocket unbounded buffer** | `core/websocket.nim` | 1 MiB frame / 4 MiB message / 125-byte control |
| M12 | 🟡 MEDIUM | **SUBSCRIBE без table auth** | `core/websocket.nim`, `httpserver.nim` | `canSubscribe` + `hasPrivilegeFor` SELECT |
| L1 | 🟢 LOW | **SCRAM timing user enumeration** | `protocol/auth.nim` | Dummy nonce+encode work за unknown users |
| L2 | 🟢 LOW | **SCRAM `c=` не се верифицира** | `protocol/auth.nim`, `scram.nim` | `c=` must match gs2 header (`biws` за `n,,`) |
## Поправени — батч 4 (2)
| # | Severity | Проблем | Файл | Fix |
|---|----------|---------|------|-----|
| H10 | 🟠 HIGH | **B-tree `remove` separator convention**`splitChild`/search са left-max (`key > sep → right`); `removeRec` копираше first key на дясното дете (right-min). Naive left-max върху *internal* ключове чупи `prop_test`. | `storage/btree.nim` | Leaf borrow/remove пишат left-max; merge underflow се качва нагоре; invariant `max(left) <= sep` (boundary duplicates са позволени, `next` ги събира) |
| L4 | 🟢 LOW | **NULL equality**`NULL = NULL` / `col = NULL` бяха true през string sentinel | `query/exec/eval.nim` | Сравнения, LIKE, IN, NOT, AND/OR: NULL operand → unknown (`\N`); `IS NULL` непроменен |
---
## Остават (12)
## Остават (0)
### 🟠 HIGH (2)
| # | Проблем | Файл | Предложен fix |
|---|---------|------|---------------|
| H2 | **TLS client връзките между възли не верифицират сертификата**`forwardQueryToLeader` ползва `verifyMode = CVerifyNone` → MITM на клъстър линка. Raft client dials са със същия default (`raftTlsVerifyPeer: false`). | `core/server.nim:70` | Verify peer cert срещу CA при client handshake (fail-closed при enabled TLS) |
| H10 | **B-tree `remove` separator convention** — audit: `splitChild` left-max vs `removeRec` right-min. Naive left-max rewrite of separators/borrows **fails** `prop_test` interleaved insert/remove; needs careful multi-level fix + more targeted repro first. | `storage/btree.nim:377` | Repro + full-tree separator invariant; keep borrow/merge/search consistent |
### 🟡 MEDIUM (7)
| # | Проблем | Файл | Предложен fix |
|---|---------|------|---------------|
| M2 | **disttxn `connectWithTimeout` без SO_ERROR + uncaught RPC** — refused connect е "writable" → връща true; `sendDistTxnRpc` няма try/except → OSError wedge-ва 2PC състояние. (BUG-042 fix-нат в replication, не тук.) | `core/disttxn.nim:88` | `getsockopt(SO_ERROR)` + try/except около per-participant RPC |
| M7 | **Compaction unlink-ва input-ите преди output-ът да е loadable в каталога** — verifySSTable вече е преди unlink; остава catalog re-load ordering в caller. | `storage/compaction.nim` / LSM apply | Load/verify output в каталога ПРЕДИ unlink на input-ите |
| M8 | **`OFFSET n` без `LIMIT` връща 0 реда; negative `LIMIT` чупи** — `limitCount = 0` е sentinel и за "няма limit", и за "LIMIT 0"; `sourceRows[start..<endIdx]` с endIdx<start → IndexDefect. | `query/exec/lower.nim:411`, `plan_exec.nim:219` | Отделен sentinel (-1 = unlimited); clamp negative |
| M9 | **Aggregate window функции връщат NULL**`SUM/AVG/COUNT/MIN/MAX OVER (...)` попадат в `else` клона (`"\N"`); само ranking/lead/lag се handle-ват. | `query/exec/window.nim` | `of "sum","avg","count","min","max"` с `resolveFrameBounds` |
| M10 | **WebSocket приема unmasked client frames** — RFC 6455 §5.1 изисква server да затвори връзката при unmasked client frame (cache-poisoning защита). | `core/websocket.nim:85` | Затвори връзката при `masked == false` |
| M11 | **WebSocket без frame/message size limit → DoS**`buf.add(chunk)` расте неограничено; няма 125-byte control-frame cap. | `core/websocket.nim:215` | Max frame/message size + control-frame cap |
| M12 | **WebSocket SUBSCRIBE bypass-ва table-level auth** — всеки автентикиран клиент subscribe-ва към произволна таблица и получава всички insert/delete. | `core/websocket.nim:232` | Table read authorization при subscribe |
### 🟢 LOW (3)
| # | Проблем | Файл | Предложен fix |
|---|---------|------|---------------|
| L1 | **SCRAM timing user enumeration** — unknown user връща веднага, known user прави urandom+HMAC/PBKDF2 работа → timing delta (BUG-049 fix-на съобщението, не timing-а). | `protocol/auth.nim:227` | Equivalent dummy work за unknown users |
| L2 | **SCRAM channel-binding не се верифицира**`c=` се приема verbatim; RFC 5802 изисква валидация. Не е exploitable днес (няма TLS-CB). | `protocol/auth.nim:259` | Enforce expected `c=` (напр. `biws`) |
| L4 | **NULL equality semantics**`NULL = NULL` и `col = NULL` → true (string sentinel сравнение), не unknown/false. Системно за string-based value модела. | `query/exec/eval.nim:431` | Three-valued logic за NULL (по-голям рефакторинг) |
Няма отворени находки от този одит.
---
+11 -1
View File
@@ -8,6 +8,10 @@ All notable changes to BaraDB are documented in this file.
- **MIGRATE auth bypass (CRITICAL)** — the internal `MIGRATE` text-protocol handler now requires authentication (matching the `REP`/`DISTTXN` handlers); previously an unauthenticated client could inject arbitrary key/value rows (`core/server.nim`)
- **Pre-auth wire-length DoS (HIGH)** — `parseHeader` rejects messages larger than the 64 MB wire cap before allocating the receive buffer (`core/server.nim`)
- **TLS peer verify on cluster forwarding (HIGH)** — follower→leader SQL forwarding uses a dedicated client TLS context; `BARADB_TLS_CA_FILE` auto-enables `BARADB_TLS_VERIFY_PEER`; production fails closed if TLS is on without CA+verify (`core/config.nim`, `core/server.nim`)
- **WebSocket unmasked frames / unbounded buffers (MEDIUM)** — unmasked client frames are rejected (RFC 6455 §5.1); frame/message size and control-frame caps bound memory (`core/websocket.nim`)
- **WebSocket SUBSCRIBE table auth (MEDIUM)** — with auth enabled, SUBSCRIBE requires table SELECT privilege (`core/websocket.nim`, `core/httpserver.nim`)
- **SCRAM timing + channel-binding (LOW)** — unknown users do equivalent dummy work; `c=` must match the gs2 header (`protocol/auth.nim`, `protocol/scram.nim`)
### Fixed
@@ -28,6 +32,12 @@ All notable changes to BaraDB are documented in this file.
- **REP replication put/delete encoding** — the legacy REP payload carries an explicit op tag so PK-only inserts (empty value) replicate as puts instead of vanishing as deletes (`core/replication.nim`, `core/server.nim`)
- **REP receiver secondary indexes** — the legacy REP receiver applies via `applyReplicatedPut/Delete` under the storage gate, keeping B-tree/FTS/HNSW/graph indexes consistent on the replica (`core/server.nim`)
- **Snapshot send stall (partial)** — the leader's snapshot send runs gzip off the event loop on a worker thread (`gzipFileAsync`), so heartbeats keep firing during compression; tar (send) and the restore path still run on the loop (`core/backup.nim`, `core/raft.nim`)
- **disttxn refused-connect wedge (MEDIUM)** — `connectWithTimeout` checks `SO_ERROR`; `sendDistTxnRpc` catches `CatchableError` so a refused peer cannot leave 2PC stuck (`core/disttxn.nim`)
- **Compaction catalog order (MEDIUM)** — inputs stay on disk until the output is loaded and MANIFEST is written; then they are unlinked (`storage/compaction.nim`, `baradadb.nim`)
- **`OFFSET n` without `LIMIT` (MEDIUM)** — no longer treated as `LIMIT 0`; negative LIMIT/OFFSET are clamped (`query/exec/lower.nim`, `plan_exec.nim`)
- **Window `SUM`/`AVG`/`COUNT`/`MIN`/`MAX` (MEDIUM)** — aggregate window functions compute over the frame instead of returning NULL (`query/exec/window.nim`)
- **B-tree leaf separators (HIGH)** — leaf borrow/remove now keep the left-max convention used by `splitChild` and search (`key > sep → right`); merge underflow rebalances up the tree; `checkInvariants` guards `max(left) <= sep` (`storage/btree.nim`)
- **NULL comparison three-valued logic (LOW)** — `NULL = NULL` / `col = NULL` / `col != x` yield unknown (`\N`) so `WHERE` excludes them; `IS NULL` is unchanged (`query/exec/eval.nim`)
### Removed
@@ -35,7 +45,7 @@ All notable changes to BaraDB are documented in this file.
### Added
- Deep audit report `BUG_AUDIT_2026-08.md` (~28 findings; 17 fixed across batches 12, ~12 tracked)
- Deep audit report `BUG_AUDIT_2026-08.md` (~28 findings; all fixed across batches 14)
---
+6 -2
View File
@@ -165,7 +165,11 @@
**Батч 2 — поправени (12):** semi-sync partial-ack (H3), COUNT/SUM/AVG(DISTINCT) (H6), UNION/INTERSECT/EXCEPT (H7), MERGE THEN DELETE (H8), WAL torn-record recovery (H9), MVCC write iteration (M1), checkpoint lock leak (M3), flushUnsafe order (M4), compaction empty-key (M5), rewriteLive atomic replace (M6), mmap overflow (L3), stray `protocol/scram` ELF.
**Остават (~12):** вж. `BUG_AUDIT_2026-08.md` — TLS peer verify (H2), B-tree separator (H10, needs careful repro), disttxn SO_ERROR (M2), compaction catalog order (M7), OFFSET-без-LIMIT (M8), window агрегати (M9), WebSocket (M10M12), SCRAM (L1L2), NULL equality (L4).
**Батч 3 — поправени (10):** TLS peer verify (H2), disttxn SO_ERROR (M2), compaction catalog order (M7), OFFSET-без-LIMIT (M8), window агрегати (M9), WebSocket (M10M12), SCRAM (L1L2).
**Батч 4 — поправени (2):** B-tree leaf left-max separators (H10), NULL three-valued comparisons (L4).
**Остават (0):** вж. `BUG_AUDIT_2026-08.md`.
---
@@ -181,7 +185,7 @@
| **Този план** — Сесии 10, 11, 12 | ✅ Завършен |
| Raft C3a/C3b + DDL/forward/compact/metrics (2026-07-30) | ✅ Завършен на `main``docs/superpowers/specs/2026-07-30-raft-cluster-status.md` |
| **Production GA v1.2.0** (single-node) | ✅ `docs/superpowers/plans/2026-07-30-production-ga.md` |
| **Сесия 13** — Stabilization & Deep Audit (2026-08) | 🔄 В процес — батч 1+2 (17 поправки); остават ~12; `BUG_AUDIT_2026-08.md` |
| **Сесия 13** — Stabilization & Deep Audit (2026-08) | ✅ Батч 1–4 (28 поправки); `BUG_AUDIT_2026-08.md` |
---
+26 -1
View File
@@ -14,6 +14,8 @@ type
tlsEnabled*: bool
certFile*: string
keyFile*: string
tlsCaFile*: string
tlsVerifyPeer*: bool
idleTimeoutMs*: int
queryTimeoutMs*: int
slowQueryThresholdMs*: int
@@ -67,6 +69,8 @@ proc defaultConfig*(): BaraConfig =
tlsEnabled: false,
certFile: "",
keyFile: "",
tlsCaFile: "",
tlsVerifyPeer: false,
idleTimeoutMs: 300_000,
queryTimeoutMs: 30_000,
slowQueryThresholdMs: 1_000,
@@ -134,6 +138,8 @@ proc loadConfigFromJson*(path: string, cfg: var BaraConfig) =
if s.hasKey("enabled"): cfg.tlsEnabled = s["enabled"].getBool()
if s.hasKey("cert_file"): cfg.certFile = s["cert_file"].getStr()
if s.hasKey("key_file"): cfg.keyFile = s["key_file"].getStr()
if s.hasKey("ca_file"): cfg.tlsCaFile = s["ca_file"].getStr()
if s.hasKey("verify_peer"): cfg.tlsVerifyPeer = s["verify_peer"].getBool()
if j.hasKey("auth"):
let s = j["auth"]
if s.hasKey("enabled"): cfg.authEnabled = s["enabled"].getBool()
@@ -177,6 +183,13 @@ proc loadConfigFromEnv*(cfg: var BaraConfig) =
cfg.tlsEnabled = parseEnvBool(getEnv("BARADB_TLS_ENABLED", ""), cfg.tlsEnabled)
cfg.certFile = getEnv("BARADB_CERT_FILE", cfg.certFile)
cfg.keyFile = getEnv("BARADB_KEY_FILE", cfg.keyFile)
cfg.tlsCaFile = getEnv("BARADB_TLS_CA_FILE", cfg.tlsCaFile)
let tlsVerifyEnv = getEnv("BARADB_TLS_VERIFY_PEER", "")
if tlsVerifyEnv.len > 0:
cfg.tlsVerifyPeer = parseEnvBool(tlsVerifyEnv, cfg.tlsVerifyPeer)
elif cfg.tlsCaFile.len > 0:
# CA present and verify flag unset → verify (fail-closed for MITM).
cfg.tlsVerifyPeer = true
cfg.idleTimeoutMs = parseEnvInt(getEnv("BARADB_IDLE_TIMEOUT_MS", ""), cfg.idleTimeoutMs)
cfg.queryTimeoutMs = parseEnvInt(getEnv("BARADB_QUERY_TIMEOUT_MS", ""), cfg.queryTimeoutMs)
cfg.slowQueryThresholdMs = parseEnvInt(getEnv("BARADB_SLOW_QUERY_THRESHOLD_MS", ""), cfg.slowQueryThresholdMs)
@@ -233,7 +246,11 @@ proc loadConfigFromEnv*(cfg: var BaraConfig) =
cfg.raftTlsCertFile = getEnv("BARADB_RAFT_TLS_CERT_FILE", cfg.raftTlsCertFile)
cfg.raftTlsKeyFile = getEnv("BARADB_RAFT_TLS_KEY_FILE", cfg.raftTlsKeyFile)
cfg.raftTlsCaFile = getEnv("BARADB_RAFT_TLS_CA_FILE", cfg.raftTlsCaFile)
cfg.raftTlsVerifyPeer = parseEnvBool(getEnv("BARADB_RAFT_TLS_VERIFY_PEER", ""), cfg.raftTlsVerifyPeer)
let raftVerifyEnv = getEnv("BARADB_RAFT_TLS_VERIFY_PEER", "")
if raftVerifyEnv.len > 0:
cfg.raftTlsVerifyPeer = parseEnvBool(raftVerifyEnv, cfg.raftTlsVerifyPeer)
elif cfg.raftTlsCaFile.len > 0:
cfg.raftTlsVerifyPeer = true
# Optional: client (SQL) addresses for leader write forwarding.
# Same id@host:port shape as BARADB_RAFT_PEERS, but ports are BARADB_PORT values.
let clientPeersEnv = getEnv("BARADB_RAFT_CLIENT_PEERS", "")
@@ -292,6 +309,14 @@ proc validateProductionConfig*(cfg: BaraConfig) =
if cfg.jwtSecret in ["change-me", "change-me-to-random-32-char-string", "secret", "default"]:
raise newException(ValueError,
"Production refuses insecure JWT secret placeholder. Set a strong BARADB_JWT_SECRET.")
if cfg.tlsEnabled:
if not cfg.tlsVerifyPeer or cfg.tlsCaFile.len == 0:
raise newException(ValueError,
"Production TLS requires peer verification. Set BARADB_TLS_VERIFY_PEER=true and BARADB_TLS_CA_FILE.")
if cfg.raftTlsEnabled:
if not cfg.raftTlsVerifyPeer or cfg.raftTlsCaFile.len == 0:
raise newException(ValueError,
"Production raft TLS requires peer verification. Set BARADB_RAFT_TLS_VERIFY_PEER=true and BARADB_RAFT_TLS_CA_FILE.")
proc getEffectiveJwtSecret*(cfg: BaraConfig): string =
if cfg.jwtSecret.len > 0:
+20 -8
View File
@@ -5,6 +5,8 @@ import std/monotimes
import std/net
import std/strutils
import std/nativesockets
when defined(posix):
import std/posix
type
DistTxnState* = enum
@@ -89,6 +91,13 @@ proc connectWithTimeout(sock: Socket, host: string, port: Port, timeoutMs: int):
var fds = @[sock.getFd]
if selectWrite(fds, timeoutMs) <= 0:
return false
when defined(posix):
# selectWrite reports a refused connect as writable; SO_ERROR tells the truth.
var err: cint = 0
var errLen = SockLen(sizeof(err))
discard posix.getsockopt(sock.getFd, 1'i32, 4'i32, addr err, addr errLen)
if err != 0:
return false
sock.getFd.setBlocking(true)
return true
@@ -96,15 +105,18 @@ proc sendDistTxnRpc(host: string, port: int, txnId: uint64, action: string, time
## Send 2PC RPC to participant node via TCP text protocol.
## Protocol: "DISTTXN <txnId> <action>\n" where action = PREPARE|COMMIT|ROLLBACK
## Response: "OK\n" or "ERR <msg>\n"
var sock = newSocket()
defer: sock.close()
if not connectWithTimeout(sock, host, Port(port), timeoutMs):
try:
var sock = newSocket()
defer: sock.close()
if not connectWithTimeout(sock, host, Port(port), timeoutMs):
return false
let msg = "DISTTXN " & $txnId & " " & action & "\n"
sock.send(msg)
var response = ""
sock.readLine(response)
return response.strip() == "OK"
except CatchableError:
return false
let msg = "DISTTXN " & $txnId & " " & action & "\n"
sock.send(msg)
var response = ""
sock.readLine(response)
return response.strip() == "OK"
type
ParticipantInfo = object
+9
View File
@@ -19,6 +19,7 @@ import ../storage/gate
import ../core/mvcc
import ../protocol/wire
import ../core/websocket
import ../query/exec/rls
import jwt as jwtlib
import ../protocol/auth
import ../protocol/ratelimit
@@ -55,6 +56,14 @@ proc newHttpServerWithRegistry*(config: BaraConfig, registry: DatabaseRegistry):
ctx.txnManager = newTxnManager()
let secret = config.getEffectiveJwtSecret()
let ws = newWsServer(config, secret)
block:
let wsRef {.cursor.} = ws
let ctxRef {.cursor.} = ctx
wsRef.canSubscribe = proc(username, table: string): bool {.gcsafe.} =
if username.len == 0:
return false
{.cast(gcsafe).}:
return hasPrivilegeFor(ctxRef, username, table, "SELECT")
let rl = newRateLimiter(rlaTokenBucket, config.rateLimitGlobal, config.rateLimitPerClient)
ctx.onChange = proc(ev: ChangeEvent) =
let msg = $ev.kind & " " & ev.table
+28 -9
View File
@@ -1,6 +1,7 @@
## BaraDB Server — async TCP server with wire protocol
import std/asyncdispatch
import std/asyncnet
import std/os
import std/strutils
import std/sequtils
import std/tables
@@ -50,6 +51,10 @@ type
clusterMembership*: ClusterMembership
gossipProtocol*: GossipProtocol
tls*: TLSContext
## Dedicated client-role TLS context for follower→leader forwarding.
## Must not reuse `tls` with verifyPeer — OpenSSL contexts are role-agnostic
## and enabling verify on the server context would break inbound handshakes.
tlsClient*: TLSContext
rateLimiter*: RateLimiter
activeConnections*: int
activeConnectionsLock*: Lock
@@ -66,9 +71,22 @@ proc newServerWithRegistry*(config: BaraConfig, registry: DatabaseRegistry): Ser
let ctx = cast[ExecutionContext](cast[pointer](dbInfo.ctx))
ctx.txnManager = newTxnManager()
var tls: TLSContext = nil
var tlsClient: TLSContext = nil
if config.tlsEnabled and config.certFile.len > 0 and config.keyFile.len > 0:
let tlsConfig = newTLSConfig(config.certFile, config.keyFile)
tls = newTLSContext(tlsConfig)
if config.tlsVerifyPeer and config.tlsCaFile.len == 0:
raise newException(ValueError,
"tlsVerifyPeer is true but CA file is missing. Set BARADB_TLS_CA_FILE")
if config.tlsVerifyPeer and config.tlsCaFile.len > 0 and
not fileExists(config.tlsCaFile):
raise newException(ValueError,
"BARADB_TLS_VERIFY_PEER=true but CA file missing: " & config.tlsCaFile)
tls = newTLSContext(newTLSConfig(config.certFile, config.keyFile))
if config.tlsVerifyPeer:
tlsClient = newTLSContext(newTLSConfig(
config.certFile, config.keyFile,
caFile = config.tlsCaFile, verifyPeer = true))
else:
tlsClient = tls
# Initialize sharding / gossip. Server fields own the refs; locals used inside
# callback closures are {.cursor.} so ARC does not form uncollectable cycles
@@ -85,6 +103,7 @@ proc newServerWithRegistry*(config: BaraConfig, registry: DatabaseRegistry): Ser
clusterMembership: nil,
gossipProtocol: newGossipProtocol(localId, config.address, config.port, gossipPort = gossipPort),
tls: tls,
tlsClient: tlsClient,
rateLimiter: rl)
result.clusterMembership = newClusterMembership(result.shardRouter, localId)
initLock(result.activeConnectionsLock)
@@ -227,12 +246,12 @@ proc forwardQueryToLeader*(host: string, port: int, query: string,
timeoutMs: int = 5000): Future[(bool, QueryResult, string)] {.async.} =
## Proxy a write/DDL to the known leader's SQL port. Used by followers when
## BARADB_RAFT_CLIENT_PEERS maps leader id → host:clientPort.
## `tls` is the local server's client-port TLS context: when the wire port
## `tls` is a *client-role* context (Server.tlsClient). When the wire port
## serves TLS, the leader's does too, so the forwarding dial must complete a
## client handshake. The context is reused as-is (verifyMode stays
## CVerifyNone — do NOT enable verifyPeer on the reused context); OpenSSL
## contexts are role-agnostic in Nim's stdlib, wrapConnectedSocket with
## handshakeAsClient sets the role.
## client handshake. Peer verification is honoured when that context was
## built with verifyPeer (BARADB_TLS_VERIFY_PEER + BARADB_TLS_CA_FILE).
## Do NOT pass Server.tls (the inbound/server context) with verifyPeer
## flipped on — OpenSSL contexts are role-agnostic in Nim's stdlib.
var sock: AsyncSocket = nil
try:
sock = newAsyncSocket()
@@ -806,7 +825,7 @@ proc handleClient(server: Server, client: AsyncSocket, clientId: int) {.async.}
replication=server.replicationManager, raftNode=server.raftNode,
raftWriteTimeoutMs=server.config.raftWriteTimeoutMs,
raftPeerClientAddrs=server.config.raftPeerClientAddrs,
forwardTls=server.tls)
forwardTls=server.tlsClient)
let durationMs = int((getMonoTime().ticks() - startTicks) div 1_000_000)
if durationMs >= slowThreshold:
@@ -830,7 +849,7 @@ proc handleClient(server: Server, client: AsyncSocket, clientId: int) {.async.}
replication=server.replicationManager, raftNode=server.raftNode,
raftWriteTimeoutMs=server.config.raftWriteTimeoutMs,
raftPeerClientAddrs=server.config.raftPeerClientAddrs,
forwardTls=server.tls)
forwardTls=server.tlsClient)
let durationMs = int((getMonoTime().ticks() - startTicks) div 1_000_000)
if durationMs >= slowThreshold:
+91 -38
View File
@@ -15,18 +15,25 @@ else:
import config
import jwt as jwtlib
const
## RFC 6455: reject oversized frames/messages to bound memory.
MaxWsFrameBytes* = 1 * 1024 * 1024
MaxWsMessageBytes* = 4 * 1024 * 1024
MaxWsControlPayload* = 125
type
WsFrame = object
fin: bool
opcode: uint8
masked: bool
payloadLen: uint64
maskKey: array[4, byte]
payload: string
WsFrame* = object
fin*: bool
opcode*: uint8
masked*: bool
payloadLen*: uint64
maskKey*: array[4, byte]
payload*: string
WsClient* = ref object
socket: AsyncSocket
id: int
username: string
subscriptions: HashSet[string]
WsServer* = ref object
@@ -35,6 +42,8 @@ type
running: bool
config*: BaraConfig
secretKey*: string
## Table-level read authorization for SUBSCRIBE. Nil + authEnabled → deny.
canSubscribe*: proc(username, table: string): bool {.closure, gcsafe.}
onInsert*: proc (table, key, value: string) {.closure.}
onDelete*: proc (table, key: string) {.closure.}
@@ -46,20 +55,19 @@ proc newWsServer*(cfg: BaraConfig = defaultConfig(), secret: string = ""): WsSer
# WebSocket frame encoding/decoding (RFC 6455)
# ----------------------------------------------------------------------
proc encodeFrame(opcode: uint8, payload: string): string =
proc encodeFrame*(opcode: uint8, payload: string, masked = false,
maskKey: array[4, byte] = [0'u8, 0, 0, 0]): string =
result = ""
let isMasked = false
var b0 = 0x80'u8 or opcode
result.add(char(b0))
var b1 = 0'u8
if not isMasked:
if payload.len < 126:
b1 = uint8(payload.len)
elif payload.len <= 65535:
b1 = 126
else:
b1 = 127
var b1 = if masked: 0x80'u8 else: 0'u8
if payload.len < 126:
b1 = b1 or uint8(payload.len)
elif payload.len <= 65535:
b1 = b1 or 126
else:
b1 = b1 or 127
result.add(char(b1))
if payload.len >= 126 and payload.len <= 65535:
@@ -71,9 +79,17 @@ proc encodeFrame(opcode: uint8, payload: string): string =
for i in countdown(7, 0):
result.add(char((len64 shr (i * 8)) and 0xFF))
result.add(payload)
if masked:
for i in 0..3:
result.add(char(maskKey[i]))
for i, c in payload:
result.add(char(byte(c) xor maskKey[i mod 4]))
else:
result.add(payload)
proc decodeFrame(data: string): (WsFrame, int) =
proc decodeFrame*(data: string): (WsFrame, int) =
## Returns (frame, consumed). consumed == 0 → need more bytes;
## consumed < 0 → protocol error (close the connection).
if data.len < 2:
return (WsFrame(), 0)
@@ -84,6 +100,10 @@ proc decodeFrame(data: string): (WsFrame, int) =
frame.opcode = b0 and 0x0F
frame.masked = (b1 and 0x80) != 0
# RFC 6455 §5.1 — client-to-server frames MUST be masked.
if not frame.masked:
return (WsFrame(), -1)
var len = uint64(b1 and 0x7F)
var offset = 2
@@ -98,23 +118,24 @@ proc decodeFrame(data: string): (WsFrame, int) =
len = (len shl 8) or uint64(uint8(data[2 + i]))
offset = 10
if frame.masked:
if data.len < offset + 4: return (WsFrame(), 0)
for i in 0..3:
frame.maskKey[i] = byte(data[offset + i])
offset += 4
let isControl = frame.opcode == 0x8 or frame.opcode == 0x9 or frame.opcode == 0xA
if isControl and (not frame.fin or len > uint64(MaxWsControlPayload)):
return (WsFrame(), -1)
if len > uint64(MaxWsFrameBytes):
return (WsFrame(), -1)
if data.len < offset + 4: return (WsFrame(), 0)
for i in 0..3:
frame.maskKey[i] = byte(data[offset + i])
offset += 4
if uint64(data.len) < uint64(offset) + len:
return (Wsframe(), 0)
return (WsFrame(), 0)
if len > uint64(high(int) - 1):
return (Wsframe(), 0)
let plen = int(len)
if frame.masked:
for i in 0..<plen:
frame.payload.add(char(byte(data[offset + i]) xor frame.maskKey[i mod 4]))
else:
frame.payload = data[offset..offset + plen - 1]
frame.payloadLen = len
for i in 0..<plen:
frame.payload.add(char(byte(data[offset + i]) xor frame.maskKey[i mod 4]))
return (frame, offset + plen)
@@ -179,6 +200,14 @@ proc computeAcceptKey(key: string): string =
# Subscription management
# ----------------------------------------------------------------------
proc validSubscribeTable(table: string): bool =
if table.len == 0 or table.len > 128:
return false
for c in table:
if c notin {'a'..'z', 'A'..'Z', '0'..'9', '_'}:
return false
return true
proc subscribe*(client: WsClient, table: string) =
client.subscriptions.incl(table)
@@ -201,9 +230,11 @@ proc broadcastToTable*(server: WsServer, table: string, msg: string) {.async.} =
# WebSocket client handler
# ----------------------------------------------------------------------
proc handleWsClient(server: WsServer, client: AsyncSocket, id: int) {.async.} =
proc handleWsClient(server: WsServer, client: AsyncSocket, id: int,
username: string = "") {.async.} =
echo "WebSocket client ", id, " connected"
var wsClient = WsClient(socket: client, id: id, subscriptions: initHashSet[string]())
var wsClient = WsClient(socket: client, id: id, username: username,
subscriptions: initHashSet[string]())
server.clients[id] = wsClient
var buf = ""
@@ -212,12 +243,22 @@ proc handleWsClient(server: WsServer, client: AsyncSocket, id: int) {.async.} =
let chunk = await client.recv(4096)
if chunk.len == 0:
break
if buf.len + chunk.len > MaxWsMessageBytes:
let closeF = encodeFrame(0x8, "")
try: await client.send(closeF) except CatchableError: discard
break
buf.add(chunk)
while buf.len >= 2:
let (frame, consumed) = decodeFrame(buf)
if consumed == 0:
break
if consumed < 0:
let closeF = encodeFrame(0x8, "")
try: await client.send(closeF) except CatchableError: discard
client.close()
server.clients.del(id)
return
case frame.opcode
of 0x8: # close
@@ -231,9 +272,18 @@ proc handleWsClient(server: WsServer, client: AsyncSocket, id: int) {.async.} =
let msg = frame.payload
if msg.startsWith("SUBSCRIBE "):
let table = msg[10..^1].strip()
wsClient.subscribe(table)
let ack = encodeFrame(0x1, "OK subscribed to " & table)
await client.send(ack)
var allowed = validSubscribeTable(table)
if allowed and server.config.authEnabled:
if username.len == 0 or server.canSubscribe == nil or
not server.canSubscribe(username, table):
allowed = false
if allowed:
wsClient.subscribe(table)
let ack = encodeFrame(0x1, "OK subscribed to " & table)
await client.send(ack)
else:
let nack = encodeFrame(0x1, "ERR subscribe denied for " & table)
await client.send(nack)
elif msg.startsWith("UNSUBSCRIBE "):
let table = msg[12..^1].strip()
wsClient.unsubscribe(table)
@@ -285,6 +335,7 @@ proc handleConnection(server: WsServer, client: AsyncSocket) {.async.} =
return
# Auth check
var username = ""
if server.config.authEnabled:
let authHeader = headers.getOrDefault("authorization", "")
if authHeader.len == 0 or not authHeader.startsWith("Bearer "):
@@ -305,6 +356,8 @@ proc handleConnection(server: WsServer, client: AsyncSocket) {.async.} =
await client.send("HTTP/1.1 401 Unauthorized\r\n\r\n")
client.close()
return
if "sub" in token.claims:
username = token.claims["sub"].node.str
except CatchableError:
await client.send("HTTP/1.1 401 Unauthorized\r\n\r\n")
client.close()
@@ -321,7 +374,7 @@ proc handleConnection(server: WsServer, client: AsyncSocket) {.async.} =
await client.send(response)
inc server.nextId
asyncCheck server.handleWsClient(client, server.nextId)
asyncCheck server.handleWsClient(client, server.nextId, username)
proc setTcpNoDelay(sock: AsyncSocket) =
## Enable TCP_NODELAY using the correct protocol level (IPPROTO_TCP).
+13 -2
View File
@@ -221,10 +221,17 @@ proc registerScramUser*(am: AuthManager, username, password: string,
let cred = createScramCredential(password, iterationCount = iterationCount)
am.scramUsers[username] = cred
proc dummyScramStartWork() =
## Match known-user startScram work (urandom nonce + base64) so unknown
## users cannot be enumerated by timing.
discard generateNonce()
discard encode("0123456789abcdef0123456789abcdef")
proc startScram*(am: AuthManager, clientFirstMessage: string): string =
## Start SCRAM authentication. Returns server-first-message.
let (_, username, clientNonce) = parseClientFirst(clientFirstMessage)
if username notin am.scramUsers:
let (gs2, username, clientNonce) = parseClientFirst(clientFirstMessage)
if username notin am.scramUsers or gs2 notin ["n", "y"]:
dummyScramStartWork()
raise newException(ValueError, "Authentication failed")
let cred = am.scramUsers[username]
@@ -239,6 +246,7 @@ proc startScram*(am: AuthManager, clientFirstMessage: string): string =
var state = ScramServerState(
username: username,
gs2Flag: gs2,
clientFirstMessageBare: clientFirstMessageBare,
serverFirstMessage: serverFirst,
authMessage: authMessage,
@@ -264,6 +272,9 @@ proc finishScram*(am: AuthManager, clientFinalMessage: string): (bool, string) =
var state = am.scramSessions[nonce]
am.scramSessions.del(nonce)
if stripB64Padding(cbind) != expectedChannelBinding(state.gs2Flag):
return (false, "e=channel-bindings-dont-match")
# Update authMessage with client-final-message-without-proof
let clientFinalWithoutProof = "c=" & cbind & ",r=" & nonce
state.authMessage = state.authMessage & "," & clientFinalWithoutProof
+14
View File
@@ -19,6 +19,7 @@ type
ScramServerState* = object
username*: string
gs2Flag*: string
clientFirstMessageBare*: string
serverFirstMessage*: string
authMessage*: string
@@ -189,6 +190,19 @@ proc createScramCredential*(password: string, salt: string = "",
# SCRAM message parsing / building
# ---------------------------------------------------------------------------
proc expectedChannelBinding*(gs2Flag: string): string =
## RFC 5802 cbind-input with no cbind-data is the gs2-header: flag + "," + authzid + ",".
## Authzid is unsupported, so the header is `n,,` or `y,,`.
let header = gs2Flag & ",,"
result = encode(header)
while result.endsWith("="):
result.setLen(result.len - 1)
proc stripB64Padding*(s: string): string =
result = s
while result.endsWith("="):
result.setLen(result.len - 1)
proc parseClientFirst*(msg: string): (string, string, string) =
## Parse client-first-message: gs2-header,username,nonce
## Returns: (gs2_header, username, nonce)
+37 -4
View File
@@ -429,6 +429,8 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
let right = evalExprOld(expr.binRight, row, ctx)
case expr.binOp
of irEq:
# SQL three-valued logic: any NULL operand → unknown, not true.
if isNull(left) or isNull(right): return "\\N"
if left == right: return "true"
# Try numeric comparison
try:
@@ -436,6 +438,7 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
except CatchableError: discard
return "false"
of irNeq:
if isNull(left) or isNull(right): return "\\N"
# Numeric-first so `!=` is the exact complement of `=` (irEq): string
# inequality alone would make `5 != 5.0` true while `5 = 5.0` is true.
try:
@@ -443,26 +446,37 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
except CatchableError:
return if left != right: "true" else: "false"
of irLt:
if isNull(left) or isNull(right): return "\\N"
try:
return if parseFloat(left) < parseFloat(right): "true" else: "false"
except CatchableError: return if left < right: "true" else: "false"
of irLte:
if isNull(left) or isNull(right): return "\\N"
try:
return if parseFloat(left) <= parseFloat(right): "true" else: "false"
except CatchableError: return if left <= right: "true" else: "false"
of irGt:
if isNull(left) or isNull(right): return "\\N"
try:
return if parseFloat(left) > parseFloat(right): "true" else: "false"
except CatchableError: return if left > right: "true" else: "false"
of irGte:
if isNull(left) or isNull(right): return "\\N"
try:
return if parseFloat(left) >= parseFloat(right): "true" else: "false"
except CatchableError: return if left >= right: "true" else: "false"
of irAnd:
if left == "true" and right == "true": return "true"
return "false"
# false AND x = false; unknown AND true/unknown = unknown; else both true.
let lNull = isNull(left)
let rNull = isNull(right)
let lTrue = left == "true"
let rTrue = right == "true"
if (not lNull and not lTrue) or (not rNull and not rTrue): return "false"
if lNull or rNull: return "\\N"
return "true"
of irOr:
if left == "true" or right == "true": return "true"
if isNull(left) or isNull(right): return "\\N"
return "false"
of irAdd, irSub, irMul, irDiv, irMod, irPow:
let v = evalExpr(expr, stringTableToValueRow(row), ctx)
@@ -475,6 +489,7 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
of vkString: return v.strVal
else: return "\\N"
of irLike:
if isNull(left) or isNull(right): return "\\N"
proc escapeRe(s: string): string =
result = ""
for ch in s:
@@ -490,6 +505,7 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
except CatchableError: discard
return "false"
of irILike:
if isNull(left) or isNull(right): return "\\N"
proc escapeRe(s: string): string =
result = ""
for ch in s:
@@ -505,8 +521,10 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
except CatchableError: discard
return "false"
of irIn:
if isNull(left): return "\\N"
if expr.binRight.kind == irekSubquery:
let subRows = requireExecutePlanHook()(ctx, expr.binRight.subqueryPlan)
var sawNull = false
for row in subRows:
# Compare against the first non-internal column only (SQL semantics)
var firstVal = ""
@@ -516,8 +534,14 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
firstVal = valueToString(v)
found = true
break
if found and firstVal == left: return "true"
if not found: continue
if isNull(firstVal):
sawNull = true
continue
if firstVal == left: return "true"
if sawNull: return "\\N"
return "false"
if isNull(right): return "\\N"
try:
let lv = parseFloat(left)
let rv = parseFloat(right)
@@ -525,8 +549,10 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
except CatchableError: discard
return if left == right: "true" else: "false"
of irNotIn:
if isNull(left): return "\\N"
if expr.binRight.kind == irekSubquery:
let subRows = requireExecutePlanHook()(ctx, expr.binRight.subqueryPlan)
var sawNull = false
for row in subRows:
# Compare against the first non-internal column only (SQL semantics)
var firstVal = ""
@@ -536,8 +562,14 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
firstVal = valueToString(v)
found = true
break
if found and firstVal == left: return "false"
if not found: continue
if isNull(firstVal):
sawNull = true
continue
if firstVal == left: return "false"
if sawNull: return "\\N"
return "true"
if isNull(right): return "\\N"
try:
let lv = parseFloat(left)
let rv = parseFloat(right)
@@ -662,6 +694,7 @@ proc evalExprOld*(expr: IRExpr, row: Table[string, string], ctx: ExecutionContex
case expr.unOp
of irNot:
let v = evalExprOld(expr.unExpr, row, ctx)
if isNull(v): return "\\N"
return if v == "true": "false" else: "true"
of irIsNull:
let v = evalExprOld(expr.unExpr, row, ctx)
+13 -4
View File
@@ -411,9 +411,18 @@ proc lowerSelect*(node: Node): IRPlan =
if node.selLimit != nil or node.selOffset != nil:
let limitPlan = IRPlan(kind: irpkLimit)
limitPlan.limitSource = result
limitPlan.limitCount = if node.selLimit != nil and node.selLimit.limitExpr.kind == nkIntLit:
node.selLimit.limitExpr.intVal else: 0
limitPlan.limitOffset = if node.selOffset != nil and node.selOffset.offsetExpr.kind == nkIntLit:
node.selOffset.offsetExpr.intVal else: 0
# limitCount: -1 = unlimited (OFFSET without LIMIT). LIMIT 0 is empty.
# Negative LIMIT/OFFSET are clamped so slicing cannot IndexDefect.
if node.selLimit != nil:
if node.selLimit.limitExpr.kind == nkIntLit:
limitPlan.limitCount = max(0'i64, node.selLimit.limitExpr.intVal)
else:
limitPlan.limitCount = 0
else:
limitPlan.limitCount = -1
if node.selOffset != nil and node.selOffset.offsetExpr.kind == nkIntLit:
limitPlan.limitOffset = max(0'i64, node.selOffset.offsetExpr.intVal)
else:
limitPlan.limitOffset = 0
result = limitPlan
+6
View File
@@ -255,12 +255,18 @@ proc executePlan*(ctx: ExecutionContext, plan: IRPlan): seq[Row] =
of irpkLimit:
let sourceRows = executePlan(ctx, plan.limitSource)
var start = int(plan.limitOffset)
if start < 0: start = 0
if start > sourceRows.len: start = sourceRows.len
if plan.limitCount < 0:
# OFFSET without LIMIT — return the remainder.
return sourceRows[start ..< sourceRows.len]
if plan.limitCount == 0:
return @[]
var endIdx = start + int(plan.limitCount)
if endIdx > sourceRows.len:
endIdx = sourceRows.len
if endIdx < start:
endIdx = start
return sourceRows[start..<endIdx]
of irpkGroupBy:
+8 -6
View File
@@ -11,20 +11,22 @@ import lower
# Row-Level Security
# ----------------------------------------------------------------------
proc hasPrivilege*(ctx: ExecutionContext, tableName, command: string): bool =
if ctx.currentUser.len == 0: return true
let user = ctx.users.getOrDefault(ctx.currentUser)
proc hasPrivilegeFor*(ctx: ExecutionContext, username, tableName, command: string): bool =
## Privilege check for an explicit username (does not mutate ctx.currentUser).
if username.len == 0: return true
let user = ctx.users.getOrDefault(username)
if user.isSuperuser: return true
# Check table-level policies for user or PUBLIC
# For now: if no policies exist, allow everything (backward compatible)
if tableName notin ctx.policies: return true
let policies = ctx.policies[tableName]
# If RLS is enabled (policies exist), check if user matches any policy
for pol in policies:
if pol.command == "ALL" or pol.command == command:
return true
return false
proc hasPrivilege*(ctx: ExecutionContext, tableName, command: string): bool =
if ctx.currentUser.len == 0: return true
hasPrivilegeFor(ctx, ctx.currentUser, tableName, command)
proc passesPolicy*(ctx: ExecutionContext, tableName, command: string, row: Row): bool =
if ctx.currentUser.len == 0: return true
let user = ctx.users.getOrDefault(ctx.currentUser)
+58
View File
@@ -177,6 +177,64 @@ proc computeWindowValues*(rows: seq[Row], expr: IRExpr, ctx: ExecutionContext =
for pos, rowIdx in sortedIdxs:
let (_, fEnd) = resolveFrameBounds(pos, sortedIdxs.len, frameStart, frameEnd)
result[rowIdx] = valueToString(evalExpr(expr.wfArgs[0], rows[sortedIdxs[fEnd]], ctx))
of "sum", "avg", "count", "min", "max":
let countAll = wfName == "count" and
(expr.wfArgs.len == 0 or expr.wfArgs[0].kind == irekStar)
for pos, rowIdx in sortedIdxs:
let (fStart, fEnd) = resolveFrameBounds(pos, sortedIdxs.len, frameStart, frameEnd)
if wfName == "count" and countAll:
result[rowIdx] = $(fEnd - fStart + 1)
continue
if wfName == "count":
var cnt = 0
if expr.wfArgs.len > 0:
for i in fStart .. fEnd:
let s = valueToString(evalExpr(expr.wfArgs[0], rows[sortedIdxs[i]], ctx))
if not isNull(s) and s.len > 0:
inc cnt
result[rowIdx] = $cnt
continue
if expr.wfArgs.len == 0:
result[rowIdx] = "\\N"
continue
var sum = 0.0
var cnt = 0
var minF = 0.0
var maxF = 0.0
var minS = ""
var maxS = ""
var allNumeric = true
for i in fStart .. fEnd:
let s = valueToString(evalExpr(expr.wfArgs[0], rows[sortedIdxs[i]], ctx))
if isNull(s) or s.len == 0: continue
inc cnt
if cnt == 1 or s < minS: minS = s
if cnt == 1 or s > maxS: maxS = s
try:
let f = parseFloat(s)
sum += f
if cnt == 1:
minF = f
maxF = f
else:
if f < minF: minF = f
if f > maxF: maxF = f
except CatchableError:
allNumeric = false
if cnt == 0:
result[rowIdx] = "\\N"
else:
case wfName
of "sum":
result[rowIdx] = if allNumeric: $sum else: "\\N"
of "avg":
result[rowIdx] = if allNumeric: $(sum / float(cnt)) else: "\\N"
of "min":
result[rowIdx] = if allNumeric: $minF else: minS
of "max":
result[rowIdx] = if allNumeric: $maxF else: maxS
else:
result[rowIdx] = "\\N"
else:
# Unknown window function — fill with null
for rowIdx in sortedIdxs:
+5
View File
@@ -194,6 +194,11 @@ proc parsePrimary(p: var Parser): Node =
discard p.expect(tkWhere)
node.funcFilter = p.parseExpr()
discard p.expect(tkRParen)
# Window aggregate: SUM/AVG/COUNT/MIN/MAX(...) OVER (...)
if p.peek().kind == tkOver:
let overClause = p.parseOverClause()
return Node(kind: nkWindowExpr, winFunc: funcName.toLower(), winArgs: args,
winOver: overClause, line: tok.line, col: tok.col)
return node
of tkCase:
discard p.advance()
+73 -9
View File
@@ -175,6 +175,61 @@ proc scan*[K, V](btree: BTreeIndex[K, V], startKey, endKey: K): seq[(K, seq[V])]
finally:
release(btree.lock)
proc subtreeMinMax[K, V](node: BTreeNode[K, V]): (bool, K, K) =
## Inclusive min/max of keys stored in this subtree's leaves.
if node == nil:
return (false, default(K), default(K))
if node.isLeaf:
if node.keys.len == 0:
return (false, default(K), default(K))
return (true, node.keys[0], node.keys[^1])
var have = false
var mn, mx: K
for c in node.children:
let (ok, a, b) = subtreeMinMax(c)
if ok:
if not have:
mn = a
mx = b
have = true
else:
if a < mn: mn = a
if b > mx: mx = b
return (have, mn, mx)
proc collectSeparatorErrors[K, V](node: BTreeNode[K, V], errors: var seq[string]) =
## Search uses `key > separator → right child`, so every key in the left
## subtree must be <= sep (otherwise it is routed right and missed — no
## prev-leaf pointer). Boundary duplicates are allowed: the same key may be
## max(left) and min(right); get/remove walk `next` to collect them.
if node == nil or node.isLeaf:
return
for i in 0..<node.keys.len:
if i + 1 >= node.children.len:
errors.add("internal node has fewer children than keys+1")
break
let (lok, _, lmax) = subtreeMinMax(node.children[i])
let (rok, rmin, _) = subtreeMinMax(node.children[i + 1])
let sep = node.keys[i]
if lok and lmax > sep:
errors.add("separator[" & $i & "]=" & $sep &
" < max(left)=" & $lmax & " (search would miss left keys)")
if lok and rok and rmin < lmax:
errors.add("separator[" & $i & "]=" & $sep &
" leaf order inverted: max(left)=" & $lmax & " > min(right)=" & $rmin)
collectSeparatorErrors(node.children[i], errors)
if node.children.len > 0:
collectSeparatorErrors(node.children[^1], errors)
proc checkInvariants*[K, V](btree: BTreeIndex[K, V]): seq[string] =
## Returns a list of separator/search-routing violations (empty = healthy).
acquire(btree.lock)
try:
result = @[]
collectSeparatorErrors(btree.root, result)
finally:
release(btree.lock)
proc len*[K, V](btree: BTreeIndex[K, V]): int =
acquire(btree.lock)
try:
@@ -209,7 +264,9 @@ proc borrowFromLeft[K, V](node: BTreeNode[K, V], parent: BTreeNode[K, V], parent
node.values.insert(borrowVal, 0)
sibling.keys.setLen(sibling.keys.len - 1)
sibling.values.setLen(sibling.values.len - 1)
parent.keys[parentIdx - 1] = node.keys[0]
# Search is `key > sep → right` (left-max). After lending, sep is the
# left sibling's new max — not the borrowed key now sitting in `node`.
parent.keys[parentIdx - 1] = sibling.keys[^1]
else:
# Borrow from internal sibling
let borrowKey = sibling.keys[^1]
@@ -231,7 +288,8 @@ proc borrowFromRight[K, V](node: BTreeNode[K, V], parent: BTreeNode[K, V], paren
node.values.add(borrowVal)
sibling.keys.delete(0)
sibling.values.delete(0)
parent.keys[parentIdx] = sibling.keys[0]
# Borrowed key is now left's max; search must keep it on the left.
parent.keys[parentIdx] = node.keys[^1]
else:
let borrowKey = sibling.keys[0]
let borrowChild = sibling.children[0]
@@ -325,10 +383,14 @@ proc rebalanceAfterDelete[K, V](node: BTreeNode[K, V], root: var BTreeNode[K, V]
mergeWithLeft(node, parent, parentIdx)
elif hasRight:
mergeWithRight(node, parent, parentIdx)
else:
return
# Recursively rebalance parent if it fell below minimum
# Merge dropped a separator from parent — rebalance up the tree.
if parent == root and parent.keys.len == 0 and parent.children.len == 1:
root = parent.children[0]
elif parent.keys.len < minKeysForLeaf(parent, order):
rebalanceAfterDelete(parent, root, order)
proc remove*[K, V](btree: var BTreeIndex[K, V], key: K, value: V) =
acquire(btree.lock)
@@ -368,15 +430,17 @@ proc remove*[K, V](btree: var BTreeIndex[K, V], key: K, value: V) =
else:
# Internal node: recurse into child
let child = node.children[i]
let oldFirstKey = if child.keys.len > 0: child.keys[0] else: default(K)
let found = removeRec(child, root, order)
if found:
# Update separator if child's first key changed.
# Separator node.keys[i-1] represents child's first key (for i > 0).
if i > 0 and child.keys.len > 0 and child.keys[0] != oldFirstKey:
node.keys[i - 1] = child.keys[0]
# Rebalance the child if needed
# Rebalance first — merge/borrow rewrite parent separators.
rebalanceAfterDelete(child, root, order)
# Leaf children only: refresh left-max separator. Skip if merge
# already unlinked this child. Internal keys are promoted
# separators, not copies of child.keys[^1]; copying those (the
# naive H10 rewrite) makes sep < max(left) and misses keys.
if i < node.children.len and node.children[i] == child and
child.isLeaf and child.keys.len > 0 and i < node.keys.len:
node.keys[i] = child.keys[^1]
return found
if removeRec(btree.root, btree.root, btree.order):
+3 -8
View File
@@ -158,20 +158,15 @@ proc compact*(cs: CompactionStrategy, level: int): CompactionResult =
createdAt: tables[^1].createdAt,
)
# Verify output SSTable before deleting sources
# Verify output SSTable before mutating the catalog. Input files stay on
# disk until the caller loads the output and writes MANIFEST (crash-safe
# order: output durable + catalog updated, then unlink inputs).
let (ok, msg) = verifySSTable(outputPath)
if not ok:
echo "[ERROR] Compaction output verification failed: ", msg
try: removeFile(outputPath) except CatchableError: discard
return CompactionResult()
# Remove old SSTable files
for t in tables:
try:
removeFile(t.path)
except CatchableError as e:
echo "[WARN] Failed to remove old SSTable: ", t.path, ": ", e.msg
# Update level arrays
var newTables: seq[SSTableMeta] = @[]
for t in cs.levels[level]:
+32 -15
View File
@@ -41,27 +41,18 @@ proc newCompactionManager*(db: LSMTree): CompactionManager =
result.strategy.rebuildFromLSM(db)
proc applyCompactionResult(db: LSMTree, result: compaction.CompactionResult) =
## Apply compaction output under the caller's lock: update sstables + MANIFEST.
## On Linux, compact may already have unlinked inputs; we still close our mmaps.
## Crash-safe apply: load output while inputs still exist, swap catalog,
## write MANIFEST, then unlink inputs. A crash before MANIFEST leaves the
## old set intact (orphan output is ignored); a crash after MANIFEST leaves
## at worst unlinked-but-closed input files.
if result.outputTables.len == 0:
return
var newSSTables: seq[SSTable] = @[]
var removedPaths = initTable[string, bool]()
for t in result.inputTables:
removedPaths[t.path] = true
for sst in db.sstables.mitems:
if sst.path notin removedPaths:
newSSTables.add(sst)
else:
# Drop mmap after compact unlinked the path (fd remains valid until close)
sst.close()
var loaded: seq[SSTable] = @[]
for meta in result.outputTables:
try:
var sst = loadSSTable(meta.path)
let name = splitFile(meta.path).name
# Prefer numeric id from filename; otherwise allocate
let parsed = try: parseInt(name) except CatchableError: -1
if parsed >= 0:
sst.id = parsed
@@ -69,11 +60,30 @@ proc applyCompactionResult(db: LSMTree, result: compaction.CompactionResult) =
sst.id = db.nextSSTableId
inc db.nextSSTableId
sst.level = meta.level
newSSTables.add(sst)
loaded.add(sst)
db.nextSSTableId = max(db.nextSSTableId, sst.id + 1)
except CatchableError as e:
warn("Compaction output SSTable failed to load: " & meta.path & "" & e.msg)
for s in loaded.mitems:
s.close()
for m in result.outputTables:
try: removeFile(m.path) except CatchableError: discard
return
var removedPaths = initTable[string, bool]()
for t in result.inputTables:
removedPaths[t.path] = true
var newSSTables: seq[SSTable] = @[]
var toDrop: seq[SSTable] = @[]
for sst in db.sstables.mitems:
if sst.path notin removedPaths:
newSSTables.add(sst)
else:
toDrop.add(sst)
for sst in loaded:
newSSTables.add(sst)
newSSTables.sort(proc(a, b: SSTable): int = cmp(a.id, b.id))
db.sstables = newSSTables
db.needsCompaction = db.countL0() >= L0CompactionTrigger
@@ -84,6 +94,13 @@ proc applyCompactionResult(db: LSMTree, result: compaction.CompactionResult) =
except CatchableError as e:
warn("Failed to write MANIFEST after compaction: " & e.msg)
for sst in toDrop.mitems:
try:
removeFile(sst.path)
except CatchableError as e:
warn("Failed to remove compacted SSTable: " & sst.path & "" & e.msg)
sst.close()
proc compact*(cm: CompactionManager) =
# Gate first (cross-thread), then per-DB write lock
withStorageGate:
+225
View File
@@ -8,7 +8,15 @@ import ../src/barabadb/query/exec/dml
import ../src/barabadb/core/types
import ../src/barabadb/core/config
import ../src/barabadb/core/replication
import ../src/barabadb/core/disttxn
import ../src/barabadb/core/websocket
import ../src/barabadb/protocol/auth
import ../src/barabadb/protocol/scram
import ../src/barabadb/storage/lsm
import ../src/barabadb/storage/compaction
import ../src/barabadb/storage/btree
import std/random
import std/sets
const testDir = "/tmp/baradb_bugfix_test"
@@ -721,3 +729,220 @@ suite "Query correctness — audit batch 2":
rm.connectReplica("r1")
let lsn = rm.writeLsn(@[1'u8, 2, 3])
check lsn == 0
suite "Audit batch 3 — remaining 2026-08 findings":
test "TLS CA file auto-enables peer verify":
putEnv("BARADB_TLS_CA_FILE", "/tmp/ca.crt")
defer: delEnv("BARADB_TLS_CA_FILE")
var cfg = defaultConfig()
loadConfigFromEnv(cfg)
check cfg.tlsCaFile == "/tmp/ca.crt"
check cfg.tlsVerifyPeer == true
test "explicit BARADB_TLS_VERIFY_PEER=false wins over CA auto-enable":
putEnv("BARADB_TLS_CA_FILE", "/tmp/ca.crt")
putEnv("BARADB_TLS_VERIFY_PEER", "false")
defer:
delEnv("BARADB_TLS_CA_FILE")
delEnv("BARADB_TLS_VERIFY_PEER")
var cfg = defaultConfig()
loadConfigFromEnv(cfg)
check cfg.tlsVerifyPeer == false
test "production TLS without verify is rejected":
putEnv("BARADB_ENV", "production")
defer: delEnv("BARADB_ENV")
var cfg = defaultConfig()
cfg.authEnabled = true
cfg.jwtSecret = "a".repeat(32)
cfg.tlsEnabled = true
var msg = ""
try:
validateProductionConfig(cfg)
except ValueError as e:
msg = e.msg
check "TLS" in msg or "verify" in msg.toLower()
test "OFFSET without LIMIT returns remaining rows":
var ctx = setupCtx()
defer: teardown(ctx)
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (1, 'a')"))
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (2, 'b')"))
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (3, 'c')"))
let r = executeQuery(ctx, parse("SELECT id FROM users ORDER BY id OFFSET 1"))
check r.success
check r.rows.len == 2
check valueToString(r.rows[0]["id"]) == "2"
check valueToString(r.rows[1]["id"]) == "3"
test "LIMIT 0 returns no rows":
var ctx = setupCtx()
defer: teardown(ctx)
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (1, 'a')"))
let r = executeQuery(ctx, parse("SELECT id FROM users LIMIT 0"))
check r.success
check r.rows.len == 0
test "negative LIMIT is clamped to empty, not IndexDefect":
var ctx = setupCtx()
defer: teardown(ctx)
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (1, 'a')"))
let r = executeQuery(ctx, parse("SELECT id FROM users LIMIT -5"))
check r.success
check r.rows.len == 0
test "WebSocket decode rejects unmasked client frames":
let raw = encodeFrame(0x1, "SUBSCRIBE t", masked = false)
let (_, n) = decodeFrame(raw)
check n < 0
test "WebSocket decode accepts masked client frames":
let raw = encodeFrame(0x1, "SUBSCRIBE t", masked = true,
maskKey = [1'u8, 2, 3, 4])
let (frame, n) = decodeFrame(raw)
check n > 0
check frame.masked
check frame.payload == "SUBSCRIBE t"
test "WebSocket decode rejects oversized frame length":
var raw = newString(10)
raw[0] = char(0x81) # FIN + text
raw[1] = char(0xFF) # MASK + 127
# 8-byte length = 2 GiB
raw[2] = char(0)
raw[3] = char(0)
raw[4] = char(0)
raw[5] = char(0)
raw[6] = char(0x80)
raw[7] = char(0)
raw[8] = char(0)
raw[9] = char(0)
let (_, n) = decodeFrame(raw)
check n < 0
test "disttxn prepare against refused port fails closed":
var tm = newDistTxnManager()
let txn = tm.beginTransaction("coord")
txn.addParticipant("n1", "127.0.0.1", 1)
check txn.prepare() == false
check txn.isAborted
test "compact leaves input files on disk for catalog apply":
let testDir = "/tmp/baradb_bugfix_compact_order"
removeDir(testDir)
var db = newLSMTree(testDir, 128)
defer:
db.close()
removeDir(testDir)
for round in 0 ..< L0CompactionTrigger:
db.put("r" & $round, cast[seq[byte]]("v" & $round))
db.flush()
var cs = newCompactionStrategy(testDir)
cs.rebuildFromLSM(db)
let cr = cs.compact(0)
check cr.outputTables.len == 1
check fileExists(cr.outputTables[0].path)
for t in cr.inputTables:
check fileExists(t.path)
test "SCRAM unknown user fails without leaking existence":
var am = newAuthManager()
am.registerScramUser("alice", "wonderland")
var msg = ""
try:
discard am.startScram("n,,n=eve,r=abcnonceabcnonceabcn")
except ValueError as e:
msg = e.msg
check msg == "Authentication failed"
test "SCRAM rejects mismatched channel binding":
var am = newAuthManager()
am.registerScramUser("alice", "wonderland")
let clientNonce = generateNonce()
let clientFirst = "n,,n=alice,r=" & clientNonce
let serverFirst = am.startScram(clientFirst)
var combinedNonce = ""
for part in serverFirst.split(","):
if part.startsWith("r="): combinedNonce = part[2..^1]
let (ok, err) = am.finishScram("c=AAAA,r=" & combinedNonce & ",p=AA")
check ok == false
check err == "e=channel-bindings-dont-match"
test "SCRAM expected cbind for gs2 n is biws":
check expectedChannelBinding("n") == "biws"
suite "Audit batch 4 — B-tree separator (H10) and NULL equality (L4)":
test "B-tree separators stay valid after interleaved insert/remove":
## Search routes with `key > sep → right`. After delete, removeRec used to
## copy the right child's first key into the separator (right-min), which
## breaks max(left) <= sep < min(right).
var rng = initRand(20260828)
var btree = newBTreeIndex[int, string](order = 5)
var tracker = initTable[int, seq[string]]()
for i in 0..<400:
let k = rng.rand(0..80)
if rng.rand(0..2) < 2:
let v = "v" & $i
btree.insert(k, v)
if k notin tracker: tracker[k] = @[]
tracker[k].add(v)
else:
if k in tracker and tracker[k].len > 0:
let v = tracker[k][0]
btree.remove(k, v)
tracker[k].del(0)
if tracker[k].len == 0: tracker.del(k)
if i mod 50 == 49:
check btree.checkInvariants().len == 0
for k, vals in tracker:
check btree.get(k).toHashSet == vals.toHashSet
test "B-tree sequential fill + prefix delete keeps separators":
var btree = newBTreeIndex[int, string](order = 4)
for i in 0..<60:
btree.insert(i, "v" & $i)
for i in 0..<30:
btree.remove(i, "v" & $i)
check btree.checkInvariants().len == 0
for i in 30..<60:
check btree.get(i) == @["v" & $i]
for i in 0..<30:
check btree.get(i).len == 0
test "NULL = NULL is unknown, not true":
var ctx = setupCtx()
defer: teardown(ctx)
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (1, NULL)"))
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (2, 'bob')"))
let eqNull = executeQuery(ctx, parse("SELECT id FROM users WHERE name = NULL"))
check eqNull.success
check eqNull.rows.len == 0
let isNull = executeQuery(ctx, parse("SELECT id FROM users WHERE name IS NULL"))
check isNull.success
check isNull.rows.len == 1
check valueToString(isNull.rows[0]["id"]) == "1"
test "NULL != value is unknown so WHERE excludes the row":
var ctx = setupCtx()
defer: teardown(ctx)
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (1, NULL)"))
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (2, 'bob')"))
let r = executeQuery(ctx, parse("SELECT id FROM users WHERE name != 'bob'"))
check r.success
check r.rows.len == 0
test "SELECT NULL = NULL yields NULL, IS NULL is true":
var ctx = setupCtx()
defer: teardown(ctx)
discard executeQuery(ctx, parse("INSERT INTO users (id, name) VALUES (1, 'a')"))
let eq = executeQuery(ctx, parse("SELECT (NULL = NULL) AS x FROM users"))
check eq.success
check eq.rows.len == 1
check valueToString(eq.rows[0]["x"]) == "\\N"
let isn = executeQuery(ctx, parse("SELECT (NULL IS NULL) AS x FROM users"))
check isn.success
check valueToString(isn.rows[0]["x"]) == "true"
+15
View File
@@ -4298,6 +4298,21 @@ suite "Window Functions":
if row["name"] == "Bob":
check row["last_sal"] == "90000"
test "SUM/AVG/COUNT window aggregates over a running frame":
## Default frame is UNBOUNDED PRECEDING .. CURRENT ROW (lower.nim).
let r = qexec.executeQuery(ctx, parse(
"SELECT name, salary, SUM(salary) OVER (ORDER BY salary) AS running, COUNT(*) OVER (ORDER BY salary) AS cnt FROM employees"))
check r.success
check r.rows.len == 5
for row in r.rows:
if row["name"] == "Charlie":
check parseFloat($row["running"]) == 70000.0
check $row["cnt"] == "1"
if row["name"] == "Eve":
# 70000+75000+80000+90000+95000
check parseFloat($row["running"]) == 410000.0
check $row["cnt"] == "5"
suite "GROUP BY Aggregates":
var db: LSMTree
var ctx: qexec.ExecutionContext