dimgigov
09f462f467
release: v1.2.0 Production GA (single-node)
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- known-limitations, deployment runbook, release checklist
- prod compose requires JWT secret; BARADB_ENV=production fail-closed
- scripts/backup-restore-drill.sh (backup → wipe → restore → verify)
- version bump 1.2.0 (nimble, Dockerfile, health, CHANGELOG dated)
2026-07-30 21:52:21 +03:00
dimgigov
1b3c26123a
feat(raft): expose raft metrics on /metrics and /health
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- RaftMetrics counters: elections, term changes, appends, commit waits/ms,
timeouts, lost leadership, forwards, applies, compactions
- Gauges via prometheusText: is_leader, term, log size, commit/applied,
apply lag, snapshot index
- Wire httpServer.raftNode; extend GET /metrics and GET /health
2026-07-30 21:38:44 +03:00
dimgigov
53704e1036
feat(raft): safe log compaction with snapshot metadata
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- lastSnapshotIndex/Term bound the compacted prefix; lastLogIndex/Term
and AppendEntries prevLog checks respect the snapshot base
- compactLog drops entries only through min(matchIndex, lastApplied) on
the leader so lagging peers can still catch up via AppendEntries
- Persist snapshot fields in raft_state.bin; BARADB_RAFT_LOG_MAX_ENTRIES
- Fix commit-index scan to use findLogEntryByIndex (works after compact)
2026-07-30 21:35:41 +03:00
dimgigov
9df8316305
feat(raft): transparent leader write/DDL forwarding
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- BARADB_RAFT_CLIENT_PEERS maps node id → SQL client host:port
- Followers proxy DML/DDL to the known leader over the wire protocol
(falls back to "not leader" when the map is missing)
- E2E: follower CREATE/INSERT succeed via forward; docs updated
2026-07-30 21:31:05 +03:00
dimgigov
095698ba82
feat(raft): replicate schema DDL through the raft log
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- isRaftDdl + leader-only gate for CREATE/DROP/ALTER (not DATABASE)
- appendDdlToRaft ships original SQL; applyCommand re-executes via
applyReplicatedDdl (idempotent on leader double-apply)
- Mixed DDL+DML batches use the DDL path so order is preserved
- Fix secondary-index point lookup to use entry.lsmKey (not filter col)
- E2E: CREATE only on leader, schema + index SELECT on follower
2026-07-30 21:25:58 +03:00
dimgigov
50f827f8cf
fix(raft): graph apply + reject non-default DB writes
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- applyReplicatedPut/Delete keep in-memory graphs in sync with node/edge
table rows (idempotent edges via addEdgeWithIdIfAbsent).
- When Raft is enabled, DML is refused on any database other than
'default' (the only DB the state machine is wired to).
- Docs updated (en/bg); unit test for graph apply.
2026-07-30 21:19:01 +03:00
dimgigov
0d51497f57
fix(raft): multi-stmt write gate, rich apply, delete kv convention
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- Reject follower writes if any statement in the batch is DML/COMMIT
(not only stmts[0]).
- COMMIT always emits empty-valued kvPairs for isDelete entries.
- applyCommand updates LSM plus secondary B-tree/FTS/HNSW indexes
(applyReplicatedPut/Delete) so follower index scans see replicated rows.
- Tests: not-leader append, commit timeout, index apply unit, E2E
index-backed SELECT on follower.
2026-07-30 21:14:46 +03:00
dimgigov
44060701b7
test(raft): E2E replicated writes; wire SQL path to raft node
...
- Fix runTcpServer to run the already-wired Server (raftNode was assigned
on a different instance that never accepted clients).
- Cap raft peer connect at 200ms and fan out heartbeats in parallel so a
dead peer cannot stall AppendEntries to the live majority.
- Add raft_writes_e2e_test: 3-node write replication, follower rejection,
and post-failover writes; wire into nimble test + gitignore.
2026-07-30 21:06:49 +03:00
dimgigov
333941ab65
feat(raft): leader appends writes to raft log and waits for commit
2026-07-30 19:36:16 +03:00
dimgigov
38c1c01841
feat(raft): classify writes, reject them on follower nodes
2026-07-30 19:07:58 +03:00
dimgigov
b6eccf284e
test(raft): end-to-end 3-node cluster election and failover
2026-07-30 18:35:18 +03:00
dimgigov
f01586354a
fix(raft): partial-read-safe frame reassembly
2026-07-30 18:01:53 +03:00
dimgigov
3f8537eaad
feat(raft): run election timer in production, reset on AppendEntries
2026-07-30 17:47:05 +03:00
dimgigov
853ec7dd3b
feat(raft): parse id@host:port peers, enable raft state persistence
2026-07-30 17:34:39 +03:00
dimgigov
ed5a71913c
fix: harden exception handling, break ARC cycles, sync license/client
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Replace bare except: with CatchableError across storage, query, Raft,
backup, and protocol code so Defects are not swallowed. Break uncollectable
ARC cycles in server shard/gossip callbacks via Server-owned refs and
cursor locals. Align package license with LICENSE (BSD-3-Clause), sync
README version, and point test_all at the canonical clients/nim baradb
client (parseConnectionString + aliases).
2026-07-23 01:07:51 +03:00
dimgigov
8db5cfe7e1
feat: harden storage, schema persistence, fair benches, fix wire crash
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Core storage: hash MemTable, WAL group commit, L0 compaction rebuild,
reader-writer lock, and a global StorageGate so HTTP workers and TCP
share the LSM safely under multi-thread access.
Schema: durable CREATE/ALTER/DROP under _schema:tables:* with full LSM
restore on open. Executor types/values/schema split into query/exec/.
Wire protocol: switch default MM to ARC — ORC cycle collector segfaulted
after ~20 async INSERTs. Fair multi-tier benchmarks (SQLite/HTTP/wire/PG)
and honesty docs for mixed-tier comparisons.
2026-07-18 16:55:50 +03:00
dimgigov
ef264d7d69
feat: add unified search engine — HNSW heap-opt, segment index, boolean/phrase/ngram/facet
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
New src/barabadb/search/ module with 9 components:
- priority_queue.nim: BoundedHeap for O(log n) search
- hnsw_opt.nim: heap-based searchLayer (2.4x faster, 92-99% recall@10)
- inverted.nim: segment-based index with soft-delete and compaction
- phrase.nim: positional phrase + proximity search
- boolean.nim: recursive descent parser (AND/OR/NOT/ranges/wildcards)
- ngram.nim: trigram index for O(1) fuzzy/prefix/wildcard
- stemmer.nim: Porter2 stemmers (EN/BG/DE/FR/RU)
- facet.nim: faceted search with filter pushdown
- engine.nim: UnifiedSearchEngine combining all search types
Performance (dim=128, efConstruction=200):
N=1K: 0.30ms search, 99.6% recall@10
N=10K: 1.09ms search, 92.6% recall@10
N=50K: 2.26ms search, 75.5% recall@10
Includes search benchmarks (benchmarks/search_bench.nim), updated docs
(en/bg fts.md, en/bg search.md), and crossmodal engine integration.
2026-05-30 13:42:08 +03:00
dimgigov
42043f3946
v1.1.7: deep security & reliability audit — 33 bugs fixed
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Critical (5):
- Reject empty JWT secret when authEnabled (server.nim)
- Fix 2PC marking uncontacted participants as prepared/committed (disttxn.nim)
- Fix Raft commit index calculation for even-sized clusters (raft.nim)
- Fix REP/DISTTXN protocol auth bypass (server.nim)
- Fix HTTP backup/restore path traversal (httpserver.nim)
High (11):
- Fix WAL write race with flush (lsm.nim)
- Fix MVCC savepoint/rollback deep-copy writeSet (mvcc.nim)
- Fix table mutation during deadlock iteration (mvcc.nim)
- Fix LIMIT 0 returning all rows (executor.nim)
- Fix COUNT(col) counting NULL values — 3 locations (executor.nim)
- Fix EXISTS subquery lowering missing subqueryPlan (executor.nim)
- Fix Raft appendEntries/applyCommitted array vs logical index (raft.nim)
- Fix timing attacks on constantTimeCompare and SCRAM (auth.nim, scram.nim)
- Fix B-tree leaf merge phantom separator key (btree.nim)
- Fix SSL verifyPeer not applied to newContext (ssl.nim)
- Fix sharding connectWithTimeout missing SO_ERROR check (sharding.nim)
- Fix sync replication returning success on partial ack (replication.nim)
- Fix WebSocket JWT expiration not validated (websocket.nim)
Medium (13):
- Fix writeSSTable partial file → tmp + atomic rename (lsm.nim)
- Fix multi-CTE table loss (executor.nim)
- Fix nl_to_sql DML restricted to superuser (executor.nim)
- Fix unbounded plan cache — max 10000 (adaptive.nim)
- Fix migration lock crash persistence — timestamp + stale detection (executor.nim)
- Fix admin panel auth (httpserver.nim)
- Fix MVCC unbounded txn tracking — prune in compactVersions (mvcc.nim)
- Fix connection pool maxLifetime check (pool.nim)
- Fix JWT JSON parser backslash escapes (auth.nim)
- Fix substr(s, start) returning single char (udf.nim)
- Fix loadSSTable minimum file-size check (lsm.nim)
- Fix compaction mmap leak (compaction.nim)
- Fix JSON injection in hybrid_search_filtered (executor.nim)
Low (4):
- Raft loadState logs error instead of silent discard
- Replication healthCheck double-close fixed
- Lexer readIdent double column counting fixed
- WebSocket frame 32-bit overflow guard
All 448 tests passing, 0 failures. Bump version to 1.1.7.
2026-05-29 14:17:41 +03:00
dimgigov
a5abb6031b
fix(backup): fix 7 bugs in backup/restore — strip-components, multi-db path, verify+log in HTTP handler, parseBackupFilename for .tar, getArchiveSize gzip -l, dead code removal
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-25 20:44:14 +03:00
dimgigov
a37a62c69e
clients/nim-allographer/src/allographer/query_builder/models/baradb/baradb_exec.nim src/barabadb/core/mvcc.nim src/barabadb/query/codegen.nim src/barabadb/query/lexer.nim src/barabadb/query/parser.nim src/barabadb/query/udf.nim tests/test_all.nim
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-25 18:56:02 +03:00
dimgigov
eb62b381a1
feat: multi-database backup, HTTP API endpoints, admin panel UI
...
- backup CLI: add --all-databases, --database, --data-root flags
- backup format: tar.gz with databases/<name>/ + backup.json metadata
- HTTP API: new endpoints /databases, /backup, /backups, /restore
- HTTP API: database switching via X-Database header
- Admin panel: database selector dropdown, Databases tab, Backups tab
- Admin panel: backup/restore UI with fetch() to new endpoints
- Update deps: hunos 1.3.0, jwt-nim-baraba 2.1.0
- Docker: backup cron uses --all-databases --data-root=/data/databases
- Docs: update api-http.md, protocol.md with correct endpoints
- OpenAPI spec updated to v1.1.6
Known limitation: DELETE /databases disabled due to ORC SIGSEGV
in LSMTree.close(). Use SQL DROP DATABASE instead.
2026-05-25 18:46:07 +03:00
dimgigov
2d310a33a1
fix: resolve all 55 identified bugs across the codebase
...
Comprehensive bug fix pass across all subsystems:
- Critical: lexer infinite loop, WAL lock discipline, checkpoint safety,
compaction verification, HMAC key truncation, healthCheck leak
- High: MVCC lock safety, B-Tree delete rebalancing, Raft stale term
handling, replication socket leaks and ack cleanup, sharding migration
with old assignments, 2PC recovery, JWT/SCRAM auth fixes, wire protocol
bounds checks
- Medium: config error handling, MERGE parser completeness, IR/codegen
correctness, UDF bounds checks, LIMIT cost accuracy, mmap safety
- Low: timeout handling, float parsing edge cases, uint32 truncation,
cache entry cleanup
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-21 10:59:51 +03:00
dimgigov
1ff6fabaff
fix(backup): resolve relative SSTable paths in incremental backup
...
MANIFEST stores relative paths like 'sstables/1.sst'.
The old code checked fileExists against CWD instead of dataDir,
silently omitting SSTables when run from another directory.
Now resolves paths relative to dataDir.
2026-05-21 09:44:28 +03:00
dimgigov
372e5cf627
fix(query,types): refactor Row to Value-typed map, fix IN list, nkPath, multi-table joins
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- Refactored Row from Table[string, string] to Table[string, Value]
- Added Value operators: ==, !=, $, in for string interop
- Fixed valueToString for vkNull to return '\\N' instead of ''
- Fixed evalExpr irekField to return vkNull when field not found
- Added IN (val1, val2, ...) parser support
- Fixed nkPath column names in multi-table joins
- Fixed LATERAL JOIN null padding when no matching rows
- Added CREATE/DROP/USE/SHOW DATABASE parser support
- Adapted all tests for new Value type
2026-05-20 23:22:14 +03:00
dimgigov
57d2908066
fix(executor,mvcc,tests): dedup aliases, fix txn delete, fix MIN/MAX null, add regression tests
...
- Add isDelete flag to VersionedRecord to fix COMMIT treating empty values
as DELETE. Previously version.xmax != TxnId(0) was never true for active
transactions, so deletes inside transactions never removed rows.
- Fix MIN/MAX aggregates skipping NULL: was checking for "\\N" instead
of correct "\N" sentinel, so NULL values were never skipped.
- Add duplicate alias deduplication in lowerSelect() and getSelectColumns().
Identical aliases now get _1, _2 suffixes (e.g. count(*), count(*)_1).
- Add nil guard for expr.subqueryPlan in irekSubquery evalExpr handler.
- Initialize txnManager in newExecutionContext to prevent segfault on BEGIN.
- Add regression tests: DELETE in transaction, PK-only row survival,
MIN/MAX NULL skipping.
2026-05-20 21:36:15 +03:00
dimgigov
cd46edcb67
Bump version to 1.1.6; fix storage bugs, tests, CI pipeline and Docker config
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Storage fixes:
- Fix bloom filter hash2 OverflowDefect (uint64 instead of int Hash)
- Fix MemTable.put size calculation on overwrite (was leaking size)
- Fix fuzz_test duplicate-key issues in LSM delete and SSTable tests
Test fixes:
- Fix stress_test.nim: replace deprecated threadpool with std/threads
- Fix fuzz_test.nim: add missing imports (tables, algorithm, sets)
- Fix fuzz_test.nim: var sst for close() compatibility
- Remove unused imports from test_all.nim and bench_all.nim
Docker / CI fixes:
- Fix Dockerfile.source: invalid nimlang/nim:2.2.10-alpine tag → 2.2.10 + ubuntu:24.04 runtime
- Fix Dockerfile.source healthcheck (--spider → -qO- for 200 OK)
- Fix Dockerfile run comment ports (9470/9471 → 9912/9913)
- Fix scripts/docker-run.sh healthcheck port (9470 → 9912)
- Add ARG BUILD_DATE/VCS_REF to Dockerfile for docker-build.sh
- Update all version strings 1.1.4 → 1.1.6 across nimble/docker/source/docs
2026-05-19 12:13:33 +03:00
dimgigov
8afa998516
feat: rate limiting, shared DB instance, TCP_NODELAY fix
...
- Add rate limiter to TCP and HTTP servers
- Share single LSMTree between TCP and HTTP servers
- Replace unsafe cast[string]/cast[seq[byte]] with safe helpers
- Stabilize gossip UDP socket with exponential backoff
- Fix TCP_NODELAY via low-level setSockOptInt(IPPROTO_TCP)
(asyncnet.setSockOpt(OptNoDelay) uses SOL_SOCKET and fails)
- Apply TCP_NODELAY to server and websocket sockets
2026-05-18 19:33:40 +03:00
dimgigov
7e6a45e6b7
Стабилизация на storage слоя — 6 фази
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Фаза 1: SSTable Integrity
- SSTable v3 формат с CRC32 footer (data/index/bloom CRC)
- Нов модул storage/crc32.nim (zero-dep, IEEE 802.3)
- verifySSTable() — проверка на magic, version, CRC
- loadSSTable strict mode — отхвърля корумпирани файлове
- newLSMTree логва [WARN] при corrupt SSTables
Фаза 2: baradb repair
- Нов модул tools/repair.nim
- Сканира SSTables, проверява CRC, премества битите в corrupt/
- WAL replay (CrashRecovery) за възстановяване на данни
- CLI: ./baradadb repair --data-dir=... [--dry-run]
Фаза 3: MANIFEST File
- Atomic write MANIFEST (tmp + rename)
- newLSMTree зарежда от MANIFEST, fallback към walkDir
- checkStorageConsistency() — orphan/missing detection
- flushUnsafe и compaction записват MANIFEST
Фаза 4: WAL Rotation & Incremental Backup
- WAL segment rotation при 64MB (wal_archive/wal.000NNN.log)
- maybeRotate() на всеки 1000 записа + при flush
- backup incremental — архивира MANIFEST + SSTables + WAL
- CRC verification на SSTables преди архивиране
Фаза 5: Online Consistent Backup
- checkpoint() — freeze memtable + flush + WAL rotate
- ./baradadb checkpoint — offline consistent snapshot
- backup --online — checkpoint + incremental backup
Фаза 6: SSTable Version Migration
- SSTable.fileVersion поле
- listLegacySSTables() — намира v1/v2 файлове
- migrateSSTable() — пренаписва към v3 (tmp + rename)
- ./baradadb migrate [--dry-run] — offline migration
Документация:
- Обновени docs/en/backup.md, docs/bg/backup.md
- Обновени docs/en/storage.md, docs/bg/storage.md
- Добавени тестове в tests/test_all.nim
2026-05-18 15:04:49 +03:00
dimgigov
8596cea548
Stabilization: replication semi-sync, MVCC aborted deleters, SSTable level, SCRAM fix
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
P0-P1 fixes:
- replication.nim: rmSync/rmSemiSync now wait for replica ACKs
- mvcc.nim: aborted deleters no longer hide records (return true for visibility)
- storage/lsm.nim: SSTable level persisted in header (version 2, backward compat)
- protocol/auth.nim: removed insecure SCRAM raw hash fallback
Build: 0 warnings, 0 errors. All tests pass.
2026-05-18 12:04:45 +03:00
dimgigov
47c4393a7e
Complete plan execution: gossip/raft bounds, deadlock fix, JSON escaping, memtable limits
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Remaining plan items:
- 1.4 hybrid_search: JSON built with std/json instead of string concat
- 2.3 lsm.nim: reject oversized memtable entries
- 3.3 gossip.nim: bounds checking for idLen/nodeCount/hostLen
- 3.4 raft.nim: max length caps for cmdLen/dataLen/logLen
- 4.3 deadlock.nim: per-path seq stack instead of global parent table
2026-05-18 11:42:46 +03:00
dimgigov
967c0855a5
Bug fixes: composite PK, nl_to_sql sandbox, FK check, SQL injection, storage correctness
...
Critical fixes:
- Composite PK: execInsert + validateConstraints use all PK columns
- nl_to_sql: non-SELECT SQL no longer executed directly during validation
- FK check: removed O(N) scanMemTable fallback, uses db.get() with SSTables
- exprToSql: nkIdent wrapped in quotes to prevent SQL injection
- restoreSchema: try/except around tokenize/parse for crash resilience
- recovery.nim: lastTxnId tracking + putUnsafe/deleteUnsafe without WAL
- SCRAM: verifyClientProof length check + DefaultIterationCount restored
Storage fixes:
- lsm.nim: SSTable sort order fixed (ascending), close() flushes all memtables
- compaction.nim: tombstones preserved during compaction
- wal.nim: header written for empty existing files, readEntries checks magic
- btree.nim: B+ tree leaf split keeps boundary key
- bloom.nim: deserialize raises on short data
- mmap.nim: bounds checks for adviseWillNeed/DontNeed + posix.close()
Protocol fixes:
- zerocopy.nim: readString bounds check
- wire.nim: deserializeValue 32-bit underflow check
- auth.nim: JWT JSON escaping for claims
- server.nim: readUint32BE bounds check + specific exception handling
- raft.nim: readData checks + specific exception handling
Tests:
- Added Composite Primary Key test suite (4 tests)
Build: 0 warnings, 0 errors
2026-05-18 11:33:11 +03:00
dimgigov
a28c845476
Bump version to 1.1.4 across all components
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-17 20:57:19 +03:00
dimgigov
19fa760604
Week 4: Production Hardening — Thread Safety, Fuzz Tests, Property-Based Tests, NimForum Smoke Test
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Thread Safety:
- Add lock to BTreeIndex (insert/get/scan/remove/contains)
- Add lock to InvertedIndex (addDocument/removeDocument/search/bm25/tfidf/fuzzy/regex)
- Add ctxLock to ExecutionContext for DDL/schema metadata, autoIncCounters, sequences
- Deep-copy sessionVars in cloneForConnection() to isolate per-connection state
- Fix activeConnections check-then-act race with Lock
Wire Protocol Hardening:
- Fix RangeDefect in deserializeValue for invalid FieldKind bytes
- Fix IndexDefect in fkFloat32/fkFloat64 by reusing readUint32/readUint64
- Fix int32/int64 cast from uint32/uint64 to use cast[] instead of constructor
- Fix parseHeader to validate MsgKind instead of holey enum cast
Property-Based Tests (tests/prop_test.nim):
- 8 invariants for evalExprValue: literal types, commutativity, identity,
double negation, NULL propagation
Wire Protocol Fuzz (tests/fuzz_test.nim):
- 7 fuzz tests: random bytes deserialization, truncated buffers,
MsgKind casts, parseHeader logic, message roundtrips
NimForum Adapter Smoke Test (tests/nimforum_smoke_test.nim):
- Full TCP server lifecycle with adapter CRUD and parameterized queries
Test Results: 423 tests pass, 0 failures
2026-05-17 11:57:55 +03:00
dimgigov
073ec41652
fix: issue #9 reserved keyword 'key' + issue #10 empty string treated as NULL
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Issue 9: Add backtick-quoted identifier support in lexer.nim
- Users can now write `key` to use reserved keywords as column/table names
Issue 10: Use \N as internal NULL marker instead of empty string
- isNull() now checks for \N instead of value.len == 0
- Empty strings can now be stored in NOT NULL columns
- NULL values properly distinguished from empty strings in storage, wire protocol, and HTTP JSON responses
2026-05-17 01:59:25 +03:00
dimgigov
652ed1b477
feat: add ID generators, fix 6 forum-discovered deficiencies
...
ID Generators:
- AUTO_INCREMENT for INTEGER columns
- SERIAL / BIGSERIAL as syntactic sugar
- gen_random_uuid() / uuid() for UUID v4 generation
- nextval() / currval() for sequence support
- snowflake_id(node_id) for distributed 64-bit IDs
- RETURNING clause for INSERT
Deficiency Fixes:
- Comma join: FROM t1, t2 now works as implicit CROSS JOIN
- DEFAULT in restoreSchema: defaults survive server restart
- Duplicate column names: t.id instead of id in joins
- Empty result sets: always send column metadata to client
- GROUP BY non-agg columns: return first row value instead of empty
- Aggregate column names: count(*) instead of count()
Bug fix: PK uniqueness check now uses correct key format
2026-05-16 02:57:30 +03:00
dimgigov
2e945c1dcb
release: bump version to 1.1.2 (forum-tested, nested SQL fixes)
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-15 18:34:47 +03:00
dimgigov
c5751b4a0c
fix: clean up compiler warnings and unused variables
2026-05-15 18:01:11 +03:00
dimgigov
f7d4961125
feat: Multi-tenant ERP support via session variables + RLS
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- SET var = value / current_setting('var') for session-scoped variables
- current_user / current_role SQL keywords with auth bridge
- server.nim + httpserver.nim populate ExecutionContext.currentUser/currentRole
- RLS policies can reference current_setting('app.tenant_id') for tenant isolation
- Fixed evalExpr to propagate ctx recursively (fixes current_user in sub-expressions)
- Fixed GROUPING SETS execution (lowerSelect checks selGroupingSetsKind)
- Fixed FTS CREATE INDEX docId mismatch (hash of tableName.$key)
- Fixed all test suites to use isolated temp directories
- Added 5 multi-tenant tests (355 total, all green)
- Updated docs: PLAN_SQL_ADVANCED.md, baraql.md, changelog.md
2026-05-14 16:28:41 +03:00
dimgigov
71dcffecce
fix(gossip): use async UDP socket to avoid blocking the event loop
...
The gossip listener used synchronous newSocket + recvFrom,
which blocks the async event loop and prevents other async
operations from running while waiting for UDP packets.
Switch to newAsyncSocket + async recvFrom so the loop can
continue processing other tasks between gossip rounds.
2026-05-14 02:22:07 +03:00
dimgigov
4e7e568525
release: v1.1.0 — bug fixes, Docker, clients, SCRAM auth
...
- Bump version to 1.1.0 across all components
- Remove debug CI artifacts (debug.yml, test_all.nim.bak, test_lock)
- Update CHANGELOG with v1.1.0 release notes
- Bug fixes: irNeg, distributed txn, sharding, Raft majority, MVCC,
LSM-Tree, auth (JWT + SCRAM-SHA-256), wire protocol, SQL injection,
ReDoS, graph, vector, FTS, build warnings
- Client SDKs: JS/TS, Python, Nim, Rust with tests and examples
- Docker: production + source + test compose configs
- TLA+: crossmodal, backup, recovery specs with symmetry reduction
2026-05-13 15:05:59 +03:00
dimgigov
42c675224c
fix: distributed transaction and sharding bugs
...
- disttxn: rollback now correctly updates participant state (was modifying a copy)
- disttxn: bare except replaced with CatchableError
- sharding: migrateData condition was always false, now correctly migrates data
- sharding: unsafe cast[string] replaced with manual byte-to-char conversion
- sharding: bare except replaced with CatchableError
- server: parseUInt replaced with parseBiggestUint for uint64 portability
2026-05-13 14:01:13 +03:00
dimgigov
c0bd3dff86
feat: distributed gaps filled — gossip UDP transport, sharding data migration, inter-module wiring (raft-disttxn, gossip-sharding, replication-disttxn)
2026-05-13 11:11:53 +03:00
dimgigov
ac55237b72
feat: real SCRAM-SHA-256 authentication (Option C)
...
Security:
- New scram.nim module with full SCRAM-SHA-256 per RFC 7677
* PBKDF2-HMAC-SHA-256 key derivation
* HMAC-SHA-256 with multiple overloads for bytes/strings
* Secure nonce/salt generation via /dev/urandom
* Client/server message parsing and proof verification
AuthManager updates:
- registerScramUser(): stores salted credentials (salt + iterations + storedKey + serverKey)
- startScram(): initiates challenge-response handshake
- finishScram(): verifies client proof and returns server signature
- Legacy amSCRAMSHA256 path kept for backward compatibility
HTTP endpoints:
- POST /auth/scram/start — accepts client-first-message, returns server-first
- POST /auth/scram/finish — accepts client-final-message, returns server-final
Tests:
- SCRAM-SHA-256 full handshake test (register → start → compute proof → finish)
- SCRAM-SHA-256 invalid proof rejection test
Build: 0 warnings, all tests pass
2026-05-13 09:50:07 +03:00
dimgigov
422df08ab9
feat: clean build + crossmodal.tla + TLA+ symmetry (v1.0.0-prep)
...
Build warnings cleanup (0 warnings):
- Suppress threadpool deprecation warning (baradadb.nim)
- Remove unused 'os' import (logging.nim)
- Fix ImplicitDefaultValue for newNode params (ast.nim)
- Add explicit cstring cast for posix.open (wal.nim)
- Fix HoleEnumConv for MsgKind parsing (server.nim)
TLA+ Formal Verification:
- Add symmetry reduction (Permutations) to all 9 existing specs
- Add SYMMETRY directive to all .cfg files
- New crossmodal.tla: cross-modal consistency spec
* MetadataVectorConsistency, HybridResultValid
* CommittedAtomicity, AbortedAtomicity, TxnStateValid
- New models/crossmodal.cfg
Tests:
- Add Cross-Modal TLA+ Faithfulness tests (4 tests)
Docs:
- Update PLAN.md and BUG_AUDIT.md with completed tasks
2026-05-13 09:24:04 +03:00
dimgigov
a72e17f063
fix: последните 4 критични/средни бъга — LSM-Tree, MVCC, DistTxn, Raft
...
Поправени проблеми:
- LSM-Tree: WAL write вече е извън db.lock (отделен walLock) → по-добра concurrency
- MVCC: добавен compactVersions() който се изпълнява на всеки 100 commits
Премахва стари overwritten версии, които не са видими за active транзакции
- DistTxn: commit() вече НЕ rollback-ва commit-нали participants
Ако някой participant е commit-нал, транзакцията се маркира като committed
- Raft: добавен disk persistence за currentTerm, votedFor, log
saveState() при всяка промяна; loadState() при стартиране
292 теста, 0 failure-а.
2026-05-12 23:30:58 +03:00
dimgigov
5e585dd347
fix: deadlock detector, graph/query/lexer бъгове
...
Поправени проблеми:
- Deadlock detector: cycle reconstruction вече проверява пълен цикъл
- Graph cypher: executeCypher acquire-ва lock при четене
- Graph engine: loadFromFile acquire-ва lock при зареждане
- Graph engine: lock поле е exported за външен достъп
- Query: parseRowData поддържа escape-ване на , и = в стойности
- Query: execUpdateRow escape-ва стойности при сериализация
- Lexer: readNumber спира при втора точка (1.2.3 вече не е валиден float)
- Lexer: skipBlockComment хвърля ValueError при незатворен коментар
292 теста, 0 failure-а.
2026-05-12 23:21:29 +03:00
dimgigov
80d57a36bd
fix: rate limiter, graph/vector/query/server бъгове
...
Поправени проблеми:
- Rate limiter: globalRate вече се enforce-ва; cleanupStaleClients() за memory leak
- Graph engine: addEdge проверява дали src/dst nodes съществуват
- Vector engine: distance() хвърля ValueError при dimension mismatch
- Query: aggregate + * на empty result вече не crash-ва
- Query: exprToSql funcCall вече работи с 0 аргумента
- Query: INSERT trigger вече не crash-ва при празни VALUES
- Server: recvWithTimeout() за всички network reads
- Server: защита от negative activeConnections
292 теста, 0 failure-а.
2026-05-12 23:15:09 +03:00
dimgigov
131541a0e5
fix: критични бъгове в storage, MVCC, auth, Raft, query executor
...
Поправени проблеми:
- MVCC: aborted транзакции вече не стават видими (добавено abortedTxns множество)
- LSM-Tree: поправена загуба на данни при immutable memtable overwrite
- LSM-Tree: поправена SSTable search order (сортиране по id вместо minKey)
- WAL: sync()/close() вече извикват posix.fsync()
- Auth: simpleHash (djb2) заменен с HMAC-SHA256
- Recovery: summary() вече не мутира базата данни
- Raft: поправена majority calculation за четен брой нодове
- Query: EXISTS subqueries вече изпълняват подзаявката
- Nimble: поправен build (-d:ssl) и bench task
Добавен BUG_AUDIT.md с пълен доклад и план за подобрения.
292 теста, 0 failure-а.
2026-05-12 22:45:50 +03:00
dimgigov
8d39d633cf
chore: update version strings to v1.0.0 in Docker and Nim code
...
- Dockerfile, Dockerfile.source: LABEL version=1.0.0
- Dockerfile.source: update Nim base image 2.2.2 -> 2.2.10
- src/baradadb.nim, src/barabadb/core/httpserver.nim: v0.1.0 -> v1.0.0
- Rebuild baradadb binary with v1.0.0
2026-05-07 22:06:04 +03:00
dimgigov
c8633b9589
fix: non-blocking replication/disttxn timeouts, OTLP epoch time, test paths, compiler warnings
2026-05-07 15:41:33 +03:00