dimgigov
ed5a71913c
fix: harden exception handling, break ARC cycles, sync license/client
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Replace bare except: with CatchableError across storage, query, Raft,
backup, and protocol code so Defects are not swallowed. Break uncollectable
ARC cycles in server shard/gossip callbacks via Server-owned refs and
cursor locals. Align package license with LICENSE (BSD-3-Clause), sync
README version, and point test_all at the canonical clients/nim baradb
client (parseConnectionString + aliases).
2026-07-23 01:07:51 +03:00
dimgigov
8db5cfe7e1
feat: harden storage, schema persistence, fair benches, fix wire crash
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Core storage: hash MemTable, WAL group commit, L0 compaction rebuild,
reader-writer lock, and a global StorageGate so HTTP workers and TCP
share the LSM safely under multi-thread access.
Schema: durable CREATE/ALTER/DROP under _schema:tables:* with full LSM
restore on open. Executor types/values/schema split into query/exec/.
Wire protocol: switch default MM to ARC — ORC cycle collector segfaulted
after ~20 async INSERTs. Fair multi-tier benchmarks (SQLite/HTTP/wire/PG)
and honesty docs for mixed-tier comparisons.
2026-07-18 16:55:50 +03:00
dimgigov
42043f3946
v1.1.7: deep security & reliability audit — 33 bugs fixed
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Critical (5):
- Reject empty JWT secret when authEnabled (server.nim)
- Fix 2PC marking uncontacted participants as prepared/committed (disttxn.nim)
- Fix Raft commit index calculation for even-sized clusters (raft.nim)
- Fix REP/DISTTXN protocol auth bypass (server.nim)
- Fix HTTP backup/restore path traversal (httpserver.nim)
High (11):
- Fix WAL write race with flush (lsm.nim)
- Fix MVCC savepoint/rollback deep-copy writeSet (mvcc.nim)
- Fix table mutation during deadlock iteration (mvcc.nim)
- Fix LIMIT 0 returning all rows (executor.nim)
- Fix COUNT(col) counting NULL values — 3 locations (executor.nim)
- Fix EXISTS subquery lowering missing subqueryPlan (executor.nim)
- Fix Raft appendEntries/applyCommitted array vs logical index (raft.nim)
- Fix timing attacks on constantTimeCompare and SCRAM (auth.nim, scram.nim)
- Fix B-tree leaf merge phantom separator key (btree.nim)
- Fix SSL verifyPeer not applied to newContext (ssl.nim)
- Fix sharding connectWithTimeout missing SO_ERROR check (sharding.nim)
- Fix sync replication returning success on partial ack (replication.nim)
- Fix WebSocket JWT expiration not validated (websocket.nim)
Medium (13):
- Fix writeSSTable partial file → tmp + atomic rename (lsm.nim)
- Fix multi-CTE table loss (executor.nim)
- Fix nl_to_sql DML restricted to superuser (executor.nim)
- Fix unbounded plan cache — max 10000 (adaptive.nim)
- Fix migration lock crash persistence — timestamp + stale detection (executor.nim)
- Fix admin panel auth (httpserver.nim)
- Fix MVCC unbounded txn tracking — prune in compactVersions (mvcc.nim)
- Fix connection pool maxLifetime check (pool.nim)
- Fix JWT JSON parser backslash escapes (auth.nim)
- Fix substr(s, start) returning single char (udf.nim)
- Fix loadSSTable minimum file-size check (lsm.nim)
- Fix compaction mmap leak (compaction.nim)
- Fix JSON injection in hybrid_search_filtered (executor.nim)
Low (4):
- Raft loadState logs error instead of silent discard
- Replication healthCheck double-close fixed
- Lexer readIdent double column counting fixed
- WebSocket frame 32-bit overflow guard
All 448 tests passing, 0 failures. Bump version to 1.1.7.
2026-05-29 14:17:41 +03:00
dimgigov
7d19e06805
fix(btree): critical seq.del vs seq.delete bug and B+ tree leaf traversal
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Root cause: seq.del(i) does swap-with-last (destroys sorted order)
while seq.delete(i) preserves order. This caused massive data loss
during B-tree removes.
Changes:
- btree.nim: Replace seq.del with seq.delete (critical data corruption fix)
- btree.nim: search() traverses leaf linked list for duplicate boundary keys
- btree.nim: splitChild() consolidates duplicate boundary key values before split
- btree.nim: len() counts unique keys via leaf traversal
- btree.nim: remove() traverses leaf linked list to remove ALL occurrences
- btree.nim: Fix separator update in internal node (node.keys[i-1] not [i])
- prop_test.nim: Use set equality for interleaved test (rebalancing reorders)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-23 22:01:20 +03:00
dimgigov
2d310a33a1
fix: resolve all 55 identified bugs across the codebase
...
Comprehensive bug fix pass across all subsystems:
- Critical: lexer infinite loop, WAL lock discipline, checkpoint safety,
compaction verification, HMAC key truncation, healthCheck leak
- High: MVCC lock safety, B-Tree delete rebalancing, Raft stale term
handling, replication socket leaks and ack cleanup, sharding migration
with old assignments, 2PC recovery, JWT/SCRAM auth fixes, wire protocol
bounds checks
- Medium: config error handling, MERGE parser completeness, IR/codegen
correctness, UDF bounds checks, LIMIT cost accuracy, mmap safety
- Low: timeout handling, float parsing edge cases, uint32 truncation,
cache entry cleanup
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-21 10:59:51 +03:00
dimgigov
cd46edcb67
Bump version to 1.1.6; fix storage bugs, tests, CI pipeline and Docker config
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Storage fixes:
- Fix bloom filter hash2 OverflowDefect (uint64 instead of int Hash)
- Fix MemTable.put size calculation on overwrite (was leaking size)
- Fix fuzz_test duplicate-key issues in LSM delete and SSTable tests
Test fixes:
- Fix stress_test.nim: replace deprecated threadpool with std/threads
- Fix fuzz_test.nim: add missing imports (tables, algorithm, sets)
- Fix fuzz_test.nim: var sst for close() compatibility
- Remove unused imports from test_all.nim and bench_all.nim
Docker / CI fixes:
- Fix Dockerfile.source: invalid nimlang/nim:2.2.10-alpine tag → 2.2.10 + ubuntu:24.04 runtime
- Fix Dockerfile.source healthcheck (--spider → -qO- for 200 OK)
- Fix Dockerfile run comment ports (9470/9471 → 9912/9913)
- Fix scripts/docker-run.sh healthcheck port (9470 → 9912)
- Add ARG BUILD_DATE/VCS_REF to Dockerfile for docker-build.sh
- Update all version strings 1.1.4 → 1.1.6 across nimble/docker/source/docs
2026-05-19 12:13:33 +03:00
dimgigov
7e6a45e6b7
Стабилизация на storage слоя — 6 фази
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Фаза 1: SSTable Integrity
- SSTable v3 формат с CRC32 footer (data/index/bloom CRC)
- Нов модул storage/crc32.nim (zero-dep, IEEE 802.3)
- verifySSTable() — проверка на magic, version, CRC
- loadSSTable strict mode — отхвърля корумпирани файлове
- newLSMTree логва [WARN] при corrupt SSTables
Фаза 2: baradb repair
- Нов модул tools/repair.nim
- Сканира SSTables, проверява CRC, премества битите в corrupt/
- WAL replay (CrashRecovery) за възстановяване на данни
- CLI: ./baradadb repair --data-dir=... [--dry-run]
Фаза 3: MANIFEST File
- Atomic write MANIFEST (tmp + rename)
- newLSMTree зарежда от MANIFEST, fallback към walkDir
- checkStorageConsistency() — orphan/missing detection
- flushUnsafe и compaction записват MANIFEST
Фаза 4: WAL Rotation & Incremental Backup
- WAL segment rotation при 64MB (wal_archive/wal.000NNN.log)
- maybeRotate() на всеки 1000 записа + при flush
- backup incremental — архивира MANIFEST + SSTables + WAL
- CRC verification на SSTables преди архивиране
Фаза 5: Online Consistent Backup
- checkpoint() — freeze memtable + flush + WAL rotate
- ./baradadb checkpoint — offline consistent snapshot
- backup --online — checkpoint + incremental backup
Фаза 6: SSTable Version Migration
- SSTable.fileVersion поле
- listLegacySSTables() — намира v1/v2 файлове
- migrateSSTable() — пренаписва към v3 (tmp + rename)
- ./baradadb migrate [--dry-run] — offline migration
Документация:
- Обновени docs/en/backup.md, docs/bg/backup.md
- Обновени docs/en/storage.md, docs/bg/storage.md
- Добавени тестове в tests/test_all.nim
2026-05-18 15:04:49 +03:00
dimgigov
132e8c7a31
Phase 5: Storage fuzz tests + WAL entryCount fix + SSTable overflow checks + compaction sizeBytes accuracy
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- WAL: entryCount is now accurate after restart by counting existing entries
- LSM writeSSTable: adds uint32 overflow checks for key/value/entry counts
- Compaction: sizeBytes now uses actual file size instead of rough guess
- Fuzz tests: 14 new tests covering WAL truncation, LSM put/get/delete,
recovery, scan, overwrites, SSTable roundtrip, corruption, tombstones
2026-05-18 12:34:26 +03:00
dimgigov
3f29c519d4
Storage hardening: btree size fix, WAL entryCount note, LSM put error handling
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- btree.nim: insertNonFull returns bool; size only increments for new keys
- wal.nim: comment about entryCount reset on append
- lsm.nim: recovery/put/delete/putUnsafe raise IOError if put fails after flush
2026-05-18 12:14:30 +03:00
dimgigov
8596cea548
Stabilization: replication semi-sync, MVCC aborted deleters, SSTable level, SCRAM fix
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
P0-P1 fixes:
- replication.nim: rmSync/rmSemiSync now wait for replica ACKs
- mvcc.nim: aborted deleters no longer hide records (return true for visibility)
- storage/lsm.nim: SSTable level persisted in header (version 2, backward compat)
- protocol/auth.nim: removed insecure SCRAM raw hash fallback
Build: 0 warnings, 0 errors. All tests pass.
2026-05-18 12:04:45 +03:00
dimgigov
47c4393a7e
Complete plan execution: gossip/raft bounds, deadlock fix, JSON escaping, memtable limits
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Remaining plan items:
- 1.4 hybrid_search: JSON built with std/json instead of string concat
- 2.3 lsm.nim: reject oversized memtable entries
- 3.3 gossip.nim: bounds checking for idLen/nodeCount/hostLen
- 3.4 raft.nim: max length caps for cmdLen/dataLen/logLen
- 4.3 deadlock.nim: per-path seq stack instead of global parent table
2026-05-18 11:42:46 +03:00
dimgigov
967c0855a5
Bug fixes: composite PK, nl_to_sql sandbox, FK check, SQL injection, storage correctness
...
Critical fixes:
- Composite PK: execInsert + validateConstraints use all PK columns
- nl_to_sql: non-SELECT SQL no longer executed directly during validation
- FK check: removed O(N) scanMemTable fallback, uses db.get() with SSTables
- exprToSql: nkIdent wrapped in quotes to prevent SQL injection
- restoreSchema: try/except around tokenize/parse for crash resilience
- recovery.nim: lastTxnId tracking + putUnsafe/deleteUnsafe without WAL
- SCRAM: verifyClientProof length check + DefaultIterationCount restored
Storage fixes:
- lsm.nim: SSTable sort order fixed (ascending), close() flushes all memtables
- compaction.nim: tombstones preserved during compaction
- wal.nim: header written for empty existing files, readEntries checks magic
- btree.nim: B+ tree leaf split keeps boundary key
- bloom.nim: deserialize raises on short data
- mmap.nim: bounds checks for adviseWillNeed/DontNeed + posix.close()
Protocol fixes:
- zerocopy.nim: readString bounds check
- wire.nim: deserializeValue 32-bit underflow check
- auth.nim: JWT JSON escaping for claims
- server.nim: readUint32BE bounds check + specific exception handling
- raft.nim: readData checks + specific exception handling
Tests:
- Added Composite Primary Key test suite (4 tests)
Build: 0 warnings, 0 errors
2026-05-18 11:33:11 +03:00
dimgigov
19fa760604
Week 4: Production Hardening — Thread Safety, Fuzz Tests, Property-Based Tests, NimForum Smoke Test
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Thread Safety:
- Add lock to BTreeIndex (insert/get/scan/remove/contains)
- Add lock to InvertedIndex (addDocument/removeDocument/search/bm25/tfidf/fuzzy/regex)
- Add ctxLock to ExecutionContext for DDL/schema metadata, autoIncCounters, sequences
- Deep-copy sessionVars in cloneForConnection() to isolate per-connection state
- Fix activeConnections check-then-act race with Lock
Wire Protocol Hardening:
- Fix RangeDefect in deserializeValue for invalid FieldKind bytes
- Fix IndexDefect in fkFloat32/fkFloat64 by reusing readUint32/readUint64
- Fix int32/int64 cast from uint32/uint64 to use cast[] instead of constructor
- Fix parseHeader to validate MsgKind instead of holey enum cast
Property-Based Tests (tests/prop_test.nim):
- 8 invariants for evalExprValue: literal types, commutativity, identity,
double negation, NULL propagation
Wire Protocol Fuzz (tests/fuzz_test.nim):
- 7 fuzz tests: random bytes deserialization, truncated buffers,
MsgKind casts, parseHeader logic, message roundtrips
NimForum Adapter Smoke Test (tests/nimforum_smoke_test.nim):
- Full TCP server lifecycle with adapter CRUD and parameterized queries
Test Results: 423 tests pass, 0 failures
2026-05-17 11:57:55 +03:00
dimgigov
07a22d305d
chore: fix all build warnings and add stabilization roadmap
...
- Fix 3× ResultShadowed in cypher.nim, recovery.nim, shell.nim
- Remove 6× UnusedImport in test files
- Build is now clean: 0 warnings, 294 tests pass
- Add 4-week stabilization plan to PLAN.md (Sessions 9.1–9.4)
2026-05-17 03:47:17 +03:00
dimgigov
c5751b4a0c
fix: clean up compiler warnings and unused variables
2026-05-15 18:01:11 +03:00
dimgigov
c0bd3dff86
feat: distributed gaps filled — gossip UDP transport, sharding data migration, inter-module wiring (raft-disttxn, gossip-sharding, replication-disttxn)
2026-05-13 11:11:53 +03:00
dimgigov
422df08ab9
feat: clean build + crossmodal.tla + TLA+ symmetry (v1.0.0-prep)
...
Build warnings cleanup (0 warnings):
- Suppress threadpool deprecation warning (baradadb.nim)
- Remove unused 'os' import (logging.nim)
- Fix ImplicitDefaultValue for newNode params (ast.nim)
- Add explicit cstring cast for posix.open (wal.nim)
- Fix HoleEnumConv for MsgKind parsing (server.nim)
TLA+ Formal Verification:
- Add symmetry reduction (Permutations) to all 9 existing specs
- Add SYMMETRY directive to all .cfg files
- New crossmodal.tla: cross-modal consistency spec
* MetadataVectorConsistency, HybridResultValid
* CommittedAtomicity, AbortedAtomicity, TxnStateValid
- New models/crossmodal.cfg
Tests:
- Add Cross-Modal TLA+ Faithfulness tests (4 tests)
Docs:
- Update PLAN.md and BUG_AUDIT.md with completed tasks
2026-05-13 09:24:04 +03:00
dimgigov
a72e17f063
fix: последните 4 критични/средни бъга — LSM-Tree, MVCC, DistTxn, Raft
...
Поправени проблеми:
- LSM-Tree: WAL write вече е извън db.lock (отделен walLock) → по-добра concurrency
- MVCC: добавен compactVersions() който се изпълнява на всеки 100 commits
Премахва стари overwritten версии, които не са видими за active транзакции
- DistTxn: commit() вече НЕ rollback-ва commit-нали participants
Ако някой participant е commit-нал, транзакцията се маркира като committed
- Raft: добавен disk persistence за currentTerm, votedFor, log
saveState() при всяка промяна; loadState() при стартиране
292 теста, 0 failure-а.
2026-05-12 23:30:58 +03:00
dimgigov
6aaabb518d
fix: високо и средно приоритетни бъгове в query, protocol, storage, vector, fts
...
Поправени проблеми:
- Wire protocol: bounds checking, max length limits, recursion depth limit
- SQL injection: escape-ване на quotes в exprToSql string literals
- ReDoS: escape-ване на regex metachars в LIKE/ILIKE
- Stale BTree indexes: добавен BTree.remove() + изтриване при UPDATE/DELETE
- SSL: quoteShell за всички shell команди с пътища
- Boolean literals: parser вече разпознава tkTrue/tkFalse
- Unary minus: lowerExpr map-ва ukNeg към irNeg (вместо irNot)
- Non-aggregate UDFs: lowerExpr създава irekFuncCall вместо NULL literal
- UTF-8 FTS: tokenize използва runes вместо байтове
- HNSW: добавен Lock за thread-safe insert/search
292 теста, 0 failure-а.
2026-05-12 23:02:54 +03:00
dimgigov
131541a0e5
fix: критични бъгове в storage, MVCC, auth, Raft, query executor
...
Поправени проблеми:
- MVCC: aborted транзакции вече не стават видими (добавено abortedTxns множество)
- LSM-Tree: поправена загуба на данни при immutable memtable overwrite
- LSM-Tree: поправена SSTable search order (сортиране по id вместо minKey)
- WAL: sync()/close() вече извикват posix.fsync()
- Auth: simpleHash (djb2) заменен с HMAC-SHA256
- Recovery: summary() вече не мутира базата данни
- Raft: поправена majority calculation за четен брой нодове
- Query: EXISTS subqueries вече изпълняват подзаявката
- Nimble: поправен build (-d:ssl) и bench task
Добавен BUG_AUDIT.md с пълен доклад и план за подобрения.
292 теста, 0 failure-а.
2026-05-12 22:45:50 +03:00
dimgigov
651375a156
feat: production hardening — JWT security, WAL reader, tracing, 2PC real RPC, stress test fix
...
- JWT: getEffectiveJwtSecret() helper + warning log when not configured
- WAL: readEntries() with timestamp filter for PITR
- Tracing: core/tracing.nim with span recording + executeQuery integration
- 2PC: real TCP RPC via sendDistTxnRpc (host='' = local fallback)
- Stress test: updated comment, 10K ops confirmed with internal locks
- SSTable metadata comments clarified (offsets patched correctly)
- addParticipant has default params (host='', port=0)
2026-05-07 13:31:29 +03:00
dimgigov
08f73598a5
Fix WAL crash recovery — data survives power loss
...
- wal.nim: open existing WAL with fmAppend instead of fmWrite (was truncating!)
- lsm.nim: replay WAL entries into memTable on startup before accepting writes
- Fixes real data loss on crash — previously WAL was wiped on every restart
2026-05-07 00:29:43 +03:00
dimgigov
f9f77b3a18
feat: production blockers — JOINs, deadlock detection, TLS, parameterized queries
...
- JOIN execution: INNER/LEFT/RIGHT/FULL/CROSS with column disambiguation
- Deadlock detection: wait-for graph wired into TxnManager.write()
- TLS/SSL: OpenSSL via std/net for TCP wire protocol, auto self-signed certs
- Parameterized queries: ? placeholders with WireValue binding
- Wire protocol: mkQueryParams message support
- HTTP /query endpoint accepts JSON params array
- Nim client: query(sql, params) overload
- Tests: 262 passing (15 new)
All PLAN.md production blockers resolved.
2026-05-06 16:42:53 +03:00
dimgigov
856a07c030
feat: HTTP server in main entry point + background compaction scheduling
...
- baradadb.nim now starts both TCP wire protocol and HTTP REST servers
- HTTP server runs in background thread via hunos on port TCP+440
- WebSocket server auto-starts on HTTP port + 1
- CompactionManager with background async loop (default 60s interval)
- CompactionStrategy wired to LSMTree SSTables
- Fixed forward declaration for lowerExpr in executor.nim
- 216 tests passing
2026-05-06 14:23:13 +03:00
dimgigov
1fb715a1d4
feat: integrate hunos web server + jwt-nim-baraba + real WAL recovery + compaction
...
HTTP Server:
- Replaced asynchttpserver with hunos (multi-threaded, trie router, CORS middleware)
- JWT authentication via jwt-nim-baraba (HS256 with BearSSL crypto)
- POST /query, GET /health, GET /metrics, POST /auth, GET /api (OpenAPI spec)
- Proper CORS headers via hunos corsMiddleware
WAL Recovery:
- recover() now actually applies committed entries (REDO)
- Skips uncommitted entries (UNDO)
- Takes optional LSMTree parameter to apply recovery
SSTable Compaction:
- Real compaction: reads SSTable entries, merges by key (newest wins), writes merged file
- Deduplication: keeps only newest version per key
- Tombstone cleanup: removes deleted entries
- Old SSTable files deleted after merge
Dependencies:
- Added hunos >= 1.2.0 and jwt >= 2.1.0 to nimble
SSTable fields exported for compaction access.
All 216 tests pass
2026-05-06 13:42:37 +03:00
dimgigov
096c8347cf
feat: complete Phase 3 + new production roadmap for Web/ERP
...
- Thread-safety: locks in LSMTree and Graph engines
- Raft network transport: async TCP, serialization, heartbeat, 3-node election test
- CI/CD: GitHub Actions workflow
- Cleanup: remove dead code, unused imports, build artifacts
- New PLAN.md targeting production Web/ERP readiness
- 216 tests passing
2026-05-06 10:40:34 +03:00
dimgigov
f8909f155c
feat: wire protocol integration in TCP server
...
- Server now reads binary wire protocol messages (header + payload)
- Implements recvExact for reliable message framing
- Query execution against LSM-Tree for SELECT/INSERT/DELETE
- SELECT supports point reads (WHERE key = 'value') and full scans
- INSERT parses simple EdgeDB-style syntax
- DELETE with WHERE clause
- Wire protocol responses: Data, Complete, Error, Pong
- Export wire protocol read/write helpers for external use
- Add scanMemTable to LSMTree for full scans
- Add GEL/ to .gitignore
- All 214 tests pass
2026-05-06 03:38:10 +03:00
dimgigov
5dba7b5699
feat: Phase 1 — SSTable persistence, README honesty, benchmark fix
...
- Add Current Status / Limitations section to README
- Fix benchmark compilation (Duration.ticks → inNanoseconds)
- Implement real SSTable binary format with write/read/mmap support
- Add BloomFilter serialize/deserialize for disk storage
- Fix mmap.nim to use posix.open instead of system.open
- New PLAN.md with improvement roadmap
- All 214 tests pass
2026-05-06 03:16:39 +03:00
dimgigov
805dbc8ad6
fix: comprehensive bug audit — 15 critical/high/medium fixes
...
Critical fixes:
- B-Tree splitChild: fixed out-of-bounds on child.children.len (mid+1..len → mid+1..<len)
- Wire protocol: all deserializeValue cases now properly assign to result
- Wire protocol: fkBool/fkInt8 now increment pos after reading
- MVCC: write-write conflict now detected for completed transactions
- MVCC: commit only marks visible versions as deleted (not concurrent uncommitted)
- Lexer: buffer overrun on backslash escape at end of input — added bounds check
- Bulgarian/Russian stemmers: fixed UTF-8 byte offsets for Cyrillic suffixes (2 bytes per char)
High fixes:
- WebSocket: added missing std/tables import
- Raft: added randomize() call for election timeouts
- MemTable.get: binary search O(log n) instead of linear scan O(n)
Medium fixes:
- Zerocopy: string length now written as 4-byte int32 (was single byte)
- Raft appendEntries: guard against nextIdx underflow
- Columnar: null tracking with nulls seq on ColumnPtr
- Removed dead code in lexer readIdent (redundant true/false checks)
- Updated Bulgarian stemming test to match corrected byte offsets
All 214 tests pass across 48 suites with 0 failures.
2026-05-06 02:46:37 +03:00
dimgigov
b9f6059cbf
feat: crash recovery, Raft election timer, CLI autocomplete, roadmap update — 246 tests
...
Crash Recovery:
- WAL file scanning with magic/version/entry parsing
- REDO/UNDO analysis — identify committed vs uncommitted txns
- Summary reporting with entry/txn/apply status
Raft Election Timer:
- ElectionTimer with configurable timeout and tick/check/reset
- Automatic election start on follower timeout
- Candidate timeout restart on failed elections
CLI Autocomplete:
- Full autocomplete engine for commands and SQL keywords
- Suggest function for single/multi completions
- 60+ keywords (SELECT, FROM, WHERE, JOIN, GROUP BY, etc.)
ROADMAP.md:
- All completed items properly marked
- Updated status table — 9 out of 12 phases at 90%+
14 new tests (246 total, all passing)
2026-05-06 02:12:49 +03:00
dimgigov
b0a760c0ab
feat: schema inheritance, codegen, replication, mmap — 136 tests
...
Schema:
- Inheritance: multi-level, property merging, override, isSubtype
- Computed properties with expressions
- getSubtypes for polymorphic queries
Codegen:
- IR plan → storage operations compilation
- Predicate pushdown optimization (filter → point read)
- Cost estimation and EXPLAIN output
Replication:
- Sync replication (wait for all replicas)
- Async replication (fire and forget)
- Semi-sync (wait for N replicas)
- Replica state tracking and lag monitoring
Storage:
- Memory-mapped I/O for SSTable access
- Sequential/random access hints (madvise)
- Page-level read operations
29 new tests (136 total, all passing)
2026-05-06 01:23:28 +03:00
dimgigov
67213826a8
feat: compaction, page cache, WebSocket, rate limiter, TF-IDF, fuzzy search, regex, metadata filter — 73 tests
...
- SSTable compaction: size-tiered strategy, level-based scheduling
- Page cache: LRU eviction, hit rate tracking, capacity management
- WebSocket: full duplex streaming, frame encoding, ping/pong
- Rate limiter: token bucket + sliding window algorithms
- FTS: TF-IDF ranking, Levenshtein fuzzy matching, wildcard regex
- Vector: metadata filtering on HNSW search
- 16 new tests (73 total, all passing)
2026-05-06 01:03:58 +03:00
dimgigov
07a37d8e78
feat: B-Tree, columnar engine, IR/type checker, connection pool, JWT auth, quantization, Louvain, pattern matching — 57 tests
...
- B-Tree index: insert, get, scan range, duplicate keys
- Columnar engine: batch ops, RLE/dict encoding, GroupBy, aggregates
- IR (Intermediate Representation): plan nodes, expressions, type checker
- Connection pool: load-balanced eviction, min/max connections
- JWT authentication with token verify and claims parsing
- Vector quantization: scalar 8-bit/4-bit, product quantization, binary
- Louvain community detection algorithm
- Graph pattern matching (subgraph isomorphism)
- 18 new test suites (57 total, all passing)
2026-05-06 00:57:30 +03:00
dimgigov
6935889877
feat: initial BaraDB — multimodal database engine in Nim
...
- LSM-Tree storage engine with WAL, bloom filter, MemTable
- BaraQL query language: lexer (80+ tokens), recursive descent parser, AST
- Vector engine: HNSW + IVF-PQ indexes, 4 distance metrics
- Graph engine: adjacency list, BFS/DFS, Dijkstra, PageRank
- Full-Text Search: inverted index, BM25 ranking, stemming, stop words
- Type system: 17 types (int/float/string/uuid/json/vector/...)
- Async TCP server
- 21 passing tests
2026-05-06 00:22:12 +03:00