dimgigov
0c7847abba
docs: update backup.md with multi-database and HTTP API docs
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-25 18:49:26 +03:00
dimgigov
eb62b381a1
feat: multi-database backup, HTTP API endpoints, admin panel UI
...
- backup CLI: add --all-databases, --database, --data-root flags
- backup format: tar.gz with databases/<name>/ + backup.json metadata
- HTTP API: new endpoints /databases, /backup, /backups, /restore
- HTTP API: database switching via X-Database header
- Admin panel: database selector dropdown, Databases tab, Backups tab
- Admin panel: backup/restore UI with fetch() to new endpoints
- Update deps: hunos 1.3.0, jwt-nim-baraba 2.1.0
- Docker: backup cron uses --all-databases --data-root=/data/databases
- Docs: update api-http.md, protocol.md with correct endpoints
- OpenAPI spec updated to v1.1.6
Known limitation: DELETE /databases disabled due to ORC SIGSEGV
in LSMTree.close(). Use SQL DROP DATABASE instead.
2026-05-25 18:46:07 +03:00
dimgigov
a9ea80a1f8
fix(prop_test): "Nil expr evaluates to NULL" test — compare Value.kind, not Value == ""
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
The test was comparing a Value to a string literal which always failed.
Changed to check s.kind == vkNull on both calls.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-23 22:06:37 +03:00
dimgigov
7d19e06805
fix(btree): critical seq.del vs seq.delete bug and B+ tree leaf traversal
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Root cause: seq.del(i) does swap-with-last (destroys sorted order)
while seq.delete(i) preserves order. This caused massive data loss
during B-tree removes.
Changes:
- btree.nim: Replace seq.del with seq.delete (critical data corruption fix)
- btree.nim: search() traverses leaf linked list for duplicate boundary keys
- btree.nim: splitChild() consolidates duplicate boundary key values before split
- btree.nim: len() counts unique keys via leaf traversal
- btree.nim: remove() traverses leaf linked list to remove ALL occurrences
- btree.nim: Fix separator update in internal node (node.keys[i-1] not [i])
- prop_test.nim: Use set equality for interleaved test (rebalancing reorders)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-23 22:01:20 +03:00
dimgigov
b25fea4d21
fix: all 5 bugs — reserved words, cloneForConnection, SQL injection, SHOW TABLES, getRowPlain
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-21 22:51:04 +03:00
dimgigov
837b4d56fc
fix(nim-allographer/baradb): critical client fixes and bug audit
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- Export hidden symbols in baradb_client.nim (socket, nextId, readQueryResponse, toString, toBytes)
- Add missing imports in baradb_exec.nim (asyncnet, enums, baradb_query)
- Rewrite toJson() to use wire protocol columnTypes instead of guessing types from strings
- Fix aggregate functions (count, avg, sum) to handle JInt/JFloat/JString correctly
- Fix paginate/fastPaginate discard issues
- Add placeholder arguments for whereIn/whereNotIn/whereBetween/whereNotBetween
- Add placeholder count validation in formatSql()
- Fix getRowPlain() to avoid IndexDefect on empty results
- Harden transactionStart/transactionEnd against server crashes
- Reset placeHolder in insertSql() to avoid stale values
- Fix test_query.nim to use backtick quoting for BaraDB compatibility
Known server bugs documented in BARADB_CLIENT_BUGS.md
2026-05-21 22:01:55 +03:00
dimgigov
bb843b9a03
feat: migrate system + cross-DB engine + IMPORT/EXPORT syntax -- 22 files, client+server+docs
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-21 19:32:14 +03:00
dimgigov
f427ba5968
fix(baradb): schema builder, insertId RETURNING, whereNull, rename bugs
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- create_column_query.nim: full RdbTypeKind -> SQL mapping (~350 lines)
- create_table.nim, add_column.nim, change_column.nim: use createColumnString
- insertId: use RETURNING clause instead of SELECT MAX race condition
- whereNull: add missing 'symbol': 'is' key to prevent runtime crash
- rename_column/rename_table: fix changeTo -> previousName
- create_migration_table: use backtick quoting consistently
- PLAN_BARADB.md: update status to reflect completed fixes
2026-05-21 16:06:45 +03:00
dimgigov
8a385cba6a
docs(readme): clarify this is a BaraDB fork of nim-allographer
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-21 15:54:56 +03:00
dimgigov
6bfc5b3a3c
feat(clients): add nim-allographer client with BaraDB driver support
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- Pure Nim wire protocol client (async + sync, no C FFI)
- Query Builder: fluent API, SQL generator, execution, transactions
- Schema Builder: create/alter/drop tables, column operations
- Connection pool with aging and timeout handling
- ORM integration via compile-time DB_BARADB switch
- Tests: test_open, test_query
- Documentation: README.md, PLAN_BARADB.md
2026-05-21 15:49:11 +03:00
dimgigov
2d310a33a1
fix: resolve all 55 identified bugs across the codebase
...
Comprehensive bug fix pass across all subsystems:
- Critical: lexer infinite loop, WAL lock discipline, checkpoint safety,
compaction verification, HMAC key truncation, healthCheck leak
- High: MVCC lock safety, B-Tree delete rebalancing, Raft stale term
handling, replication socket leaks and ack cleanup, sharding migration
with old assignments, 2PC recovery, JWT/SCRAM auth fixes, wire protocol
bounds checks
- Medium: config error handling, MERGE parser completeness, IR/codegen
correctness, UDF bounds checks, LIMIT cost accuracy, mmap safety
- Low: timeout handling, float parsing edge cases, uint32 truncation,
cache entry cleanup
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-21 10:59:51 +03:00
dimgigov
7fc6adbe47
chore: remove old planning and bug report files
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
These files are outdated and only cause confusion.
Keep only the current PLAN.md.
2026-05-21 09:47:40 +03:00
dimgigov
072e7a5d13
test(join): add test for duplicate columns in SELECT *
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Verify that JOIN with overlapping column names preserves
both left (bare) and right (qualified) values.
2026-05-21 09:44:32 +03:00
dimgigov
d4dc433e48
fix(query): preserve both sides of duplicate column names in JOIN
...
When SELECT * joins tables with overlapping column names
(e.g. 'id' in both users and orders), the right-side value
was being dropped because irekStar filtered out dotted keys.
- Add expandStarRow() that keeps bare columns and adds
qualified aliases (u.id, o.id) only for duplicated names
- Fix r.columns expansion to include all joined tables,
qualifying duplicates from the right side
2026-05-21 09:44:31 +03:00
dimgigov
1ff6fabaff
fix(backup): resolve relative SSTable paths in incremental backup
...
MANIFEST stores relative paths like 'sstables/1.sst'.
The old code checked fileExists against CWD instead of dataDir,
silently omitting SSTables when run from another directory.
Now resolves paths relative to dataDir.
2026-05-21 09:44:28 +03:00
dimgigov
4a83eb6783
docs: update distributed/backup/schema/storage for multi-database
...
- Add multi-database limitations to distributed docs
- Fix backup commands and add multi-database backup guidance
- Add SQL migration section to schema docs
- Update storage examples to use per-database paths
2026-05-21 09:44:25 +03:00
dimgigov
a526a3aef9
docs: remove non-English/Bulgarian language docs, fix links
...
Remove ar, de, fa, ru, tr, zh documentation folders.
Update docs/index.md and docs/bg/index.md to only list en/bg.
Update README badge from 7 to 2 languages.
Update changelog entry for Bulgarian docs.
2026-05-21 09:44:17 +03:00
dimgigov
372e5cf627
fix(query,types): refactor Row to Value-typed map, fix IN list, nkPath, multi-table joins
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- Refactored Row from Table[string, string] to Table[string, Value]
- Added Value operators: ==, !=, $, in for string interop
- Fixed valueToString for vkNull to return '\\N' instead of ''
- Fixed evalExpr irekField to return vkNull when field not found
- Added IN (val1, val2, ...) parser support
- Fixed nkPath column names in multi-table joins
- Fixed LATERAL JOIN null padding when no matching rows
- Added CREATE/DROP/USE/SHOW DATABASE parser support
- Adapted all tests for new Value type
2026-05-20 23:22:14 +03:00
dimgigov
57d2908066
fix(executor,mvcc,tests): dedup aliases, fix txn delete, fix MIN/MAX null, add regression tests
...
- Add isDelete flag to VersionedRecord to fix COMMIT treating empty values
as DELETE. Previously version.xmax != TxnId(0) was never true for active
transactions, so deletes inside transactions never removed rows.
- Fix MIN/MAX aggregates skipping NULL: was checking for "\\N" instead
of correct "\N" sentinel, so NULL values were never skipped.
- Add duplicate alias deduplication in lowerSelect() and getSelectColumns().
Identical aliases now get _1, _2 suffixes (e.g. count(*), count(*)_1).
- Add nil guard for expr.subqueryPlan in irekSubquery evalExpr handler.
- Initialize txnManager in newExecutionContext to prevent segfault on BEGIN.
- Add regression tests: DELETE in transaction, PK-only row survival,
MIN/MAX NULL skipping.
2026-05-20 21:36:15 +03:00
dimgigov
edd7e728f9
fix(query): add correlated subquery support
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
evalExpr was missing irekSubquery case — scalar subqueries in SELECT/WHERE
fell through to else:return "" producing empty strings. Add:
- irekSubquery handler in evalExpr that executes the subquery plan
- outerRow + subqueryPlan fields on ExecutionContext for correlation
- qualified column injection in execScan for correlated filter refs
- collectCorrelatedTablesFromPlan to discover outer table references
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-19 23:16:31 +03:00
dimgigov
14596c9584
fix(adapter,executor,tests): harden nimforum adapter and fix transaction/aggregate bugs
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Adapter (baradb_sqlite.nim):
- getRow now returns @[] for missing rows (db_sqlite compat)
- insertID/nextId handle NULL (\N) values correctly
- preprocessQuery replaces SQLite-specific syntax:
TIMESTAMP->DATETIME, DATETIME('now')->current timestamp, INET->STRING
- add dbBegin/dbCommit/dbRollback for transaction support
Executor:
- fix COMMIT treating empty values as DELETE (rows with only PK were lost)
- fix MIN/MAX aggregates including NULL (\N) which skewed results
- add numeric comparison for irNeq
Tests:
- stress_test: shared DB, 16 workers, correctness scan after run
- nimforum_smoke_test: 10 tests including schema, transactions, iterators
2026-05-19 20:43:00 +03:00
dimgigov
d3b77174da
fix: use numeric comparison for MIN/MAX aggregates
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-19 15:19:47 +03:00
dimgigov
cd46edcb67
Bump version to 1.1.6; fix storage bugs, tests, CI pipeline and Docker config
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Storage fixes:
- Fix bloom filter hash2 OverflowDefect (uint64 instead of int Hash)
- Fix MemTable.put size calculation on overwrite (was leaking size)
- Fix fuzz_test duplicate-key issues in LSM delete and SSTable tests
Test fixes:
- Fix stress_test.nim: replace deprecated threadpool with std/threads
- Fix fuzz_test.nim: add missing imports (tables, algorithm, sets)
- Fix fuzz_test.nim: var sst for close() compatibility
- Remove unused imports from test_all.nim and bench_all.nim
Docker / CI fixes:
- Fix Dockerfile.source: invalid nimlang/nim:2.2.10-alpine tag → 2.2.10 + ubuntu:24.04 runtime
- Fix Dockerfile.source healthcheck (--spider → -qO- for 200 OK)
- Fix Dockerfile run comment ports (9470/9471 → 9912/9913)
- Fix scripts/docker-run.sh healthcheck port (9470 → 9912)
- Add ARG BUILD_DATE/VCS_REF to Dockerfile for docker-build.sh
- Update all version strings 1.1.4 → 1.1.6 across nimble/docker/source/docs
2026-05-19 12:13:33 +03:00
dimgigov
343f479127
docs(bg,en): fix ports and WebSocket example for new server model
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- Update architecture diagram to include WebSocket server (port 9913)
- Fix HTTP port from 9470 to 9912 (TCP + 440) in protocol docs
- Fix WebSocket port from 9471 to 9913 (TCP + 441) in protocol docs
- Fix WebSocket Nim example: remove non-existent onMessage API and
unsafe cast[string](data), use correct WsServer API with safe helpers
- Clarify TCP_NODELAY is also handled internally by hunos (HTTP)
- Note that bytesToString/stringToBytes are wire-protocol helpers
2026-05-18 19:41:21 +03:00
dimgigov
8afa998516
feat: rate limiting, shared DB instance, TCP_NODELAY fix
...
- Add rate limiter to TCP and HTTP servers
- Share single LSMTree between TCP and HTTP servers
- Replace unsafe cast[string]/cast[seq[byte]] with safe helpers
- Stabilize gossip UDP socket with exponential backoff
- Fix TCP_NODELAY via low-level setSockOptInt(IPPROTO_TCP)
(asyncnet.setSockOpt(OptNoDelay) uses SOL_SOCKET and fails)
- Apply TCP_NODELAY to server and websocket sockets
2026-05-18 19:33:40 +03:00
dimgigov
7e6a45e6b7
Стабилизация на storage слоя — 6 фази
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Фаза 1: SSTable Integrity
- SSTable v3 формат с CRC32 footer (data/index/bloom CRC)
- Нов модул storage/crc32.nim (zero-dep, IEEE 802.3)
- verifySSTable() — проверка на magic, version, CRC
- loadSSTable strict mode — отхвърля корумпирани файлове
- newLSMTree логва [WARN] при corrupt SSTables
Фаза 2: baradb repair
- Нов модул tools/repair.nim
- Сканира SSTables, проверява CRC, премества битите в corrupt/
- WAL replay (CrashRecovery) за възстановяване на данни
- CLI: ./baradadb repair --data-dir=... [--dry-run]
Фаза 3: MANIFEST File
- Atomic write MANIFEST (tmp + rename)
- newLSMTree зарежда от MANIFEST, fallback към walkDir
- checkStorageConsistency() — orphan/missing detection
- flushUnsafe и compaction записват MANIFEST
Фаза 4: WAL Rotation & Incremental Backup
- WAL segment rotation при 64MB (wal_archive/wal.000NNN.log)
- maybeRotate() на всеки 1000 записа + при flush
- backup incremental — архивира MANIFEST + SSTables + WAL
- CRC verification на SSTables преди архивиране
Фаза 5: Online Consistent Backup
- checkpoint() — freeze memtable + flush + WAL rotate
- ./baradadb checkpoint — offline consistent snapshot
- backup --online — checkpoint + incremental backup
Фаза 6: SSTable Version Migration
- SSTable.fileVersion поле
- listLegacySSTables() — намира v1/v2 файлове
- migrateSSTable() — пренаписва към v3 (tmp + rename)
- ./baradadb migrate [--dry-run] — offline migration
Документация:
- Обновени docs/en/backup.md, docs/bg/backup.md
- Обновени docs/en/storage.md, docs/bg/storage.md
- Добавени тестове в tests/test_all.nim
2026-05-18 15:04:49 +03:00
dimgigov
cf2aba104f
Phase 5 complete: B-tree property tests + benchmark regression suite
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
- prop_test.nim: 11 new property-based B-Tree invariants
(size accuracy, get roundtrip, scan ordering, range correctness,
contains, remove, large order, duplicates, empty tree, interleaved ops)
- bench_all.nim: JSON-based benchmark result tracking with regression
comparison against previous runs; each benchmark reports ops/sec
delta vs last run
2026-05-18 12:52:44 +03:00
dimgigov
132e8c7a31
Phase 5: Storage fuzz tests + WAL entryCount fix + SSTable overflow checks + compaction sizeBytes accuracy
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- WAL: entryCount is now accurate after restart by counting existing entries
- LSM writeSSTable: adds uint32 overflow checks for key/value/entry counts
- Compaction: sizeBytes now uses actual file size instead of rough guess
- Fuzz tests: 14 new tests covering WAL truncation, LSM put/get/delete,
recovery, scan, overwrites, SSTable roundtrip, corruption, tombstones
2026-05-18 12:34:26 +03:00
dimgigov
3f29c519d4
Storage hardening: btree size fix, WAL entryCount note, LSM put error handling
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- btree.nim: insertNonFull returns bool; size only increments for new keys
- wal.nim: comment about entryCount reset on append
- lsm.nim: recovery/put/delete/putUnsafe raise IOError if put fails after flush
2026-05-18 12:14:30 +03:00
dimgigov
8596cea548
Stabilization: replication semi-sync, MVCC aborted deleters, SSTable level, SCRAM fix
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
P0-P1 fixes:
- replication.nim: rmSync/rmSemiSync now wait for replica ACKs
- mvcc.nim: aborted deleters no longer hide records (return true for visibility)
- storage/lsm.nim: SSTable level persisted in header (version 2, backward compat)
- protocol/auth.nim: removed insecure SCRAM raw hash fallback
Build: 0 warnings, 0 errors. All tests pass.
2026-05-18 12:04:45 +03:00
dimgigov
47c4393a7e
Complete plan execution: gossip/raft bounds, deadlock fix, JSON escaping, memtable limits
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Remaining plan items:
- 1.4 hybrid_search: JSON built with std/json instead of string concat
- 2.3 lsm.nim: reject oversized memtable entries
- 3.3 gossip.nim: bounds checking for idLen/nodeCount/hostLen
- 3.4 raft.nim: max length caps for cmdLen/dataLen/logLen
- 4.3 deadlock.nim: per-path seq stack instead of global parent table
2026-05-18 11:42:46 +03:00
dimgigov
967c0855a5
Bug fixes: composite PK, nl_to_sql sandbox, FK check, SQL injection, storage correctness
...
Critical fixes:
- Composite PK: execInsert + validateConstraints use all PK columns
- nl_to_sql: non-SELECT SQL no longer executed directly during validation
- FK check: removed O(N) scanMemTable fallback, uses db.get() with SSTables
- exprToSql: nkIdent wrapped in quotes to prevent SQL injection
- restoreSchema: try/except around tokenize/parse for crash resilience
- recovery.nim: lastTxnId tracking + putUnsafe/deleteUnsafe without WAL
- SCRAM: verifyClientProof length check + DefaultIterationCount restored
Storage fixes:
- lsm.nim: SSTable sort order fixed (ascending), close() flushes all memtables
- compaction.nim: tombstones preserved during compaction
- wal.nim: header written for empty existing files, readEntries checks magic
- btree.nim: B+ tree leaf split keeps boundary key
- bloom.nim: deserialize raises on short data
- mmap.nim: bounds checks for adviseWillNeed/DontNeed + posix.close()
Protocol fixes:
- zerocopy.nim: readString bounds check
- wire.nim: deserializeValue 32-bit underflow check
- auth.nim: JWT JSON escaping for claims
- server.nim: readUint32BE bounds check + specific exception handling
- raft.nim: readData checks + specific exception handling
Tests:
- Added Composite Primary Key test suite (4 tests)
Build: 0 warnings, 0 errors
2026-05-18 11:33:11 +03:00
dimgigov
a28c845476
Bump version to 1.1.4 across all components
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-17 20:57:19 +03:00
dimgigov
3ac53ecda2
docs(readme): add AI-Native Data Platform features (Sessions 10-12)
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
- Update Why BaraDB comparison table with Hybrid RAG, AI Agents,
MCP Server, LangChain integration, Graph SQL, Cypher
- Add Hybrid RAG Search section under Vector Engine
- Add Graph SQL Integration section with CREATE GRAPH, GRAPH_TABLE,
Cypher translation, and similarity/node2vec algorithms
- Add new AI-Native Data Platform section covering:
- Natural Language → SQL (nl_to_sql, schema_prompt)
- Text Chunking & Auto-Embedding (chunk, embed_text)
- MCP Server (Model Context Protocol over STDIO)
- LangChain Vector Store (Python + JS)
- Chat Message History with RLS
- Update Project Structure with ai/ and mcp/ directories
- Update Roadmap Progress with v1.1.2 features
2026-05-17 17:05:47 +03:00
dimgigov
1e38e29f25
fix(security): SQL injection in Python/JS clients + bare except + session leak
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
- Fix SQL injection vulnerabilities in chat_history.py, langchain_store.py,
rag_pipeline.py, baradb_langchain.js by switching to parameterized queries
- Replace dangerous bare except: with except CatchableError:/ValueError:
in llm.nim, embed.nim, cypher.nim, mcp/server.nim, executor.nim
- Fix session variable leak in MCP handleVectorSearch/handleSchemaInspect
- Build: 0 errors, all tests pass
2026-05-17 16:58:05 +03:00
dimgigov
c95bc4cd44
docs: synchronize documentation across all languages
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
- Add mcp.md to: bg, fa, ru, tr, zh, ar
- Add index.md to Bulgarian (bg)
- Add 24 missing German (de) documentation files
Translations for all supported languages now complete.
2026-05-17 16:29:28 +03:00
dimgigov
a5d34c001a
docs: add German (DE) documentation + update all docs for Sessions 10-12
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
New German documentation (docs/de/):
- index.md, quickstart.md, installation.md
- baraql.md, graph.md, vector.md, mcp.md
Updated English documentation:
- changelog.md: all Sessions 10-12 features
- graph.md: SQL GRAPH_TABLE, CREATE GRAPH, all 8 algorithms, Cypher, similarity_nodes, node2vec
- vector.md: hybrid RAG, chunk(), embed_text(), auto-embed, nl_to_sql(), schema_prompt()
- baraql.md: new AI & Cross-Modal Functions section, updated keyword tables
- mcp.md: MCP Server documentation (new file)
- index.md: added German (DE) language link
2026-05-17 16:15:45 +03:00
dimgigov
e783215276
docs: mark all sessions 10-11-12 complete — plan finished
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-17 16:01:20 +03:00
dimgigov
a0c5ce8598
feat: Session 11 finale — similarity_nodes, node2vec, Cypher layer
...
- similarity_nodes(graph, metric): Jaccard / Adamic-Adar similarity between all node pairs
- node2vec_embed(graph, dimensions): Random-walk based graph embeddings
- cypher(query): Translate Cypher MATCH...RETURN → BaraQL GRAPH_TABLE SQL
- Fixed recursive lock deadlock in similarity functions
- Fixed graph start node parsing (pattern variable vs numeric ID)
- Added sequtils import to graph engine
- All 340+ existing tests pass
2026-05-17 16:00:15 +03:00
dimgigov
898f108963
docs: update PLAN.md - all sessions 10, 11, 12 marked complete
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
2026-05-17 15:39:20 +03:00
dimgigov
e23b1d61d2
feat: Session 12 AI Agents & NL->SQL
...
- src/barabadb/ai/llm.nim: LLM client for NL->SQL generation
- Supports OpenAI-compatible and Ollama APIs
- Configurable via BARADB_LLM_ENDPOINT, BARADB_LLM_MODEL, BARADB_LLM_API_KEY
- extractSQL() parses SQL from LLM responses (handles markdown blocks)
- Temperature 0.1 for deterministic SQL generation
- nl_to_sql() SQL function: natural language -> SQL
- Schema-aware prompt with table column definitions + indexes + RLS
- Query validation layer: wraps generated SQL in LIMIT 0 subquery
- Self-correction loop: on error, feeds error back to LLM for fix
- Tenant-aware: respects current session variables
- schema_prompt() SQL function: generates DDL + sample data + indexes
- Returns full CREATE TABLE statement with column types and constraints
- Includes up to 5 sample rows for context
- Lists indexes, RLS policies, foreign keys
- Perfect for feeding into LLM context
- All 340+ existing tests pass
2026-05-17 15:38:14 +03:00
dimgigov
80c3fee9de
feat: 10.2.3 ChatMessageHistory + 10.2.4 RAG pipeline example
...
- clients/python/baradb/chat_history.py: BaraDBChatHistory class
- Stores conversation threads in BaraDB with multi-tenant RLS
- session_id + tenant_id + user_id isolation
- Auto-creates table and index
- Compatible with LangChain message format
- examples/rag_pipeline.py: End-to-end RAG pipeline example
- PDF/text ingestion -> chunking -> embedding -> BaraDB storage
- Hybrid search with vector distance
- LLM generation (OpenAI / Ollama)
- Supports --file and --query modes
- Configurable chunk size, overlap, top-k
- PLAN.md: Updated all Session 10 tasks as complete
2026-05-17 15:30:20 +03:00
dimgigov
13bc17cfa8
feat: 10.1.4 Chunking + embedding pipeline
...
- New modules: src/barabadb/ai/chunk.nim (text chunking) and embed.nim (HTTP embedding client)
- chunk() SQL function: returns JSON array of chunks with configurable size/overlap
- embed_text() SQL function: calls external embedding API (OpenAI/Ollama compatible)
- Auto-embedding on INSERT: when VECTOR column is null but TEXT column is populated,
generates embeddings via configured embedder
- Configurable via env vars: BARADB_EMBED_ENDPOINT, BARADB_EMBED_MODEL, BARADB_EMBED_API_KEY
- All 340+ existing tests pass
2026-05-17 15:26:24 +03:00
dimgigov
8a395225c0
feat: Session 10.3 MCP Server + Session 11 Graph Engine Deep Integration
...
MCP Server (10.3):
- STDIO JSON-RPC 2.0 transport with 3 AI tools: query, vector_search, schema_inspect
- Multi-tenant session vars (tenant_id, user_id) with RLS support
- Standalone binary: build/baramcp
- Tested with all 3 tools + parameterized queries + vector search + schema inspect
Graph Engine Deep Integration (Session 11):
- CREATE GRAPH / DROP GRAPH DDL support
- Graph engine wired to SQL executor via GRAPH_TABLE() function
- 7 algorithms: BFS, DFS, PageRank, ShortestPath, Dijkstra, Louvain, Community
- INSERT into _nodes/_edges tables auto-syncs with native Graph adjacency lists
- Optional MATCH pattern, ALGORITHM, START, END, MAXDEPTH in GRAPH_TABLE syntax
- All 340+ existing tests pass
2026-05-17 15:18:31 +03:00
dimgigov
55bc3e862a
feat(langchain): Session 10.2 — LangChain Vector Store (Python + JS)
...
- BaraDBStore for Python: add_texts, similarity_search, max_marginal_relevance_search, delete
- BaraDBStore for JS: addDocuments, addTexts, similaritySearch, maxMarginalRelevanceSearch, delete
- Both use hybrid_search() / hybrid_search_filtered() for vector+FTS+RRF
- Multi-tenant support via tenant_id session variable + metadata filter
- Embedding function is injected by user (OpenAI, sentence-transformers, etc.)
- MMR reranking for result diversity
2026-05-17 13:46:42 +03:00
dimgigov
67965ffa8b
feat(hybrid): Session 10.1.3 — Metadata pre-filtering in vector search
...
- Populate HNSW metadata with all relational columns during INSERT and CREATE INDEX
- Add doHybridSearchFiltered() using searchWithFilter() for HNSW pre-filtering
- Add SQL function hybrid_search_filtered(table, vec_col, text_col, query, vector, k, filter_col, filter_val)
- Enforce k-limit on both doHybridSearch and doHybridSearchFiltered results
- 2 tests: tenant isolation + empty filter fallback
2026-05-17 13:41:30 +03:00
dimgigov
836d30d84a
feat(hybrid): Session 10.1 — Hybrid RAG Search with RRF reranking
...
- Add searchEx() to vector engine returning metadata
- Add reciprocalRankFusion(), doHybridSearch(), findRealIdByDocId() helpers
- Add SQL functions: hybrid_search(), hybrid_search_ids(), rerank()
- Fix CREATE INDEX HNSW docId to use hash(fullKey) matching INSERT
- 5 tests covering hybrid search, ids, RRF ranking, rerank, missing indexes
2026-05-17 13:30:19 +03:00
dimgigov
f622c8f82c
docs(plan): archive old plan, create AI-Native Data Platform roadmap
...
- PLAN.md → PLAN_old_3.md (sessions 1-9 completed)
- New PLAN.md: Sessions 10-12 focused on:
- Vector AI Native Integration (Hybrid RAG, LangChain, MCP Server)
- Graph Engine Deep Integration (Native storage, Cypher, Algorithms)
- AI Agents & NL → SQL
- All features preserve multi-tenant RLS isolation and MVCC safety
2026-05-17 13:08:24 +03:00
dimgigov
d2ac485b2e
docs(plan): mark Phase 2 (JOIN) and Phase 3 (FK Enforcement) as complete
2026-05-17 13:03:15 +03:00
dimgigov
2e0969245c
feat(fk): Foreign Key Enforcement — ON DELETE/UPDATE CASCADE, SET NULL, RESTRICT
...
- Add tkRestrict token to lexer
- Parse ON DELETE CASCADE/SET NULL/RESTRICT and ON UPDATE CASCADE/SET NULL/RESTRICT
in both table-level and column-level FK constraints
- Add fkOnDelete/fkOnUpdate to ColumnDef and ForeignKeyDef
- Fix table-level FK constraint application (third pass after columns are created)
- Implement enforceFkOnDelete, enforceFkOnUpdate, enforceFkOnChildUpdate helpers
- Wire FK enforcement into DELETE and UPDATE execution paths
- Add 9 regression tests covering all FK actions
2026-05-17 13:02:39 +03:00