129 Commits

Author	SHA1	Message	Date
dimgigov	e28c1f4896	refactor(exec): extract table scans into exec/scan.nim	2026-07-30 14:19:11 +03:00
dimgigov	f97e72314f	refactor(exec): extract RLS/privileges into exec/rls.nim	2026-07-30 14:11:08 +03:00
dimgigov	26475058bf	refactor(exec): extract AST->IR lowering into exec/lower.nim	2026-07-30 14:05:16 +03:00
dimgigov	b5e9636fd2	refactor(exec): extract expression evaluation + hybrid search into exec/eval.nim	2026-07-30 13:52:56 +03:00
dimgigov	2b8cc98348	refactor(exec): extract migration storage into exec/migrations.nim	2026-07-30 13:41:52 +03:00
dimgigov	2efcddba19	refactor(exec): extract param binding into exec/params.nim	2026-07-30 13:36:39 +03:00
dimgigov	46e3d7f51e	refactor(exec): extract join/vector helpers into exec/helpers.nim	2026-07-30 13:27:17 +03:00
dimgigov	08fb391ac1	refactor(exec): extract context management into exec/context.nim	2026-07-30 13:18:10 +03:00
dimgigov	2d09edd9f7	fix+refactor: soft keywords as identifiers, full test wiring, ORC crash docs ... - parser: clause keywords (header, format, status, user, csv, ...) now work as identifiers everywhere; IMPORT/EXPORT accept FORMAT csv/HEADER true - nimble test + CI run all 13 test suites (650 checks green) - ExecutionContext.registry is now {.cursor.} (breaks registry<->ctx cycle) - ORC crash reproduced and bisected (tests/orc_repro.py); ARC stays the MM	2026-07-30 13:11:28 +03:00
dimgigov	ed5a71913c	fix: harden exception handling, break ARC cycles, sync license/client ... Replace bare except: with CatchableError across storage, query, Raft, backup, and protocol code so Defects are not swallowed. Break uncollectable ARC cycles in server shard/gossip callbacks via Server-owned refs and cursor locals. Align package license with LICENSE (BSD-3-Clause), sync README version, and point test_all at the canonical clients/nim baradb client (parseConnectionString + aliases).	2026-07-23 01:07:51 +03:00
dimgigov	8db5cfe7e1	feat: harden storage, schema persistence, fair benches, fix wire crash ... Core storage: hash MemTable, WAL group commit, L0 compaction rebuild, reader-writer lock, and a global StorageGate so HTTP workers and TCP share the LSM safely under multi-thread access. Schema: durable CREATE/ALTER/DROP under _schema:tables:* with full LSM restore on open. Executor types/values/schema split into query/exec/. Wire protocol: switch default MM to ARC — ORC cycle collector segfaulted after ~20 async INSERTs. Fair multi-tier benchmarks (SQLite/HTTP/wire/PG) and honesty docs for mixed-tier comparisons.	2026-07-18 16:55:50 +03:00
dimgigov	aa4ab11210	feat: canonical Nim client refactor with typed rows, pool, and allographer wrapper ... - Extract wire protocol into clients/nim/src/baradb/wire.nim - Add BaraError exception hierarchy - Refactor BaraClient/SyncClient with typedRows, AsyncLock request queue, timeouts, TLS config - Add BaraPool and optional HTTP fallback - Add mock-server wire and pool unit tests - Bump baradb nimble package to 1.2.0 - Make nim-allographer depend on canonical baradb client - Use typed rows in allographer toJson - Deprecate src/barabadb/client/client.nim - Update docs/en/clients.md and clients/nim/README.md	2026-06-18 21:29:55 +03:00
dimgigov	1c42eff7ef	fix(vector): use float64 accumulators for distance functions ... Vector distance functions (cosine, euclidean, dot, manhattan) returned float64 but accumulated sums in float32, causing precision loss in SQL results (e.g. sqrt(2) truncated to float32). Use float64 accumulators and casts throughout. docs(bugs): mark BUG-023 and BUG-024 as fixed The code already cleans up pendingAcks and passes oldAssignments from rebalance to migrateData; update BUGS.md to match the implementation.	2026-06-12 23:06:19 +03:00
dimgigov	ef264d7d69	feat: add unified search engine — HNSW heap-opt, segment index, boolean/phrase/ngram/facet ... New src/barabadb/search/ module with 9 components: - priority_queue.nim: BoundedHeap for O(log n) search - hnsw_opt.nim: heap-based searchLayer (2.4x faster, 92-99% recall@10) - inverted.nim: segment-based index with soft-delete and compaction - phrase.nim: positional phrase + proximity search - boolean.nim: recursive descent parser (AND/OR/NOT/ranges/wildcards) - ngram.nim: trigram index for O(1) fuzzy/prefix/wildcard - stemmer.nim: Porter2 stemmers (EN/BG/DE/FR/RU) - facet.nim: faceted search with filter pushdown - engine.nim: UnifiedSearchEngine combining all search types Performance (dim=128, efConstruction=200): N=1K: 0.30ms search, 99.6% recall@10 N=10K: 1.09ms search, 92.6% recall@10 N=50K: 2.26ms search, 75.5% recall@10 Includes search benchmarks (benchmarks/search_bench.nim), updated docs (en/bg fts.md, en/bg search.md), and crossmodal engine integration.	2026-05-30 13:42:08 +03:00
dimgigov	965ed2f675	perf: optimize FTS and HNSW engines + real PostgreSQL benchmarks ... FTS Engine (src/barabadb/fts/engine.nim): - Fix bm25Score doing O(n) linear scan per document - Cache IDF per token instead of recomputing for each doc - Use entry.termFreq directly instead of searching postings again - Result: FTS search +438% (249 -> 1360 queries/s) HNSW Vector Engine (src/barabadb/vector/engine.nim): - Optimize distance functions with float32 + 4x loop unrolling - Rewrite searchLayer: swap+pop instead of O(n) del, track worst-nearest instead of sorting nearest on every iteration - Result: HNSW insert +117% (245 -> 543 ops/s), search 2.2x faster Benchmarks: - Add real PostgreSQL comparison script (benchmarks/pg_bench.py) - Add report generator (benchmarks/generate_report.py) - Fix compare.nim cpuTime() bug (was dividing by 1M incorrectly) - Add nimble tasks: bench_pg, bench_report Docs: - Update README.md and docs/en/performance.md with real measured numbers - Add benchmarks/REAL_COMPARISON.md Version bump: 1.1.7 -> 1.1.8	2026-05-29 17:11:22 +03:00
dimgigov	42043f3946	v1.1.7: deep security & reliability audit — 33 bugs fixed ... Critical (5): - Reject empty JWT secret when authEnabled (server.nim) - Fix 2PC marking uncontacted participants as prepared/committed (disttxn.nim) - Fix Raft commit index calculation for even-sized clusters (raft.nim) - Fix REP/DISTTXN protocol auth bypass (server.nim) - Fix HTTP backup/restore path traversal (httpserver.nim) High (11): - Fix WAL write race with flush (lsm.nim) - Fix MVCC savepoint/rollback deep-copy writeSet (mvcc.nim) - Fix table mutation during deadlock iteration (mvcc.nim) - Fix LIMIT 0 returning all rows (executor.nim) - Fix COUNT(col) counting NULL values — 3 locations (executor.nim) - Fix EXISTS subquery lowering missing subqueryPlan (executor.nim) - Fix Raft appendEntries/applyCommitted array vs logical index (raft.nim) - Fix timing attacks on constantTimeCompare and SCRAM (auth.nim, scram.nim) - Fix B-tree leaf merge phantom separator key (btree.nim) - Fix SSL verifyPeer not applied to newContext (ssl.nim) - Fix sharding connectWithTimeout missing SO_ERROR check (sharding.nim) - Fix sync replication returning success on partial ack (replication.nim) - Fix WebSocket JWT expiration not validated (websocket.nim) Medium (13): - Fix writeSSTable partial file → tmp + atomic rename (lsm.nim) - Fix multi-CTE table loss (executor.nim) - Fix nl_to_sql DML restricted to superuser (executor.nim) - Fix unbounded plan cache — max 10000 (adaptive.nim) - Fix migration lock crash persistence — timestamp + stale detection (executor.nim) - Fix admin panel auth (httpserver.nim) - Fix MVCC unbounded txn tracking — prune in compactVersions (mvcc.nim) - Fix connection pool maxLifetime check (pool.nim) - Fix JWT JSON parser backslash escapes (auth.nim) - Fix substr(s, start) returning single char (udf.nim) - Fix loadSSTable minimum file-size check (lsm.nim) - Fix compaction mmap leak (compaction.nim) - Fix JSON injection in hybrid_search_filtered (executor.nim) Low (4): - Raft loadState logs error instead of silent discard - Replication healthCheck double-close fixed - Lexer readIdent double column counting fixed - WebSocket frame 32-bit overflow guard All 448 tests passing, 0 failures. Bump version to 1.1.7.	2026-05-29 14:17:41 +03:00
dimgigov	a5abb6031b	fix(backup): fix 7 bugs in backup/restore — strip-components, multi-db path, verify+log in HTTP handler, parseBackupFilename for .tar, getArchiveSize gzip -l, dead code removal	2026-05-25 20:44:14 +03:00
dimgigov	a37a62c69e	clients/nim-allographer/src/allographer/query_builder/models/baradb/baradb_exec.nim src/barabadb/core/mvcc.nim src/barabadb/query/codegen.nim src/barabadb/query/lexer.nim src/barabadb/query/parser.nim src/barabadb/query/udf.nim tests/test_all.nim	2026-05-25 18:56:02 +03:00
dimgigov	eb62b381a1	feat: multi-database backup, HTTP API endpoints, admin panel UI ... - backup CLI: add --all-databases, --database, --data-root flags - backup format: tar.gz with databases/<name>/ + backup.json metadata - HTTP API: new endpoints /databases, /backup, /backups, /restore - HTTP API: database switching via X-Database header - Admin panel: database selector dropdown, Databases tab, Backups tab - Admin panel: backup/restore UI with fetch() to new endpoints - Update deps: hunos 1.3.0, jwt-nim-baraba 2.1.0 - Docker: backup cron uses --all-databases --data-root=/data/databases - Docs: update api-http.md, protocol.md with correct endpoints - OpenAPI spec updated to v1.1.6 Known limitation: DELETE /databases disabled due to ORC SIGSEGV in LSMTree.close(). Use SQL DROP DATABASE instead.	2026-05-25 18:46:07 +03:00
dimgigov	7d19e06805	fix(btree): critical seq.del vs seq.delete bug and B+ tree leaf traversal ... Root cause: seq.del(i) does swap-with-last (destroys sorted order) while seq.delete(i) preserves order. This caused massive data loss during B-tree removes. Changes: - btree.nim: Replace seq.del with seq.delete (critical data corruption fix) - btree.nim: search() traverses leaf linked list for duplicate boundary keys - btree.nim: splitChild() consolidates duplicate boundary key values before split - btree.nim: len() counts unique keys via leaf traversal - btree.nim: remove() traverses leaf linked list to remove ALL occurrences - btree.nim: Fix separator update in internal node (node.keys[i-1] not [i]) - prop_test.nim: Use set equality for interleaved test (rebalancing reorders) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-23 22:01:20 +03:00
dimgigov	b25fea4d21	fix: all 5 bugs — reserved words, cloneForConnection, SQL injection, SHOW TABLES, getRowPlain	2026-05-21 22:51:04 +03:00
dimgigov	bb843b9a03	feat: migrate system + cross-DB engine + IMPORT/EXPORT syntax -- 22 files, client+server+docs	2026-05-21 19:32:14 +03:00
dimgigov	2d310a33a1	fix: resolve all 55 identified bugs across the codebase ... Comprehensive bug fix pass across all subsystems: - Critical: lexer infinite loop, WAL lock discipline, checkpoint safety, compaction verification, HMAC key truncation, healthCheck leak - High: MVCC lock safety, B-Tree delete rebalancing, Raft stale term handling, replication socket leaks and ack cleanup, sharding migration with old assignments, 2PC recovery, JWT/SCRAM auth fixes, wire protocol bounds checks - Medium: config error handling, MERGE parser completeness, IR/codegen correctness, UDF bounds checks, LIMIT cost accuracy, mmap safety - Low: timeout handling, float parsing edge cases, uint32 truncation, cache entry cleanup Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-21 10:59:51 +03:00
dimgigov	d4dc433e48	fix(query): preserve both sides of duplicate column names in JOIN ... When SELECT * joins tables with overlapping column names (e.g. 'id' in both users and orders), the right-side value was being dropped because irekStar filtered out dotted keys. - Add expandStarRow() that keeps bare columns and adds qualified aliases (u.id, o.id) only for duplicated names - Fix r.columns expansion to include all joined tables, qualifying duplicates from the right side	2026-05-21 09:44:31 +03:00
dimgigov	1ff6fabaff	fix(backup): resolve relative SSTable paths in incremental backup ... MANIFEST stores relative paths like 'sstables/1.sst'. The old code checked fileExists against CWD instead of dataDir, silently omitting SSTables when run from another directory. Now resolves paths relative to dataDir.	2026-05-21 09:44:28 +03:00
dimgigov	372e5cf627	fix(query,types): refactor Row to Value-typed map, fix IN list, nkPath, multi-table joins ... - Refactored Row from Table[string, string] to Table[string, Value] - Added Value operators: ==, !=, $, in for string interop - Fixed valueToString for vkNull to return '\\N' instead of '' - Fixed evalExpr irekField to return vkNull when field not found - Added IN (val1, val2, ...) parser support - Fixed nkPath column names in multi-table joins - Fixed LATERAL JOIN null padding when no matching rows - Added CREATE/DROP/USE/SHOW DATABASE parser support - Adapted all tests for new Value type	2026-05-20 23:22:14 +03:00
dimgigov	57d2908066	fix(executor,mvcc,tests): dedup aliases, fix txn delete, fix MIN/MAX null, add regression tests ... - Add isDelete flag to VersionedRecord to fix COMMIT treating empty values as DELETE. Previously version.xmax != TxnId(0) was never true for active transactions, so deletes inside transactions never removed rows. - Fix MIN/MAX aggregates skipping NULL: was checking for "\\N" instead of correct "\N" sentinel, so NULL values were never skipped. - Add duplicate alias deduplication in lowerSelect() and getSelectColumns(). Identical aliases now get _1, _2 suffixes (e.g. count(*), count(*)_1). - Add nil guard for expr.subqueryPlan in irekSubquery evalExpr handler. - Initialize txnManager in newExecutionContext to prevent segfault on BEGIN. - Add regression tests: DELETE in transaction, PK-only row survival, MIN/MAX NULL skipping.	2026-05-20 21:36:15 +03:00
dimgigov	edd7e728f9	fix(query): add correlated subquery support ... evalExpr was missing irekSubquery case — scalar subqueries in SELECT/WHERE fell through to else:return "" producing empty strings. Add: - irekSubquery handler in evalExpr that executes the subquery plan - outerRow + subqueryPlan fields on ExecutionContext for correlation - qualified column injection in execScan for correlated filter refs - collectCorrelatedTablesFromPlan to discover outer table references Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-19 23:16:31 +03:00
dimgigov	14596c9584	fix(adapter,executor,tests): harden nimforum adapter and fix transaction/aggregate bugs ... Adapter (baradb_sqlite.nim): - getRow now returns @[] for missing rows (db_sqlite compat) - insertID/nextId handle NULL (\N) values correctly - preprocessQuery replaces SQLite-specific syntax: TIMESTAMP->DATETIME, DATETIME('now')->current timestamp, INET->STRING - add dbBegin/dbCommit/dbRollback for transaction support Executor: - fix COMMIT treating empty values as DELETE (rows with only PK were lost) - fix MIN/MAX aggregates including NULL (\N) which skewed results - add numeric comparison for irNeq Tests: - stress_test: shared DB, 16 workers, correctness scan after run - nimforum_smoke_test: 10 tests including schema, transactions, iterators	2026-05-19 20:43:00 +03:00
dimgigov	d3b77174da	fix: use numeric comparison for MIN/MAX aggregates	2026-05-19 15:19:47 +03:00
dimgigov	cd46edcb67	Bump version to 1.1.6; fix storage bugs, tests, CI pipeline and Docker config ... Storage fixes: - Fix bloom filter hash2 OverflowDefect (uint64 instead of int Hash) - Fix MemTable.put size calculation on overwrite (was leaking size) - Fix fuzz_test duplicate-key issues in LSM delete and SSTable tests Test fixes: - Fix stress_test.nim: replace deprecated threadpool with std/threads - Fix fuzz_test.nim: add missing imports (tables, algorithm, sets) - Fix fuzz_test.nim: var sst for close() compatibility - Remove unused imports from test_all.nim and bench_all.nim Docker / CI fixes: - Fix Dockerfile.source: invalid nimlang/nim:2.2.10-alpine tag → 2.2.10 + ubuntu:24.04 runtime - Fix Dockerfile.source healthcheck (--spider → -qO- for 200 OK) - Fix Dockerfile run comment ports (9470/9471 → 9912/9913) - Fix scripts/docker-run.sh healthcheck port (9470 → 9912) - Add ARG BUILD_DATE/VCS_REF to Dockerfile for docker-build.sh - Update all version strings 1.1.4 → 1.1.6 across nimble/docker/source/docs	2026-05-19 12:13:33 +03:00
dimgigov	8afa998516	feat: rate limiting, shared DB instance, TCP_NODELAY fix ... - Add rate limiter to TCP and HTTP servers - Share single LSMTree between TCP and HTTP servers - Replace unsafe cast[string]/cast[seq[byte]] with safe helpers - Stabilize gossip UDP socket with exponential backoff - Fix TCP_NODELAY via low-level setSockOptInt(IPPROTO_TCP) (asyncnet.setSockOpt(OptNoDelay) uses SOL_SOCKET and fails) - Apply TCP_NODELAY to server and websocket sockets	2026-05-18 19:33:40 +03:00
dimgigov	7e6a45e6b7	Стабилизация на storage слоя — 6 фази ... Фаза 1: SSTable Integrity - SSTable v3 формат с CRC32 footer (data/index/bloom CRC) - Нов модул storage/crc32.nim (zero-dep, IEEE 802.3) - verifySSTable() — проверка на magic, version, CRC - loadSSTable strict mode — отхвърля корумпирани файлове - newLSMTree логва [WARN] при corrupt SSTables Фаза 2: baradb repair - Нов модул tools/repair.nim - Сканира SSTables, проверява CRC, премества битите в corrupt/ - WAL replay (CrashRecovery) за възстановяване на данни - CLI: ./baradadb repair --data-dir=... [--dry-run] Фаза 3: MANIFEST File - Atomic write MANIFEST (tmp + rename) - newLSMTree зарежда от MANIFEST, fallback към walkDir - checkStorageConsistency() — orphan/missing detection - flushUnsafe и compaction записват MANIFEST Фаза 4: WAL Rotation & Incremental Backup - WAL segment rotation при 64MB (wal_archive/wal.000NNN.log) - maybeRotate() на всеки 1000 записа + при flush - backup incremental — архивира MANIFEST + SSTables + WAL - CRC verification на SSTables преди архивиране Фаза 5: Online Consistent Backup - checkpoint() — freeze memtable + flush + WAL rotate - ./baradadb checkpoint — offline consistent snapshot - backup --online — checkpoint + incremental backup Фаза 6: SSTable Version Migration - SSTable.fileVersion поле - listLegacySSTables() — намира v1/v2 файлове - migrateSSTable() — пренаписва към v3 (tmp + rename) - ./baradadb migrate [--dry-run] — offline migration Документация: - Обновени docs/en/backup.md, docs/bg/backup.md - Обновени docs/en/storage.md, docs/bg/storage.md - Добавени тестове в tests/test_all.nim	2026-05-18 15:04:49 +03:00
dimgigov	132e8c7a31	Phase 5: Storage fuzz tests + WAL entryCount fix + SSTable overflow checks + compaction sizeBytes accuracy ... - WAL: entryCount is now accurate after restart by counting existing entries - LSM writeSSTable: adds uint32 overflow checks for key/value/entry counts - Compaction: sizeBytes now uses actual file size instead of rough guess - Fuzz tests: 14 new tests covering WAL truncation, LSM put/get/delete, recovery, scan, overwrites, SSTable roundtrip, corruption, tombstones	2026-05-18 12:34:26 +03:00
dimgigov	3f29c519d4	Storage hardening: btree size fix, WAL entryCount note, LSM put error handling ... - btree.nim: insertNonFull returns bool; size only increments for new keys - wal.nim: comment about entryCount reset on append - lsm.nim: recovery/put/delete/putUnsafe raise IOError if put fails after flush	2026-05-18 12:14:30 +03:00
dimgigov	8596cea548	Stabilization: replication semi-sync, MVCC aborted deleters, SSTable level, SCRAM fix ... P0-P1 fixes: - replication.nim: rmSync/rmSemiSync now wait for replica ACKs - mvcc.nim: aborted deleters no longer hide records (return true for visibility) - storage/lsm.nim: SSTable level persisted in header (version 2, backward compat) - protocol/auth.nim: removed insecure SCRAM raw hash fallback Build: 0 warnings, 0 errors. All tests pass.	2026-05-18 12:04:45 +03:00
dimgigov	47c4393a7e	Complete plan execution: gossip/raft bounds, deadlock fix, JSON escaping, memtable limits ... Remaining plan items: - 1.4 hybrid_search: JSON built with std/json instead of string concat - 2.3 lsm.nim: reject oversized memtable entries - 3.3 gossip.nim: bounds checking for idLen/nodeCount/hostLen - 3.4 raft.nim: max length caps for cmdLen/dataLen/logLen - 4.3 deadlock.nim: per-path seq stack instead of global parent table	2026-05-18 11:42:46 +03:00
dimgigov	967c0855a5	Bug fixes: composite PK, nl_to_sql sandbox, FK check, SQL injection, storage correctness ... Critical fixes: - Composite PK: execInsert + validateConstraints use all PK columns - nl_to_sql: non-SELECT SQL no longer executed directly during validation - FK check: removed O(N) scanMemTable fallback, uses db.get() with SSTables - exprToSql: nkIdent wrapped in quotes to prevent SQL injection - restoreSchema: try/except around tokenize/parse for crash resilience - recovery.nim: lastTxnId tracking + putUnsafe/deleteUnsafe without WAL - SCRAM: verifyClientProof length check + DefaultIterationCount restored Storage fixes: - lsm.nim: SSTable sort order fixed (ascending), close() flushes all memtables - compaction.nim: tombstones preserved during compaction - wal.nim: header written for empty existing files, readEntries checks magic - btree.nim: B+ tree leaf split keeps boundary key - bloom.nim: deserialize raises on short data - mmap.nim: bounds checks for adviseWillNeed/DontNeed + posix.close() Protocol fixes: - zerocopy.nim: readString bounds check - wire.nim: deserializeValue 32-bit underflow check - auth.nim: JWT JSON escaping for claims - server.nim: readUint32BE bounds check + specific exception handling - raft.nim: readData checks + specific exception handling Tests: - Added Composite Primary Key test suite (4 tests) Build: 0 warnings, 0 errors	2026-05-18 11:33:11 +03:00
dimgigov	a28c845476	Bump version to 1.1.4 across all components	2026-05-17 20:57:19 +03:00
dimgigov	1e38e29f25	fix(security): SQL injection in Python/JS clients + bare except + session leak ... - Fix SQL injection vulnerabilities in chat_history.py, langchain_store.py, rag_pipeline.py, baradb_langchain.js by switching to parameterized queries - Replace dangerous bare except: with except CatchableError:/ValueError: in llm.nim, embed.nim, cypher.nim, mcp/server.nim, executor.nim - Fix session variable leak in MCP handleVectorSearch/handleSchemaInspect - Build: 0 errors, all tests pass	2026-05-17 16:58:05 +03:00
dimgigov	a0c5ce8598	feat: Session 11 finale — similarity_nodes, node2vec, Cypher layer ... - similarity_nodes(graph, metric): Jaccard / Adamic-Adar similarity between all node pairs - node2vec_embed(graph, dimensions): Random-walk based graph embeddings - cypher(query): Translate Cypher MATCH...RETURN → BaraQL GRAPH_TABLE SQL - Fixed recursive lock deadlock in similarity functions - Fixed graph start node parsing (pattern variable vs numeric ID) - Added sequtils import to graph engine - All 340+ existing tests pass	2026-05-17 16:00:15 +03:00
dimgigov	e23b1d61d2	feat: Session 12 AI Agents & NL->SQL ... - src/barabadb/ai/llm.nim: LLM client for NL->SQL generation - Supports OpenAI-compatible and Ollama APIs - Configurable via BARADB_LLM_ENDPOINT, BARADB_LLM_MODEL, BARADB_LLM_API_KEY - extractSQL() parses SQL from LLM responses (handles markdown blocks) - Temperature 0.1 for deterministic SQL generation - nl_to_sql() SQL function: natural language -> SQL - Schema-aware prompt with table column definitions + indexes + RLS - Query validation layer: wraps generated SQL in LIMIT 0 subquery - Self-correction loop: on error, feeds error back to LLM for fix - Tenant-aware: respects current session variables - schema_prompt() SQL function: generates DDL + sample data + indexes - Returns full CREATE TABLE statement with column types and constraints - Includes up to 5 sample rows for context - Lists indexes, RLS policies, foreign keys - Perfect for feeding into LLM context - All 340+ existing tests pass	2026-05-17 15:38:14 +03:00
dimgigov	13bc17cfa8	feat: 10.1.4 Chunking + embedding pipeline ... - New modules: src/barabadb/ai/chunk.nim (text chunking) and embed.nim (HTTP embedding client) - chunk() SQL function: returns JSON array of chunks with configurable size/overlap - embed_text() SQL function: calls external embedding API (OpenAI/Ollama compatible) - Auto-embedding on INSERT: when VECTOR column is null but TEXT column is populated, generates embeddings via configured embedder - Configurable via env vars: BARADB_EMBED_ENDPOINT, BARADB_EMBED_MODEL, BARADB_EMBED_API_KEY - All 340+ existing tests pass	2026-05-17 15:26:24 +03:00
dimgigov	8a395225c0	feat: Session 10.3 MCP Server + Session 11 Graph Engine Deep Integration ... MCP Server (10.3): - STDIO JSON-RPC 2.0 transport with 3 AI tools: query, vector_search, schema_inspect - Multi-tenant session vars (tenant_id, user_id) with RLS support - Standalone binary: build/baramcp - Tested with all 3 tools + parameterized queries + vector search + schema inspect Graph Engine Deep Integration (Session 11): - CREATE GRAPH / DROP GRAPH DDL support - Graph engine wired to SQL executor via GRAPH_TABLE() function - 7 algorithms: BFS, DFS, PageRank, ShortestPath, Dijkstra, Louvain, Community - INSERT into _nodes/_edges tables auto-syncs with native Graph adjacency lists - Optional MATCH pattern, ALGORITHM, START, END, MAXDEPTH in GRAPH_TABLE syntax - All 340+ existing tests pass	2026-05-17 15:18:31 +03:00
dimgigov	67965ffa8b	feat(hybrid): Session 10.1.3 — Metadata pre-filtering in vector search ... - Populate HNSW metadata with all relational columns during INSERT and CREATE INDEX - Add doHybridSearchFiltered() using searchWithFilter() for HNSW pre-filtering - Add SQL function hybrid_search_filtered(table, vec_col, text_col, query, vector, k, filter_col, filter_val) - Enforce k-limit on both doHybridSearch and doHybridSearchFiltered results - 2 tests: tenant isolation + empty filter fallback	2026-05-17 13:41:30 +03:00
dimgigov	836d30d84a	feat(hybrid): Session 10.1 — Hybrid RAG Search with RRF reranking ... - Add searchEx() to vector engine returning metadata - Add reciprocalRankFusion(), doHybridSearch(), findRealIdByDocId() helpers - Add SQL functions: hybrid_search(), hybrid_search_ids(), rerank() - Fix CREATE INDEX HNSW docId to use hash(fullKey) matching INSERT - 5 tests covering hybrid search, ids, RRF ranking, rerank, missing indexes	2026-05-17 13:30:19 +03:00
dimgigov	2e0969245c	feat(fk): Foreign Key Enforcement — ON DELETE/UPDATE CASCADE, SET NULL, RESTRICT ... - Add tkRestrict token to lexer - Parse ON DELETE CASCADE/SET NULL/RESTRICT and ON UPDATE CASCADE/SET NULL/RESTRICT in both table-level and column-level FK constraints - Add fkOnDelete/fkOnUpdate to ColumnDef and ForeignKeyDef - Fix table-level FK constraint application (third pass after columns are created) - Implement enforceFkOnDelete, enforceFkOnUpdate, enforceFkOnChildUpdate helpers - Wire FK enforcement into DELETE and UPDATE execution paths - Add 9 regression tests covering all FK actions	2026-05-17 13:02:39 +03:00
dimgigov	dc4ad86ee1	Сесия 9: Production Hardening — prop tests (58 invariant-а), fuzz tests (35 сценария), thread safety (SharedLock ref)	2026-05-17 12:37:50 +03:00
dimgigov	19fa760604	Week 4: Production Hardening — Thread Safety, Fuzz Tests, Property-Based Tests, NimForum Smoke Test ... Thread Safety: - Add lock to BTreeIndex (insert/get/scan/remove/contains) - Add lock to InvertedIndex (addDocument/removeDocument/search/bm25/tfidf/fuzzy/regex) - Add ctxLock to ExecutionContext for DDL/schema metadata, autoIncCounters, sequences - Deep-copy sessionVars in cloneForConnection() to isolate per-connection state - Fix activeConnections check-then-act race with Lock Wire Protocol Hardening: - Fix RangeDefect in deserializeValue for invalid FieldKind bytes - Fix IndexDefect in fkFloat32/fkFloat64 by reusing readUint32/readUint64 - Fix int32/int64 cast from uint32/uint64 to use cast[] instead of constructor - Fix parseHeader to validate MsgKind instead of holey enum cast Property-Based Tests (tests/prop_test.nim): - 8 invariants for evalExprValue: literal types, commutativity, identity, double negation, NULL propagation Wire Protocol Fuzz (tests/fuzz_test.nim): - 7 fuzz tests: random bytes deserialization, truncated buffers, MsgKind casts, parseHeader logic, message roundtrips NimForum Adapter Smoke Test (tests/nimforum_smoke_test.nim): - Full TCP server lifecycle with adapter CRUD and parameterized queries Test Results: 423 tests pass, 0 failures	2026-05-17 11:57:55 +03:00
dimgigov	6021bfcb10	Сесия 9: Седмица 3 — JOIN Performance ... - Добавен IRJoinStrategy enum (nestedLoop, hash, indexNestedLoop) - Hash Join: build hash table върху по-малката страна, probe от другата - Index Nested Loop Join: използва B-Tree индекс за point lookup - Query planner chooseJoinStrategy избира стратегия според наличие на индекс - LEFT/RIGHT/FULL JOIN fallback-ват към Nested Loop - PK индексите се игнорират за INL (само explicit индекси) - Benchmark: JOIN 10K ~115ms (Hash), ~90ms (Index NL) - 5 нови теста за join performance и planner избор Тестове: 316 — 0 failures Build: 0 warnings	2026-05-17 10:50:12 +03:00