273 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dimgigov	5858a9da17	docs: UNIQUE index enforcement plan	2026-07-30 16:59:26 +03:00
dimgigov	8d2d97ad94	feat(persist): standalone B-tree indexes survive restart	2026-07-30 16:53:25 +03:00
dimgigov	fcb6237caf	docs: B-tree index persistence plan	2026-07-30 16:46:04 +03:00
dimgigov	8df0e02d3f	docs: engine persistence (C1) done	2026-07-30 16:36:49 +03:00
dimgigov	821b668d87	fix(persist): unnamed FTS/HNSW indexes survive restart (nameless replay DDL)	2026-07-30 16:27:36 +03:00
dimgigov	214b9cf346	fix: DROP INDEX/TABLE clean up FTS/HNSW indexes and their schema keys	2026-07-30 16:20:23 +03:00
dimgigov	ce6e7aa707	feat(persist): graphs survive restart (rebuild from backing tables)	2026-07-30 16:11:52 +03:00
dimgigov	214e44abd7	feat(persist): HNSW vector indexes survive restart	2026-07-30 16:05:25 +03:00
dimgigov	6703ca2b29	feat(persist): FTS indexes survive restart (schema key + restore replay)	2026-07-30 15:56:48 +03:00
dimgigov	9088dc1381	docs: engine persistence (C1) spec + implementation plan	2026-07-30 15:47:03 +03:00
dimgigov	ba7cf195c6	chore: stop tracking compiled test binaries (bugfix_test, nimforum_smoke_test)	2026-07-30 15:36:41 +03:00
dimgigov	62504aa348	docs(exec): document module layering after executor split	2026-07-30 15:17:31 +03:00
dimgigov	b282a59fb5	refactor(exec): slim executor.nim to dispatcher + hook wiring	2026-07-30 15:12:41 +03:00
dimgigov	a5976f648a	refactor(exec): extract IR plan execution into exec/plan_exec.nim	2026-07-30 15:02:34 +03:00
dimgigov	a147d4b620	refactor(exec): extract window functions into exec/window.nim	2026-07-30 14:52:17 +03:00
dimgigov	e618266325	refactor(exec): extract triggers/constraints into exec/triggers.nim	2026-07-30 14:43:49 +03:00
dimgigov	70c7297e33	refactor(exec): extract FK enforcement into exec/fk.nim	2026-07-30 14:35:15 +03:00
dimgigov	70b7ec7f08	refactor(exec): extract DML row operations into exec/dml.nim	2026-07-30 14:28:33 +03:00
dimgigov	e28c1f4896	refactor(exec): extract table scans into exec/scan.nim	2026-07-30 14:19:11 +03:00
dimgigov	f97e72314f	refactor(exec): extract RLS/privileges into exec/rls.nim	2026-07-30 14:11:08 +03:00
dimgigov	26475058bf	refactor(exec): extract AST->IR lowering into exec/lower.nim	2026-07-30 14:05:16 +03:00
dimgigov	b5e9636fd2	refactor(exec): extract expression evaluation + hybrid search into exec/eval.nim	2026-07-30 13:52:56 +03:00
dimgigov	2b8cc98348	refactor(exec): extract migration storage into exec/migrations.nim	2026-07-30 13:41:52 +03:00
dimgigov	2efcddba19	refactor(exec): extract param binding into exec/params.nim	2026-07-30 13:36:39 +03:00
dimgigov	46e3d7f51e	refactor(exec): extract join/vector helpers into exec/helpers.nim	2026-07-30 13:27:17 +03:00
dimgigov	08fb391ac1	refactor(exec): extract context management into exec/context.nim	2026-07-30 13:18:10 +03:00
dimgigov	2d09edd9f7	fix+refactor: soft keywords as identifiers, full test wiring, ORC crash docs ... - parser: clause keywords (header, format, status, user, csv, ...) now work as identifiers everywhere; IMPORT/EXPORT accept FORMAT csv/HEADER true - nimble test + CI run all 13 test suites (650 checks green) - ExecutionContext.registry is now {.cursor.} (breaks registry<->ctx cycle) - ORC crash reproduced and bisected (tests/orc_repro.py); ARC stays the MM	2026-07-30 13:11:28 +03:00
dimgigov	ed5a71913c	fix: harden exception handling, break ARC cycles, sync license/client ... Replace bare except: with CatchableError across storage, query, Raft, backup, and protocol code so Defects are not swallowed. Break uncollectable ARC cycles in server shard/gossip callbacks via Server-owned refs and cursor locals. Align package license with LICENSE (BSD-3-Clause), sync README version, and point test_all at the canonical clients/nim baradb client (parseConnectionString + aliases).	2026-07-23 01:07:51 +03:00
dimgigov	8db5cfe7e1	feat: harden storage, schema persistence, fair benches, fix wire crash ... Core storage: hash MemTable, WAL group commit, L0 compaction rebuild, reader-writer lock, and a global StorageGate so HTTP workers and TCP share the LSM safely under multi-thread access. Schema: durable CREATE/ALTER/DROP under _schema:tables:* with full LSM restore on open. Executor types/values/schema split into query/exec/. Wire protocol: switch default MM to ARC — ORC cycle collector segfaulted after ~20 async INSERTs. Fair multi-tier benchmarks (SQLite/HTTP/wire/PG) and honesty docs for mixed-tier comparisons.	2026-07-18 16:55:50 +03:00
dimgigov	aa4ab11210	feat: canonical Nim client refactor with typed rows, pool, and allographer wrapper ... - Extract wire protocol into clients/nim/src/baradb/wire.nim - Add BaraError exception hierarchy - Refactor BaraClient/SyncClient with typedRows, AsyncLock request queue, timeouts, TLS config - Add BaraPool and optional HTTP fallback - Add mock-server wire and pool unit tests - Bump baradb nimble package to 1.2.0 - Make nim-allographer depend on canonical baradb client - Use typed rows in allographer toJson - Deprecate src/barabadb/client/client.nim - Update docs/en/clients.md and clients/nim/README.md	2026-06-18 21:29:55 +03:00
dimgigov	1c42eff7ef	fix(vector): use float64 accumulators for distance functions ... Vector distance functions (cosine, euclidean, dot, manhattan) returned float64 but accumulated sums in float32, causing precision loss in SQL results (e.g. sqrt(2) truncated to float32). Use float64 accumulators and casts throughout. docs(bugs): mark BUG-023 and BUG-024 as fixed The code already cleans up pendingAcks and passes oldAssignments from rebalance to migrateData; update BUGS.md to match the implementation.	2026-06-12 23:06:19 +03:00
dimgigov	ef264d7d69	feat: add unified search engine — HNSW heap-opt, segment index, boolean/phrase/ngram/facet ... New src/barabadb/search/ module with 9 components: - priority_queue.nim: BoundedHeap for O(log n) search - hnsw_opt.nim: heap-based searchLayer (2.4x faster, 92-99% recall@10) - inverted.nim: segment-based index with soft-delete and compaction - phrase.nim: positional phrase + proximity search - boolean.nim: recursive descent parser (AND/OR/NOT/ranges/wildcards) - ngram.nim: trigram index for O(1) fuzzy/prefix/wildcard - stemmer.nim: Porter2 stemmers (EN/BG/DE/FR/RU) - facet.nim: faceted search with filter pushdown - engine.nim: UnifiedSearchEngine combining all search types Performance (dim=128, efConstruction=200): N=1K: 0.30ms search, 99.6% recall@10 N=10K: 1.09ms search, 92.6% recall@10 N=50K: 2.26ms search, 75.5% recall@10 Includes search benchmarks (benchmarks/search_bench.nim), updated docs (en/bg fts.md, en/bg search.md), and crossmodal engine integration.	2026-05-30 13:42:08 +03:00
dimgigov	965ed2f675	perf: optimize FTS and HNSW engines + real PostgreSQL benchmarks ... FTS Engine (src/barabadb/fts/engine.nim): - Fix bm25Score doing O(n) linear scan per document - Cache IDF per token instead of recomputing for each doc - Use entry.termFreq directly instead of searching postings again - Result: FTS search +438% (249 -> 1360 queries/s) HNSW Vector Engine (src/barabadb/vector/engine.nim): - Optimize distance functions with float32 + 4x loop unrolling - Rewrite searchLayer: swap+pop instead of O(n) del, track worst-nearest instead of sorting nearest on every iteration - Result: HNSW insert +117% (245 -> 543 ops/s), search 2.2x faster Benchmarks: - Add real PostgreSQL comparison script (benchmarks/pg_bench.py) - Add report generator (benchmarks/generate_report.py) - Fix compare.nim cpuTime() bug (was dividing by 1M incorrectly) - Add nimble tasks: bench_pg, bench_report Docs: - Update README.md and docs/en/performance.md with real measured numbers - Add benchmarks/REAL_COMPARISON.md Version bump: 1.1.7 -> 1.1.8	2026-05-29 17:11:22 +03:00
dimgigov	42043f3946	v1.1.7: deep security & reliability audit — 33 bugs fixed ... Critical (5): - Reject empty JWT secret when authEnabled (server.nim) - Fix 2PC marking uncontacted participants as prepared/committed (disttxn.nim) - Fix Raft commit index calculation for even-sized clusters (raft.nim) - Fix REP/DISTTXN protocol auth bypass (server.nim) - Fix HTTP backup/restore path traversal (httpserver.nim) High (11): - Fix WAL write race with flush (lsm.nim) - Fix MVCC savepoint/rollback deep-copy writeSet (mvcc.nim) - Fix table mutation during deadlock iteration (mvcc.nim) - Fix LIMIT 0 returning all rows (executor.nim) - Fix COUNT(col) counting NULL values — 3 locations (executor.nim) - Fix EXISTS subquery lowering missing subqueryPlan (executor.nim) - Fix Raft appendEntries/applyCommitted array vs logical index (raft.nim) - Fix timing attacks on constantTimeCompare and SCRAM (auth.nim, scram.nim) - Fix B-tree leaf merge phantom separator key (btree.nim) - Fix SSL verifyPeer not applied to newContext (ssl.nim) - Fix sharding connectWithTimeout missing SO_ERROR check (sharding.nim) - Fix sync replication returning success on partial ack (replication.nim) - Fix WebSocket JWT expiration not validated (websocket.nim) Medium (13): - Fix writeSSTable partial file → tmp + atomic rename (lsm.nim) - Fix multi-CTE table loss (executor.nim) - Fix nl_to_sql DML restricted to superuser (executor.nim) - Fix unbounded plan cache — max 10000 (adaptive.nim) - Fix migration lock crash persistence — timestamp + stale detection (executor.nim) - Fix admin panel auth (httpserver.nim) - Fix MVCC unbounded txn tracking — prune in compactVersions (mvcc.nim) - Fix connection pool maxLifetime check (pool.nim) - Fix JWT JSON parser backslash escapes (auth.nim) - Fix substr(s, start) returning single char (udf.nim) - Fix loadSSTable minimum file-size check (lsm.nim) - Fix compaction mmap leak (compaction.nim) - Fix JSON injection in hybrid_search_filtered (executor.nim) Low (4): - Raft loadState logs error instead of silent discard - Replication healthCheck double-close fixed - Lexer readIdent double column counting fixed - WebSocket frame 32-bit overflow guard All 448 tests passing, 0 failures. Bump version to 1.1.7.	2026-05-29 14:17:41 +03:00
dimgigov	37a8ed52ba	deps: bump jwt-nim-baraba to v2.1.2 (security fixes & Nim 2.2 compat)	2026-05-26 13:23:54 +03:00
dimgigov	a5abb6031b	fix(backup): fix 7 bugs in backup/restore — strip-components, multi-db path, verify+log in HTTP handler, parseBackupFilename for .tar, getArchiveSize gzip -l, dead code removal	2026-05-25 20:44:14 +03:00
dimgigov	a37a62c69e	clients/nim-allographer/src/allographer/query_builder/models/baradb/baradb_exec.nim src/barabadb/core/mvcc.nim src/barabadb/query/codegen.nim src/barabadb/query/lexer.nim src/barabadb/query/parser.nim src/barabadb/query/udf.nim tests/test_all.nim	2026-05-25 18:56:02 +03:00
dimgigov	0c7847abba	docs: update backup.md with multi-database and HTTP API docs	2026-05-25 18:49:26 +03:00
dimgigov	eb62b381a1	feat: multi-database backup, HTTP API endpoints, admin panel UI ... - backup CLI: add --all-databases, --database, --data-root flags - backup format: tar.gz with databases/<name>/ + backup.json metadata - HTTP API: new endpoints /databases, /backup, /backups, /restore - HTTP API: database switching via X-Database header - Admin panel: database selector dropdown, Databases tab, Backups tab - Admin panel: backup/restore UI with fetch() to new endpoints - Update deps: hunos 1.3.0, jwt-nim-baraba 2.1.0 - Docker: backup cron uses --all-databases --data-root=/data/databases - Docs: update api-http.md, protocol.md with correct endpoints - OpenAPI spec updated to v1.1.6 Known limitation: DELETE /databases disabled due to ORC SIGSEGV in LSMTree.close(). Use SQL DROP DATABASE instead.	2026-05-25 18:46:07 +03:00
dimgigov	a9ea80a1f8	fix(prop_test): "Nil expr evaluates to NULL" test — compare Value.kind, not Value == "" ... The test was comparing a Value to a string literal which always failed. Changed to check s.kind == vkNull on both calls. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-23 22:06:37 +03:00
dimgigov	7d19e06805	fix(btree): critical seq.del vs seq.delete bug and B+ tree leaf traversal ... Root cause: seq.del(i) does swap-with-last (destroys sorted order) while seq.delete(i) preserves order. This caused massive data loss during B-tree removes. Changes: - btree.nim: Replace seq.del with seq.delete (critical data corruption fix) - btree.nim: search() traverses leaf linked list for duplicate boundary keys - btree.nim: splitChild() consolidates duplicate boundary key values before split - btree.nim: len() counts unique keys via leaf traversal - btree.nim: remove() traverses leaf linked list to remove ALL occurrences - btree.nim: Fix separator update in internal node (node.keys[i-1] not [i]) - prop_test.nim: Use set equality for interleaved test (rebalancing reorders) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-23 22:01:20 +03:00
dimgigov	b25fea4d21	fix: all 5 bugs — reserved words, cloneForConnection, SQL injection, SHOW TABLES, getRowPlain	2026-05-21 22:51:04 +03:00
dimgigov	837b4d56fc	fix(nim-allographer/baradb): critical client fixes and bug audit ... - Export hidden symbols in baradb_client.nim (socket, nextId, readQueryResponse, toString, toBytes) - Add missing imports in baradb_exec.nim (asyncnet, enums, baradb_query) - Rewrite toJson() to use wire protocol columnTypes instead of guessing types from strings - Fix aggregate functions (count, avg, sum) to handle JInt/JFloat/JString correctly - Fix paginate/fastPaginate discard issues - Add placeholder arguments for whereIn/whereNotIn/whereBetween/whereNotBetween - Add placeholder count validation in formatSql() - Fix getRowPlain() to avoid IndexDefect on empty results - Harden transactionStart/transactionEnd against server crashes - Reset placeHolder in insertSql() to avoid stale values - Fix test_query.nim to use backtick quoting for BaraDB compatibility Known server bugs documented in BARADB_CLIENT_BUGS.md	2026-05-21 22:01:55 +03:00
dimgigov	bb843b9a03	feat: migrate system + cross-DB engine + IMPORT/EXPORT syntax -- 22 files, client+server+docs	2026-05-21 19:32:14 +03:00
dimgigov	f427ba5968	fix(baradb): schema builder, insertId RETURNING, whereNull, rename bugs ... - create_column_query.nim: full RdbTypeKind -> SQL mapping (~350 lines) - create_table.nim, add_column.nim, change_column.nim: use createColumnString - insertId: use RETURNING clause instead of SELECT MAX race condition - whereNull: add missing 'symbol': 'is' key to prevent runtime crash - rename_column/rename_table: fix changeTo -> previousName - create_migration_table: use backtick quoting consistently - PLAN_BARADB.md: update status to reflect completed fixes	2026-05-21 16:06:45 +03:00
dimgigov	8a385cba6a	docs(readme): clarify this is a BaraDB fork of nim-allographer	2026-05-21 15:54:56 +03:00
dimgigov	6bfc5b3a3c	feat(clients): add nim-allographer client with BaraDB driver support ... - Pure Nim wire protocol client (async + sync, no C FFI) - Query Builder: fluent API, SQL generator, execution, transactions - Schema Builder: create/alter/drop tables, column operations - Connection pool with aging and timeout handling - ORM integration via compile-time DB_BARADB switch - Tests: test_open, test_query - Documentation: README.md, PLAN_BARADB.md	2026-05-21 15:49:11 +03:00
dimgigov	2d310a33a1	fix: resolve all 55 identified bugs across the codebase ... Comprehensive bug fix pass across all subsystems: - Critical: lexer infinite loop, WAL lock discipline, checkpoint safety, compaction verification, HMAC key truncation, healthCheck leak - High: MVCC lock safety, B-Tree delete rebalancing, Raft stale term handling, replication socket leaks and ack cleanup, sharding migration with old assignments, 2PC recovery, JWT/SCRAM auth fixes, wire protocol bounds checks - Medium: config error handling, MERGE parser completeness, IR/codegen correctness, UDF bounds checks, LIMIT cost accuracy, mmap safety - Low: timeout handling, float parsing edge cases, uint32 truncation, cache entry cleanup Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-21 10:59:51 +03:00
dimgigov	7fc6adbe47	chore: remove old planning and bug report files ... These files are outdated and only cause confusion. Keep only the current PLAN.md.	2026-05-21 09:47:40 +03:00
dimgigov	072e7a5d13	test(join): add test for duplicate columns in SELECT * ... Verify that JOIN with overlapping column names preserves both left (bare) and right (qualified) values.	2026-05-21 09:44:32 +03:00