12 Commits

Author	SHA1	Message	Date
dimgigov	1c42eff7ef	fix(vector): use float64 accumulators for distance functions ... Vector distance functions (cosine, euclidean, dot, manhattan) returned float64 but accumulated sums in float32, causing precision loss in SQL results (e.g. sqrt(2) truncated to float32). Use float64 accumulators and casts throughout. docs(bugs): mark BUG-023 and BUG-024 as fixed The code already cleans up pendingAcks and passes oldAssignments from rebalance to migrateData; update BUGS.md to match the implementation.	2026-06-12 23:06:19 +03:00
dimgigov	ef264d7d69	feat: add unified search engine — HNSW heap-opt, segment index, boolean/phrase/ngram/facet ... New src/barabadb/search/ module with 9 components: - priority_queue.nim: BoundedHeap for O(log n) search - hnsw_opt.nim: heap-based searchLayer (2.4x faster, 92-99% recall@10) - inverted.nim: segment-based index with soft-delete and compaction - phrase.nim: positional phrase + proximity search - boolean.nim: recursive descent parser (AND/OR/NOT/ranges/wildcards) - ngram.nim: trigram index for O(1) fuzzy/prefix/wildcard - stemmer.nim: Porter2 stemmers (EN/BG/DE/FR/RU) - facet.nim: faceted search with filter pushdown - engine.nim: UnifiedSearchEngine combining all search types Performance (dim=128, efConstruction=200): N=1K: 0.30ms search, 99.6% recall@10 N=10K: 1.09ms search, 92.6% recall@10 N=50K: 2.26ms search, 75.5% recall@10 Includes search benchmarks (benchmarks/search_bench.nim), updated docs (en/bg fts.md, en/bg search.md), and crossmodal engine integration.	2026-05-30 13:42:08 +03:00
dimgigov	965ed2f675	perf: optimize FTS and HNSW engines + real PostgreSQL benchmarks ... FTS Engine (src/barabadb/fts/engine.nim): - Fix bm25Score doing O(n) linear scan per document - Cache IDF per token instead of recomputing for each doc - Use entry.termFreq directly instead of searching postings again - Result: FTS search +438% (249 -> 1360 queries/s) HNSW Vector Engine (src/barabadb/vector/engine.nim): - Optimize distance functions with float32 + 4x loop unrolling - Rewrite searchLayer: swap+pop instead of O(n) del, track worst-nearest instead of sorting nearest on every iteration - Result: HNSW insert +117% (245 -> 543 ops/s), search 2.2x faster Benchmarks: - Add real PostgreSQL comparison script (benchmarks/pg_bench.py) - Add report generator (benchmarks/generate_report.py) - Fix compare.nim cpuTime() bug (was dividing by 1M incorrectly) - Add nimble tasks: bench_pg, bench_report Docs: - Update README.md and docs/en/performance.md with real measured numbers - Add benchmarks/REAL_COMPARISON.md Version bump: 1.1.7 -> 1.1.8	2026-05-29 17:11:22 +03:00
dimgigov	836d30d84a	feat(hybrid): Session 10.1 — Hybrid RAG Search with RRF reranking ... - Add searchEx() to vector engine returning metadata - Add reciprocalRankFusion(), doHybridSearch(), findRealIdByDocId() helpers - Add SQL functions: hybrid_search(), hybrid_search_ids(), rerank() - Fix CREATE INDEX HNSW docId to use hash(fullKey) matching INSERT - 5 tests covering hybrid search, ids, RRF ranking, rerank, missing indexes	2026-05-17 13:30:19 +03:00
dimgigov	80d57a36bd	fix: rate limiter, graph/vector/query/server бъгове ... Поправени проблеми: - Rate limiter: globalRate вече се enforce-ва; cleanupStaleClients() за memory leak - Graph engine: addEdge проверява дали src/dst nodes съществуват - Vector engine: distance() хвърля ValueError при dimension mismatch - Query: aggregate + * на empty result вече не crash-ва - Query: exprToSql funcCall вече работи с 0 аргумента - Query: INSERT trigger вече не crash-ва при празни VALUES - Server: recvWithTimeout() за всички network reads - Server: защита от negative activeConnections 292 теста, 0 failure-а.	2026-05-12 23:15:09 +03:00
dimgigov	6aaabb518d	fix: високо и средно приоритетни бъгове в query, protocol, storage, vector, fts ... Поправени проблеми: - Wire protocol: bounds checking, max length limits, recursion depth limit - SQL injection: escape-ване на quotes в exprToSql string literals - ReDoS: escape-ване на regex metachars в LIKE/ILIKE - Stale BTree indexes: добавен BTree.remove() + изтриване при UPDATE/DELETE - SSL: quoteShell за всички shell команди с пътища - Boolean literals: parser вече разпознава tkTrue/tkFalse - Unary minus: lowerExpr map-ва ukNeg към irNeg (вместо irNot) - Non-aggregate UDFs: lowerExpr създава irekFuncCall вместо NULL literal - UTF-8 FTS: tokenize използва runes вместо байтове - HNSW: добавен Lock за thread-safe insert/search 292 теста, 0 failure-а.	2026-05-12 23:02:54 +03:00
dimgigov	2a13066a0d	feat: real HNSW search with hierarchical graph navigation ... - Implement proper HNSW insert with level-based neighbor linking - Implement searchLayer with greedy beam search - Implement hierarchical search: top-level→level 0 with ef beam - Add bidirectional neighbor pruning (maxM limit) - searchWithFilter now uses HNSW + post-filtering - Achieves ~99% recall@10 on 2K vectors (dim=128) - All 214 tests pass	2026-05-06 03:25:04 +03:00
dimgigov	d80ec4e449	feat: zero-copy serialization, adaptive query, distributed txns, vector batch/rebuild — 222 tests ... Zero-Copy Serialization: - Direct memory buffer with schema-based field offsets - Write/read int32/int64/float/bool/string without copies - FastMem copy operations (fastCopy, fastCopyFrom, slice) - ZcTable for batch columnar records Adaptive Query Execution: - Cardinality estimation with exponential moving average - Reoptimize triggers when actual/estimated row ratio exceeds threshold - Plan caching with hash-based lookup - Execution context with parallelism hints and explain Distributed Transactions: - Two-phase commit across multiple nodes - Saga pattern with step-by-step execute/compensate - DistTxnManager with cleanup lifecycle Vector Batch Operations: - batchInsert/batchSearch for HNSW and IVF-PQ - IndexWatcher with auto-rebuild based on unindexed count and ratio - Rebuild statistics tracking 26 new tests (222 total, all passing)	2026-05-06 01:57:28 +03:00
dimgigov	eecd846df9	feat: UDF stdlib, SIMD vector ops, benchmarks — 162 tests ... - User Defined Functions: register/call/deregister, stdlib (math, string, type conversion, array) - SIMD vector operations: unrolled dot product, L2, cosine, manhattan, normalize, batch distance - TopK and batch distance for vector search - Performance benchmarks (LSM, B-Tree, HNSW, FTS, Graph) - All roadmap phases marked complete except cluster/optimizations tail - 26 new tests (162 total, all passing)	2026-05-06 01:33:51 +03:00
dimgigov	67213826a8	feat: compaction, page cache, WebSocket, rate limiter, TF-IDF, fuzzy search, regex, metadata filter — 73 tests ... - SSTable compaction: size-tiered strategy, level-based scheduling - Page cache: LRU eviction, hit rate tracking, capacity management - WebSocket: full duplex streaming, frame encoding, ping/pong - Rate limiter: token bucket + sliding window algorithms - FTS: TF-IDF ranking, Levenshtein fuzzy matching, wildcard regex - Vector: metadata filtering on HNSW search - 16 new tests (73 total, all passing)	2026-05-06 01:03:58 +03:00
dimgigov	07a37d8e78	feat: B-Tree, columnar engine, IR/type checker, connection pool, JWT auth, quantization, Louvain, pattern matching — 57 tests ... - B-Tree index: insert, get, scan range, duplicate keys - Columnar engine: batch ops, RLE/dict encoding, GroupBy, aggregates - IR (Intermediate Representation): plan nodes, expressions, type checker - Connection pool: load-balanced eviction, min/max connections - JWT authentication with token verify and claims parsing - Vector quantization: scalar 8-bit/4-bit, product quantization, binary - Louvain community detection algorithm - Graph pattern matching (subgraph isomorphism) - 18 new test suites (57 total, all passing)	2026-05-06 00:57:30 +03:00
dimgigov	6935889877	feat: initial BaraDB — multimodal database engine in Nim ... - LSM-Tree storage engine with WAL, bloom filter, MemTable - BaraQL query language: lexer (80+ tokens), recursive descent parser, AST - Vector engine: HNSW + IVF-PQ indexes, 4 distance metrics - Graph engine: adjacency list, BFS/DFS, Dijkstra, PageRank - Full-Text Search: inverted index, BM25 ranking, stemming, stop words - Type system: 17 types (int/float/string/uuid/json/vector/...) - Async TCP server - 21 passing tests	2026-05-06 00:22:12 +03:00