dimgigov
ef264d7d69
feat: add unified search engine — HNSW heap-opt, segment index, boolean/phrase/ngram/facet
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
New src/barabadb/search/ module with 9 components:
- priority_queue.nim: BoundedHeap for O(log n) search
- hnsw_opt.nim: heap-based searchLayer (2.4x faster, 92-99% recall@10)
- inverted.nim: segment-based index with soft-delete and compaction
- phrase.nim: positional phrase + proximity search
- boolean.nim: recursive descent parser (AND/OR/NOT/ranges/wildcards)
- ngram.nim: trigram index for O(1) fuzzy/prefix/wildcard
- stemmer.nim: Porter2 stemmers (EN/BG/DE/FR/RU)
- facet.nim: faceted search with filter pushdown
- engine.nim: UnifiedSearchEngine combining all search types
Performance (dim=128, efConstruction=200):
N=1K: 0.30ms search, 99.6% recall@10
N=10K: 1.09ms search, 92.6% recall@10
N=50K: 2.26ms search, 75.5% recall@10
Includes search benchmarks (benchmarks/search_bench.nim), updated docs
(en/bg fts.md, en/bg search.md), and crossmodal engine integration.
2026-05-30 13:42:08 +03:00
dimgigov
965ed2f675
perf: optimize FTS and HNSW engines + real PostgreSQL benchmarks
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
FTS Engine (src/barabadb/fts/engine.nim):
- Fix bm25Score doing O(n) linear scan per document
- Cache IDF per token instead of recomputing for each doc
- Use entry.termFreq directly instead of searching postings again
- Result: FTS search +438% (249 -> 1360 queries/s)
HNSW Vector Engine (src/barabadb/vector/engine.nim):
- Optimize distance functions with float32 + 4x loop unrolling
- Rewrite searchLayer: swap+pop instead of O(n) del, track worst-nearest
instead of sorting nearest on every iteration
- Result: HNSW insert +117% (245 -> 543 ops/s), search 2.2x faster
Benchmarks:
- Add real PostgreSQL comparison script (benchmarks/pg_bench.py)
- Add report generator (benchmarks/generate_report.py)
- Fix compare.nim cpuTime() bug (was dividing by 1M incorrectly)
- Add nimble tasks: bench_pg, bench_report
Docs:
- Update README.md and docs/en/performance.md with real measured numbers
- Add benchmarks/REAL_COMPARISON.md
Version bump: 1.1.7 -> 1.1.8
2026-05-29 17:11:22 +03:00
dimgigov
cd46edcb67
Bump version to 1.1.6; fix storage bugs, tests, CI pipeline and Docker config
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Storage fixes:
- Fix bloom filter hash2 OverflowDefect (uint64 instead of int Hash)
- Fix MemTable.put size calculation on overwrite (was leaking size)
- Fix fuzz_test duplicate-key issues in LSM delete and SSTable tests
Test fixes:
- Fix stress_test.nim: replace deprecated threadpool with std/threads
- Fix fuzz_test.nim: add missing imports (tables, algorithm, sets)
- Fix fuzz_test.nim: var sst for close() compatibility
- Remove unused imports from test_all.nim and bench_all.nim
Docker / CI fixes:
- Fix Dockerfile.source: invalid nimlang/nim:2.2.10-alpine tag → 2.2.10 + ubuntu:24.04 runtime
- Fix Dockerfile.source healthcheck (--spider → -qO- for 200 OK)
- Fix Dockerfile run comment ports (9470/9471 → 9912/9913)
- Fix scripts/docker-run.sh healthcheck port (9470 → 9912)
- Add ARG BUILD_DATE/VCS_REF to Dockerfile for docker-build.sh
- Update all version strings 1.1.4 → 1.1.6 across nimble/docker/source/docs
2026-05-19 12:13:33 +03:00
dimgigov
cf2aba104f
Phase 5 complete: B-tree property tests + benchmark regression suite
...
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
- prop_test.nim: 11 new property-based B-Tree invariants
(size accuracy, get roundtrip, scan ordering, range correctness,
contains, remove, large order, duplicates, empty tree, interleaved ops)
- bench_all.nim: JSON-based benchmark result tracking with regression
comparison against previous runs; each benchmark reports ops/sec
delta vs last run
2026-05-18 12:52:44 +03:00
dimgigov
5dba7b5699
feat: Phase 1 — SSTable persistence, README honesty, benchmark fix
...
- Add Current Status / Limitations section to README
- Fix benchmark compilation (Duration.ticks → inNanoseconds)
- Implement real SSTable binary format with write/read/mmap support
- Add BloomFilter serialize/deserialize for disk storage
- Fix mmap.nim to use posix.open instead of system.open
- New PLAN.md with improvement roadmap
- All 214 tests pass
2026-05-06 03:16:39 +03:00
dimgigov
da7e3afb44
feat: comparative benchmarks, Python/JS client libraries
...
Comparative Benchmarks ():
- KV write/read comparison (vs Redis)
- B-Tree insert/scan (vs PostgreSQL)
- Vector HNSW search (vs pgvector)
- FTS index/search (vs PG GIN)
- Graph BFS traversal (vs PG CTE)
- SIMD vector distance (vs numpy)
- Bar chart visualization with speedup metrics
- Overall: BaraDB 1.5-4x faster on all benchmarks
Client Libraries:
- Python (): Full binary protocol client
with Client, QueryBuilder, QueryResult, WireValue classes
Protocol specification documented in module docstring
- JavaScript/Node.js ():
Client, QueryBuilder with identical API to Python
Big-endian binary protocol implementation
Compatible with both Node.js and browser
2026-05-06 02:24:54 +03:00
dimgigov
eecd846df9
feat: UDF stdlib, SIMD vector ops, benchmarks — 162 tests
...
- User Defined Functions: register/call/deregister, stdlib (math, string, type conversion, array)
- SIMD vector operations: unrolled dot product, L2, cosine, manhattan, normalize, batch distance
- TopK and batch distance for vector search
- Performance benchmarks (LSM, B-Tree, HNSW, FTS, Graph)
- All roadmap phases marked complete except cluster/optimizations tail
- 26 new tests (162 total, all passing)
2026-05-06 01:33:51 +03:00
dimgigov