feat: harden storage, schema persistence, fair benches, fix wire crash
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
CI / test (push) Has been cancelled
CI / verify (push) Has been cancelled
Clients CI / build-server (push) Has been cancelled
Clients CI / test-python (push) Has been cancelled
Clients CI / test-javascript (push) Has been cancelled
Clients CI / test-nim (push) Has been cancelled
Clients CI / test-rust (push) Has been cancelled
Core storage: hash MemTable, WAL group commit, L0 compaction rebuild, reader-writer lock, and a global StorageGate so HTTP workers and TCP share the LSM safely under multi-thread access. Schema: durable CREATE/ALTER/DROP under _schema:tables:* with full LSM restore on open. Executor types/values/schema split into query/exec/. Wire protocol: switch default MM to ARC — ORC cycle collector segfaulted after ~20 async INSERTs. Fair multi-tier benchmarks (SQLite/HTTP/wire/PG) and honesty docs for mixed-tier comparisons.
This commit is contained in:
@@ -0,0 +1,80 @@
|
||||
# BaraDB Benchmarks
|
||||
|
||||
## Tiers (read this first)
|
||||
|
||||
| Tier | What is measured | Fair peers |
|
||||
|------|------------------|------------|
|
||||
| **embedded** | In-process storage API | BaraDB LSM ↔ SQLite |
|
||||
| **client_server** | Network + query protocol | BaraDB HTTP / **wire** ↔ PostgreSQL |
|
||||
|
||||
**Never** quote “BaraDB is Nx faster than Postgres” using embedded LSM numbers.
|
||||
That comparison mixes tiers and is meaningless as a product claim.
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# 1) Embedded micro-benches (Nim, in-process)
|
||||
nimble bench
|
||||
# or: nim c -d:release -r benchmarks/bench_all.nim
|
||||
|
||||
# 2) Optional: start server for client_server tier (HTTP + wire)
|
||||
./build/baradadb
|
||||
|
||||
# 3) Fair multi-tier suite (Python)
|
||||
# - always: SQLite embedded (+ batch)
|
||||
# - optional: BaraDB HTTP (:9912), wire TCP (:9472), PostgreSQL
|
||||
python3 benchmarks/fair_bench.py
|
||||
|
||||
# 3) Markdown report
|
||||
nimble bench_report
|
||||
# or: python3 benchmarks/generate_report.py --fair
|
||||
```
|
||||
|
||||
Outputs:
|
||||
|
||||
- `benchmark_results.json` — Nim embedded suite
|
||||
- `fair_benchmark_results.json` — multi-tier fair suite
|
||||
- `benchmarks/FAIR_COMPARISON.md` — human-readable fair report
|
||||
- `pg_benchmark_results.json` — optional PG-only micro suite
|
||||
|
||||
## Environment
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|----------|---------|---------|
|
||||
| `FAIR_N_KV` | 20000 | embedded KV ops |
|
||||
| `FAIR_N_SQL` | 5000 | SQL loops (HTTP/wire/PG) |
|
||||
| `FAIR_BATCH` | 100 | multi-row INSERT batch size |
|
||||
| `BARADB_HTTP_HOST` | 127.0.0.1 | HTTP host |
|
||||
| `BARADB_HTTP_PORT` | 9912 | HTTP port (`TCP+440`) |
|
||||
| `BARADB_WIRE_HOST` | 127.0.0.1 | wire protocol host |
|
||||
| `BARADB_WIRE_PORT` | 9472 | wire protocol TCP port |
|
||||
| `FAIR_SKIP_HTTP=1` | — | skip BaraDB HTTP |
|
||||
| `FAIR_SKIP_WIRE=1` | — | skip BaraDB wire |
|
||||
| `FAIR_SKIP_PG=1` | — | skip PostgreSQL |
|
||||
| `PGHOST` / `PGUSER` / `PGPASSWORD` / … | — | libpq-style |
|
||||
|
||||
## Files
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `bench_all.nim` | Embedded: LSM, WAL modes, BTree, vector, FTS, graph |
|
||||
| `fair_bench.py` | Fair multi-tier runner + markdown |
|
||||
| `pg_bench.py` | PostgreSQL client-server micro suite |
|
||||
| `generate_report.py` | `--fair` report; legacy mixed report without flag |
|
||||
| `compare.nim` | **Synthetic** — do not publish |
|
||||
| `search_bench.nim` | Search-focused micro suite |
|
||||
|
||||
## Durability knobs
|
||||
|
||||
- BaraDB: `wal_sync_mode` = `none` \| `group` \| `every` (see WAL-* rows from `bench_all`)
|
||||
- SQLite: `PRAGMA synchronous = OFF` vs `FULL`
|
||||
- PostgreSQL: `synchronous_commit = off` vs `on`
|
||||
|
||||
Match durability stories when claiming write speedups.
|
||||
|
||||
## Wire protocol note
|
||||
|
||||
The Python wire client (`clients/python`) is exercised by `fair_bench.py`.
|
||||
Builds use **`--mm:arc`** (see `nim.cfg`) because Nim **ORC** cycle collection
|
||||
crashed under async wire INSERT load (`markGray` SIGSEGV). With ARC, sequential
|
||||
wire INSERTs + batch multi-row INSERT are stable.
|
||||
Reference in New Issue
Block a user