Compare commits
18 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 92e6b99df2 | |||
| f8d466d8f3 | |||
| 40616ad3c0 | |||
| 8c29e700c7 | |||
| 956066631e | |||
| d11e99ab5a | |||
| 359f945170 | |||
| c55d3080cf | |||
| f7d4961125 | |||
| b0978812cb | |||
| d076cfde3b | |||
| 96dfaaecb1 | |||
| e2a526df6f | |||
| 18f3c16b2a | |||
| 71dcffecce | |||
| 398769ff97 | |||
| 8fb5dde858 | |||
| a0d2ca7776 |
@@ -80,8 +80,7 @@ jobs:
|
|||||||
working-directory: clients/python
|
working-directory: clients/python
|
||||||
run: |
|
run: |
|
||||||
python -m pip install --upgrade pip
|
python -m pip install --upgrade pip
|
||||||
pip install pytest
|
pip install -e ".[dev]"
|
||||||
pip install -e .
|
|
||||||
|
|
||||||
- name: Run Python unit tests
|
- name: Run Python unit tests
|
||||||
working-directory: clients/python
|
working-directory: clients/python
|
||||||
|
|||||||
+2
-1
@@ -37,4 +37,5 @@ clients/rust/target/
|
|||||||
# Nim compiled binaries
|
# Nim compiled binaries
|
||||||
clients/nim/src/baradb/client
|
clients/nim/src/baradb/client
|
||||||
src/barabadb/storage/compaction
|
src/barabadb/storage/compaction
|
||||||
docker-compose.test.yml
|
src/barabadb/query/executor
|
||||||
|
tests/join_tests
|
||||||
|
|||||||
@@ -0,0 +1,351 @@
|
|||||||
|
# BaraDB — Универсален план за Advanced SQL Engine
|
||||||
|
|
||||||
|
> **Визия**: BaraDB е самостоятелен, универсален SQL engine с Nim ядро, поддържащ модерни SQL:2023 разширения — Property Graph, Vector Search, JSON документи и прозоречни функции, в една вградена или клиент/сървър конфигурация.
|
||||||
|
>
|
||||||
|
> **Принцип**: Само основи. Не се добавят нови светове — само стабилизираме и документираме съществуващите.
|
||||||
|
>
|
||||||
|
> **Multi-Tenant фокус**: BaraDB е проектирана да поддържа ERP сценарии с много фирми (tenants) в една база данни. Всеки tenant се изолира чрез Row-Level Security (RLS) + session variables (`SET app.tenant_id = 'X'`), а не чрез отделни бази.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## История на разработката
|
||||||
|
|
||||||
|
- **Фаза 1 (Base SQL + MVCC + Raft)**: BaraDB core engine
|
||||||
|
- **Фаза 2 (Advanced SQL)**: Разработена с **Xiaomi Mimo** (`mimo-v2.5-pro`) — Window Functions, MERGE, LATERAL JOIN, Advanced Aggregates, PIVOT/UNPIVOT, SQL/PGQ Property Graph
|
||||||
|
- **Фаза 3 (Stabilization + Multi-Tenant)**: Текуща — Vector SQL Integration, Session Variables, `current_user`/`current_role`, RLS tenant isolation, тестове, документация
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Част 1: BaraDB Advanced SQL Engine
|
||||||
|
|
||||||
|
### 1.1 Window Functions ✅ ГОТОВО
|
||||||
|
|
||||||
|
Нови AST nodes: `nkWindowExpr`, `nkOverClause`, `nkFrameSpec`. Нов IR plan: `irpkWindow`.
|
||||||
|
|
||||||
|
| Функция | Описание | Статус |
|
||||||
|
|---------|----------|--------|
|
||||||
|
| `ROW_NUMBER()` | Пореден номер в партишъна | ✅ |
|
||||||
|
| `RANK()` / `DENSE_RANK()` | Класиране с/без gaps | ✅ |
|
||||||
|
| `LEAD(col, n, default)` / `LAG(col, n, default)` | Достъп до съседни редове | ✅ |
|
||||||
|
| `FIRST_VALUE(col)` / `LAST_VALUE(col)` | Краен елемент във frame | ✅ |
|
||||||
|
| `NTILE(n)` | Bucket-ване в n части | ✅ |
|
||||||
|
|
||||||
|
Frame поддръжка: `ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW` ✅
|
||||||
|
|
||||||
|
Файлове: `lexer.nim`, `ast.nim`, `ir.nim`, `parser.nim`, `executor.nim`, `codegen.nim`
|
||||||
|
Тестове: 5 теста в `tests/test_all.nim`, всички зелени.
|
||||||
|
|
||||||
|
### 1.2 MERGE / UPSERT ✅ ГОТОВО
|
||||||
|
|
||||||
|
```sql
|
||||||
|
MERGE INTO inventory AS target
|
||||||
|
USING updates AS source
|
||||||
|
ON target.sku = source.sku
|
||||||
|
WHEN MATCHED THEN UPDATE SET qty = target.qty + source.delta
|
||||||
|
WHEN NOT MATCHED THEN INSERT (sku, qty) VALUES (source.sku, source.delta);
|
||||||
|
```
|
||||||
|
|
||||||
|
- Поддържа таблица или subquery като source
|
||||||
|
- WHEN MATCHED UPDATE с eval на изрази (target.col + source.col)
|
||||||
|
- WHEN NOT MATCHED INSERT с eval на value изрази
|
||||||
|
- Trigger support (BEFORE/AFTER UPDATE/INSERT)
|
||||||
|
|
||||||
|
Файлове: `lexer.nim`, `ast.nim`, `ir.nim`, `parser.nim`, `executor.nim`, `codegen.nim`
|
||||||
|
Тестове: 2 теста в `tests/test_all.nim`, всички зелени.
|
||||||
|
|
||||||
|
### 1.3 LATERAL JOIN / CROSS APPLY ✅ ГОТОВО
|
||||||
|
|
||||||
|
Позволява correlated subquery във FROM clause с достъп до лявата таблица.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT u.name, recent_orders.*
|
||||||
|
FROM users u,
|
||||||
|
LATERAL (
|
||||||
|
SELECT order_id, total FROM orders o
|
||||||
|
WHERE o.user_id = u.id ORDER BY created_at DESC LIMIT 3
|
||||||
|
) recent_orders;
|
||||||
|
```
|
||||||
|
|
||||||
|
- Поддържа `JOIN LATERAL`, `LEFT JOIN LATERAL`, `CROSS JOIN LATERAL`
|
||||||
|
- Correlated references (e.g. `u.id`) чрез scan + merge + filter стратегия
|
||||||
|
- Sort и Limit от subquery се прилагат след merge
|
||||||
|
- LEFT LATERAL запазва unmatched редове с NULL padding
|
||||||
|
|
||||||
|
Файлове: `lexer.nim`, `ast.nim`, `ir.nim`, `parser.nim`, `executor.nim`
|
||||||
|
Тестове: 4 execution теста + 3 parser теста, всички зелени.
|
||||||
|
|
||||||
|
### 1.4 Advanced Aggregates ✅ ГОТОВО
|
||||||
|
|
||||||
|
- `ARRAY_AGG(col ORDER BY ...)`
|
||||||
|
- `STRING_AGG(col, delimiter)`
|
||||||
|
- `COUNT(*) FILTER (WHERE ...)`
|
||||||
|
- `GROUPING SETS`, `CUBE`, `ROLLUP`
|
||||||
|
|
||||||
|
#### GROUP BY + HAVING ✅ ГОТОВО
|
||||||
|
|
||||||
|
- SUM/AVG/MIN/MAX оценяват се в групите
|
||||||
|
- HAVING филтрира групите по aggregate условия
|
||||||
|
- Pre-computed aggregates се съхраняват в group rows
|
||||||
|
- evalExpr поддържа irekAggregate lookup
|
||||||
|
|
||||||
|
Тестове: 6 теста в `tests/test_all.nim`, всички зелени.
|
||||||
|
|
||||||
|
#### FILTER (WHERE ...) ✅ ГОТОВО
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT COUNT(*) FILTER (WHERE active = true) FROM users;
|
||||||
|
SELECT dept, SUM(amount) FILTER (WHERE amount > 100) FROM sales GROUP BY dept;
|
||||||
|
```
|
||||||
|
|
||||||
|
- Parser: `FILTER (WHERE ...)` след aggregate function call
|
||||||
|
- AST: `funcFilter*: Node` на `nkFuncCall`
|
||||||
|
- IR: `aggFilter*: IRExpr` на `irekAggregate`
|
||||||
|
- Executor: филтрира редове преди aggregate computation
|
||||||
|
|
||||||
|
Тестове: 2 execution теста + 1 parser тест, всички зелени.
|
||||||
|
|
||||||
|
#### ARRAY_AGG / STRING_AGG ✅ ГОТОВО
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT dept, ARRAY_AGG(amount) AS amounts FROM sales GROUP BY dept;
|
||||||
|
SELECT dept, STRING_AGG(name, ', ') AS names FROM employees GROUP BY dept;
|
||||||
|
```
|
||||||
|
|
||||||
|
- Нови IR aggregate ops: `irArrayAgg`, `irStringAgg`
|
||||||
|
- Multi-argument aggregate parsing (delimiter за STRING_AGG)
|
||||||
|
- FILTER support за двете функции
|
||||||
|
|
||||||
|
Тестове: 2 теста, всички зелени.
|
||||||
|
|
||||||
|
#### GROUPING SETS / ROLLUP / CUBE ✅ ГОТОВО
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT dept, SUM(amount) FROM sales GROUP BY ROLLUP (dept);
|
||||||
|
SELECT dept, job, SUM(amount) FROM sales GROUP BY CUBE (dept, job);
|
||||||
|
SELECT dept, job, SUM(amount) FROM sales GROUP BY GROUPING SETS ((dept), (job), ());
|
||||||
|
```
|
||||||
|
|
||||||
|
- ROLLUP(a, b) → GROUPING SETS ((a,b), (a), ())
|
||||||
|
- CUBE(a, b) → GROUPING SETS ((a,b), (a), (b), ())
|
||||||
|
- Генериране на subsets за CUBE чрез powerset алгоритъм
|
||||||
|
|
||||||
|
Тестове: 4 parser теста + 1 execution тест, всички зелени.
|
||||||
|
|
||||||
|
### 1.5 PIVOT / UNPIVOT ✅ ГОТОВО
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT * FROM (SELECT name, dept, salary FROM emp)
|
||||||
|
PIVOT (SUM(salary) FOR dept IN ('Eng', 'Sales'));
|
||||||
|
|
||||||
|
SELECT * FROM emp
|
||||||
|
UNPIVOT (salary FOR dept IN (eng_salary, sales_salary));
|
||||||
|
```
|
||||||
|
|
||||||
|
- Parser: PIVOT/UNPIVOT в FROM clause
|
||||||
|
- IR: `irpkPivot`, `irpkUnpivot`
|
||||||
|
- Executor: group by identity cols → aggregate per pivot value → create columns
|
||||||
|
- Subquery storage в `nkFrom.fromSubquery`
|
||||||
|
|
||||||
|
Тестове: 1 parser + 1 execution тест, всички зелени.
|
||||||
|
|
||||||
|
### 1.6 SQL:2023 Property Graph (SQL/PGQ) ✅ ГОТОВО (Parser)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT * FROM GRAPH_TABLE(org_chart
|
||||||
|
MATCH (e)-[r]->(d)
|
||||||
|
COLUMNS (e.name, d.name)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
- Lexer: `tkVertex`, `tkEdge`, `tkLabels`, `tkGraphTable`, `tkMatch`, `tkColumns`, `tkSrc`, `tkDst`
|
||||||
|
- AST: `nkGraphTraversal` с `gtGraphName`, `gtReturnCols`
|
||||||
|
- IR: `irpkGraphTraversal` с `graphName`, `graphAlgo`, `graphReturnCols`
|
||||||
|
- Executor: table-based graph storage (`graph_nodes`, `graph_edges`)
|
||||||
|
- Parser: `GRAPH_TABLE(name MATCH (pattern) COLUMNS (cols))`
|
||||||
|
|
||||||
|
Тестове: 1 parser тест, всички зелени.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Част 1.5: Multi-Tenant ERP Support ✅ ГОТОВО
|
||||||
|
|
||||||
|
BaraDB поддържа multi-tenant архитектура, при която множество фирми (tenants) работят в една физическа база данни. Това е критично за ERP сценарии, където поддръжката на "сто бази" не е опция.
|
||||||
|
|
||||||
|
### Механизъм
|
||||||
|
|
||||||
|
| Компонент | Описание |
|
||||||
|
|-----------|----------|
|
||||||
|
| **Session Variables** | `SET app.tenant_id = 'company-123'` — задава tenant за текущата сесия |
|
||||||
|
| **current_setting()** | `current_setting('app.tenant_id')` — чете session променлива в SQL израз |
|
||||||
|
| **current_user** | `current_user` — връща автентикирания потребител от JWT/SCRAM |
|
||||||
|
| **current_role** | `current_role` — връща ролята на автентикирания потребител |
|
||||||
|
| **RLS Policies** | `CREATE POLICY tenant_isolation ON invoices FOR SELECT USING (tenant_id = current_setting('app.tenant_id'))` |
|
||||||
|
| **Auth Bridge** | `server.nim` и `httpserver.nim` попълват `ExecutionContext.currentUser`/`currentRole` след верификация |
|
||||||
|
|
||||||
|
### Пример
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Една таблица за всички фирми
|
||||||
|
CREATE TABLE invoices (
|
||||||
|
id SERIAL PRIMARY KEY,
|
||||||
|
tenant_id TEXT NOT NULL,
|
||||||
|
data JSONB
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Изолация чрез RLS
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- Всяка сесия вижда само своя tenant
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- → само фактури на company-a
|
||||||
|
```
|
||||||
|
|
||||||
|
### Архитектурни предимства
|
||||||
|
|
||||||
|
- **JSONB документи** — schema-flexible, лесно се добавят нови полета без миграции (като ArangoDB)
|
||||||
|
- **RLS изолация** — базата данни гарантира, че всеки tenant вижда само своите данни
|
||||||
|
- **Един instance** — един BaraDB сървър обслужва всички tenants, вместо сто отделни бази
|
||||||
|
- **Auth integration** — JWT/SCRAM токените носят `sub` (user) и `role`, които се пропагират до executor-а
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Част 2: Мултимодални Възможности (Core Only)
|
||||||
|
|
||||||
|
### 2.1 JSON / JSONB Документи ✅ ГОТОВО
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT data->>'name' FROM users WHERE data->'tags' @> '["admin"]';
|
||||||
|
```
|
||||||
|
|
||||||
|
- Типове: `JSON`, `JSONB` колони в таблици
|
||||||
|
- Оператори: `->`, `->>`, `#>`, `#>>`, `@>`, `<@`, `?`, `?&`, `?|`
|
||||||
|
- Функции: `jsonb_array_elements`, `jsonb_object_keys`, `jsonb_extract_path`
|
||||||
|
- Съхранение: двоично parsed tree (не plain text)
|
||||||
|
|
||||||
|
### 2.2 Vector Search ⚠️ ЧАСТИЧНО (Engine ✅, SQL Integration 🔄)
|
||||||
|
|
||||||
|
**Вектор Engine (готов):**
|
||||||
|
- `src/barabadb/vector/engine.nim` — HNSW index с cosine/euclidean distance
|
||||||
|
- `src/barabadb/vector/quant.nim` — IVF-PQ quantization
|
||||||
|
- `src/barabadb/vector/simd.nim` — SIMD оптимизации
|
||||||
|
- `src/barabadb/core/crossmodal.nim` — CrossModalEngine за хибридно търсене (vector + text)
|
||||||
|
|
||||||
|
**Липсваща SQL интеграция (базова — за стабилизация):**
|
||||||
|
```sql
|
||||||
|
-- Тип и колона
|
||||||
|
CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(768));
|
||||||
|
|
||||||
|
-- Index
|
||||||
|
CREATE VECTOR INDEX idx_items_vec ON items(embedding)
|
||||||
|
USING hnsw WITH (m = 16, ef_construction = 200, metric = 'cosine');
|
||||||
|
|
||||||
|
-- Query functions
|
||||||
|
SELECT id, cosine_distance(embedding, '[0.1, 0.2, ...]') AS dist
|
||||||
|
FROM items
|
||||||
|
ORDER BY dist ASC
|
||||||
|
LIMIT 10;
|
||||||
|
```
|
||||||
|
|
||||||
|
**Задачи за стабилизация (всички изпълнени):**
|
||||||
|
- [x] `VECTOR(n)` тип в CREATE TABLE (parser + storage)
|
||||||
|
- [x] `CREATE VECTOR INDEX ... USING hnsw` (DDL)
|
||||||
|
- [x] `cosine_distance()`, `euclidean_distance()`, `inner_product()` в SQL expression evaluator
|
||||||
|
- [x] `<->` nearest-neighbor оператор в ORDER BY / WHERE
|
||||||
|
- [x] Executor integration: HNSW index population при CREATE INDEX и DML
|
||||||
|
|
||||||
|
**Статус:** ✅ ГОТОВО. 8 SQL-level vector теста зелени.
|
||||||
|
|
||||||
|
### 2.3 Full-Text Search ✅ ГОТОВО
|
||||||
|
|
||||||
|
- Inverted Index в `src/barabadb/fts/`
|
||||||
|
- `MATCH(column, query)` функция
|
||||||
|
- BM25 scoring
|
||||||
|
- Интеграция с CrossModalEngine за hybrid search
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Част 3: Транзакции и Протоколи ✅ ГОТОВО
|
||||||
|
|
||||||
|
- MVCC с snapshot isolation
|
||||||
|
- WAL + checkpoint
|
||||||
|
- Distributed transactions (2PC) — `txn.addParticipant("vector")`
|
||||||
|
- Wire protocol: binary за vectors, JSON за queries
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Имплементационен ред (финален статус)
|
||||||
|
|
||||||
|
1. ✅ **Window Functions** (AST → Parser → IR → Executor → Tests)
|
||||||
|
2. ✅ **MERGE statement** (Parser → Executor → Tests)
|
||||||
|
3. ✅ **LATERAL JOIN** (Parser → Executor, correlated subquery strategy)
|
||||||
|
4. ✅ **GROUP BY + HAVING** (SUM/AVG/MIN/MAX, HAVING filter)
|
||||||
|
5. ✅ **FILTER clause** (COUNT/SUM/AVG FILTER (WHERE ...))
|
||||||
|
6. ✅ **ARRAY_AGG / STRING_AGG** (multi-arg aggregates)
|
||||||
|
7. ✅ **GROUPING SETS / ROLLUP / CUBE** (powerset generation)
|
||||||
|
8. ✅ **PIVOT / UNPIVOT** (row-to-column transformation)
|
||||||
|
9. ✅ **SQL/PGQ Property Graph** (GRAPH_TABLE MATCH parser)
|
||||||
|
10. ✅ **JSON/JSONB** (operators + functions)
|
||||||
|
11. ✅ **Full-Text Search** (inverted index + BM25)
|
||||||
|
12. ✅ **Vector Engine** (HNSW + IVF-PQ + SIMD)
|
||||||
|
13. ✅ **Vector SQL Integration** (тип, index, distance functions, <-> operator, ORDER BY)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Крайно състояние
|
||||||
|
|
||||||
|
**340+ теста зелени.** Всички фундаментални SQL:2023 features имплементирани.
|
||||||
|
|
||||||
|
**Четирите свята:**
|
||||||
|
|
||||||
|
| Свят | Features | Статус |
|
||||||
|
|------|----------|--------|
|
||||||
|
| **SQL** | Window, MERGE, LATERAL, GROUP BY/HAVING, FILTER, ARRAY_AGG, STRING_AGG, GROUPING SETS/ROLLUP/CUBE, PIVOT/UNPIVOT | ✅ |
|
||||||
|
| **JSON** | JSON/JSONB колони, `->` / `->>` оператори | ✅ |
|
||||||
|
| **Graph** | BFS/DFS/PageRank/Dijkstra engine + SQL/PGQ GRAPH_TABLE | ✅ |
|
||||||
|
| **Vector** | HNSW index, cosine/euclidean distance, IVF-PQ, SIMD | ✅ Engine<br>🔄 SQL glue |
|
||||||
|
| **FTS** | Inverted index, BM25, hybrid search | ✅ |
|
||||||
|
|
||||||
|
**Файлове модифицирани:**
|
||||||
|
- `lexer.nim` — tkLateral, tkFilter, tkPivot, tkUnpivot, tkVertex, tkEdge, tkGraphTable, tkMatch, tkColumns, tkArrayAgg, tkStringAgg, tkGrouping, tkSets, tkRollup, tkCube, tkVector
|
||||||
|
- `ast.nim` — joinLateral, funcFilter, nkPivot, nkUnpivot, GroupingSetsKind, nkGraphTraversal fields
|
||||||
|
- `ir.nim` — joinLateral, aggFilter, irArrayAgg, irStringAgg, IRGroupingSetsKind, irpkGroupBy grouping sets, irpkPivot, irpkUnpivot, irpkGraphTraversal
|
||||||
|
- `parser.nim` — LATERAL, FILTER, multi-arg aggregates, GROUPING SETS/ROLLUP/CUBE, PIVOT/UNPIVOT, GRAPH_TABLE
|
||||||
|
- `executor.nim` — LATERAL correlated strategy, GROUP BY aggregates + HAVING, FILTER in aggregates, ARRAY_AGG/STRING_AGG, GROUPING SETS/ROLLUP/CUBE, PIVOT/UNPIVOT, GRAPH_TABLE, fromTable kind checks
|
||||||
|
- `codegen.nim` — irpkPivot, irpkUnpivot, irpkGraphTraversal
|
||||||
|
- `tests/test_all.nim` — 25+ нови теста
|
||||||
|
- `tests/join_tests.nim` — 4 LATERAL теста
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Тестова стратегия
|
||||||
|
|
||||||
|
- **Unit**: Всеки нов AST/IR/Parser тест — property-based (генериране на случайни partition/order)
|
||||||
|
- **Integration**: HTTP server + клиент тестове
|
||||||
|
- **TLA+**: `windowfunctions.tla` — deterministic partitioning semantics
|
||||||
|
- **Benchmark**: Window function performance vs PostgreSQL (опционално)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Поправени грешки при тази сесия
|
||||||
|
|
||||||
|
- **Vector SQL Integration** — имплементиран пълен SQL glue за вектори (тип, индекс, функции, оператор)
|
||||||
|
- **MERGE тестове** — поправени чрез изолиране на тестовата директория (unique temp dir per suite)
|
||||||
|
- **Row storage escape** — `escapeRowVal()` в `execInsert` за стойности със запетай (vector literals)
|
||||||
|
- **ORDER BY + projection** — `irpkSort` сега е преди `irpkProject` в `lowerSelect`, което позволява `ORDER BY` по колони извън `SELECT`
|
||||||
|
- **GROUPING SETS execution** — `lowerSelect` сега проверява `selGroupingSetsKind != gskNone` освен `selGroupBy.len > 0`, което позволява изпълнение на GROUPING SETS без традиционен GROUP BY
|
||||||
|
- **FTS CREATE INDEX docId** — поправено несъответствие в изчислението на `docId` при `CREATE INDEX ... USING FTS` (сега използва хеш на `tableName.$key`, съвместим с DML операциите)
|
||||||
|
- **Тестова изолация (всички suite-ове)** — всички `newLSMTree("")` заменени с уникални temp директории; setup/teardown за suite-ове с изолирана state
|
||||||
|
- **Multi-tenant ERP support** — имплементирани критични градивни елементи:
|
||||||
|
- `SET var = value` — session variables за tenant isolation
|
||||||
|
- `current_setting('var')` — четене на session променливи в SQL изрази
|
||||||
|
- `current_user` / `current_role` — SQL keywords, които се оценяват от `ExecutionContext`
|
||||||
|
- Auth bridge — `server.nim` и `httpserver.nim` попълват `currentUser`/`currentRole` след JWT/SCRAM верификация
|
||||||
|
- RLS tenant isolation тест — `CREATE POLICY` + `current_setting('app.tenant_id')` работи за multi-tenant филтрация
|
||||||
|
- `evalExpr` вече предава `ctx` рекурсивно — поправен бъг, при който `current_user`/`current_setting` връщаха празни стойности в под-изрази
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
> **Бележка**: Този план е *замразен* за нови светове. Следващата работа е само стабилизация на съществуващото и документация.
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
# Nodebara Compatibility Fixes
|
||||||
|
|
||||||
|
This PR bundles three independent fixes discovered while integrating BaraDB with [NodeBB](https://nodebb.org/) (a large Node.js forum application). Each commit is self-contained and can be reviewed separately.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. fix(protocol): serialize float32/float64 in big-endian
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
The JavaScript client reads `FLOAT32`/`FLOAT64` wire values with `readFloatBE()` / `readDoubleBE()` (big-endian), but the Nim server was writing them with `copyMem(..., unsafeAddr fl, N)` — i.e. **native byte order**. On little-endian machines (virtually all x86_64 servers) the client deserializes garbage. Example: a zset `score = 1.0` becomes `3.03865e-319`, which breaks any application relying on numeric scores (user IDs, timestamps, rankings, etc.).
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
Cast floats to `int32`/`int64` and route them through the existing `bigEndian32`/`bigEndian64` helpers, exactly like the integer paths already do. Same change applied to deserialization.
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
Breaking fix for cross-platform wire compatibility. No API changes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. feat(client): add TCP request queue for safe concurrency
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
`Client.query()`, `execute()`, and `ping()` were async methods that wrote directly to the TCP socket. When NodeBB fired multiple parallel DB operations (common on startup), their binary frames interleaved on the wire, causing parse errors, wrong request/response pairing, and random crashes.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
Introduce an internal `_requestQueue` + `_requestLock`. Every public async method enqueues a closure; a tiny drain loop processes them one at a time via `setImmediate()`.
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
No breaking API change. Existing single-request usage is unchanged; concurrent usage now works safely.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. fix(gossip): use async UDP socket to avoid blocking the event loop
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
`startGossipListener` created a **synchronous** UDP socket (`newSocket`) and called blocking `recvFrom` inside an `async` proc. This freezes the entire async event loop until a UDP packet arrives, stalling all other async I/O.
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
Replace `newSocket` with `newAsyncSocket` and `recvFrom` with `await recvFrom`.
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
Non-breaking. Gossip remains optional; when enabled it no longer blocks the main loop.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- [x] NodeBB v4.11.2 setup completes end-to-end
|
||||||
|
- [x] Admin login works (relies on correct FLOAT64 score deserialization)
|
||||||
|
- [x] Concurrent DB queries during startup no longer corrupt frames
|
||||||
|
- [x] Gossip listener no longer blocks other async tasks
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Checklist
|
||||||
|
|
||||||
|
- [x] Each commit is atomic and compiles on its own
|
||||||
|
- [x] No Nim compiler warnings introduced
|
||||||
|
- [x] JS client backward-compatible for single-request callers
|
||||||
|
- [x] Existing tests (`nimble test`) still pass
|
||||||
@@ -285,8 +285,23 @@ let range = btree.scan("key_a", "key_z")
|
|||||||
|
|
||||||
### Vector Engine
|
### Vector Engine
|
||||||
|
|
||||||
Native HNSW and IVF-PQ indexes for similarity search.
|
Native HNSW and IVF-PQ indexes for similarity search with full SQL integration.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- SQL vector search
|
||||||
|
CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(768));
|
||||||
|
INSERT INTO items (id, embedding) VALUES (1, '[0.1, 0.2, 0.3, ...]');
|
||||||
|
|
||||||
|
-- Nearest neighbor search
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY cosine_distance(embedding, '[0.1, 0.2, 0.3, ...]') ASC
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- With HNSW index
|
||||||
|
CREATE INDEX idx_vec ON items(embedding) USING hnsw;
|
||||||
|
```
|
||||||
|
|
||||||
|
Native Nim API:
|
||||||
```nim
|
```nim
|
||||||
import barabadb/vector/engine
|
import barabadb/vector/engine
|
||||||
|
|
||||||
@@ -301,7 +316,10 @@ let filtered = idx.searchWithFilter(queryVector, k = 10,
|
|||||||
```
|
```
|
||||||
|
|
||||||
Features:
|
Features:
|
||||||
- **HNSW** — hierarchical navigable small world graph
|
- **SQL vector types** — `VECTOR(n)` with dimension validation
|
||||||
|
- **SQL distance functions** — `cosine_distance()`, `euclidean_distance()`, `inner_product()`, `l1_distance()`, `l2_distance()`
|
||||||
|
- **`<->` operator** — Euclidean distance nearest-neighbor shorthand
|
||||||
|
- **HNSW index** — `CREATE INDEX ... USING hnsw` with automatic maintenance
|
||||||
- **IVF-PQ** — inverted file index with product quantization
|
- **IVF-PQ** — inverted file index with product quantization
|
||||||
- **Distance metrics** — cosine, euclidean, dot product, Manhattan
|
- **Distance metrics** — cosine, euclidean, dot product, Manhattan
|
||||||
- **Quantization** — scalar 8-bit/4-bit, product, binary
|
- **Quantization** — scalar 8-bit/4-bit, product, binary
|
||||||
@@ -606,6 +624,23 @@ See [docs/en/docker.md](docs/en/docker.md) for full Docker documentation.
|
|||||||
| `BARADB_CERT_FILE` | — | TLS certificate path |
|
| `BARADB_CERT_FILE` | — | TLS certificate path |
|
||||||
| `BARADB_KEY_FILE` | — | TLS private key path |
|
| `BARADB_KEY_FILE` | — | TLS private key path |
|
||||||
|
|
||||||
|
## Built with BaraDB
|
||||||
|
|
||||||
|
### NodeBara
|
||||||
|
|
||||||
|
**[NodeBara](https://codeberg.org/baraDB/nodebara)** is the first large-scale application running on BaraDB — a modern forum platform forked from NodeBB and fully adapted for BaraDB's native multimodal engine.
|
||||||
|
|
||||||
|
- **Concurrent query safety** — TCP request queue in the JS client handles NodeBara's parallel startup queries without frame corruption
|
||||||
|
- **Numeric accuracy** — Big-endian float serialization guarantees correct zset scores, timestamps, and rankings across platforms
|
||||||
|
- **Non-blocking cluster gossip** — Async UDP sockets keep the event loop free under load
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://codeberg.org/baraDB/nodebara
|
||||||
|
cd nodebara
|
||||||
|
npm install
|
||||||
|
npm run setup # uses BaraDB as the default database
|
||||||
|
```
|
||||||
|
|
||||||
## Client SDKs
|
## Client SDKs
|
||||||
|
|
||||||
BaraDB provides official clients for multiple languages:
|
BaraDB provides official clients for multiple languages:
|
||||||
@@ -1214,7 +1249,7 @@ src/barabadb/
|
|||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Run all tests (262 tests, 56 suites)
|
# Run all tests (340+ tests, 60+ suites)
|
||||||
nim c --path:src -r tests/test_all.nim
|
nim c --path:src -r tests/test_all.nim
|
||||||
|
|
||||||
# Run benchmarks
|
# Run benchmarks
|
||||||
@@ -1232,6 +1267,7 @@ nim c -d:release -r benchmarks/bench_all.nim
|
|||||||
| Protocol (binary + HTTP + WS + pool + auth + ratelimit) | ✅ | 100% | v1.0.0 |
|
| Protocol (binary + HTTP + WS + pool + auth + ratelimit) | ✅ | 100% | v1.0.0 |
|
||||||
| Schema (inheritance + computed + migrations) | ✅ | 100% | v1.0.0 |
|
| Schema (inheritance + computed + migrations) | ✅ | 100% | v1.0.0 |
|
||||||
| Vector engine (HNSW + IVF-PQ + quant + SIMD + metadata) | ✅ | 100% | v1.0.0 |
|
| Vector engine (HNSW + IVF-PQ + quant + SIMD + metadata) | ✅ | 100% | v1.0.0 |
|
||||||
|
| Vector SQL Integration (VECTOR type, distance functions, <->, HNSW indexes) | ✅ | 100% | v1.1.0 |
|
||||||
| Graph engine (all algorithms + pattern matching) | ✅ | 100% | v1.0.0 |
|
| Graph engine (all algorithms + pattern matching) | ✅ | 100% | v1.0.0 |
|
||||||
| FTS (BM25 + TF-IDF + fuzzy + regex + multi-language) | ✅ | 100% | v1.0.0 |
|
| FTS (BM25 + TF-IDF + fuzzy + regex + multi-language) | ✅ | 100% | v1.0.0 |
|
||||||
| CLI shell | ✅ | 100% | v1.0.0 |
|
| CLI shell | ✅ | 100% | v1.0.0 |
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ npm install baradb
|
|||||||
Or from source:
|
Or from source:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/barabadb/baradadb.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd clients/javascript
|
cd clients/javascript
|
||||||
npm link
|
npm link
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -244,6 +244,8 @@ class Client {
|
|||||||
this.requestId = 0;
|
this.requestId = 0;
|
||||||
this._buffer = Buffer.alloc(0);
|
this._buffer = Buffer.alloc(0);
|
||||||
this._pendingResolve = null;
|
this._pendingResolve = null;
|
||||||
|
this._requestQueue = [];
|
||||||
|
this._requestLock = false;
|
||||||
}
|
}
|
||||||
|
|
||||||
async connect() {
|
async connect() {
|
||||||
@@ -478,70 +480,98 @@ class Client {
|
|||||||
throw new Error(`Unexpected auth response: 0x${header.kind.toString(16)}`);
|
throw new Error(`Unexpected auth response: 0x${header.kind.toString(16)}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async _processQueue() {
|
||||||
|
if (this._requestLock || this._requestQueue.length === 0) return;
|
||||||
|
this._requestLock = true;
|
||||||
|
const { task, resolve, reject } = this._requestQueue.shift();
|
||||||
|
try {
|
||||||
|
const result = await task();
|
||||||
|
resolve(result);
|
||||||
|
} catch (err) {
|
||||||
|
reject(err);
|
||||||
|
} finally {
|
||||||
|
this._requestLock = false;
|
||||||
|
setImmediate(() => this._processQueue());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_enqueue(task) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
this._requestQueue.push({ task, resolve, reject });
|
||||||
|
this._processQueue();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
async ping() {
|
async ping() {
|
||||||
if (!this.connected) throw new Error('Not connected');
|
if (!this.connected) throw new Error('Not connected');
|
||||||
const msg = this._build(MsgKind.PING, Buffer.alloc(0));
|
return this._enqueue(async () => {
|
||||||
this.socket.write(msg);
|
const msg = this._build(MsgKind.PING, Buffer.alloc(0));
|
||||||
|
this.socket.write(msg);
|
||||||
|
|
||||||
const header = await this._readHeader();
|
const header = await this._readHeader();
|
||||||
if (header.kind === MsgKind.PONG) return true;
|
if (header.kind === MsgKind.PONG) return true;
|
||||||
if (header.kind === MsgKind.ERROR) throw await this._readError(header.length);
|
if (header.kind === MsgKind.ERROR) throw await this._readError(header.length);
|
||||||
return false;
|
return false;
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
async query(sql) {
|
async query(sql) {
|
||||||
if (!this.connected) throw new Error('Not connected');
|
if (!this.connected) throw new Error('Not connected');
|
||||||
const queryBuf = Buffer.from(sql, 'utf-8');
|
return this._enqueue(async () => {
|
||||||
const payload = Buffer.alloc(4 + queryBuf.length + 1);
|
const queryBuf = Buffer.from(sql, 'utf-8');
|
||||||
payload.writeUInt32BE(queryBuf.length, 0);
|
const payload = Buffer.alloc(4 + queryBuf.length + 1);
|
||||||
queryBuf.copy(payload, 4);
|
payload.writeUInt32BE(queryBuf.length, 0);
|
||||||
payload[4 + queryBuf.length] = ResultFormat.BINARY;
|
queryBuf.copy(payload, 4);
|
||||||
|
payload[4 + queryBuf.length] = ResultFormat.BINARY;
|
||||||
|
|
||||||
const msg = this._build(MsgKind.QUERY, payload);
|
const msg = this._build(MsgKind.QUERY, payload);
|
||||||
this.socket.write(msg);
|
this.socket.write(msg);
|
||||||
|
|
||||||
const header = await this._readHeader();
|
const header = await this._readHeader();
|
||||||
if (header.kind === MsgKind.ERROR) throw await this._readError(header.length);
|
if (header.kind === MsgKind.ERROR) throw await this._readError(header.length);
|
||||||
if (header.kind === MsgKind.DATA) return await this._readDataResponse(header.length);
|
if (header.kind === MsgKind.DATA) return await this._readDataResponse(header.length);
|
||||||
if (header.kind === MsgKind.COMPLETE) {
|
if (header.kind === MsgKind.COMPLETE) {
|
||||||
const data = await this._recvExact(header.length);
|
const data = await this._recvExact(header.length);
|
||||||
const result = new QueryResult();
|
const result = new QueryResult();
|
||||||
result.affectedRows = data.readUInt32BE(0);
|
result.affectedRows = data.readUInt32BE(0);
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
return new QueryResult();
|
return new QueryResult();
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
async queryParams(sql, params = []) {
|
async queryParams(sql, params = []) {
|
||||||
if (!this.connected) throw new Error('Not connected');
|
if (!this.connected) throw new Error('Not connected');
|
||||||
const queryBuf = Buffer.from(sql, 'utf-8');
|
return this._enqueue(async () => {
|
||||||
const paramParts = [];
|
const queryBuf = Buffer.from(sql, 'utf-8');
|
||||||
for (const p of params) {
|
const paramParts = [];
|
||||||
paramParts.push(p.serialize());
|
for (const p of params) {
|
||||||
}
|
paramParts.push(p.serialize());
|
||||||
const paramsBuf = Buffer.concat(paramParts);
|
}
|
||||||
|
const paramsBuf = Buffer.concat(paramParts);
|
||||||
|
|
||||||
const payload = Buffer.alloc(4 + queryBuf.length + 1 + 4 + paramsBuf.length);
|
const payload = Buffer.alloc(4 + queryBuf.length + 1 + 4 + paramsBuf.length);
|
||||||
let pos = 0;
|
let pos = 0;
|
||||||
payload.writeUInt32BE(queryBuf.length, pos); pos += 4;
|
payload.writeUInt32BE(queryBuf.length, pos); pos += 4;
|
||||||
queryBuf.copy(payload, pos); pos += queryBuf.length;
|
queryBuf.copy(payload, pos); pos += queryBuf.length;
|
||||||
payload[pos] = ResultFormat.BINARY; pos++;
|
payload[pos] = ResultFormat.BINARY; pos++;
|
||||||
payload.writeUInt32BE(params.length, pos); pos += 4;
|
payload.writeUInt32BE(params.length, pos); pos += 4;
|
||||||
paramsBuf.copy(payload, pos);
|
paramsBuf.copy(payload, pos);
|
||||||
|
|
||||||
const msg = this._build(MsgKind.QUERY_PARAMS, payload);
|
const msg = this._build(MsgKind.QUERY_PARAMS, payload);
|
||||||
this.socket.write(msg);
|
this.socket.write(msg);
|
||||||
|
|
||||||
const header = await this._readHeader();
|
const header = await this._readHeader();
|
||||||
if (header.kind === MsgKind.ERROR) throw await this._readError(header.length);
|
if (header.kind === MsgKind.ERROR) throw await this._readError(header.length);
|
||||||
if (header.kind === MsgKind.DATA) return await this._readDataResponse(header.length);
|
if (header.kind === MsgKind.DATA) return await this._readDataResponse(header.length);
|
||||||
if (header.kind === MsgKind.COMPLETE) {
|
if (header.kind === MsgKind.COMPLETE) {
|
||||||
const data = await this._recvExact(header.length);
|
const data = await this._recvExact(header.length);
|
||||||
const result = new QueryResult();
|
const result = new QueryResult();
|
||||||
result.affectedRows = data.readUInt32BE(0);
|
result.affectedRows = data.readUInt32BE(0);
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
return new QueryResult();
|
return new QueryResult();
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
async execute(sql) {
|
async execute(sql) {
|
||||||
|
|||||||
@@ -8,8 +8,8 @@ const assert = require('node:assert');
|
|||||||
const net = require('net');
|
const net = require('net');
|
||||||
const { Client, WireValue, QueryBuilder } = require('../baradb');
|
const { Client, WireValue, QueryBuilder } = require('../baradb');
|
||||||
|
|
||||||
const HOST = 'localhost';
|
const HOST = process.env.BARADB_HOST || 'localhost';
|
||||||
const PORT = 9472;
|
const PORT = parseInt(process.env.BARADB_PORT || '9472', 10);
|
||||||
|
|
||||||
function serverAvailable() {
|
function serverAvailable() {
|
||||||
return new Promise((resolve) => {
|
return new Promise((resolve) => {
|
||||||
|
|||||||
@@ -15,13 +15,13 @@ Official Nim client for **BaraDB** — a multimodal database engine.
|
|||||||
Add to your `.nimble` file:
|
Add to your `.nimble` file:
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
requires "baradb >= 1.0.0"
|
requires "baradb >= 1.1.0"
|
||||||
```
|
```
|
||||||
|
|
||||||
Or clone locally:
|
Or clone locally:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/barabadb/baradadb.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd clients/nim
|
cd clients/nim
|
||||||
nimble develop
|
nimble develop
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -1,15 +1,17 @@
|
|||||||
## BaraDB Nim Client — Integration Tests
|
## BaraDB Nim Client — Integration Tests
|
||||||
## Requires a running BaraDB server on localhost:9472.
|
## Requires a running BaraDB server.
|
||||||
|
## Set BARADB_HOST / BARADB_PORT env vars to override defaults.
|
||||||
|
|
||||||
import std/unittest
|
import std/unittest
|
||||||
import std/asyncdispatch
|
import std/asyncdispatch
|
||||||
import std/asyncnet
|
import std/asyncnet
|
||||||
import std/strutils
|
import std/strutils
|
||||||
|
import std/os
|
||||||
import baradb/client
|
import baradb/client
|
||||||
|
|
||||||
const
|
const
|
||||||
TestHost = "127.0.0.1"
|
TestHost = getEnv("BARADB_HOST", "127.0.0.1")
|
||||||
TestPort = 9472
|
TestPort = parseInt(getEnv("BARADB_PORT", "9472"))
|
||||||
|
|
||||||
proc serverAvailable(): bool =
|
proc serverAvailable(): bool =
|
||||||
try:
|
try:
|
||||||
|
|||||||
+71
-49
@@ -1,15 +1,16 @@
|
|||||||
# BaraDB Python Client
|
# BaraDB Python Async Client
|
||||||
|
|
||||||
Official Python client for **BaraDB** — a multimodal database engine written in Nim.
|
Official async Python client for **BaraDB** — a multimodal database engine written in Nim.
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
|
- **Async/await** — fully non-blocking, concurrent query support
|
||||||
- **Binary wire protocol** — fast, compact TCP communication
|
- **Binary wire protocol** — fast, compact TCP communication
|
||||||
- **Sync & blocking API** — simple to use in scripts and apps
|
- **Request queueing** — sequential processing with concurrent execution
|
||||||
- **Query builder** — fluent SQL construction
|
- **Query builder** — fluent SQL construction
|
||||||
- **Parameterized queries** — safe from SQL injection
|
- **Parameterized queries** — safe from SQL injection
|
||||||
- **Vector & JSON support** — first-class multimodal types
|
- **Vector & JSON support** — first-class multimodal types
|
||||||
- **Context managers** — `with` statement support
|
- **Context managers** — async `async with` statement support
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@@ -20,7 +21,7 @@ pip install baradb
|
|||||||
Or from source:
|
Or from source:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/barabadb/baradadb.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd clients/python
|
cd clients/python
|
||||||
pip install -e ".[dev]"
|
pip install -e ".[dev]"
|
||||||
```
|
```
|
||||||
@@ -28,71 +29,92 @@ pip install -e ".[dev]"
|
|||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
import asyncio
|
||||||
from baradb import Client
|
from baradb import Client
|
||||||
|
|
||||||
client = Client("localhost", 9472)
|
async def main():
|
||||||
client.connect()
|
client = Client("localhost", 9472)
|
||||||
|
await client.connect()
|
||||||
|
|
||||||
result = client.query("SELECT name, age FROM users WHERE age > 18")
|
result = await client.query("SELECT name, age FROM users WHERE age > 18")
|
||||||
for row in result:
|
for row in result:
|
||||||
print(row["name"], row["age"])
|
print(row["name"], row["age"])
|
||||||
|
|
||||||
client.close()
|
await client.close()
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
```
|
```
|
||||||
|
|
||||||
### Context Manager
|
### Context Manager
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
import asyncio
|
||||||
from baradb import Client
|
from baradb import Client
|
||||||
|
|
||||||
with Client("localhost", 9472) as client:
|
async def main():
|
||||||
result = client.query("SELECT 1")
|
async with Client("localhost", 9472) as client:
|
||||||
print(result.row_count)
|
result = await client.query("SELECT 1")
|
||||||
|
print(result.row_count)
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
```
|
```
|
||||||
|
|
||||||
### Parameterized Queries
|
### Parameterized Queries
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
import asyncio
|
||||||
from baradb import Client, WireValue
|
from baradb import Client, WireValue
|
||||||
|
|
||||||
with Client("localhost", 9472) as client:
|
async def main():
|
||||||
result = client.query_params(
|
async with Client("localhost", 9472) as client:
|
||||||
"SELECT * FROM users WHERE age > $1 AND country = $2",
|
result = await client.query_params(
|
||||||
[WireValue.int64(18), WireValue.string("BG")],
|
"SELECT * FROM users WHERE age > $1 AND country = $2",
|
||||||
)
|
[WireValue.int64(18), WireValue.string("BG")],
|
||||||
for row in result:
|
)
|
||||||
print(row)
|
for row in result:
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
```
|
```
|
||||||
|
|
||||||
### Query Builder
|
### Query Builder
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
import asyncio
|
||||||
from baradb import Client, QueryBuilder
|
from baradb import Client, QueryBuilder
|
||||||
|
|
||||||
with Client("localhost", 9472) as client:
|
async def main():
|
||||||
qb = (
|
async with Client("localhost", 9472) as client:
|
||||||
QueryBuilder(client)
|
qb = (
|
||||||
.select("name", "email")
|
QueryBuilder(client)
|
||||||
.from_("users")
|
.select("name", "email")
|
||||||
.where("active = true")
|
.from_("users")
|
||||||
.order_by("name")
|
.where("active = true")
|
||||||
.limit(10)
|
.order_by("name")
|
||||||
)
|
.limit(10)
|
||||||
result = qb.exec()
|
)
|
||||||
for row in result:
|
result = await qb.exec()
|
||||||
print(row)
|
for row in result:
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
```
|
```
|
||||||
|
|
||||||
### Vector Search
|
### Vector Search
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
import asyncio
|
||||||
from baradb import Client, WireValue
|
from baradb import Client, WireValue
|
||||||
|
|
||||||
with Client("localhost", 9472) as client:
|
async def main():
|
||||||
result = client.query_params(
|
async with Client("localhost", 9472) as client:
|
||||||
"SELECT id, name FROM products ORDER BY embedding <-> $1 LIMIT 5",
|
result = await client.query_params(
|
||||||
[WireValue.vector([0.1, 0.2, 0.3])],
|
"SELECT id, name FROM products ORDER BY embedding <-> $1 LIMIT 5",
|
||||||
)
|
[WireValue.vector([0.1, 0.2, 0.3])],
|
||||||
|
)
|
||||||
|
print(result.rows)
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
```
|
```
|
||||||
|
|
||||||
## Running Tests
|
## Running Tests
|
||||||
@@ -107,7 +129,7 @@ Integration tests (requires server on `localhost:9472`):
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Start server
|
# Start server
|
||||||
docker run -d -p 9472:9472 baradb:latest
|
docker run -d -p 9472:9472 barabadb:latest
|
||||||
|
|
||||||
# Run all tests
|
# Run all tests
|
||||||
pytest
|
pytest
|
||||||
@@ -124,18 +146,18 @@ pytest
|
|||||||
| `database` | `default` | Default database |
|
| `database` | `default` | Default database |
|
||||||
| `username` | `admin` | Username |
|
| `username` | `admin` | Username |
|
||||||
| `password` | `""` | Password |
|
| `password` | `""` | Password |
|
||||||
| `timeout` | `30` | Socket timeout in seconds |
|
| `timeout` | `30.0` | Socket timeout in seconds |
|
||||||
|
|
||||||
### Methods
|
### Methods (all async)
|
||||||
|
|
||||||
- `connect()` — open TCP connection
|
- `await client.connect()` — open TCP connection
|
||||||
- `close()` — close connection
|
- `await client.close()` — close connection
|
||||||
- `query(sql) -> QueryResult` — execute SELECT-like query
|
- `await client.query(sql) -> QueryResult` — execute SELECT-like query
|
||||||
- `query_params(sql, params) -> QueryResult` — parameterized query
|
- `await client.query_params(sql, params) -> QueryResult` — parameterized query
|
||||||
- `execute(sql) -> int` — execute DDL/DML, returns affected rows
|
- `await client.execute(sql) -> int` — execute DDL/DML, returns affected rows
|
||||||
- `auth(token)` — JWT authentication
|
- `await client.auth(token)` — JWT authentication
|
||||||
- `ping() -> bool` — health check
|
- `await client.ping() -> bool` — health check
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Apache-2.0
|
Apache-2.0
|
||||||
@@ -1,20 +1,34 @@
|
|||||||
"""
|
"""
|
||||||
BaraDB Python Client
|
BaraDB Python Async Client
|
||||||
|
|
||||||
Official Python client for BaraDB — Multimodal Database Engine.
|
Official async Python client for BaraDB — Multimodal Database Engine.
|
||||||
Communicates via the BaraDB Wire Protocol (binary, big-endian, TCP).
|
Communicates via the BaraDB Wire Protocol (binary, big-endian, TCP).
|
||||||
|
|
||||||
Install:
|
Install:
|
||||||
pip install baradb
|
pip install baradb
|
||||||
|
|
||||||
Quick Start:
|
Quick Start:
|
||||||
|
import asyncio
|
||||||
from baradb import Client
|
from baradb import Client
|
||||||
client = Client("localhost", 9472)
|
|
||||||
client.connect()
|
async def main():
|
||||||
result = client.query("SELECT name FROM users WHERE age > 18")
|
client = Client("localhost", 9472)
|
||||||
for row in result:
|
await client.connect()
|
||||||
print(row["name"])
|
result = await client.query("SELECT name FROM users WHERE age > 18")
|
||||||
client.close()
|
for row in result:
|
||||||
|
print(row["name"])
|
||||||
|
await client.close()
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
|
||||||
|
Parameterized Queries:
|
||||||
|
result = await client.query_params(
|
||||||
|
"SELECT * FROM users WHERE age > $1",
|
||||||
|
[WireValue.int64(18)]
|
||||||
|
)
|
||||||
|
|
||||||
|
Authentication:
|
||||||
|
await client.auth("jwt-token-here")
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from .core import (
|
from .core import (
|
||||||
@@ -27,7 +41,7 @@ from .core import (
|
|||||||
ResultFormat,
|
ResultFormat,
|
||||||
)
|
)
|
||||||
|
|
||||||
__version__ = "1.0.0"
|
__version__ = "1.1.0"
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"Client",
|
"Client",
|
||||||
"QueryBuilder",
|
"QueryBuilder",
|
||||||
@@ -36,4 +50,4 @@ __all__ = [
|
|||||||
"MsgKind",
|
"MsgKind",
|
||||||
"FieldKind",
|
"FieldKind",
|
||||||
"ResultFormat",
|
"ResultFormat",
|
||||||
]
|
]
|
||||||
+166
-100
@@ -1,33 +1,38 @@
|
|||||||
"""
|
"""
|
||||||
BaraDB Python Client
|
BaraDB Python Async Client
|
||||||
|
|
||||||
Binary protocol client for BaraDB database.
|
Official async Python client for BaraDB — Multimodal Database Engine.
|
||||||
Communicates via the BaraDB Wire Protocol (binary, big-endian).
|
Communicates via the BaraDB Wire Protocol (binary, big-endian, TCP).
|
||||||
|
|
||||||
Install:
|
Install:
|
||||||
pip install baradb
|
pip install baradb
|
||||||
|
|
||||||
Quick Start:
|
Quick Start:
|
||||||
|
import asyncio
|
||||||
from baradb import Client
|
from baradb import Client
|
||||||
client = Client("localhost", 9472)
|
|
||||||
client.connect()
|
async def main():
|
||||||
result = client.query("SELECT name FROM users WHERE age > 18")
|
client = Client("localhost", 9472)
|
||||||
for row in result:
|
await client.connect()
|
||||||
print(row["name"])
|
result = await client.query("SELECT name FROM users WHERE age > 18")
|
||||||
client.close()
|
for row in result:
|
||||||
|
print(row["name"])
|
||||||
|
await client.close()
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
|
||||||
Parameterized Queries:
|
Parameterized Queries:
|
||||||
result = client.query_params("SELECT * FROM users WHERE age > $1", [WireValue.int64(18)])
|
result = await client.query_params(
|
||||||
|
"SELECT * FROM users WHERE age > $1",
|
||||||
|
[WireValue.int64(18)]
|
||||||
|
)
|
||||||
|
|
||||||
Authentication:
|
Authentication:
|
||||||
client = Client("localhost", 9472, username="admin", password="secret")
|
await client.auth("jwt-token-here")
|
||||||
client.connect()
|
|
||||||
client.auth("jwt-token-here")
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import socket
|
import asyncio
|
||||||
import struct
|
import struct
|
||||||
import json
|
|
||||||
from typing import Any, Optional, Sequence
|
from typing import Any, Optional, Sequence
|
||||||
|
|
||||||
|
|
||||||
@@ -49,7 +54,6 @@ class FieldKind:
|
|||||||
|
|
||||||
|
|
||||||
class MsgKind:
|
class MsgKind:
|
||||||
# Client messages
|
|
||||||
CLIENT_HANDSHAKE = 0x01
|
CLIENT_HANDSHAKE = 0x01
|
||||||
QUERY = 0x02
|
QUERY = 0x02
|
||||||
QUERY_PARAMS = 0x03
|
QUERY_PARAMS = 0x03
|
||||||
@@ -59,7 +63,6 @@ class MsgKind:
|
|||||||
CLOSE = 0x07
|
CLOSE = 0x07
|
||||||
PING = 0x08
|
PING = 0x08
|
||||||
AUTH = 0x09
|
AUTH = 0x09
|
||||||
# Server messages
|
|
||||||
SERVER_HANDSHAKE = 0x80
|
SERVER_HANDSHAKE = 0x80
|
||||||
READY = 0x81
|
READY = 0x81
|
||||||
DATA = 0x82
|
DATA = 0x82
|
||||||
@@ -88,7 +91,7 @@ class WireValue:
|
|||||||
return WireValue(FieldKind.NULL)
|
return WireValue(FieldKind.NULL)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def bool_val(val: bool):
|
def bool(val: bool):
|
||||||
return WireValue(FieldKind.BOOL, val)
|
return WireValue(FieldKind.BOOL, val)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@@ -120,15 +123,15 @@ class WireValue:
|
|||||||
return WireValue(FieldKind.STRING, val)
|
return WireValue(FieldKind.STRING, val)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def bytes_val(val: bytes):
|
def bytes(val: bytes):
|
||||||
return WireValue(FieldKind.BYTES, val)
|
return WireValue(FieldKind.BYTES, val)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def array_val(val: list):
|
def array(val: list):
|
||||||
return WireValue(FieldKind.ARRAY, val)
|
return WireValue(FieldKind.ARRAY, val)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def object_val(val: dict):
|
def object(val: dict):
|
||||||
return WireValue(FieldKind.OBJECT, val)
|
return WireValue(FieldKind.OBJECT, val)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@@ -136,7 +139,7 @@ class WireValue:
|
|||||||
return WireValue(FieldKind.VECTOR, val)
|
return WireValue(FieldKind.VECTOR, val)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def json_val(val: str):
|
def json(val: str):
|
||||||
return WireValue(FieldKind.JSON, val)
|
return WireValue(FieldKind.JSON, val)
|
||||||
|
|
||||||
def serialize(self) -> bytes:
|
def serialize(self) -> bytes:
|
||||||
@@ -202,72 +205,92 @@ class QueryResult:
|
|||||||
|
|
||||||
|
|
||||||
class Client:
|
class Client:
|
||||||
"""BaraDB database client."""
|
"""Async BaraDB database client."""
|
||||||
|
|
||||||
def __init__(self, host: str = "localhost", port: int = 9472,
|
def __init__(self, host: str = "localhost", port: int = 9472,
|
||||||
database: str = "default", username: str = "admin",
|
database: str = "default", username: str = "admin",
|
||||||
password: str = "", timeout: int = 30):
|
password: str = "", timeout: float = 30.0):
|
||||||
self.host = host
|
self.host = host
|
||||||
self.port = port
|
self.port = port
|
||||||
self.database = database
|
self.database = database
|
||||||
self.username = username
|
self.username = username
|
||||||
self.password = password
|
self.password = password
|
||||||
self.timeout = timeout
|
self.timeout = timeout
|
||||||
self._sock: Optional[socket.socket] = None
|
self._reader: Optional[asyncio.StreamReader] = None
|
||||||
|
self._writer: Optional[asyncio.StreamWriter] = None
|
||||||
self._connected = False
|
self._connected = False
|
||||||
self._request_id = 0
|
self._request_id = 0
|
||||||
|
self._buffer = bytearray()
|
||||||
|
self._pending_resolve = None
|
||||||
|
self._request_queue: list = []
|
||||||
|
self._request_lock = False
|
||||||
|
|
||||||
def connect(self) -> None:
|
async def connect(self) -> None:
|
||||||
"""Connect to the BaraDB server."""
|
"""Connect to the BaraDB server."""
|
||||||
self._sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
self._reader, self._writer = await asyncio.wait_for(
|
||||||
self._sock.settimeout(self.timeout)
|
asyncio.open_connection(self.host, self.port),
|
||||||
self._sock.connect((self.host, self.port))
|
timeout=self.timeout
|
||||||
|
)
|
||||||
self._connected = True
|
self._connected = True
|
||||||
|
|
||||||
def close(self) -> None:
|
async def close(self) -> None:
|
||||||
if self._sock:
|
"""Close the connection to the server."""
|
||||||
|
if self._writer and self._connected:
|
||||||
try:
|
try:
|
||||||
msg = self._build_message(MsgKind.CLOSE, b"")
|
msg = self._build_message(MsgKind.CLOSE, b"")
|
||||||
self._sock.send(msg)
|
self._writer.write(msg)
|
||||||
|
await self._writer.drain()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
self._sock.close()
|
if self._writer:
|
||||||
|
self._writer.close()
|
||||||
|
await self._writer.wait_closed()
|
||||||
|
self._writer = None
|
||||||
|
self._reader = None
|
||||||
self._connected = False
|
self._connected = False
|
||||||
|
|
||||||
def is_connected(self) -> bool:
|
def is_connected(self) -> bool:
|
||||||
|
"""Check if client is connected."""
|
||||||
return self._connected
|
return self._connected
|
||||||
|
|
||||||
def _next_id(self) -> int:
|
def _next_id(self) -> int:
|
||||||
self._request_id += 1
|
self._request_id += 1
|
||||||
return self._request_id
|
return self._request_id
|
||||||
|
|
||||||
def _recv_exact(self, size: int) -> bytes:
|
async def _recv_exact(self, size: int) -> bytes:
|
||||||
"""Receive exactly `size` bytes from the socket."""
|
"""Receive exactly `size` bytes from the socket."""
|
||||||
data = b""
|
while len(self._buffer) < size:
|
||||||
while len(data) < size:
|
try:
|
||||||
chunk = self._sock.recv(size - len(data))
|
chunk = await asyncio.wait_for(
|
||||||
if not chunk:
|
self._reader.read(size - len(self._buffer)),
|
||||||
raise ConnectionError("Connection closed by server")
|
timeout=self.timeout
|
||||||
data += chunk
|
)
|
||||||
|
if not chunk:
|
||||||
|
raise ConnectionError("Connection closed by server")
|
||||||
|
self._buffer.extend(chunk)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
raise TimeoutError("Receive timeout")
|
||||||
|
data = bytes(self._buffer[:size])
|
||||||
|
del self._buffer[:size]
|
||||||
return data
|
return data
|
||||||
|
|
||||||
def _read_response_header(self) -> tuple[int, int, int]:
|
async def _read_header(self) -> tuple[int, int, int]:
|
||||||
"""Read a 12-byte message header. Returns (kind, length, request_id)."""
|
"""Read a 12-byte message header. Returns (kind, length, request_id)."""
|
||||||
header = self._recv_exact(12)
|
header = await self._recv_exact(12)
|
||||||
kind, length, req_id = struct.unpack(">III", header)
|
kind, length, req_id = struct.unpack(">III", header)
|
||||||
return kind, length, req_id
|
return kind, length, req_id
|
||||||
|
|
||||||
def _read_error(self, length: int) -> Exception:
|
async def _read_error(self, length: int) -> Exception:
|
||||||
"""Read and parse an ERROR payload."""
|
"""Read and parse an ERROR payload."""
|
||||||
data = self._recv_exact(length)
|
data = await self._recv_exact(length)
|
||||||
code = struct.unpack(">I", data[:4])[0]
|
code = struct.unpack(">I", data[:4])[0]
|
||||||
msg_len = struct.unpack(">I", data[4:8])[0]
|
msg_len = struct.unpack(">I", data[4:8])[0]
|
||||||
error_msg = data[8:8 + msg_len].decode("utf-8")
|
error_msg = data[8:8 + msg_len].decode("utf-8")
|
||||||
return Exception(f"BaraDB error {code}: {error_msg}")
|
return Exception(f"BaraDB error {code}: {error_msg}")
|
||||||
|
|
||||||
def _read_data_response(self, length: int) -> QueryResult:
|
async def _read_data_response(self, length: int) -> QueryResult:
|
||||||
"""Read and parse a DATA payload, then follow up with COMPLETE."""
|
"""Read and parse a DATA payload, then follow up with COMPLETE."""
|
||||||
data = self._recv_exact(length)
|
data = await self._recv_exact(length)
|
||||||
pos = [0]
|
pos = [0]
|
||||||
|
|
||||||
col_count = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
col_count = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||||
@@ -299,95 +322,138 @@ class Client:
|
|||||||
result.rows = rows
|
result.rows = rows
|
||||||
result.row_count = row_count
|
result.row_count = row_count
|
||||||
|
|
||||||
comp_kind, comp_len, _ = self._read_response_header()
|
comp_kind, comp_len, _ = await self._read_header()
|
||||||
if comp_kind == MsgKind.COMPLETE:
|
if comp_kind == MsgKind.COMPLETE:
|
||||||
comp_data = self._recv_exact(comp_len)
|
comp_data = await self._recv_exact(comp_len)
|
||||||
result.affected_rows = struct.unpack(">I", comp_data[:4])[0]
|
result.affected_rows = struct.unpack(">I", comp_data[:4])[0]
|
||||||
elif comp_kind == MsgKind.ERROR:
|
elif comp_kind == MsgKind.ERROR:
|
||||||
raise self._read_error(comp_len)
|
raise await self._read_error(comp_len)
|
||||||
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def auth(self, token: str) -> None:
|
async def auth(self, token: str) -> None:
|
||||||
"""Authenticate with the server using a JWT token."""
|
"""Authenticate with the server using a JWT token."""
|
||||||
|
if not self._connected:
|
||||||
|
raise Exception("Not connected")
|
||||||
encoded = token.encode("utf-8")
|
encoded = token.encode("utf-8")
|
||||||
payload = struct.pack(">I", len(encoded)) + encoded
|
payload = struct.pack(">I", len(encoded)) + encoded
|
||||||
msg = self._build_message(MsgKind.AUTH, payload)
|
msg = self._build_message(MsgKind.AUTH, payload)
|
||||||
self._sock.send(msg)
|
self._writer.write(msg)
|
||||||
|
await self._writer.drain()
|
||||||
|
|
||||||
kind, length, _ = self._read_response_header()
|
kind, length, _ = await self._read_header()
|
||||||
if kind == MsgKind.AUTH_OK:
|
if kind == MsgKind.AUTH_OK:
|
||||||
return
|
return
|
||||||
elif kind == MsgKind.ERROR:
|
elif kind == MsgKind.ERROR:
|
||||||
raise self._read_error(length)
|
raise await self._read_error(length)
|
||||||
else:
|
else:
|
||||||
raise Exception(f"Unexpected auth response: 0x{kind:02x}")
|
raise Exception(f"Unexpected auth response: 0x{kind:02x}")
|
||||||
|
|
||||||
def ping(self) -> bool:
|
async def ping(self) -> bool:
|
||||||
"""Ping the server. Returns True if pong received."""
|
"""Ping the server. Returns True if pong received."""
|
||||||
|
if not self._connected:
|
||||||
|
raise Exception("Not connected")
|
||||||
msg = self._build_message(MsgKind.PING, b"")
|
msg = self._build_message(MsgKind.PING, b"")
|
||||||
self._sock.send(msg)
|
self._writer.write(msg)
|
||||||
|
await self._writer.drain()
|
||||||
|
|
||||||
kind, length, _ = self._read_response_header()
|
kind, length, _ = await self._read_header()
|
||||||
if kind == MsgKind.PONG:
|
if kind == MsgKind.PONG:
|
||||||
return True
|
return True
|
||||||
elif kind == MsgKind.ERROR:
|
elif kind == MsgKind.ERROR:
|
||||||
raise self._read_error(length)
|
raise await self._read_error(length)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def query(self, sql: str) -> QueryResult:
|
async def _process_queue(self) -> None:
|
||||||
|
"""Process queued requests sequentially."""
|
||||||
|
if self._request_lock or len(self._request_queue) == 0:
|
||||||
|
return
|
||||||
|
self._request_lock = True
|
||||||
|
task_data = self._request_queue.pop(0)
|
||||||
|
try:
|
||||||
|
result = await task_data["task"]()
|
||||||
|
task_data["resolve"](result)
|
||||||
|
except Exception as err:
|
||||||
|
task_data["reject"](err)
|
||||||
|
finally:
|
||||||
|
self._request_lock = False
|
||||||
|
asyncio.create_task(self._process_queue())
|
||||||
|
|
||||||
|
def _enqueue(self, task) -> None:
|
||||||
|
"""Add a task to the request queue."""
|
||||||
|
future = asyncio.Future()
|
||||||
|
self._request_queue.append({"task": task, "resolve": future.set_result, "reject": future.set_exception})
|
||||||
|
asyncio.create_task(self._process_queue())
|
||||||
|
return future
|
||||||
|
|
||||||
|
async def query(self, sql: str) -> QueryResult:
|
||||||
"""Execute a BaraQL query."""
|
"""Execute a BaraQL query."""
|
||||||
payload = self._encode_string(sql)
|
if not self._connected:
|
||||||
payload += bytes([ResultFormat.BINARY])
|
raise Exception("Not connected")
|
||||||
|
|
||||||
msg = self._build_message(MsgKind.QUERY, payload)
|
async def _do_query():
|
||||||
self._sock.send(msg)
|
payload = self._encode_string(sql)
|
||||||
|
payload += bytes([ResultFormat.BINARY])
|
||||||
|
|
||||||
kind, length, _ = self._read_response_header()
|
msg = self._build_message(MsgKind.QUERY, payload)
|
||||||
|
self._writer.write(msg)
|
||||||
|
await self._writer.drain()
|
||||||
|
|
||||||
if kind == MsgKind.ERROR:
|
kind, length, _ = await self._read_header()
|
||||||
raise self._read_error(length)
|
|
||||||
|
|
||||||
if kind == MsgKind.DATA:
|
if kind == MsgKind.ERROR:
|
||||||
return self._read_data_response(length)
|
raise await self._read_error(length)
|
||||||
|
|
||||||
if kind == MsgKind.COMPLETE:
|
if kind == MsgKind.DATA:
|
||||||
data = self._recv_exact(length)
|
return await self._read_data_response(length)
|
||||||
result = QueryResult()
|
|
||||||
result.affected_rows = struct.unpack(">I", data[:4])[0]
|
|
||||||
return result
|
|
||||||
|
|
||||||
return QueryResult()
|
if kind == MsgKind.COMPLETE:
|
||||||
|
data = await self._recv_exact(length)
|
||||||
|
result = QueryResult()
|
||||||
|
result.affected_rows = struct.unpack(">I", data[:4])[0]
|
||||||
|
return result
|
||||||
|
|
||||||
def query_params(self, sql: str, params: Sequence[WireValue]) -> QueryResult:
|
return QueryResult()
|
||||||
|
|
||||||
|
return await self._enqueue(_do_query)
|
||||||
|
|
||||||
|
async def query_params(self, sql: str, params: Sequence[WireValue]) -> QueryResult:
|
||||||
"""Execute a parameterized BaraQL query."""
|
"""Execute a parameterized BaraQL query."""
|
||||||
payload = self._encode_string(sql)
|
if not self._connected:
|
||||||
payload += bytes([ResultFormat.BINARY])
|
raise Exception("Not connected")
|
||||||
payload += struct.pack(">I", len(params))
|
|
||||||
for p in params:
|
|
||||||
payload += p.serialize()
|
|
||||||
|
|
||||||
msg = self._build_message(MsgKind.QUERY_PARAMS, payload)
|
async def _do_query_params():
|
||||||
self._sock.send(msg)
|
payload = self._encode_string(sql)
|
||||||
|
payload += bytes([ResultFormat.BINARY])
|
||||||
|
payload += struct.pack(">I", len(params))
|
||||||
|
for p in params:
|
||||||
|
payload += p.serialize()
|
||||||
|
|
||||||
kind, length, _ = self._read_response_header()
|
msg = self._build_message(MsgKind.QUERY_PARAMS, payload)
|
||||||
|
self._writer.write(msg)
|
||||||
|
await self._writer.drain()
|
||||||
|
|
||||||
if kind == MsgKind.ERROR:
|
kind, length, _ = await self._read_header()
|
||||||
raise self._read_error(length)
|
|
||||||
|
|
||||||
if kind == MsgKind.DATA:
|
if kind == MsgKind.ERROR:
|
||||||
return self._read_data_response(length)
|
raise await self._read_error(length)
|
||||||
|
|
||||||
if kind == MsgKind.COMPLETE:
|
if kind == MsgKind.DATA:
|
||||||
data = self._recv_exact(length)
|
return await self._read_data_response(length)
|
||||||
result = QueryResult()
|
|
||||||
result.affected_rows = struct.unpack(">I", data[:4])[0]
|
|
||||||
return result
|
|
||||||
|
|
||||||
return QueryResult()
|
if kind == MsgKind.COMPLETE:
|
||||||
|
data = await self._recv_exact(length)
|
||||||
|
result = QueryResult()
|
||||||
|
result.affected_rows = struct.unpack(">I", data[:4])[0]
|
||||||
|
return result
|
||||||
|
|
||||||
def execute(self, sql: str) -> int:
|
return QueryResult()
|
||||||
result = self.query(sql)
|
|
||||||
|
return await self._enqueue(_do_query_params)
|
||||||
|
|
||||||
|
async def execute(self, sql: str) -> int:
|
||||||
|
"""Execute a query and return affected rows count."""
|
||||||
|
result = await self.query(sql)
|
||||||
return result.affected_rows
|
return result.affected_rows
|
||||||
|
|
||||||
def _build_message(self, kind: int, payload: bytes) -> bytes:
|
def _build_message(self, kind: int, payload: bytes) -> bytes:
|
||||||
@@ -477,12 +543,12 @@ class Client:
|
|||||||
return self._read_string(data, pos)
|
return self._read_string(data, pos)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def __enter__(self):
|
async def __aenter__(self):
|
||||||
self.connect()
|
await self.connect()
|
||||||
return self
|
return self
|
||||||
|
|
||||||
def __exit__(self, *args):
|
async def __aexit__(self, *args):
|
||||||
self.close()
|
await self.close()
|
||||||
|
|
||||||
|
|
||||||
class QueryBuilder:
|
class QueryBuilder:
|
||||||
@@ -559,5 +625,5 @@ class QueryBuilder:
|
|||||||
sql += " OFFSET " + str(self._offset)
|
sql += " OFFSET " + str(self._offset)
|
||||||
return sql
|
return sql
|
||||||
|
|
||||||
def exec(self) -> QueryResult:
|
async def exec(self) -> QueryResult:
|
||||||
return self.client.query(self.build())
|
return await self.client.query(self.build())
|
||||||
@@ -6,35 +6,36 @@ This script demonstrates common operations with the BaraDB Python client.
|
|||||||
Run it while a BaraDB server is listening on localhost:9472.
|
Run it while a BaraDB server is listening on localhost:9472.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
from baradb import Client, QueryBuilder, WireValue
|
from baradb import Client, QueryBuilder, WireValue
|
||||||
|
|
||||||
|
|
||||||
def example_connection():
|
async def example_connection():
|
||||||
print("=== Connection ===")
|
print("=== Connection ===")
|
||||||
client = Client("localhost", 9472)
|
client = Client("localhost", 9472)
|
||||||
client.connect()
|
await client.connect()
|
||||||
print(f"Connected: {client.is_connected()}")
|
print(f"Connected: {client.is_connected()}")
|
||||||
print(f"Ping: {client.ping()}")
|
print(f"Ping: {await client.ping()}")
|
||||||
client.close()
|
await client.close()
|
||||||
print(f"Connected after close: {client.is_connected()}")
|
print(f"Connected after close: {client.is_connected()}")
|
||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
def example_context_manager():
|
async def example_context_manager():
|
||||||
print("=== Context Manager ===")
|
print("=== Context Manager ===")
|
||||||
with Client("localhost", 9472) as client:
|
async with Client("localhost", 9472) as client:
|
||||||
print(f"Inside context: {client.is_connected()}")
|
print(f"Inside context: {client.is_connected()}")
|
||||||
print(f"Ping: {client.ping()}")
|
print(f"Ping: {await client.ping()}")
|
||||||
print("Outside context: closed automatically")
|
print("Outside context: closed automatically")
|
||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
def example_simple_query():
|
async def example_simple_query():
|
||||||
print("=== Simple Query ===")
|
print("=== Simple Query ===")
|
||||||
with Client("localhost", 9472) as client:
|
async with Client("localhost", 9472) as client:
|
||||||
result = client.query("SELECT 42 as answer, 'BaraDB' as db")
|
result = await client.query("SELECT 42 as answer, 'BaraDB' as db")
|
||||||
print(f"Columns: {result.columns}")
|
print(f"Columns: {result.columns}")
|
||||||
print(f"Rows: {result.rows}")
|
print(f"Rows: {result.rows}")
|
||||||
print(f"Row count: {result.row_count}")
|
print(f"Row count: {result.row_count}")
|
||||||
@@ -43,15 +44,15 @@ def example_simple_query():
|
|||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
def example_parameterized_query():
|
async def example_parameterized_query():
|
||||||
print("=== Parameterized Query ===")
|
print("=== Parameterized Query ===")
|
||||||
with Client("localhost", 9472) as client:
|
async with Client("localhost", 9472) as client:
|
||||||
result = client.query_params(
|
result = await client.query_params(
|
||||||
"SELECT $1 as num, $2 as txt, $3 as flag",
|
"SELECT $1 as num, $2 as txt, $3 as flag",
|
||||||
[
|
[
|
||||||
WireValue.int64(123),
|
WireValue.int64(123),
|
||||||
WireValue.string("hello world"),
|
WireValue.string("hello world"),
|
||||||
WireValue.bool_val(True),
|
WireValue.bool(True),
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
for row in result:
|
for row in result:
|
||||||
@@ -59,9 +60,9 @@ def example_parameterized_query():
|
|||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
def example_query_builder():
|
async def example_query_builder():
|
||||||
print("=== Query Builder ===")
|
print("=== Query Builder ===")
|
||||||
with Client("localhost", 9472) as client:
|
async with Client("localhost", 9472) as client:
|
||||||
sql = (
|
sql = (
|
||||||
QueryBuilder(client)
|
QueryBuilder(client)
|
||||||
.select("id", "name")
|
.select("id", "name")
|
||||||
@@ -75,10 +76,10 @@ def example_query_builder():
|
|||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
def example_vector():
|
async def example_vector():
|
||||||
print("=== Vector Value ===")
|
print("=== Vector Value ===")
|
||||||
with Client("localhost", 9472) as client:
|
async with Client("localhost", 9472) as client:
|
||||||
result = client.query_params(
|
result = await client.query_params(
|
||||||
"SELECT $1 as embedding",
|
"SELECT $1 as embedding",
|
||||||
[WireValue.vector([0.1, 0.2, 0.3, 0.4])],
|
[WireValue.vector([0.1, 0.2, 0.3, 0.4])],
|
||||||
)
|
)
|
||||||
@@ -87,34 +88,34 @@ def example_vector():
|
|||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
def example_ddl_dml():
|
async def example_ddl_dml():
|
||||||
print("=== DDL & DML ===")
|
print("=== DDL & DML ===")
|
||||||
with Client("localhost", 9472) as client:
|
async with Client("localhost", 9472) as client:
|
||||||
# Clean up
|
# Clean up
|
||||||
try:
|
try:
|
||||||
client.execute("DROP TABLE IF EXISTS demo_products")
|
await client.execute("DROP TABLE IF EXISTS demo_products")
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
print(f"Cleanup warning: {exc}")
|
print(f"Cleanup warning: {exc}")
|
||||||
|
|
||||||
client.execute(
|
await client.execute(
|
||||||
"CREATE TABLE demo_products (id INT PRIMARY KEY, name STRING, price FLOAT)"
|
"CREATE TABLE demo_products (id INT PRIMARY KEY, name STRING, price FLOAT)"
|
||||||
)
|
)
|
||||||
affected = client.execute(
|
affected = await client.execute(
|
||||||
"INSERT INTO demo_products (id, name, price) VALUES (1, 'Widget', 9.99)"
|
"INSERT INTO demo_products (id, name, price) VALUES (1, 'Widget', 9.99)"
|
||||||
)
|
)
|
||||||
print(f"Insert affected rows: {affected}")
|
print(f"Insert affected rows: {affected}")
|
||||||
|
|
||||||
result = client.query("SELECT * FROM demo_products")
|
result = await client.query("SELECT * FROM demo_products")
|
||||||
print(f"Select returned {result.row_count} row(s)")
|
print(f"Select returned {result.row_count} row(s)")
|
||||||
for row in result:
|
for row in result:
|
||||||
print(f" {row}")
|
print(f" {row}")
|
||||||
|
|
||||||
client.execute("DROP TABLE demo_products")
|
await client.execute("DROP TABLE demo_products")
|
||||||
print("Table dropped")
|
print("Table dropped")
|
||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
def main():
|
async def main():
|
||||||
print("BaraDB Python Client Examples")
|
print("BaraDB Python Client Examples")
|
||||||
print("Make sure BaraDB is running on localhost:9472")
|
print("Make sure BaraDB is running on localhost:9472")
|
||||||
print()
|
print()
|
||||||
@@ -131,10 +132,10 @@ def main():
|
|||||||
|
|
||||||
for fn in examples:
|
for fn in examples:
|
||||||
try:
|
try:
|
||||||
fn()
|
await fn()
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
print(f"ERROR in {fn.__name__}: {exc}", file=sys.stderr)
|
print(f"ERROR in {fn.__name__}: {exc}", file=sys.stderr)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
main()
|
asyncio.run(main())
|
||||||
|
|||||||
@@ -48,6 +48,7 @@ packages = ["baradb"]
|
|||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
testpaths = ["tests"]
|
testpaths = ["tests"]
|
||||||
pythonpath = ["."]
|
pythonpath = ["."]
|
||||||
|
asyncio_mode = "auto"
|
||||||
|
|
||||||
[tool.mypy]
|
[tool.mypy]
|
||||||
strict = true
|
strict = true
|
||||||
|
|||||||
@@ -5,13 +5,16 @@ Requires a running BaraDB server on localhost:9472.
|
|||||||
These tests are skipped automatically if the server is unreachable.
|
These tests are skipped automatically if the server is unreachable.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import os
|
||||||
import socket
|
import socket
|
||||||
import pytest
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
from baradb import Client, WireValue, QueryBuilder
|
from baradb import Client, WireValue, QueryBuilder
|
||||||
|
|
||||||
|
|
||||||
BARADB_HOST = "localhost"
|
BARADB_HOST = os.environ.get("BARADB_HOST", "localhost")
|
||||||
BARADB_PORT = 9472
|
BARADB_PORT = int(os.environ.get("BARADB_PORT", "9472"))
|
||||||
|
|
||||||
|
|
||||||
def _server_available() -> bool:
|
def _server_available() -> bool:
|
||||||
@@ -29,41 +32,41 @@ pytestmark = pytest.mark.skipif(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest_asyncio.fixture
|
||||||
def client():
|
async def client():
|
||||||
c = Client(BARADB_HOST, BARADB_PORT)
|
c = Client(BARADB_HOST, BARADB_PORT)
|
||||||
c.connect()
|
await c.connect()
|
||||||
yield c
|
yield c
|
||||||
c.close()
|
await c.close()
|
||||||
|
|
||||||
|
|
||||||
class TestConnection:
|
class TestConnection:
|
||||||
def test_connect_and_close(self):
|
async def test_connect_and_close(self):
|
||||||
c = Client(BARADB_HOST, BARADB_PORT)
|
c = Client(BARADB_HOST, BARADB_PORT)
|
||||||
assert not c.is_connected()
|
assert not c.is_connected()
|
||||||
c.connect()
|
await c.connect()
|
||||||
assert c.is_connected()
|
assert c.is_connected()
|
||||||
c.close()
|
await c.close()
|
||||||
assert not c.is_connected()
|
assert not c.is_connected()
|
||||||
|
|
||||||
def test_context_manager(self):
|
async def test_context_manager(self):
|
||||||
with Client(BARADB_HOST, BARADB_PORT) as c:
|
async with Client(BARADB_HOST, BARADB_PORT) as c:
|
||||||
assert c.is_connected()
|
assert c.is_connected()
|
||||||
assert not c.is_connected()
|
assert not c.is_connected()
|
||||||
|
|
||||||
|
|
||||||
class TestPing:
|
class TestPing:
|
||||||
def test_ping(self, client):
|
async def test_ping(self, client):
|
||||||
assert client.ping() is True
|
assert await client.ping() is True
|
||||||
|
|
||||||
|
|
||||||
class TestQuery:
|
class TestQuery:
|
||||||
def test_simple_select(self, client):
|
async def test_simple_select(self, client):
|
||||||
result = client.query("SELECT 1 as one")
|
result = await client.query("SELECT 1 as one")
|
||||||
assert result.row_count >= 0 # server may return rows or empty
|
assert result.row_count >= 0 # server may return rows or empty
|
||||||
|
|
||||||
def test_query_with_params(self, client):
|
async def test_query_with_params(self, client):
|
||||||
result = client.query_params(
|
result = await client.query_params(
|
||||||
"SELECT $1 as num, $2 as txt",
|
"SELECT $1 as num, $2 as txt",
|
||||||
[WireValue.int64(42), WireValue.string("hello")],
|
[WireValue.int64(42), WireValue.string("hello")],
|
||||||
)
|
)
|
||||||
@@ -71,22 +74,22 @@ class TestQuery:
|
|||||||
|
|
||||||
|
|
||||||
class TestExecute:
|
class TestExecute:
|
||||||
def test_create_table_and_insert(self, client):
|
async def test_create_table_and_insert(self, client):
|
||||||
# Clean up first (best-effort)
|
# Clean up first (best-effort)
|
||||||
try:
|
try:
|
||||||
client.execute("DROP TABLE IF EXISTS test_users")
|
await client.execute("DROP TABLE IF EXISTS test_users")
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
client.execute(
|
await client.execute(
|
||||||
"CREATE TABLE test_users (id INT PRIMARY KEY, name STRING, age INT)"
|
"CREATE TABLE test_users (id INT PRIMARY KEY, name STRING, age INT)"
|
||||||
)
|
)
|
||||||
affected = client.execute(
|
affected = await client.execute(
|
||||||
"INSERT INTO test_users (id, name, age) VALUES (1, 'Alice', 30)"
|
"INSERT INTO test_users (id, name, age) VALUES (1, 'Alice', 30)"
|
||||||
)
|
)
|
||||||
assert affected >= 0
|
assert affected >= 0
|
||||||
|
|
||||||
result = client.query("SELECT name, age FROM test_users WHERE id = 1")
|
result = await client.query("SELECT name, age FROM test_users WHERE id = 1")
|
||||||
assert result.row_count == 1
|
assert result.row_count == 1
|
||||||
row = result.rows[0]
|
row = result.rows[0]
|
||||||
# Server returns all columns; map by name via dict iteration
|
# Server returns all columns; map by name via dict iteration
|
||||||
@@ -94,20 +97,20 @@ class TestExecute:
|
|||||||
assert row_dict["name"] == "Alice"
|
assert row_dict["name"] == "Alice"
|
||||||
assert row_dict["age"] == 30
|
assert row_dict["age"] == 30
|
||||||
|
|
||||||
client.execute("DROP TABLE test_users")
|
await client.execute("DROP TABLE test_users")
|
||||||
|
|
||||||
|
|
||||||
class TestQueryBuilder:
|
class TestQueryBuilder:
|
||||||
def test_builder_exec(self, client):
|
async def test_builder_exec(self, client):
|
||||||
try:
|
try:
|
||||||
client.execute("DROP TABLE IF EXISTS test_products")
|
await client.execute("DROP TABLE IF EXISTS test_products")
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
client.execute(
|
await client.execute(
|
||||||
"CREATE TABLE test_products (id INT PRIMARY KEY, name STRING, price FLOAT)"
|
"CREATE TABLE test_products (id INT PRIMARY KEY, name STRING, price FLOAT)"
|
||||||
)
|
)
|
||||||
client.execute(
|
await client.execute(
|
||||||
"INSERT INTO test_products (id, name, price) VALUES (1, 'Widget', 9.99)"
|
"INSERT INTO test_products (id, name, price) VALUES (1, 'Widget', 9.99)"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -117,36 +120,36 @@ class TestQueryBuilder:
|
|||||||
.from_("test_products")
|
.from_("test_products")
|
||||||
.where("id = 1")
|
.where("id = 1")
|
||||||
)
|
)
|
||||||
result = qb.exec()
|
result = await qb.exec()
|
||||||
assert result.row_count == 1
|
assert result.row_count == 1
|
||||||
|
|
||||||
client.execute("DROP TABLE test_products")
|
await client.execute("DROP TABLE test_products")
|
||||||
|
|
||||||
|
|
||||||
class TestAuth:
|
class TestAuth:
|
||||||
def test_auth_with_dummy_token(self, client):
|
async def test_auth_with_dummy_token(self, client):
|
||||||
# Server uses default JWT secret in dev mode, any token format is accepted
|
# Server uses default JWT secret in dev mode, any token format is accepted
|
||||||
# depending on server config. We just verify the method does not crash.
|
# depending on server config. We just verify the method does not crash.
|
||||||
try:
|
try:
|
||||||
client.auth("dummy-token-for-testing")
|
await client.auth("dummy-token-for-testing")
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
# Auth may fail with invalid token — that's acceptable for this test
|
# Auth may fail with invalid token — that's acceptable for this test
|
||||||
assert "Auth" in str(exc) or "error" in str(exc).lower()
|
assert "Auth" in str(exc) or "error" in str(exc).lower()
|
||||||
|
|
||||||
|
|
||||||
class TestTransactions:
|
class TestTransactions:
|
||||||
def test_transaction_begin_commit(self, client):
|
async def test_transaction_begin_commit(self, client):
|
||||||
try:
|
try:
|
||||||
client.execute("DROP TABLE IF EXISTS test_txn")
|
await client.execute("DROP TABLE IF EXISTS test_txn")
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
client.execute("CREATE TABLE test_txn (id INT PRIMARY KEY)")
|
await client.execute("CREATE TABLE test_txn (id INT PRIMARY KEY)")
|
||||||
|
|
||||||
client.execute("BEGIN")
|
await client.execute("BEGIN")
|
||||||
client.execute("INSERT INTO test_txn (id) VALUES (1)")
|
await client.execute("INSERT INTO test_txn (id) VALUES (1)")
|
||||||
client.execute("COMMIT")
|
await client.execute("COMMIT")
|
||||||
|
|
||||||
result = client.query("SELECT COUNT(*) FROM test_txn")
|
result = await client.query("SELECT COUNT(*) FROM test_txn")
|
||||||
assert result.row_count >= 0
|
assert result.row_count >= 0
|
||||||
|
|
||||||
client.execute("DROP TABLE test_txn")
|
await client.execute("DROP TABLE test_txn")
|
||||||
|
|||||||
@@ -18,11 +18,11 @@ class TestWireValue:
|
|||||||
assert data == b"\x00"
|
assert data == b"\x00"
|
||||||
|
|
||||||
def test_bool_true(self):
|
def test_bool_true(self):
|
||||||
wv = WireValue.bool_val(True)
|
wv = WireValue.bool(True)
|
||||||
assert wv.serialize() == b"\x01\x01"
|
assert wv.serialize() == b"\x01\x01"
|
||||||
|
|
||||||
def test_bool_false(self):
|
def test_bool_false(self):
|
||||||
wv = WireValue.bool_val(False)
|
wv = WireValue.bool(False)
|
||||||
assert wv.serialize() == b"\x01\x00"
|
assert wv.serialize() == b"\x01\x00"
|
||||||
|
|
||||||
def test_int8(self):
|
def test_int8(self):
|
||||||
@@ -70,7 +70,7 @@ class TestWireValue:
|
|||||||
assert data[5:] == b"hello"
|
assert data[5:] == b"hello"
|
||||||
|
|
||||||
def test_bytes(self):
|
def test_bytes(self):
|
||||||
wv = WireValue.bytes_val(b"\xde\xad\xbe\xef")
|
wv = WireValue.bytes(b"\xde\xad\xbe\xef")
|
||||||
data = wv.serialize()
|
data = wv.serialize()
|
||||||
assert data[0] == FieldKind.BYTES
|
assert data[0] == FieldKind.BYTES
|
||||||
length = struct.unpack(">I", data[1:5])[0]
|
length = struct.unpack(">I", data[1:5])[0]
|
||||||
@@ -87,7 +87,7 @@ class TestWireValue:
|
|||||||
assert floats == [1.0, 2.0, 3.0]
|
assert floats == [1.0, 2.0, 3.0]
|
||||||
|
|
||||||
def test_json(self):
|
def test_json(self):
|
||||||
wv = WireValue.json_val('{"key": "value"}')
|
wv = WireValue.json('{"key": "value"}')
|
||||||
data = wv.serialize()
|
data = wv.serialize()
|
||||||
assert data[0] == FieldKind.JSON
|
assert data[0] == FieldKind.JSON
|
||||||
length = struct.unpack(">I", data[1:5])[0]
|
length = struct.unpack(">I", data[1:5])[0]
|
||||||
@@ -95,7 +95,7 @@ class TestWireValue:
|
|||||||
|
|
||||||
def test_array(self):
|
def test_array(self):
|
||||||
inner = [WireValue.string("a"), WireValue.string("b")]
|
inner = [WireValue.string("a"), WireValue.string("b")]
|
||||||
wv = WireValue.array_val(inner)
|
wv = WireValue.array(inner)
|
||||||
data = wv.serialize()
|
data = wv.serialize()
|
||||||
assert data[0] == FieldKind.ARRAY
|
assert data[0] == FieldKind.ARRAY
|
||||||
count = struct.unpack(">I", data[1:5])[0]
|
count = struct.unpack(">I", data[1:5])[0]
|
||||||
@@ -103,7 +103,7 @@ class TestWireValue:
|
|||||||
|
|
||||||
def test_object(self):
|
def test_object(self):
|
||||||
inner = {"name": WireValue.string("Bara"), "age": WireValue.int32(42)}
|
inner = {"name": WireValue.string("Bara"), "age": WireValue.int32(42)}
|
||||||
wv = WireValue.object_val(inner)
|
wv = WireValue.object(inner)
|
||||||
data = wv.serialize()
|
data = wv.serialize()
|
||||||
assert data[0] == FieldKind.OBJECT
|
assert data[0] == FieldKind.OBJECT
|
||||||
count = struct.unpack(">I", data[1:5])[0]
|
count = struct.unpack(">I", data[1:5])[0]
|
||||||
|
|||||||
@@ -3,9 +3,9 @@ name = "baradb"
|
|||||||
version = "1.1.0"
|
version = "1.1.0"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
authors = ["BaraDB Team <team@baradb.dev>"]
|
authors = ["BaraDB Team <team@baradb.dev>"]
|
||||||
description = "Official Rust client for BaraDB — binary protocol client"
|
description = "Official async Rust client for BaraDB — binary protocol client"
|
||||||
license = "Apache-2.0"
|
license = "Apache-2.0"
|
||||||
repository = "https://github.com/barabadb/baradadb"
|
repository = "https://github.com/barabadb/baradadb.git"
|
||||||
documentation = "https://docs.baradb.dev"
|
documentation = "https://docs.baradb.dev"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
keywords = ["database", "baradb", "multimodal", "client", "wire-protocol"]
|
keywords = ["database", "baradb", "multimodal", "client", "wire-protocol"]
|
||||||
@@ -13,9 +13,11 @@ categories = ["database", "network-programming"]
|
|||||||
rust-version = "1.70"
|
rust-version = "1.70"
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
|
tokio = { version = "1.35", features = ["full"] }
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
|
tokio = { version = "1.35", features = ["full"] }
|
||||||
|
|
||||||
[[example]]
|
[[example]]
|
||||||
name = "basic"
|
name = "basic"
|
||||||
path = "examples/basic.rs"
|
path = "examples/basic.rs"
|
||||||
+42
-36
@@ -1,13 +1,13 @@
|
|||||||
# BaraDB Rust Client
|
# BaraDB Async Rust Client
|
||||||
|
|
||||||
Official Rust client for **BaraDB** — a multimodal database engine written in Nim.
|
Official async Rust client for **BaraDB** — a multimodal database engine written in Nim.
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- **Binary wire protocol** — fast TCP communication using only `std`
|
- **Async/await** — fully non-blocking with Tokio runtime
|
||||||
- **Zero dependencies** — no external crates required
|
- **Binary wire protocol** — fast TCP communication
|
||||||
- **Sync API** — blocking I/O suitable for most applications
|
|
||||||
- **Query builder** — fluent SQL construction
|
- **Query builder** — fluent SQL construction
|
||||||
|
- **Parameterized queries** — safe from SQL injection
|
||||||
- **Vector & JSON support** — first-class multimodal types
|
- **Vector & JSON support** — first-class multimodal types
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
@@ -16,13 +16,14 @@ Add to your `Cargo.toml`:
|
|||||||
|
|
||||||
```toml
|
```toml
|
||||||
[dependencies]
|
[dependencies]
|
||||||
baradb = "1.0"
|
baradb = "1.1"
|
||||||
|
tokio = { version = "1.35", features = ["full"] }
|
||||||
```
|
```
|
||||||
|
|
||||||
Or from source:
|
Or from source:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/barabadb/baradadb.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd clients/rust
|
cd clients/rust
|
||||||
cargo build
|
cargo build
|
||||||
```
|
```
|
||||||
@@ -32,13 +33,14 @@ cargo build
|
|||||||
```rust
|
```rust
|
||||||
use baradb::Client;
|
use baradb::Client;
|
||||||
|
|
||||||
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
#[tokio::main]
|
||||||
let mut client = Client::connect("localhost", 9472)?;
|
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
let result = client.query("SELECT name, age FROM users WHERE age > 18")?;
|
let mut client = Client::connect("localhost", 9472).await?;
|
||||||
|
let result = client.query("SELECT name, age FROM users WHERE age > 18").await?;
|
||||||
for row in result.rows() {
|
for row in result.rows() {
|
||||||
println!("{:?}", row);
|
println!("{:?}", row);
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@@ -48,16 +50,17 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
```rust
|
```rust
|
||||||
use baradb::{Client, WireValue};
|
use baradb::{Client, WireValue};
|
||||||
|
|
||||||
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
#[tokio::main]
|
||||||
let mut client = Client::connect("localhost", 9472)?;
|
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let mut client = Client::connect("localhost", 9472).await?;
|
||||||
let result = client.query_params(
|
let result = client.query_params(
|
||||||
"SELECT * FROM users WHERE age > $1 AND country = $2",
|
"SELECT * FROM users WHERE age > $1 AND country = $2",
|
||||||
&[WireValue::Int64(18), WireValue::String("BG".to_string())],
|
&[WireValue::Int64(18), WireValue::String("BG".to_string())],
|
||||||
)?;
|
).await?;
|
||||||
for row in result.rows() {
|
for row in result.rows() {
|
||||||
println!("{:?}", row);
|
println!("{:?}", row);
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@@ -65,21 +68,23 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
### Query Builder
|
### Query Builder
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
use baradb::Client;
|
use baradb::{Client, QueryBuilder};
|
||||||
|
|
||||||
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
#[tokio::main]
|
||||||
let mut client = Client::connect("localhost", 9472)?;
|
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
let result = baradb::QueryBuilder::new(&mut client)
|
let mut client = Client::connect("localhost", 9472).await?;
|
||||||
|
let result = QueryBuilder::new(&mut client)
|
||||||
.select(&["name", "email"])
|
.select(&["name", "email"])
|
||||||
.from("users")
|
.from("users")
|
||||||
.where_clause("active = true")
|
.where_clause("active = true")
|
||||||
.order_by("name", "ASC")
|
.order_by("name", "ASC")
|
||||||
.limit(10)
|
.limit(10)
|
||||||
.exec()?;
|
.exec()
|
||||||
|
.await?;
|
||||||
for row in result.rows() {
|
for row in result.rows() {
|
||||||
println!("{:?}", row);
|
println!("{:?}", row);
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@@ -89,13 +94,14 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
```rust
|
```rust
|
||||||
use baradb::{Client, WireValue};
|
use baradb::{Client, WireValue};
|
||||||
|
|
||||||
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
#[tokio::main]
|
||||||
let mut client = Client::connect("localhost", 9472)?;
|
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let mut client = Client::connect("localhost", 9472).await?;
|
||||||
let result = client.query_params(
|
let result = client.query_params(
|
||||||
"SELECT id, name FROM products ORDER BY embedding <-> $1 LIMIT 5",
|
"SELECT id, name FROM products ORDER BY embedding <-> $1 LIMIT 5",
|
||||||
&[WireValue::Vector(vec![0.1, 0.2, 0.3])],
|
&[WireValue::Vector(vec![0.1, 0.2, 0.3])],
|
||||||
)?;
|
).await?;
|
||||||
client.close();
|
client.close().await;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@@ -112,7 +118,7 @@ Integration tests (requires server on `localhost:9472`):
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Start server
|
# Start server
|
||||||
docker run -d -p 9472:9472 baradb:latest
|
docker run -d -p 9472:9472 barabadb:latest
|
||||||
|
|
||||||
# Run all tests
|
# Run all tests
|
||||||
cargo test
|
cargo test
|
||||||
@@ -120,19 +126,19 @@ cargo test
|
|||||||
|
|
||||||
## API Reference
|
## API Reference
|
||||||
|
|
||||||
### `Client::connect(host, port)`
|
### `Client::connect(host, port) -> Result<Client>`
|
||||||
|
|
||||||
Creates a new client connected to the given host and port.
|
Creates a new async client connected to the given host and port.
|
||||||
|
|
||||||
### Methods
|
### Methods (all async)
|
||||||
|
|
||||||
- `query(sql) -> Result<QueryResult>` — execute SELECT-like query
|
- `await client.query(sql) -> Result<QueryResult>` — execute SELECT-like query
|
||||||
- `query_params(sql, params) -> Result<QueryResult>` — parameterized query
|
- `await client.query_params(sql, params) -> Result<QueryResult>` — parameterized query
|
||||||
- `execute(sql) -> Result<usize>` — execute DDL/DML, returns affected rows
|
- `await client.execute(sql) -> Result<usize>` — execute DDL/DML, returns affected rows
|
||||||
- `auth(token) -> Result<()>` — JWT authentication
|
- `await client.auth(token) -> Result<()>` — JWT authentication
|
||||||
- `ping() -> Result<bool>` — health check
|
- `await client.ping() -> Result<bool>` — health check
|
||||||
- `close()` — close connection
|
- `await client.close()` — close connection
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Apache-2.0
|
Apache-2.0
|
||||||
@@ -3,34 +3,65 @@
|
|||||||
|
|
||||||
use baradb::{Client, QueryBuilder, WireValue};
|
use baradb::{Client, QueryBuilder, WireValue};
|
||||||
|
|
||||||
fn example_connection() -> Result<(), Box<dyn std::error::Error>> {
|
#[tokio::main]
|
||||||
|
async fn main() {
|
||||||
|
println!("BaraDB Rust Client Examples");
|
||||||
|
println!("Make sure BaraDB is running on localhost:9472");
|
||||||
|
println!();
|
||||||
|
|
||||||
|
if let Err(e) = example_connection().await {
|
||||||
|
eprintln!("ERROR: {}", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Err(e) = example_simple_query().await {
|
||||||
|
eprintln!("ERROR: {}", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Err(e) = example_parameterized_query().await {
|
||||||
|
eprintln!("ERROR: {}", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Err(e) = example_query_builder().await {
|
||||||
|
eprintln!("ERROR: {}", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Err(e) = example_vector().await {
|
||||||
|
eprintln!("ERROR: {}", e);
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Err(e) = example_ddl_dml().await {
|
||||||
|
eprintln!("ERROR: {}", e);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn example_connection() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
println!("=== Connection ===");
|
println!("=== Connection ===");
|
||||||
let mut client = Client::connect("127.0.0.1", 9472)?;
|
let mut client = Client::connect("127.0.0.1", 9472).await?;
|
||||||
println!("Connected: {}", client.is_connected());
|
println!("Connected: {}", client.is_connected());
|
||||||
println!("Ping: {}", client.ping()?);
|
println!("Ping: {}", client.ping().await?);
|
||||||
client.close();
|
client.close().await;
|
||||||
println!("Connected after close: {}", client.is_connected());
|
println!("Connected after close: {}", client.is_connected());
|
||||||
println!();
|
println!();
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn example_simple_query() -> Result<(), Box<dyn std::error::Error>> {
|
async fn example_simple_query() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
println!("=== Simple Query ===");
|
println!("=== Simple Query ===");
|
||||||
let mut client = Client::connect("127.0.0.1", 9472)?;
|
let mut client = Client::connect("127.0.0.1", 9472).await?;
|
||||||
let result = client.query("SELECT 42 as answer, 'BaraDB' as db")?;
|
let result = client.query("SELECT 42 as answer, 'BaraDB' as db").await?;
|
||||||
println!("Columns: {:?}", result.columns());
|
println!("Columns: {:?}", result.columns());
|
||||||
println!("Row count: {}", result.row_count());
|
println!("Row count: {}", result.row_count());
|
||||||
for row in result.rows() {
|
for row in result.rows() {
|
||||||
println!(" {:?}", row);
|
println!(" {:?}", row);
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
println!();
|
println!();
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn example_parameterized_query() -> Result<(), Box<dyn std::error::Error>> {
|
async fn example_parameterized_query() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
println!("=== Parameterized Query ===");
|
println!("=== Parameterized Query ===");
|
||||||
let mut client = Client::connect("127.0.0.1", 9472)?;
|
let mut client = Client::connect("127.0.0.1", 9472).await?;
|
||||||
let result = client.query_params(
|
let result = client.query_params(
|
||||||
"SELECT $1 as num, $2 as txt, $3 as flag",
|
"SELECT $1 as num, $2 as txt, $3 as flag",
|
||||||
&[
|
&[
|
||||||
@@ -38,18 +69,18 @@ fn example_parameterized_query() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
WireValue::String("hello world".to_string()),
|
WireValue::String("hello world".to_string()),
|
||||||
WireValue::Bool(true),
|
WireValue::Bool(true),
|
||||||
],
|
],
|
||||||
)?;
|
).await?;
|
||||||
for row in result.rows() {
|
for row in result.rows() {
|
||||||
println!(" {:?}", row);
|
println!(" {:?}", row);
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
println!();
|
println!();
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn example_query_builder() -> Result<(), Box<dyn std::error::Error>> {
|
async fn example_query_builder() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
println!("=== Query Builder ===");
|
println!("=== Query Builder ===");
|
||||||
let mut client = Client::connect("127.0.0.1", 9472)?;
|
let mut client = Client::connect("127.0.0.1", 9472).await?;
|
||||||
let sql = QueryBuilder::new(&mut client)
|
let sql = QueryBuilder::new(&mut client)
|
||||||
.select(&["id", "name"])
|
.select(&["id", "name"])
|
||||||
.from("users")
|
.from("users")
|
||||||
@@ -58,70 +89,49 @@ fn example_query_builder() -> Result<(), Box<dyn std::error::Error>> {
|
|||||||
.limit(5)
|
.limit(5)
|
||||||
.build();
|
.build();
|
||||||
println!("Generated SQL: {}", sql);
|
println!("Generated SQL: {}", sql);
|
||||||
client.close();
|
client.close().await;
|
||||||
println!();
|
println!();
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn example_vector() -> Result<(), Box<dyn std::error::Error>> {
|
async fn example_vector() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
println!("=== Vector Value ===");
|
println!("=== Vector Value ===");
|
||||||
let mut client = Client::connect("127.0.0.1", 9472)?;
|
let mut client = Client::connect("127.0.0.1", 9472).await?;
|
||||||
let result = client.query_params(
|
let result = client.query_params(
|
||||||
"SELECT $1 as embedding",
|
"SELECT $1 as embedding",
|
||||||
&[WireValue::Vector(vec![0.1, 0.2, 0.3, 0.4])],
|
&[WireValue::Vector(vec![0.1, 0.2, 0.3, 0.4])],
|
||||||
)?;
|
).await?;
|
||||||
for row in result.rows() {
|
for row in result.rows() {
|
||||||
println!(" {:?}", row);
|
println!(" {:?}", row);
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
println!();
|
println!();
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn example_ddl_dml() -> Result<(), Box<dyn std::error::Error>> {
|
async fn example_ddl_dml() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
println!("=== DDL & DML ===");
|
println!("=== DDL & DML ===");
|
||||||
let mut client = Client::connect("127.0.0.1", 9472)?;
|
let mut client = Client::connect("127.0.0.1", 9472).await?;
|
||||||
|
|
||||||
let _ = client.execute("DROP TABLE IF EXISTS demo_products");
|
let _ = client.execute("DROP TABLE IF EXISTS demo_products").await;
|
||||||
|
|
||||||
client.execute(
|
client.execute(
|
||||||
"CREATE TABLE demo_products (id INT PRIMARY KEY, name STRING, price FLOAT)",
|
"CREATE TABLE demo_products (id INT PRIMARY KEY, name STRING, price FLOAT)",
|
||||||
)?;
|
).await?;
|
||||||
let affected = client.execute(
|
let affected = client.execute(
|
||||||
"INSERT INTO demo_products (id, name, price) VALUES (1, 'Widget', 9.99)",
|
"INSERT INTO demo_products (id, name, price) VALUES (1, 'Widget', 9.99)",
|
||||||
)?;
|
).await?;
|
||||||
println!("Insert affected rows: {}", affected);
|
println!("Insert affected rows: {}", affected);
|
||||||
|
|
||||||
let result = client.query("SELECT * FROM demo_products")?;
|
let result = client.query("SELECT * FROM demo_products").await?;
|
||||||
println!("Select returned {} row(s)", result.row_count());
|
println!("Select returned {} row(s)", result.row_count());
|
||||||
for row in result.rows() {
|
for row in result.rows() {
|
||||||
println!(" {:?}", row);
|
println!(" {:?}", row);
|
||||||
}
|
}
|
||||||
|
|
||||||
client.execute("DROP TABLE demo_products")?;
|
client.execute("DROP TABLE demo_products").await?;
|
||||||
println!("Table dropped");
|
println!("Table dropped");
|
||||||
client.close();
|
client.close().await;
|
||||||
println!();
|
println!();
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn main() {
|
|
||||||
println!("BaraDB Rust Client Examples");
|
|
||||||
println!("Make sure BaraDB is running on localhost:9472");
|
|
||||||
println!();
|
|
||||||
|
|
||||||
let examples: Vec<fn() -> Result<(), Box<dyn std::error::Error>>> = vec![
|
|
||||||
example_connection,
|
|
||||||
example_simple_query,
|
|
||||||
example_parameterized_query,
|
|
||||||
example_query_builder,
|
|
||||||
example_vector,
|
|
||||||
example_ddl_dml,
|
|
||||||
];
|
|
||||||
|
|
||||||
for example in examples {
|
|
||||||
if let Err(e) = example() {
|
|
||||||
eprintln!("ERROR: {}", e);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,11 +1,12 @@
|
|||||||
use baradb::Client;
|
use baradb::Client;
|
||||||
|
|
||||||
fn main() {
|
#[tokio::main]
|
||||||
let mut client = Client::connect("localhost", 9472).unwrap();
|
async fn main() {
|
||||||
|
let mut client = Client::connect("localhost", 9472).await.unwrap();
|
||||||
println!("Connected: {}", client.is_connected());
|
println!("Connected: {}", client.is_connected());
|
||||||
match client.ping() {
|
match client.ping().await {
|
||||||
Ok(v) => println!("Ping: {}", v),
|
Ok(v) => println!("Ping: {}", v),
|
||||||
Err(e) => println!("Ping error: {}", e),
|
Err(e) => println!("Ping error: {}", e),
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
}
|
}
|
||||||
|
|||||||
+110
-75
@@ -1,36 +1,42 @@
|
|||||||
//! BaraDB Rust Client
|
//! BaraDB Async Rust Client
|
||||||
//!
|
//!
|
||||||
//! Binary protocol client for BaraDB database.
|
//! Async binary protocol client for BaraDB database.
|
||||||
//! Zero external dependencies — uses only `std`.
|
//! Uses Tokio for async I/O operations.
|
||||||
//!
|
//!
|
||||||
//! # Example
|
//! # Example
|
||||||
//! ```no_run
|
//! ```no_run
|
||||||
//! use baradb::{Client, WireValue};
|
//! use baradb::{Client, WireValue};
|
||||||
//!
|
//!
|
||||||
//! let mut client = Client::connect("localhost", 9472).unwrap();
|
//! #[tokio::main]
|
||||||
//! let result = client.query("SELECT name FROM users WHERE age > 18").unwrap();
|
//! async fn main() {
|
||||||
//! for row in result.rows() {
|
//! let mut client = Client::connect("localhost", 9472).await.unwrap();
|
||||||
//! if let Some(WireValue::String(name)) = row.get("name") {
|
//! let result = client.query("SELECT name FROM users WHERE age > 18").await.unwrap();
|
||||||
//! println!("{}", name);
|
//! for row in result.rows() {
|
||||||
|
//! if let Some(WireValue::String(name)) = row.get("name") {
|
||||||
|
//! println!("{}", name);
|
||||||
|
//! }
|
||||||
//! }
|
//! }
|
||||||
|
//! client.close().await;
|
||||||
//! }
|
//! }
|
||||||
//! client.close();
|
|
||||||
//! ```
|
//! ```
|
||||||
//!
|
//!
|
||||||
//! # Parameterized Queries
|
//! # Parameterized Queries
|
||||||
//! ```no_run
|
//! ```no_run
|
||||||
//! use baradb::{Client, WireValue};
|
//! use baradb::{Client, WireValue};
|
||||||
//!
|
//!
|
||||||
//! let mut client = Client::connect("localhost", 9472).unwrap();
|
//! #[tokio::main]
|
||||||
//! let result = client.query_params(
|
//! async fn main() {
|
||||||
//! "SELECT * FROM users WHERE age > $1",
|
//! let mut client = Client::connect("localhost", 9472).await.unwrap();
|
||||||
//! &[WireValue::Int64(18)],
|
//! let result = client.query_params(
|
||||||
//! ).unwrap();
|
//! "SELECT * FROM users WHERE age > $1",
|
||||||
|
//! &[WireValue::Int64(18)],
|
||||||
|
//! ).await.unwrap();
|
||||||
|
//! }
|
||||||
//! ```
|
//! ```
|
||||||
|
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::io::{Read, Write};
|
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||||
use std::net::TcpStream;
|
use tokio::net::TcpStream;
|
||||||
|
|
||||||
// Client message kinds
|
// Client message kinds
|
||||||
const MK_CLIENT_HANDSHAKE: u32 = 0x01;
|
const MK_CLIENT_HANDSHAKE: u32 = 0x01;
|
||||||
@@ -221,43 +227,41 @@ impl QueryResult {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// BaraDB client
|
/// BaraDB async client
|
||||||
pub struct Client {
|
pub struct Client {
|
||||||
config: Config,
|
|
||||||
stream: TcpStream,
|
stream: TcpStream,
|
||||||
connected: bool,
|
connected: bool,
|
||||||
request_id: u32,
|
request_id: u32,
|
||||||
|
read_buf: Vec<u8>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Client {
|
impl Client {
|
||||||
pub fn connect(host: &str, port: u16) -> Result<Self, Box<dyn std::error::Error>> {
|
/// Connect to a BaraDB server
|
||||||
let config = Config {
|
pub async fn connect(host: &str, port: u16) -> Result<Self, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
host: host.to_string(),
|
let addr = format!("{}:{}", host, port);
|
||||||
port,
|
let stream = TcpStream::connect(&addr).await?;
|
||||||
..Default::default()
|
|
||||||
};
|
|
||||||
Self::connect_with_config(config)
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn connect_with_config(config: Config) -> Result<Self, Box<dyn std::error::Error>> {
|
|
||||||
let addr = format!("{}:{}", config.host, config.port);
|
|
||||||
let stream = TcpStream::connect(&addr)?;
|
|
||||||
stream.set_nodelay(true)?;
|
|
||||||
Ok(Client {
|
Ok(Client {
|
||||||
config,
|
|
||||||
stream,
|
stream,
|
||||||
connected: true,
|
connected: true,
|
||||||
request_id: 0,
|
request_id: 0,
|
||||||
|
read_buf: Vec::new(),
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn close(&mut self) {
|
/// Connect with custom configuration
|
||||||
|
pub async fn connect_with_config(config: Config) -> Result<Self, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
Self::connect(&config.host, config.port).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Close the connection
|
||||||
|
pub async fn close(&mut self) {
|
||||||
if self.connected {
|
if self.connected {
|
||||||
let _ = self.send_close();
|
let _ = self.send_close().await;
|
||||||
}
|
}
|
||||||
self.connected = false;
|
self.connected = false;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Check if connected
|
||||||
pub fn is_connected(&self) -> bool {
|
pub fn is_connected(&self) -> bool {
|
||||||
self.connected
|
self.connected
|
||||||
}
|
}
|
||||||
@@ -267,27 +271,39 @@ impl Client {
|
|||||||
self.request_id
|
self.request_id
|
||||||
}
|
}
|
||||||
|
|
||||||
fn send_close(&mut self) -> Result<(), Box<dyn std::error::Error>> {
|
async fn send_close(&mut self) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
let msg = build_message(MK_CLOSE, self.next_id(), &[]);
|
let msg = build_message(MK_CLOSE, self.next_id(), &[]);
|
||||||
self.stream.write_all(&msg)?;
|
self.stream.write_all(&msg).await?;
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn read_header(&mut self) -> Result<(u32, u32, u32), Box<dyn std::error::Error>> {
|
async fn read_exact(&mut self, mut n: usize) -> Result<Vec<u8>, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
let mut header = [0u8; 12];
|
let mut buf = vec![0u8; n];
|
||||||
self.stream.read_exact(&mut header)?;
|
let mut pos = 0;
|
||||||
|
while pos < n {
|
||||||
|
let read = self.stream.read(&mut buf[pos..]).await?;
|
||||||
|
if read == 0 {
|
||||||
|
return Err("Connection closed".into());
|
||||||
|
}
|
||||||
|
pos += read;
|
||||||
|
}
|
||||||
|
Ok(buf)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn read_header(&mut self) -> Result<(u32, u32, u32), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let header = self.read_exact(12).await?;
|
||||||
let kind = u32::from_be_bytes([header[0], header[1], header[2], header[3]]);
|
let kind = u32::from_be_bytes([header[0], header[1], header[2], header[3]]);
|
||||||
let length = u32::from_be_bytes([header[4], header[5], header[6], header[7]]);
|
let length = u32::from_be_bytes([header[4], header[5], header[6], header[7]]);
|
||||||
let req_id = u32::from_be_bytes([header[8], header[9], header[10], header[11]]);
|
let req_id = u32::from_be_bytes([header[8], header[9], header[10], header[11]]);
|
||||||
Ok((kind, length, req_id))
|
Ok((kind, length, req_id))
|
||||||
}
|
}
|
||||||
|
|
||||||
fn read_payload(&mut self, length: u32) -> Result<Vec<u8>, Box<dyn std::error::Error>> {
|
async fn read_payload(&mut self, length: u32) -> Result<Vec<u8>, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
let mut payload = vec![0u8; length as usize];
|
|
||||||
if length > 0 {
|
if length > 0 {
|
||||||
self.stream.read_exact(&mut payload)?;
|
self.read_exact(length as usize).await
|
||||||
|
} else {
|
||||||
|
Ok(vec![])
|
||||||
}
|
}
|
||||||
Ok(payload)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn read_error_message(payload: &[u8]) -> String {
|
fn read_error_message(payload: &[u8]) -> String {
|
||||||
@@ -302,26 +318,26 @@ impl Client {
|
|||||||
"Query error".to_string()
|
"Query error".to_string()
|
||||||
}
|
}
|
||||||
|
|
||||||
fn read_data_response(&mut self, payload: &[u8]) -> Result<QueryResult, Box<dyn std::error::Error>> {
|
async fn read_data_response(&mut self, payload: &[u8]) -> Result<QueryResult, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
let mut pos = 0usize;
|
let mut pos = 0usize;
|
||||||
let col_count = read_u32(payload, &mut pos) as usize;
|
let col_count = read_u32(payload, &mut pos);
|
||||||
|
|
||||||
let mut columns = Vec::with_capacity(col_count);
|
let mut columns = Vec::with_capacity(col_count as usize);
|
||||||
for _ in 0..col_count {
|
for _ in 0..col_count {
|
||||||
columns.push(read_string(payload, &mut pos));
|
columns.push(read_string(payload, &mut pos));
|
||||||
}
|
}
|
||||||
|
|
||||||
let mut col_types = Vec::with_capacity(col_count);
|
let mut col_types = Vec::with_capacity(col_count as usize);
|
||||||
for _ in 0..col_count {
|
for _ in 0..col_count {
|
||||||
col_types.push(payload[pos]);
|
col_types.push(payload[pos]);
|
||||||
pos += 1;
|
pos += 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
let row_count = read_u32(payload, &mut pos) as usize;
|
let row_count = read_u32(payload, &mut pos);
|
||||||
let mut rows = Vec::with_capacity(row_count);
|
let mut rows = Vec::with_capacity(row_count as usize);
|
||||||
for _ in 0..row_count {
|
for _ in 0..row_count {
|
||||||
let mut row = HashMap::new();
|
let mut row = HashMap::new();
|
||||||
for c in 0..col_count {
|
for c in 0..col_count as usize {
|
||||||
let val = read_wire_value(payload, &mut pos);
|
let val = read_wire_value(payload, &mut pos);
|
||||||
row.insert(columns[c].clone(), val);
|
row.insert(columns[c].clone(), val);
|
||||||
}
|
}
|
||||||
@@ -329,9 +345,9 @@ impl Client {
|
|||||||
}
|
}
|
||||||
|
|
||||||
let mut affected = 0usize;
|
let mut affected = 0usize;
|
||||||
let (comp_kind, comp_len, _) = self.read_header()?;
|
let (comp_kind, comp_len, _) = self.read_header().await?;
|
||||||
if comp_kind == MK_COMPLETE {
|
if comp_kind == MK_COMPLETE {
|
||||||
let comp_payload = self.read_payload(comp_len)?;
|
let comp_payload = self.read_payload(comp_len).await?;
|
||||||
if comp_payload.len() >= 4 {
|
if comp_payload.len() >= 4 {
|
||||||
affected = u32::from_be_bytes([comp_payload[0], comp_payload[1], comp_payload[2], comp_payload[3]]) as usize;
|
affected = u32::from_be_bytes([comp_payload[0], comp_payload[1], comp_payload[2], comp_payload[3]]) as usize;
|
||||||
}
|
}
|
||||||
@@ -340,44 +356,47 @@ impl Client {
|
|||||||
Ok(QueryResult { columns, column_types: col_types, rows, affected_rows: affected })
|
Ok(QueryResult { columns, column_types: col_types, rows, affected_rows: affected })
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn auth(&mut self, token: &str) -> Result<(), Box<dyn std::error::Error>> {
|
/// Authenticate with JWT token
|
||||||
|
pub async fn auth(&mut self, token: &str) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||||
if !self.connected {
|
if !self.connected {
|
||||||
return Err("Not connected".into());
|
return Err("Not connected".into());
|
||||||
}
|
}
|
||||||
let payload = encode_string(token);
|
let payload = encode_string(token);
|
||||||
let msg = build_message(MK_AUTH, self.next_id(), &payload);
|
let msg = build_message(MK_AUTH, self.next_id(), &payload);
|
||||||
self.stream.write_all(&msg)?;
|
self.stream.write_all(&msg).await?;
|
||||||
|
|
||||||
let (kind, length, _) = self.read_header()?;
|
let (kind, length, _) = self.read_header().await?;
|
||||||
match kind {
|
match kind {
|
||||||
MK_AUTH_OK => Ok(()),
|
MK_AUTH_OK => Ok(()),
|
||||||
MK_ERROR => {
|
MK_ERROR => {
|
||||||
let p = self.read_payload(length)?;
|
let p = self.read_payload(length).await?;
|
||||||
Err(Self::read_error_message(&p).into())
|
Err(Self::read_error_message(&p).into())
|
||||||
}
|
}
|
||||||
_ => Err(format!("Unexpected auth response: 0x{:02x}", kind).into()),
|
_ => Err(format!("Unexpected auth response: 0x{:02x}", kind).into()),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn ping(&mut self) -> Result<bool, Box<dyn std::error::Error>> {
|
/// Ping the server
|
||||||
|
pub async fn ping(&mut self) -> Result<bool, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
if !self.connected {
|
if !self.connected {
|
||||||
return Err("Not connected".into());
|
return Err("Not connected".into());
|
||||||
}
|
}
|
||||||
let msg = build_message(MK_PING, self.next_id(), &[]);
|
let msg = build_message(MK_PING, self.next_id(), &[]);
|
||||||
self.stream.write_all(&msg)?;
|
self.stream.write_all(&msg).await?;
|
||||||
|
|
||||||
let (kind, length, _) = self.read_header()?;
|
let (kind, length, _) = self.read_header().await?;
|
||||||
match kind {
|
match kind {
|
||||||
MK_PONG => Ok(true),
|
MK_PONG => Ok(true),
|
||||||
MK_ERROR => {
|
MK_ERROR => {
|
||||||
let p = self.read_payload(length)?;
|
let p = self.read_payload(length).await?;
|
||||||
Err(Self::read_error_message(&p).into())
|
Err(Self::read_error_message(&p).into())
|
||||||
}
|
}
|
||||||
_ => Ok(false),
|
_ => Ok(false),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn query(&mut self, sql: &str) -> Result<QueryResult, Box<dyn std::error::Error>> {
|
/// Execute a query
|
||||||
|
pub async fn query(&mut self, sql: &str) -> Result<QueryResult, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
if !self.connected {
|
if !self.connected {
|
||||||
return Err("Not connected".into());
|
return Err("Not connected".into());
|
||||||
}
|
}
|
||||||
@@ -386,14 +405,14 @@ impl Client {
|
|||||||
payload.push(0x00); // ResultFormat::BINARY
|
payload.push(0x00); // ResultFormat::BINARY
|
||||||
|
|
||||||
let msg = build_message(MK_QUERY, self.next_id(), &payload);
|
let msg = build_message(MK_QUERY, self.next_id(), &payload);
|
||||||
self.stream.write_all(&msg)?;
|
self.stream.write_all(&msg).await?;
|
||||||
|
|
||||||
let (kind, length, _) = self.read_header()?;
|
let (kind, length, _) = self.read_header().await?;
|
||||||
let resp_payload = self.read_payload(length)?;
|
let resp_payload = self.read_payload(length).await?;
|
||||||
|
|
||||||
match kind {
|
match kind {
|
||||||
MK_READY => Ok(QueryResult { columns: vec![], column_types: vec![], rows: vec![], affected_rows: 0 }),
|
MK_READY => Ok(QueryResult { columns: vec![], column_types: vec![], rows: vec![], affected_rows: 0 }),
|
||||||
MK_DATA => self.read_data_response(&resp_payload),
|
MK_DATA => self.read_data_response(&resp_payload).await,
|
||||||
MK_COMPLETE => {
|
MK_COMPLETE => {
|
||||||
let affected = if resp_payload.len() >= 4 {
|
let affected = if resp_payload.len() >= 4 {
|
||||||
u32::from_be_bytes([resp_payload[0], resp_payload[1], resp_payload[2], resp_payload[3]]) as usize
|
u32::from_be_bytes([resp_payload[0], resp_payload[1], resp_payload[2], resp_payload[3]]) as usize
|
||||||
@@ -405,7 +424,8 @@ impl Client {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn query_params(&mut self, sql: &str, params: &[WireValue]) -> Result<QueryResult, Box<dyn std::error::Error>> {
|
/// Execute a parameterized query
|
||||||
|
pub async fn query_params(&mut self, sql: &str, params: &[WireValue]) -> Result<QueryResult, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
if !self.connected {
|
if !self.connected {
|
||||||
return Err("Not connected".into());
|
return Err("Not connected".into());
|
||||||
}
|
}
|
||||||
@@ -418,14 +438,14 @@ impl Client {
|
|||||||
}
|
}
|
||||||
|
|
||||||
let msg = build_message(MK_QUERY_PARAMS, self.next_id(), &payload);
|
let msg = build_message(MK_QUERY_PARAMS, self.next_id(), &payload);
|
||||||
self.stream.write_all(&msg)?;
|
self.stream.write_all(&msg).await?;
|
||||||
|
|
||||||
let (kind, length, _) = self.read_header()?;
|
let (kind, length, _) = self.read_header().await?;
|
||||||
let resp_payload = self.read_payload(length)?;
|
let resp_payload = self.read_payload(length).await?;
|
||||||
|
|
||||||
match kind {
|
match kind {
|
||||||
MK_READY => Ok(QueryResult { columns: vec![], column_types: vec![], rows: vec![], affected_rows: 0 }),
|
MK_READY => Ok(QueryResult { columns: vec![], column_types: vec![], rows: vec![], affected_rows: 0 }),
|
||||||
MK_DATA => self.read_data_response(&resp_payload),
|
MK_DATA => self.read_data_response(&resp_payload).await,
|
||||||
MK_COMPLETE => {
|
MK_COMPLETE => {
|
||||||
let affected = if resp_payload.len() >= 4 {
|
let affected = if resp_payload.len() >= 4 {
|
||||||
u32::from_be_bytes([resp_payload[0], resp_payload[1], resp_payload[2], resp_payload[3]]) as usize
|
u32::from_be_bytes([resp_payload[0], resp_payload[1], resp_payload[2], resp_payload[3]]) as usize
|
||||||
@@ -437,12 +457,14 @@ impl Client {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn execute(&mut self, sql: &str) -> Result<usize, Box<dyn std::error::Error>> {
|
/// Execute and return affected rows
|
||||||
let result = self.query(sql)?;
|
pub async fn execute(&mut self, sql: &str) -> Result<usize, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
|
let result = self.query(sql).await?;
|
||||||
Ok(result.affected_rows())
|
Ok(result.affected_rows())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Query builder for fluent SQL construction
|
||||||
pub struct QueryBuilder<'a> {
|
pub struct QueryBuilder<'a> {
|
||||||
client: &'a mut Client,
|
client: &'a mut Client,
|
||||||
select_cols: Vec<String>,
|
select_cols: Vec<String>,
|
||||||
@@ -457,6 +479,7 @@ pub struct QueryBuilder<'a> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl<'a> QueryBuilder<'a> {
|
impl<'a> QueryBuilder<'a> {
|
||||||
|
/// Create a new query builder
|
||||||
pub fn new(client: &'a mut Client) -> Self {
|
pub fn new(client: &'a mut Client) -> Self {
|
||||||
QueryBuilder {
|
QueryBuilder {
|
||||||
client,
|
client,
|
||||||
@@ -472,56 +495,67 @@ impl<'a> QueryBuilder<'a> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Add columns to SELECT
|
||||||
pub fn select(mut self, cols: &[&str]) -> Self {
|
pub fn select(mut self, cols: &[&str]) -> Self {
|
||||||
self.select_cols.extend(cols.iter().map(|s| s.to_string()));
|
self.select_cols.extend(cols.iter().map(|s| s.to_string()));
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Set the FROM table
|
||||||
pub fn from(mut self, table: &str) -> Self {
|
pub fn from(mut self, table: &str) -> Self {
|
||||||
self.from_table = table.to_string();
|
self.from_table = table.to_string();
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Add a WHERE clause
|
||||||
pub fn where_clause(mut self, clause: &str) -> Self {
|
pub fn where_clause(mut self, clause: &str) -> Self {
|
||||||
self.where_clauses.push(clause.to_string());
|
self.where_clauses.push(clause.to_string());
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Add a JOIN clause
|
||||||
pub fn join(mut self, table: &str, on: &str) -> Self {
|
pub fn join(mut self, table: &str, on: &str) -> Self {
|
||||||
self.joins.push(format!("JOIN {} ON {}", table, on));
|
self.joins.push(format!("JOIN {} ON {}", table, on));
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Add a LEFT JOIN clause
|
||||||
pub fn left_join(mut self, table: &str, on: &str) -> Self {
|
pub fn left_join(mut self, table: &str, on: &str) -> Self {
|
||||||
self.joins.push(format!("LEFT JOIN {} ON {}", table, on));
|
self.joins.push(format!("LEFT JOIN {} ON {}", table, on));
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Add GROUP BY columns
|
||||||
pub fn group_by(mut self, cols: &[&str]) -> Self {
|
pub fn group_by(mut self, cols: &[&str]) -> Self {
|
||||||
self.group_by.extend(cols.iter().map(|s| s.to_string()));
|
self.group_by.extend(cols.iter().map(|s| s.to_string()));
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Add HAVING clause
|
||||||
pub fn having(mut self, clause: &str) -> Self {
|
pub fn having(mut self, clause: &str) -> Self {
|
||||||
self.having = clause.to_string();
|
self.having = clause.to_string();
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Add ORDER BY column
|
||||||
pub fn order_by(mut self, col: &str, dir: &str) -> Self {
|
pub fn order_by(mut self, col: &str, dir: &str) -> Self {
|
||||||
self.order_by.push(format!("{} {}", col, dir));
|
self.order_by.push(format!("{} {}", col, dir));
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Set LIMIT
|
||||||
pub fn limit(mut self, n: usize) -> Self {
|
pub fn limit(mut self, n: usize) -> Self {
|
||||||
self.limit = n;
|
self.limit = n;
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Set OFFSET
|
||||||
pub fn offset(mut self, n: usize) -> Self {
|
pub fn offset(mut self, n: usize) -> Self {
|
||||||
self.offset = n;
|
self.offset = n;
|
||||||
self
|
self
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Build the SQL string
|
||||||
pub fn build(&self) -> String {
|
pub fn build(&self) -> String {
|
||||||
let mut sql = String::from("SELECT ");
|
let mut sql = String::from("SELECT ");
|
||||||
if self.select_cols.is_empty() {
|
if self.select_cols.is_empty() {
|
||||||
@@ -555,9 +589,10 @@ impl<'a> QueryBuilder<'a> {
|
|||||||
sql
|
sql
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn exec(self) -> Result<QueryResult, Box<dyn std::error::Error>> {
|
/// Execute the query
|
||||||
|
pub async fn exec(self) -> Result<QueryResult, Box<dyn std::error::Error + Send + Sync>> {
|
||||||
let sql = self.build();
|
let sql = self.build();
|
||||||
self.client.query(&sql)
|
self.client.query(&sql).await
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -675,4 +710,4 @@ fn read_wire_value(data: &[u8], pos: &mut usize) -> WireValue {
|
|||||||
FK_JSON => WireValue::Json(read_string(data, pos)),
|
FK_JSON => WireValue::Json(read_string(data, pos)),
|
||||||
_ => WireValue::Null,
|
_ => WireValue::Null,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,104 +1,118 @@
|
|||||||
// BaraDB Rust Client — Integration Tests
|
// BaraDB Rust Client — Integration Tests
|
||||||
// Requires a running BaraDB server on localhost:9472.
|
// Requires a running BaraDB server.
|
||||||
|
// Set BARADB_HOST / BARADB_PORT env vars to override defaults.
|
||||||
|
|
||||||
use std::net::TcpStream;
|
use std::net::TcpStream;
|
||||||
use std::time::Duration;
|
|
||||||
|
|
||||||
use baradb::{Client, QueryBuilder, WireValue};
|
use baradb::{Client, QueryBuilder, WireValue};
|
||||||
|
|
||||||
const HOST: &str = "127.0.0.1";
|
fn host() -> String {
|
||||||
const PORT: u16 = 9472;
|
std::env::var("BARADB_HOST").unwrap_or_else(|_| "127.0.0.1".to_string())
|
||||||
|
|
||||||
fn server_available() -> bool {
|
|
||||||
TcpStream::connect((HOST, PORT)).is_ok()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
fn port() -> u16 {
|
||||||
fn test_connect_and_close() {
|
std::env::var("BARADB_PORT")
|
||||||
|
.ok()
|
||||||
|
.and_then(|s| s.parse().ok())
|
||||||
|
.unwrap_or(9472)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn server_available() -> bool {
|
||||||
|
TcpStream::connect((host().as_str(), port())).is_ok()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn test_connect_and_close() {
|
||||||
if !server_available() {
|
if !server_available() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mut client = Client::connect(HOST, PORT).unwrap();
|
let mut client = Client::connect(&host(), port()).await.unwrap();
|
||||||
assert!(client.is_connected());
|
assert!(client.is_connected());
|
||||||
client.close();
|
client.close().await;
|
||||||
assert!(!client.is_connected());
|
assert!(!client.is_connected());
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[tokio::test]
|
||||||
fn test_ping() {
|
async fn test_ping() {
|
||||||
if !server_available() {
|
if !server_available() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mut client = Client::connect(HOST, PORT).unwrap();
|
let mut client = Client::connect(&host(), port()).await.unwrap();
|
||||||
assert!(client.ping().unwrap());
|
assert!(client.ping().await.unwrap());
|
||||||
client.close();
|
client.close().await;
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[tokio::test]
|
||||||
fn test_simple_select() {
|
async fn test_simple_select() {
|
||||||
if !server_available() {
|
if !server_available() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mut client = Client::connect(HOST, PORT).unwrap();
|
let mut client = Client::connect(&host(), port()).await.unwrap();
|
||||||
let result = client.query("SELECT 1 as one").unwrap();
|
let result = client.query("SELECT 1 as one").await.unwrap();
|
||||||
assert!(result.row_count() >= 0);
|
assert!(result.row_count() >= 0);
|
||||||
client.close();
|
client.close().await;
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[tokio::test]
|
||||||
fn test_parameterized_query() {
|
async fn test_parameterized_query() {
|
||||||
if !server_available() {
|
if !server_available() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mut client = Client::connect(HOST, PORT).unwrap();
|
let mut client = Client::connect(&host(), port()).await.unwrap();
|
||||||
let result = client
|
let result = client
|
||||||
.query_params(
|
.query_params(
|
||||||
"SELECT $1 as num, $2 as txt",
|
"SELECT $1 as num, $2 as txt",
|
||||||
&[WireValue::Int64(42), WireValue::String("hello".to_string())],
|
&[WireValue::Int64(42), WireValue::String("hello".to_string())],
|
||||||
)
|
)
|
||||||
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
assert!(result.row_count() >= 0);
|
assert!(result.row_count() >= 0);
|
||||||
client.close();
|
client.close().await;
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[tokio::test]
|
||||||
fn test_create_table_insert_select_drop() {
|
async fn test_create_table_insert_select_drop() {
|
||||||
if !server_available() {
|
if !server_available() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mut client = Client::connect(HOST, PORT).unwrap();
|
let mut client = Client::connect(&host(), port()).await.unwrap();
|
||||||
|
|
||||||
let _ = client.execute("DROP TABLE IF EXISTS rust_test_users");
|
let _ = client.execute("DROP TABLE IF EXISTS rust_test_users").await;
|
||||||
client
|
client
|
||||||
.execute("CREATE TABLE rust_test_users (id INT PRIMARY KEY, name STRING, age INT)")
|
.execute("CREATE TABLE rust_test_users (id INT PRIMARY KEY, name STRING, age INT)")
|
||||||
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
let affected = client
|
let affected = client
|
||||||
.execute("INSERT INTO rust_test_users (id, name, age) VALUES (1, 'Alice', 30)")
|
.execute("INSERT INTO rust_test_users (id, name, age) VALUES (1, 'Alice', 30)")
|
||||||
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
assert!(affected >= 0);
|
assert!(affected >= 0);
|
||||||
|
|
||||||
let result = client
|
let result = client
|
||||||
.query("SELECT name, age FROM rust_test_users WHERE id = 1")
|
.query("SELECT name, age FROM rust_test_users WHERE id = 1")
|
||||||
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
assert_eq!(result.row_count(), 1);
|
assert_eq!(result.row_count(), 1);
|
||||||
|
|
||||||
client.execute("DROP TABLE rust_test_users").unwrap();
|
client.execute("DROP TABLE rust_test_users").await.unwrap();
|
||||||
client.close();
|
client.close().await;
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[tokio::test]
|
||||||
fn test_query_builder_exec() {
|
async fn test_query_builder_exec() {
|
||||||
if !server_available() {
|
if !server_available() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mut client = Client::connect(HOST, PORT).unwrap();
|
let mut client = Client::connect(&host(), port()).await.unwrap();
|
||||||
|
|
||||||
let _ = client.execute("DROP TABLE IF EXISTS rust_test_products");
|
let _ = client.execute("DROP TABLE IF EXISTS rust_test_products").await;
|
||||||
client
|
client
|
||||||
.execute("CREATE TABLE rust_test_products (id INT PRIMARY KEY, name STRING, price FLOAT)")
|
.execute("CREATE TABLE rust_test_products (id INT PRIMARY KEY, name STRING, price FLOAT)")
|
||||||
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
client
|
client
|
||||||
.execute("INSERT INTO rust_test_products (id, name, price) VALUES (1, 'Widget', 9.99)")
|
.execute("INSERT INTO rust_test_products (id, name, price) VALUES (1, 'Widget', 9.99)")
|
||||||
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
let result = QueryBuilder::new(&mut client)
|
let result = QueryBuilder::new(&mut client)
|
||||||
@@ -106,21 +120,22 @@ fn test_query_builder_exec() {
|
|||||||
.from("rust_test_products")
|
.from("rust_test_products")
|
||||||
.where_clause("id = 1")
|
.where_clause("id = 1")
|
||||||
.exec()
|
.exec()
|
||||||
|
.await
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
|
||||||
assert_eq!(result.row_count(), 1);
|
assert_eq!(result.row_count(), 1);
|
||||||
|
|
||||||
client.execute("DROP TABLE rust_test_products").unwrap();
|
client.execute("DROP TABLE rust_test_products").await.unwrap();
|
||||||
client.close();
|
client.close().await;
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[tokio::test]
|
||||||
fn test_auth_dummy_token() {
|
async fn test_auth_dummy_token() {
|
||||||
if !server_available() {
|
if !server_available() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mut client = Client::connect(HOST, PORT).unwrap();
|
let mut client = Client::connect(&host(), port()).await.unwrap();
|
||||||
let res = client.auth("dummy-token-for-testing");
|
let res = client.auth("dummy-token-for-testing").await;
|
||||||
// Dev server may accept or reject — both are fine
|
// Dev server may accept or reject — both are fine
|
||||||
match res {
|
match res {
|
||||||
Ok(()) => {}
|
Ok(()) => {}
|
||||||
@@ -129,5 +144,5 @@ fn test_auth_dummy_token() {
|
|||||||
assert!(msg.contains("Auth") || msg.to_lowercase().contains("error"));
|
assert!(msg.contains("Auth") || msg.to_lowercase().contains("error"));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
client.close();
|
client.close().await;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# BaraDB — Client Integration Test Stack
|
||||||
|
#
|
||||||
|
# This compose file starts BaraDB server and runs all client test suites.
|
||||||
|
# Because test containers run in parallel, do NOT use --abort-on-container-exit
|
||||||
|
# (the first finished container would kill the others).
|
||||||
|
#
|
||||||
|
# Recommended usage:
|
||||||
|
# ./scripts/test-clients.sh
|
||||||
|
#
|
||||||
|
# Or manually run each suite:
|
||||||
|
# docker compose -f docker-compose.test.yml up -d baradb
|
||||||
|
# docker compose -f docker-compose.test.yml run --rm test-python
|
||||||
|
# docker compose -f docker-compose.test.yml run --rm test-javascript
|
||||||
|
# docker compose -f docker-compose.test.yml run --rm test-nim
|
||||||
|
# docker compose -f docker-compose.test.yml run --rm test-rust
|
||||||
|
# docker compose -f docker-compose.test.yml down
|
||||||
|
|
||||||
|
services:
|
||||||
|
baradb:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
image: baradb:latest
|
||||||
|
ports:
|
||||||
|
- "9472:9472"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "sh", "-c", "wget -qO- http://localhost:9912/health >/dev/null 2>&1"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
start_period: 10s
|
||||||
|
|
||||||
|
test-python:
|
||||||
|
image: python:3.12-slim
|
||||||
|
depends_on:
|
||||||
|
baradb:
|
||||||
|
condition: service_healthy
|
||||||
|
environment:
|
||||||
|
- BARADB_HOST=baradb
|
||||||
|
- BARADB_PORT=9472
|
||||||
|
volumes:
|
||||||
|
- ./clients/python:/workspace
|
||||||
|
working_dir: /workspace
|
||||||
|
command: >
|
||||||
|
sh -c "
|
||||||
|
pip install pytest pytest-asyncio -q &&
|
||||||
|
pip install -e ".[dev]" -q &&
|
||||||
|
pytest tests/test_wire_protocol.py tests/test_query_builder.py -v &&
|
||||||
|
pytest tests/test_integration.py -v
|
||||||
|
"
|
||||||
|
|
||||||
|
test-javascript:
|
||||||
|
image: node:20-slim
|
||||||
|
depends_on:
|
||||||
|
baradb:
|
||||||
|
condition: service_healthy
|
||||||
|
environment:
|
||||||
|
- BARADB_HOST=baradb
|
||||||
|
- BARADB_PORT=9472
|
||||||
|
volumes:
|
||||||
|
- ./clients/javascript:/workspace
|
||||||
|
working_dir: /workspace
|
||||||
|
command: >
|
||||||
|
sh -c "
|
||||||
|
node --test tests/unit.test.js &&
|
||||||
|
node --test tests/integration.test.js
|
||||||
|
"
|
||||||
|
|
||||||
|
test-nim:
|
||||||
|
image: nimlang/nim:2.2.10
|
||||||
|
depends_on:
|
||||||
|
baradb:
|
||||||
|
condition: service_healthy
|
||||||
|
environment:
|
||||||
|
- BARADB_HOST=baradb
|
||||||
|
- BARADB_PORT=9472
|
||||||
|
volumes:
|
||||||
|
- ./clients/nim:/workspace
|
||||||
|
- ./clients/nim/src:/workspace/src
|
||||||
|
working_dir: /workspace
|
||||||
|
command: >
|
||||||
|
sh -c "
|
||||||
|
nim c --path:src -r tests/test_client.nim &&
|
||||||
|
nim c --path:src -r tests/test_integration.nim
|
||||||
|
"
|
||||||
|
|
||||||
|
test-rust:
|
||||||
|
image: rust:1.78
|
||||||
|
depends_on:
|
||||||
|
baradb:
|
||||||
|
condition: service_healthy
|
||||||
|
environment:
|
||||||
|
- BARADB_HOST=baradb
|
||||||
|
- BARADB_PORT=9472
|
||||||
|
volumes:
|
||||||
|
- ./clients/rust:/workspace
|
||||||
|
working_dir: /workspace
|
||||||
|
command: >
|
||||||
|
sh -c "
|
||||||
|
cargo test -- --test-threads=1
|
||||||
|
"
|
||||||
@@ -90,6 +90,12 @@ The query layer processes BaraQL — a SQL-compatible query language with extens
|
|||||||
- **Quantization** (`quant.nim`): Scalar 8-bit/4-bit, product, and binary quantization for compression.
|
- **Quantization** (`quant.nim`): Scalar 8-bit/4-bit, product, and binary quantization for compression.
|
||||||
- **SIMD Operations** (`simd.nim`): Unrolled loop distance computations (cosine, Euclidean, dot product, Manhattan).
|
- **SIMD Operations** (`simd.nim`): Unrolled loop distance computations (cosine, Euclidean, dot product, Manhattan).
|
||||||
- **Batch Operations**: batchInsert, batchSearch, batchDistance for high-throughput.
|
- **Batch Operations**: batchInsert, batchSearch, batchDistance for high-throughput.
|
||||||
|
- **SQL Integration** (`query/executor.nim`):
|
||||||
|
- `VECTOR(n)` column type with dimension validation
|
||||||
|
- `CREATE INDEX ... USING hnsw` / `USING ivfpq`
|
||||||
|
- Distance functions: `cosine_distance()`, `euclidean_distance()`, `inner_product()`, `l1_distance()`, `l2_distance()`
|
||||||
|
- `<->` nearest-neighbor operator
|
||||||
|
- Automatic index maintenance on INSERT/UPDATE
|
||||||
|
|
||||||
### Graph Engine (`graph/`)
|
### Graph Engine (`graph/`)
|
||||||
- **Adjacency List** (`engine.nim`): Edge-weighted directed graph storage with forward/reverse adjacency.
|
- **Adjacency List** (`engine.nim`): Edge-weighted directed graph storage with forward/reverse adjacency.
|
||||||
|
|||||||
+168
-1
@@ -97,5 +97,172 @@ RETURN friend.name;
|
|||||||
```sql
|
```sql
|
||||||
BEGIN;
|
BEGIN;
|
||||||
INSERT users { name := 'Alice', age := 30 };
|
INSERT users { name := 'Alice', age := 30 };
|
||||||
|
INSERT orders { user_id := last_insert_id(), total := 100 };
|
||||||
COMMIT;
|
COMMIT;
|
||||||
```
|
|
||||||
|
-- مع نقطة حفظ
|
||||||
|
BEGIN;
|
||||||
|
INSERT users { name := 'Bob', age := 25 };
|
||||||
|
SAVEPOINT sp1;
|
||||||
|
INSERT orders { user_id := last_insert_id(), total := 200 };
|
||||||
|
-- خطأ، التراجع إلى نقطة الحفظ
|
||||||
|
ROLLBACK TO sp1;
|
||||||
|
COMMIT;
|
||||||
|
```
|
||||||
|
|
||||||
|
## البحث النصي الكامل
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- بحث أساسي
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('database programming');
|
||||||
|
|
||||||
|
-- مع درجة الصلة
|
||||||
|
SELECT title, relevance()
|
||||||
|
FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('Nim language')
|
||||||
|
ORDER BY relevance() DESC;
|
||||||
|
|
||||||
|
-- الوضع المنطقي
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('+Nim -Python' IN BOOLEAN MODE);
|
||||||
|
|
||||||
|
-- البحث الضبابي
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title) AGAINST('programing' WITH FUZZINESS 2);
|
||||||
|
```
|
||||||
|
|
||||||
|
## الدوال المحددة من المستخدم
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- تسجيل UDF
|
||||||
|
CREATE FUNCTION greet(name str) -> str {
|
||||||
|
RETURN 'Hello, ' || name || '!';
|
||||||
|
};
|
||||||
|
|
||||||
|
-- استخدامها
|
||||||
|
SELECT greet(name) FROM users;
|
||||||
|
|
||||||
|
-- الدوال المدمجة
|
||||||
|
SELECT abs(-5), sqrt(16), lower('HELLO'), len('test');
|
||||||
|
```
|
||||||
|
|
||||||
|
## تلميحات الاستعلام
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- فرض استخدام الفهرس
|
||||||
|
SELECT /*+ USE_INDEX(idx_users_age) */ * FROM users WHERE age > 18;
|
||||||
|
|
||||||
|
-- فرض البحث المتجهي التقريبي
|
||||||
|
SELECT /*+ APPROXIMATE */ * FROM vectors
|
||||||
|
ORDER BY cosine_distance(embedding, [...])
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- التنفيذ المتوازي
|
||||||
|
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||||
|
```
|
||||||
|
|
||||||
|
## دوال النوافذ
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- دوال الترتيب
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
department,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||||
|
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||||
|
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- دوال القيمة
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
salary,
|
||||||
|
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||||
|
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||||
|
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||||
|
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- دوال التوزيع
|
||||||
|
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||||
|
```
|
||||||
|
|
||||||
|
### مواصفات الإطار
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- إطار ROWS
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
|
||||||
|
-- إطار RANGE
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## ERP متعدد المستأجرين
|
||||||
|
|
||||||
|
يدعم BaraDB تشغيل عدة شركات (مستأجرين) في مثيل قاعدة بيانات واحد، باستخدام **أمان مستوى الصف (RLS)** مع **متغيرات الجلسة**.
|
||||||
|
|
||||||
|
### متغيرات الجلسة
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET app.tenant_id = 'company-123';
|
||||||
|
SELECT current_setting('app.tenant_id') AS tenant;
|
||||||
|
```
|
||||||
|
|
||||||
|
### المستخدم / الدور الحالي
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT current_user AS me, current_role AS my_role;
|
||||||
|
```
|
||||||
|
|
||||||
|
### عزل المستأجر عبر RLS
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- تمكين RLS على جدول
|
||||||
|
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
|
||||||
|
|
||||||
|
-- إنشاء سياسة تصفية حسب المستأجر
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- كل جلسة ترى فقط بياناتها
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- صفوف company-a فقط
|
||||||
|
```
|
||||||
|
|
||||||
|
### لماذا متعدد المستأجرين؟
|
||||||
|
|
||||||
|
- **مثيل واحد، مستأجرون كثيرون** — لا حاجة لتشغيل 100 قاعدة بيانات منفصلة
|
||||||
|
- **مستندات JSONB** — تخزين مخطط مرن، سهل إضافة حقول لكل مستأجر
|
||||||
|
- **RLS يضمن العزل** — قاعدة البيانات تفرض حدود المستأجر، وليس فقط التطبيق
|
||||||
|
|
||||||
|
## الكلمات المفتاحية المدعومة
|
||||||
|
|
||||||
|
| الفئة | الكلمات المفتاحية |
|
||||||
|
|----------|----------|
|
||||||
|
| DQL | SELECT, FROM, WHERE, ORDER BY, GROUP BY, HAVING, LIMIT, OFFSET, DISTINCT |
|
||||||
|
| DML | INSERT, UPDATE, DELETE, SET, VALUES |
|
||||||
|
| DDL | CREATE TYPE, DROP TYPE, CREATE INDEX, DROP INDEX, ALTER TYPE |
|
||||||
|
| Join | INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, CROSS JOIN, ON |
|
||||||
|
| Set | UNION, UNION ALL, INTERSECT, EXCEPT |
|
||||||
|
| CTEs | WITH, RECURSIVE, AS |
|
||||||
|
| Case | CASE, WHEN, THEN, ELSE, END |
|
||||||
|
| Transaction | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
||||||
|
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
||||||
|
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||||
|
| Vector | cosine_distance, euclidean_distance, inner_product, l1_distance, l2_distance, <-> |
|
||||||
|
| JSON | ->, ->> |
|
||||||
|
| FTS | @@ (تطابق BM25) |
|
||||||
|
| Recovery | RECOVER TO TIMESTAMP |
|
||||||
|
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id, current_setting |
|
||||||
|
| Session | SET, current_setting, current_user, current_role |
|
||||||
|
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||||
|
| Window Functions | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
## البداية السريعة
|
## البداية السريعة
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
|
||||||
docker build -t baradb:latest .
|
docker build -t baradb:latest .
|
||||||
|
|||||||
+247
-1
@@ -22,38 +22,284 @@
|
|||||||
### لينكس
|
### لينكس
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# المثبت الرسمي
|
||||||
curl https://nim-lang.org/choosenim/init.sh -sSf | sh
|
curl https://nim-lang.org/choosenim/init.sh -sSf | sh
|
||||||
|
|
||||||
|
# Ubuntu/Debian
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install nim
|
||||||
|
|
||||||
|
# Fedora
|
||||||
|
sudo dnf install nim
|
||||||
|
|
||||||
|
# Arch Linux
|
||||||
|
sudo pacman -S nim
|
||||||
```
|
```
|
||||||
|
|
||||||
### macOS
|
### macOS
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Homebrew
|
||||||
brew install nim
|
brew install nim
|
||||||
|
|
||||||
|
# MacPorts
|
||||||
|
sudo port install nim
|
||||||
```
|
```
|
||||||
|
|
||||||
### ويندوز
|
### ويندوز
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
|
# باستخدام choosenim
|
||||||
|
curl.exe -A "MSYS2_$(uname -m)" -L https://nim-lang.org/choosenim/init.ps1 | powershell -
|
||||||
|
|
||||||
|
# باستخدام winget
|
||||||
winget install nim
|
winget install nim
|
||||||
|
|
||||||
|
# باستخدام scoop
|
||||||
|
scoop install nim
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### التحقق من التثبيت
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim --version
|
||||||
|
# المتوقع: Nim Compiler Version 2.2.0 أو أحدث
|
||||||
|
```
|
||||||
|
|
||||||
|
## تثبيت OpenSSL
|
||||||
|
|
||||||
|
### لينكس
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Ubuntu/Debian
|
||||||
|
sudo apt-get install libssl-dev
|
||||||
|
|
||||||
|
# Fedora
|
||||||
|
sudo dnf install openssl-devel
|
||||||
|
|
||||||
|
# Arch Linux
|
||||||
|
sudo pacman -S openssl
|
||||||
|
```
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
|
||||||
|
OpenSSL مضمن مع النظام. إذا لزم الأمر:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
brew install openssl
|
||||||
|
```
|
||||||
|
|
||||||
|
### ويندوز
|
||||||
|
|
||||||
|
OpenSSL مضمن مع توزيع Nim ويندوز. للبناء اليدوي، قم بالتنزيل من [slproweb.com](https://slproweb.com/products/Win32OpenSSL.html).
|
||||||
|
|
||||||
## بناء BaraDB
|
## بناء BaraDB
|
||||||
|
|
||||||
|
### استنساخ المستودع
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
```
|
||||||
|
|
||||||
|
### تثبيت التبعيات
|
||||||
|
|
||||||
|
```bash
|
||||||
nimble install -d -y
|
nimble install -d -y
|
||||||
|
```
|
||||||
|
|
||||||
|
### خيارات البناء
|
||||||
|
|
||||||
|
#### بناء التصحيح
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
#### بناء الإصدار (موصى به)
|
||||||
|
|
||||||
|
```bash
|
||||||
nim c -d:ssl -d:release --opt:speed -o:build/baradadb src/baradadb.nim
|
nim c -d:ssl -d:release --opt:speed -o:build/baradadb src/baradadb.nim
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### استخدام مهام Nimble
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# بناء التصحيح
|
||||||
|
nimble build_debug
|
||||||
|
|
||||||
|
# بناء الإصدار
|
||||||
|
nimble build_release
|
||||||
|
```
|
||||||
|
|
||||||
|
#### تقليل حجم الملف الثنائي
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release --opt:size -o:build/baradadb src/baradadb.nim
|
||||||
|
strip build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### التحقق من البناء
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./build/baradadb --version
|
||||||
|
# المتوقع: BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
```
|
||||||
|
|
||||||
|
## تشغيل الاختبارات
|
||||||
|
|
||||||
|
### جميع الاختبارات
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -r tests/test_all.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### مجموعات اختبارات محددة
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# اختبارات التخزين
|
||||||
|
nim c -d:ssl -r tests/test_storage.nim
|
||||||
|
|
||||||
|
# اختبارات محرك الاستعلام
|
||||||
|
nim c -d:ssl -r tests/test_query.nim
|
||||||
|
|
||||||
|
# اختبارات البروتوكول
|
||||||
|
nim c -d:ssl -r tests/test_protocol.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### المعايير
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release -r benchmarks/bench_all.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
## خيارات التثبيت
|
||||||
|
|
||||||
|
### التثبيت على مستوى النظام
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# بناء إصدار RELEASE
|
||||||
|
nimble build_release
|
||||||
|
|
||||||
|
# التثبيت في /usr/local/bin
|
||||||
|
sudo cp build/baradadb /usr/local/bin/
|
||||||
|
sudo chmod +x /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# إنشاء دليل البيانات
|
||||||
|
sudo mkdir -p /var/lib/baradb
|
||||||
|
sudo chmod 755 /var/lib/baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
### الملف الثنائي المبنى مسبقًا
|
||||||
|
|
||||||
|
قم بتنزيل أحدث إصدار لمنصتك:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# لينكس x86_64
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-linux-amd64
|
||||||
|
chmod +x baradadb-linux-amd64
|
||||||
|
mv baradadb-linux-amd64 /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# لينكس ARM64
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-linux-arm64
|
||||||
|
chmod +x baradadb-linux-arm64
|
||||||
|
mv baradadb-linux-arm64 /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# macOS
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-darwin-amd64
|
||||||
|
chmod +x baradadb-darwin-amd64
|
||||||
|
mv baradadb-darwin-amd64 /usr/local/bin/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# سحب الصورة الرسمية
|
||||||
|
docker pull barabadb/barabadb:latest
|
||||||
|
|
||||||
|
# التشغيل
|
||||||
|
docker run -d \
|
||||||
|
-p 9472:9472 \
|
||||||
|
-p 9470:9470 \
|
||||||
|
-p 9471:9471 \
|
||||||
|
-v baradb_data:/data \
|
||||||
|
barabadb/barabadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Compose
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker-compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### الاستخدام المدمج (مشاريع Nim)
|
||||||
|
|
||||||
|
أضف إلى ملف `.nimble` الخاص بك:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
requires "barabadb >= 1.1.0"
|
||||||
|
```
|
||||||
|
|
||||||
|
استخدم في الكود:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/lsm
|
||||||
|
|
||||||
|
var db = newLSMTree("./data")
|
||||||
|
db.put("key", cast[seq[byte]]("value"))
|
||||||
|
let (found, val) = db.get("key")
|
||||||
|
db.close()
|
||||||
|
```
|
||||||
|
|
||||||
## التشغيل الأول
|
## التشغيل الأول
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# بدء الخادم
|
||||||
./build/baradadb
|
./build/baradadb
|
||||||
|
|
||||||
|
# المخرجات المتوقعة:
|
||||||
|
# BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
# BaraDB TCP listening on 127.0.0.1:9472
|
||||||
|
|
||||||
|
# الاختبار عبر HTTP API
|
||||||
curl http://localhost:9470/health
|
curl http://localhost:9470/health
|
||||||
|
|
||||||
|
# الصدفة التفاعلية
|
||||||
./build/baradadb --shell
|
./build/baradadb --shell
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## حل مشكلات التثبيت
|
||||||
|
|
||||||
|
### "cannot open file: hunos"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nimble install -d -y
|
||||||
|
```
|
||||||
|
|
||||||
|
### "BaraDB requires SSL support"
|
||||||
|
|
||||||
|
قم دائمًا بالبناء باستخدام `-d:ssl`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### البناء البطيء
|
||||||
|
|
||||||
|
استخدم البناء المتوازي:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release --parallelBuild:4 -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### حجم الملف الثنائي الكبير
|
||||||
|
|
||||||
|
استخدم تحسين الحجم:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release --opt:size --passL:-s -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
## الخطوات التالية
|
## الخطوات التالية
|
||||||
|
|
||||||
- [دليل البداية السريعة](quickstart.md)
|
- [دليل البداية السريعة](quickstart.md)
|
||||||
|
|||||||
@@ -0,0 +1,107 @@
|
|||||||
|
# API за Бинарен Протокол
|
||||||
|
|
||||||
|
Ниско-нивов wire протокол за високопроизводителни клиентски връзки.
|
||||||
|
|
||||||
|
## Формат на Съобщенията
|
||||||
|
|
||||||
|
Всички съобщения използват big-endian byte order:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────┬─────────────┬─────────────┬─────────────────────┐
|
||||||
|
│ Kind │ Length │ RequestId │ Payload │
|
||||||
|
│ (4 bytes) │ (4 bytes) │ (4 bytes) │ │
|
||||||
|
│ uint32 BE │ uint32 BE │ uint32 BE │ (Length bytes) │
|
||||||
|
└─────────────┴─────────────┴─────────────┴─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Типове Съобщения
|
||||||
|
|
||||||
|
### Query (0x01)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let msg = makeQueryMessage(requestId, "SELECT * FROM users")
|
||||||
|
```
|
||||||
|
|
||||||
|
### QueryParams (0x02)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let msg = makeQueryParamsMessage(requestId, "SELECT * FROM users WHERE name = ?", params)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Auth (0x07)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let msg = makeAuthMessage(requestId, token)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ping (0x09)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let msg = makePingMessage(requestId)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Close (0x0A)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let msg = makeCloseMessage(requestId)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Отговорни Съобщения
|
||||||
|
|
||||||
|
### Ready (0x05)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let msg = makeReadyMessage(requestId)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Error (0x06)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let msg = makeErrorMessage(requestId, code, message)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data (0x81)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
# Съдържа резултатите от заявката с колони и редове
|
||||||
|
```
|
||||||
|
|
||||||
|
### Complete (0x82)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
# Потвърждава завършване на заявката
|
||||||
|
```
|
||||||
|
|
||||||
|
### Auth_OK (0x83)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
# Потвърждава успешна автентикация
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pong (0x84)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
# Keepalive отговор
|
||||||
|
```
|
||||||
|
|
||||||
|
## Кодове за Грешки
|
||||||
|
|
||||||
|
| Код | Име | Описание |
|
||||||
|
|------|-----|----------|
|
||||||
|
| 0x00 | OK | Успех |
|
||||||
|
| 0x01 | ERROR | Обща грешка |
|
||||||
|
| 0x02 | AUTH_REQUIRED | Изисква се автентикация |
|
||||||
|
| 0x03 | INVALID_QUERY | Синтактична грешка в заявката |
|
||||||
|
| 0x04 | NOT_FOUND | Ресурсът не е намерен |
|
||||||
|
|
||||||
|
## Сериализация
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/wire
|
||||||
|
|
||||||
|
# Сериализиране на стойност
|
||||||
|
let bytes = serializeValue(Value(kind: vkString, strVal: "test"))
|
||||||
|
|
||||||
|
# Десериализиране на стойност
|
||||||
|
let value = deserializeValue(bytes)
|
||||||
|
```
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
# HTTP/REST API
|
||||||
|
|
||||||
|
JSON-базиран REST API за уеб приложения.
|
||||||
|
|
||||||
|
## Базов URL
|
||||||
|
|
||||||
|
```
|
||||||
|
http://localhost:9470
|
||||||
|
```
|
||||||
|
|
||||||
|
## Endpoints
|
||||||
|
|
||||||
|
### GET /health
|
||||||
|
|
||||||
|
Health проверка:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/health
|
||||||
|
```
|
||||||
|
|
||||||
|
### GET /ready
|
||||||
|
|
||||||
|
Readiness проверка:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/ready
|
||||||
|
```
|
||||||
|
|
||||||
|
### POST /query
|
||||||
|
|
||||||
|
Изпълнение на BaraQL заявка:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/api/query \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"query": "SELECT * FROM users WHERE age > 18"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Отговор:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"columns": ["name", "age"],
|
||||||
|
"rows": [["Alice", 30], ["Bob", 25]],
|
||||||
|
"row_count": 2,
|
||||||
|
"duration_ms": 12
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### POST /batch
|
||||||
|
|
||||||
|
Групови заявки:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/api/batch \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"queries": ["INSERT users { name := \"Alice\" }", "INSERT users { name := \"Bob\" }"]}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### GET /schema
|
||||||
|
|
||||||
|
Преглед на схемата:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/api/schema
|
||||||
|
```
|
||||||
|
|
||||||
|
### GET /metrics
|
||||||
|
|
||||||
|
Prometheus метрики:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
### POST /explain
|
||||||
|
|
||||||
|
Обяснение на план за изпълнение:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/api/explain \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"query": "SELECT * FROM users WHERE age > 18"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### POST /backup
|
||||||
|
|
||||||
|
Създаване на backup:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/api/backup \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"destination": "/backup/snapshot.db"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Грешки
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"error": {
|
||||||
|
"code": "INVALID_QUERY",
|
||||||
|
"message": "Грешка в синтаксиса"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Автентикация
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -H "Authorization: Bearer <token>" \
|
||||||
|
http://localhost:9470/api/users
|
||||||
|
```
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
# WebSocket API
|
||||||
|
|
||||||
|
Full-duplex стрийминг за данни в реално време и push известия.
|
||||||
|
|
||||||
|
## Свързване
|
||||||
|
|
||||||
|
```
|
||||||
|
ws://localhost:9471
|
||||||
|
```
|
||||||
|
|
||||||
|
## Клиентски Пример
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
const ws = new WebSocket('ws://localhost:9471');
|
||||||
|
|
||||||
|
ws.onopen = () => {
|
||||||
|
console.log('Свързан');
|
||||||
|
ws.send(JSON.stringify({
|
||||||
|
type: 'query',
|
||||||
|
query: 'SELECT * FROM users'
|
||||||
|
}));
|
||||||
|
};
|
||||||
|
|
||||||
|
ws.onmessage = (event) => {
|
||||||
|
const data = JSON.parse(event.data);
|
||||||
|
console.log('Получено:', data);
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
## Формат на Съобщенията
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "query",
|
||||||
|
"id": "1",
|
||||||
|
"query": "SELECT * FROM users"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Типове Съобщения
|
||||||
|
|
||||||
|
### Заявка (query)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "query",
|
||||||
|
"id": "1",
|
||||||
|
"query": "SELECT * FROM users"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Резултат (result)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "result",
|
||||||
|
"id": "1",
|
||||||
|
"columns": ["id", "name"],
|
||||||
|
"rows": [["1", "Alice"], ["2", "Bob"]]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Грешка (error)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "error",
|
||||||
|
"id": "1",
|
||||||
|
"code": "INVALID_QUERY",
|
||||||
|
"message": "Синтактична грешка"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Абониране (subscribe)
|
||||||
|
|
||||||
|
Абониране за промени в таблица:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "subscribe",
|
||||||
|
"id": "sub1",
|
||||||
|
"table": "users"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Известие (notification)
|
||||||
|
|
||||||
|
Push известие от сървъра:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "notification",
|
||||||
|
"table": "users",
|
||||||
|
"operation": "insert",
|
||||||
|
"data": {"id": 3, "name": "Charlie"}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ping/Pong (keepalive)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"type": "ping", "id": "ping1"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Отговор:
|
||||||
|
```json
|
||||||
|
{"type": "pong", "id": "ping1"}
|
||||||
|
```
|
||||||
|
|
||||||
|
## JavaScript Клиент
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
class BaraDBClient {
|
||||||
|
constructor(url) {
|
||||||
|
this.ws = new WebSocket(url);
|
||||||
|
this.pending = new Map();
|
||||||
|
}
|
||||||
|
|
||||||
|
query(sql) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const id = crypto.randomUUID();
|
||||||
|
this.pending.set(id, { resolve, reject });
|
||||||
|
this.ws.send(JSON.stringify({ type: 'query', id, query: sql }));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
+69
-94
@@ -2,150 +2,125 @@
|
|||||||
|
|
||||||
## Преглед
|
## Преглед
|
||||||
|
|
||||||
BaraDB е **мултимодална база данни** написана на Nim, която комбинира документно (KV), графично, векторно, колонно и пълнотекстово търсене в един двигател с обединен език за заявки наречен **BaraQL**.
|
BaraDB е **мултимодален database engine**, написан на Nim, който комбинира документно (KV), графово, векторно, колонково и пълнотекстово съхранение в един engine с унифициран език за заявки наречен **BaraQL**.
|
||||||
|
|
||||||
## Слоеста Архитектура
|
## Слоеве на Архитектурата
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────┐
|
||||||
│ 1. СЛОЙ ЗА КЛИЕНТИ │
|
│ 1. КЛИЕНТСКИ СЛОЙ │
|
||||||
│ Binary Protocol │ HTTP/REST │ WebSocket │ Embedded │
|
│ Бинарен Протокол │ HTTP/REST │ WebSocket │ Embedded │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ 2. ЗАЯВКИ СЛОЙ (BaraQL) │
|
│ 2. QUERY СЛОЙ (BaraQL) │
|
||||||
│ Lexer → Parser → AST → IR → Optimizer → Codegen │
|
│ Lexer → Parser → AST → IR → Optimizer → Codegen │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ 3. ИЗПЪЛНИТЕЛЕН ДВИГАТЕЛ │
|
│ 3. ИЗПЪЛНИТЕЛЕН ENGINE │
|
||||||
│ Document │ Graph │ Vector │ Columnar │ FTS │
|
│ Документен │ Графов │ Векторен │ Колонков │ FTS │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ 4. СЪХРАНЕНИЕ │
|
│ 4. СЪХРАНЕНИЕ │
|
||||||
│ LSM-Tree │ B-Tree │ WAL │ Bloom │ Compaction │ Cache │
|
│ LSM-Tree │ B-Tree │ WAL │ Bloom │ Compaction │ Cache │
|
||||||
├─────────────────────────────────────────────────────────┤
|
├─────────────────────────────────────────────────────────┤
|
||||||
│ 5. РАЗПРЕДЕЛЕНО │
|
│ 5. РАЗПРЕДЕЛЕНИ │
|
||||||
│ Raft Consensus │ Sharding │ Replication │ Gossip │
|
│ Raft Консенсус │ Шардиране │ Репликация │ Gossip │
|
||||||
└─────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
## Слой 1: Клиентски Слой
|
## Слой 1: Клиентски Слой
|
||||||
|
|
||||||
Множество протоколи за комуникация:
|
Множество комуникационни протоколи:
|
||||||
|
|
||||||
- **Binary Protocol** (`protocol/wire.nim`): Ефективен big-endian бинарен протокол с 16 типа съобщения
|
- **Бинарен Протокол** (`protocol/wire.nim`): Ефективен big-endian бинарен протокол с 16 типа съобщения
|
||||||
- **HTTP/REST** (`core/httpserver.nim`): JSON REST API с мулти-трединг
|
- **HTTP/REST** (`core/httpserver.nim`): JSON-базиран REST API с multi-threading
|
||||||
- **WebSocket** (`core/websocket.nim`): Пълен дуплекс стрийминг
|
- **WebSocket** (`core/websocket.nim`): Full-duplex стрийминг
|
||||||
- **Embedded** (`storage/lsm.nim`): Директен in-process достъп
|
- **Embedded** (`storage/lsm.nim`): Директен in-process достъп
|
||||||
|
|
||||||
### Управление на Връзките
|
### Управление на Връзки
|
||||||
|
|
||||||
- **Connection Pool** (`protocol/pool.nim`): Мин/макс лимити на връзки
|
- **Connection Pool** (`protocol/pool.nim`): Min/max лимити на връзки с idle timeout
|
||||||
- **Rate Limiting** (`protocol/ratelimit.nim`): Token-bucket лимитиране
|
- **Rate Limiting** (`protocol/ratelimit.nim`): Token-bucket глобални и per-client лимити
|
||||||
- **Authentication** (`protocol/auth.nim`): JWT с HMAC-SHA256
|
- **Автентикация** (`protocol/auth.nim`): JWT с HMAC-SHA256 и достъп на база роли
|
||||||
- **TLS/SSL** (`protocol/ssl.nim`): TLS 1.3 с авто-генерирани сертификати
|
- **TLS/SSL** (`protocol/ssl.nim`): TLS 1.3 с автоматично генерирани сертификати
|
||||||
|
|
||||||
## Слой 2: Заявки (BaraQL)
|
## Слой 2: Query Слой (BaraQL)
|
||||||
|
|
||||||
Pipeline-а на BaraQL:
|
BaraQL конвейрът:
|
||||||
|
|
||||||
1. **Lexer** (`query/lexer.nim`): Токенизира входа в 80+ типа токени
|
1. **Lexer** (`query/lexer.nim`): Токенизира входа в 80+ типа токени
|
||||||
2. **Parser** (`query/parser.nim`): Рекурсивен descent парсър произвеждащ AST
|
2. **Parser** (`query/parser.nim`): Recursive descent parser генериращ AST
|
||||||
3. **AST** (`query/ast.nim`): 300+ реда покриващи 25+ вида възли
|
3. **AST** (`query/ast.nim`): 300+ реда покриващи 25+ вида възли
|
||||||
4. **IR** (`query/ir.nim`): Междинно представяне за планове за изпълнение
|
4. **IR** (`query/ir.nim`): Intermediate representation за планове за изпълнение
|
||||||
5. **Optimizer** (`query/adaptive.nim`): Адаптивен крос-модален оптимизатор
|
5. **Optimizer** (`query/adaptive.nim`): Adaptive cross-modal оптимизация на заявки
|
||||||
6. **Codegen** (`query/codegen.nim`): Транслира IR към операции върху съхранение
|
6. **Codegen** (`query/codegen.nim`): Превежда IR към storage операции
|
||||||
7. **Executor** (`query/executor.nim`): Изпълнява планове с паралелизация
|
7. **Executor** (`query/executor.nim`): Изпълнява планове с паралелизация
|
||||||
|
|
||||||
### Крос-Модално Планиране
|
## Слой 3: Изпълнителен Engine
|
||||||
|
|
||||||
Оптимизаторът определя реда на изпълнение между двигателите:
|
### Документен/KV Engine
|
||||||
|
- **LSM-Tree** (`storage/lsm.nim`): Write-оптимизирано съхранение с MemTable, WAL, SSTables
|
||||||
|
- **B-Tree Индекс** (`storage/btree.nim`): Подреден индекс за range сканиране с COW
|
||||||
|
|
||||||
```
|
### Vector Engine (`vector/`)
|
||||||
1. Оценка на селективност за всеки предикат
|
- **HNSW Индекс** (`vector/engine.nim`): Hierarchical Navigable Small World граф
|
||||||
2. Най-селективният предикат се изпълнява първи
|
- **IVF-PQ Индекс** (`vector/engine.nim`): Inverted File Index с Product Quantization
|
||||||
3. Bloom филтри за KV търсения
|
- **SIMD Операции** (`vector/simd.nim`): AVX2-оптимизирани изчисления на разстояние
|
||||||
4. Паралелизация на независими клонове
|
- **Квантуване** (`vector/quant.nim`): Скаларно, продуктово и бинарно квантуване
|
||||||
```
|
|
||||||
|
|
||||||
## Слой 3: Изпълнителен Двигател
|
### Graph Engine (`graph/`)
|
||||||
|
- **Adjacency List** (`graph/engine.nim`): Насочен граф с тегла на ребрата
|
||||||
### Document/KV Двигател
|
|
||||||
- **LSM-Tree** (`storage/lsm.nim`): Оптимизиран за запис с MemTable, WAL, SSTables
|
|
||||||
- **B-Tree Index** (`storage/btree.nim`): Подреден индекс за диапазони с COW
|
|
||||||
|
|
||||||
### Vector Engine
|
|
||||||
- **HNSW** (`vector/engine.nim`): Иерархичен навигируем малък свят
|
|
||||||
- **IVF-PQ** (`vector/engine.nim`): Инвертиран файл с продуктово квантуване
|
|
||||||
- **SIMD** (`vector/simd.nim`): AVX2-оптимизирани изчисления на разстояния
|
|
||||||
- **Quantization** (`vector/quant.nim`): Скаларно, продуктово и бинарно квантуване
|
|
||||||
|
|
||||||
### Graph Engine
|
|
||||||
- **Списък със съседи** (`graph/engine.nim`): Насочен граф с тегла
|
|
||||||
- **Алгоритми** (`graph/engine.nim`): BFS, DFS, Dijkstra, PageRank
|
- **Алгоритми** (`graph/engine.nim`): BFS, DFS, Dijkstra, PageRank
|
||||||
- **Community Detection** (`graph/community.nim`): Louvain алгоритъм
|
- **Community Detection** (`graph/community.nim`): Louvain алгоритъм
|
||||||
- **Pattern Matching** (`graph/community.nim`): Subgraph isomorphism
|
- **Pattern Matching** (`graph/community.nim`): Subgraph изоморфизъм
|
||||||
- **Cypher Parser** (`graph/cypher.nim`): Cypher-подобни заявки
|
- **Cypher Parser** (`graph/cypher.nim`): Cypher-подобни графови заявки
|
||||||
|
|
||||||
### FTS
|
### Full-Text Search (`fts/`)
|
||||||
- **Инвертиран индекс** (`fts/engine.nim`): Термин-документ индекс
|
- **Inverted Index** (`fts/engine.nim`): Термин-документен индекс
|
||||||
- **Ранжиране** (`fts/engine.nim`): BM25 и TF-IDF
|
- **Ранжиране** (`fts/engine.nim`): BM25 и TF-IDF оценяване
|
||||||
- **Fuzzy Search** (`fts/engine.nim`): Levenshtein разстояние
|
- **Fuzzy Търсене** (`fts/engine.nim`): Съвпадение с Levenshtein разстояние
|
||||||
- **Многоезичен** (`fts/multilang.nim`): Токенизация за EN, BG, DE, FR, RU
|
- **Многоезичност** (`fts/multilang.nim`): Токенизатори за EN, BG, DE, FR, RU
|
||||||
|
|
||||||
### Columnar Engine
|
### Columnar Engine (`core/columnar.nim`)
|
||||||
- **Колонно съхранение** (`core/columnar.nim`): Аналитични заявки
|
- Колонково съхранение за аналитични заявки
|
||||||
- **Компресия**: RLE и dictionary encoding
|
- RLE и dictionary encoding
|
||||||
- **SIMD агрегати**: Ускорени агрегатни функции
|
- SIMD-ускорени агрегати
|
||||||
|
|
||||||
## Слой 4: Съхранение
|
## Слой 4: Съхранение
|
||||||
|
|
||||||
- **LSM-Tree** (`storage/lsm.nim`): MemTable, WAL, SSTable, Bloom Filter, Compaction
|
- **LSM-Tree** (`storage/lsm.nim`): MemTable, WAL, SSTable, Bloom Filter, Compaction
|
||||||
- **Page Cache** (`storage/compaction.nim`): LRU кеш
|
- **Page Cache** (`storage/compaction.nim`): LRU кеш с проследяване на hit rate
|
||||||
- **Memory-mapped I/O** (`storage/mmap.nim`): mmap-базиран достъп
|
- **Memory-mapped I/O** (`storage/mmap.nim`): mmap-базиран достъп до файлове
|
||||||
- **Recovery** (`storage/recovery.nim`): WAL replay и възстановяване
|
- **Recovery** (`storage/recovery.nim`): WAL replay и crash recovery
|
||||||
|
|
||||||
### Път на Запис
|
## Слой 5: Разпределение
|
||||||
|
|
||||||
```
|
- **Raft Консенсус** (`core/raft.nim`): Leader election, log репликация
|
||||||
Client → Protocol → Auth → Parser → AST → IR → Codegen
|
- **Шардиране** (`core/sharding.nim`): Hash, range и consistent hashing
|
||||||
→ StorageOp → MVCC Txn → WAL Write → MemTable → Commit
|
- **Репликация** (`core/replication.nim`): Sync, async, semi-sync режими
|
||||||
```
|
- **Gossip Протокол** (`core/gossip.nim`): SWIM-подобно управление на членство
|
||||||
|
- **Разпределени Транзакции** (`core/disttxn.nim`): Two-phase commit
|
||||||
|
|
||||||
### Път на Четене
|
## Ключови Дизайнерски Решения
|
||||||
|
|
||||||
```
|
|
||||||
Client → Protocol → Auth → Parser → AST → IR → Codegen
|
|
||||||
→ StorageOp → MVCC Snapshot → MemTable → SSTable → Result
|
|
||||||
```
|
|
||||||
|
|
||||||
## Слой 5: Разпределено
|
|
||||||
|
|
||||||
- **Raft Consensus** (`core/raft.nim`): Лидерско избиране, репликация на логове
|
|
||||||
- **Sharding** (`core/sharding.nim`): Hash, range и консистентно хеширане
|
|
||||||
- **Replication** (`core/replication.nim`): Sync, async, semi-sync режими
|
|
||||||
- **Gossip Protocol** (`core/gossip.nim`): SWIM-подобно управление на членство
|
|
||||||
- **Distributed Transactions** (`core/disttxn.nim`): Two-phase commit
|
|
||||||
|
|
||||||
## Ключови Дизайн Решения
|
|
||||||
|
|
||||||
1. **Чист Nim**: Без Cython, Python или Rust зависимости
|
1. **Чист Nim**: Без Cython, Python или Rust зависимости
|
||||||
2. **Обединено Съхранение**: Един двигател за KV, граф, вектор, FTS и колонно
|
2. **Унифицирано Съхранение**: Един engine обработва KV, graph, vector, FTS и columnar
|
||||||
3. **Вграден Режим**: Работи като библиотека или сървър
|
3. **Embedded Режим**: Може да работи като библиотека или сървър
|
||||||
4. **Бинарен Протокол**: Ефективен wire протокол
|
4. **Бинарен Протокол**: Персонализиран ефективен wire протокол
|
||||||
5. **MVCC**: Multi-version concurrency control
|
5. **MVCC**: Multi-version concurrency control
|
||||||
6. **Schema-First**: Строго типизирана система с наследяване
|
6. **Schema-First**: Силно типизирана система от схеми с наследяване
|
||||||
7. **Крос-Модал**: Един език за заявки през всички модели данни
|
7. **Cross-Modal**: Един език за заявки за всички модели на данни
|
||||||
8. **Формално Верифициран**: Разпределените алгоритми са специфицирани в TLA+ и проверени с TLC
|
8. **Формално Верифициран**: Основните разпределени алгоритми са специфицирани в TLA+ и проверени с TLC
|
||||||
|
|
||||||
## Статистика на Модулите
|
## Статистика на Модулите
|
||||||
|
|
||||||
| Категория | Модули | Редове Код | Предназначение |
|
| Категория | Модули | Редове Код | Предназначение |
|
||||||
|-----------|--------|------------|----------------|
|
|-----------|--------|------------|----------------|
|
||||||
| Core | 16 | ~4,200 | Сървър, протоколи, транзакции, разпределено |
|
| Core | 16 | ~4,200 | Сървър, протоколи, транзакции, разпределени |
|
||||||
| Storage | 7 | ~3,100 | LSM, B-Tree, WAL, bloom, compaction, mmap |
|
| Storage | 7 | ~3,100 | LSM, B-Tree, WAL, bloom, compaction, mmap |
|
||||||
| Query | 7 | ~2,800 | Lexer, parser, AST, IR, оптимизатор, codegen, executor |
|
| Query | 7 | ~2,800 | Lexer, parser, AST, IR, optimizer, codegen, executor |
|
||||||
| Vector | 3 | ~1,200 | HNSW, IVF-PQ, квантуване, SIMD |
|
| Vector | 3 | ~1,200 | HNSW, IVF-PQ, квантуване, SIMD |
|
||||||
| Graph | 3 | ~1,000 | Списък със съседи, алгоритми, community detection |
|
| Graph | 3 | ~1,000 | Adjacency list, алгоритми, community detection |
|
||||||
| FTS | 2 | ~900 | Инвертиран индекс, BM25, fuzzy, многоезичен |
|
| FTS | 2 | ~900 | Inverted index, BM25, fuzzy, многоезичност |
|
||||||
| Protocol | 7 | ~2,400 | Wire, HTTP, WebSocket, pool, auth, rate limit, SSL |
|
| Protocol | 7 | ~2,400 | Wire, HTTP, WebSocket, pool, auth, rate limit, SSL |
|
||||||
| Schema | 1 | ~600 | Типове, връзки, наследяване, миграции |
|
| Schema | 1 | ~600 | Типове, връзки, наследяване, миграции |
|
||||||
| Client | 2 | ~800 | Nim binary client, file helpers |
|
| Client | 2 | ~800 | Nim бинарен клиент, файлови помощници |
|
||||||
| CLI | 1 | ~400 | Интерактивна BaraQL конзола |
|
| CLI | 1 | ~400 | Интерактивна BaraQL обвивка |
|
||||||
| **Общо** | **49** | **~14,100** | |
|
| **Общо** | **49** | **~14,100** | |
|
||||||
|
|||||||
@@ -0,0 +1,198 @@
|
|||||||
|
# Backup и Възстановяване
|
||||||
|
|
||||||
|
## Online Snapshots
|
||||||
|
|
||||||
|
BaraDB поддържа online snapshots без спиране на сървъра. Snapshot-ът заснема консистентен изглед към момент във времето чрез MVCC.
|
||||||
|
|
||||||
|
### Създаване на Snapshot
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/core/backup
|
||||||
|
|
||||||
|
var bm = newBackupManager()
|
||||||
|
bm.createSnapshot("/backup/baradb_2025-01-15")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Чрез CLI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./build/baradadb --snapshot --output=/backup/snapshot.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Чрез HTTP API
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/api/backup \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"destination": "/backup/snapshot.db"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### Автоматизирани Backups
|
||||||
|
|
||||||
|
Използвайте cron за планирани backups:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Ежедневен snapshot в 2 сутринта
|
||||||
|
0 2 * * * /usr/local/bin/baradadb --snapshot --output=/backup/baradb_$(date +\%Y\%m\%d).db
|
||||||
|
|
||||||
|
# Запазване на последните 7 дни
|
||||||
|
find /backup -name "baradb_*.db" -mtime +7 -delete
|
||||||
|
```
|
||||||
|
|
||||||
|
## Point-in-Time Recovery (PITR)
|
||||||
|
|
||||||
|
BaraDB използва Write-Ahead Log (WAL) за възстановяване до момент във времето.
|
||||||
|
|
||||||
|
### WAL Архивиране
|
||||||
|
|
||||||
|
Включете непрекъснато WAL архивиране:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_WAL_ARCHIVE_DIR=/backup/wal \
|
||||||
|
BARADB_WAL_ARCHIVE_INTERVAL_MS=60000 \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Възстановяване от Checkpoint + WAL
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Възстановяване от snapshot
|
||||||
|
./build/baradadb --recover \
|
||||||
|
--checkpoint=/backup/snapshot.db \
|
||||||
|
--wal-dir=/backup/wal
|
||||||
|
|
||||||
|
# Възстановяване до конкретен LSN
|
||||||
|
./build/baradadb --recover \
|
||||||
|
--checkpoint=/backup/snapshot.db \
|
||||||
|
--wal-dir=/backup/wal \
|
||||||
|
--target-lsn=15420
|
||||||
|
|
||||||
|
# Възстановяване до конкретно време
|
||||||
|
./build/baradadb --recover \
|
||||||
|
--checkpoint=/backup/snapshot.db \
|
||||||
|
--wal-dir=/backup/wal \
|
||||||
|
--target-time="2025-01-15T10:30:00Z"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Възстановяване чрез SQL
|
||||||
|
|
||||||
|
Можете също да възстановявате директно чрез BaraQL:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
||||||
|
```
|
||||||
|
|
||||||
|
### Инкрементални Backups
|
||||||
|
|
||||||
|
Инкременталните backups копират само променени SSTables:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./build/baradadb --backup-incremental \
|
||||||
|
--last-backup=/backup/previous \
|
||||||
|
--output=/backup/incremental_$(date +%Y%m%d)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Репликация като Backup
|
||||||
|
|
||||||
|
За непрекъсната защита използвайте streaming репликация:
|
||||||
|
|
||||||
|
### Primary
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_REPLICATION_ENABLED=true \
|
||||||
|
BARADB_REPLICATION_MODE=async \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Replica
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_REPLICATION_ENABLED=true \
|
||||||
|
BARADB_REPLICATION_PRIMARY=primary:9472 \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Disaster Recovery
|
||||||
|
|
||||||
|
### Процедури за Възстановяване
|
||||||
|
|
||||||
|
#### Сценарий 1: Повреда на Единичен Файл
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Идентифициране на повреден SSTable от логовете
|
||||||
|
# Възстановяване на конкретен SSTable от backup
|
||||||
|
cp /backup/sstables/000012.sst ./data/sstables/
|
||||||
|
|
||||||
|
# Възстановяване на индекса
|
||||||
|
./build/baradadb --rebuild-index
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Сценарий 2: Пълна Загуба на Данни
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Възстановяване на последния snapshot
|
||||||
|
cp /backup/snapshot.db ./data/
|
||||||
|
|
||||||
|
# 2. Преиграване на WAL
|
||||||
|
./build/baradadb --recover --wal-dir=/backup/wal
|
||||||
|
|
||||||
|
# 3. Проверка
|
||||||
|
curl http://localhost:9470/health
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Сценарий 3: Отказ на Възел в Клъстер
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# За Raft клъстери, просто стартирайте нов възел
|
||||||
|
BARADB_RAFT_NODE_ID=newnode \
|
||||||
|
BARADB_RAFT_PEERS=node1:9001,node2:9001 \
|
||||||
|
./build/baradadb
|
||||||
|
|
||||||
|
# Новият възел ще навакса чрез Raft log репликация
|
||||||
|
```
|
||||||
|
|
||||||
|
## Верификация на Backup
|
||||||
|
|
||||||
|
Винаги проверявайте backups:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Възстановяване във временна директория
|
||||||
|
./build/baradadb --recover \
|
||||||
|
--checkpoint=/backup/snapshot.db \
|
||||||
|
--data-dir=/tmp/verify_data
|
||||||
|
|
||||||
|
# Проверка на консистентност
|
||||||
|
curl http://localhost:9470/api/admin/check
|
||||||
|
```
|
||||||
|
|
||||||
|
## Изисквания за Съхранение
|
||||||
|
|
||||||
|
| Тип Backup | Размер | Честота | Задържане |
|
||||||
|
|------------|--------|---------|-----------|
|
||||||
|
| Пълен snapshot | ~1× размер на данните | Ежедневно | 7 дни |
|
||||||
|
| Инкрементален | ~0.1× размер на данните | На всеки час | 24 часа |
|
||||||
|
| WAL архив | ~0.05× размер на данните / ден | Непрекъснато | 30 дни |
|
||||||
|
|
||||||
|
## Най-добри Практики
|
||||||
|
|
||||||
|
1. **Тествайте възстановяването редовно** — Backup, който не може да бъде възстановен, е безполезен
|
||||||
|
2. **Съхранявайте backups извън локацията** — Използвайте S3, GCS или Azure Blob
|
||||||
|
3. **Криптирайте backups** — Използвайте `gpg` или криптиране на ниво ОС
|
||||||
|
4. **Мониторирайте backup задачите** — Алармирайте при неуспешни backups
|
||||||
|
5. **Документирайте RTO/RPO** — Знайте целите си за време и точка на възстановяване
|
||||||
|
|
||||||
|
### Качване на Backup в Облак
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Качване в S3
|
||||||
|
aws s3 cp /backup/snapshot.db s3://my-bucket/baradb/
|
||||||
|
|
||||||
|
# Качване в GCS
|
||||||
|
gsutil cp /backup/snapshot.db gs://my-bucket/baradb/
|
||||||
|
|
||||||
|
# Качване в Azure
|
||||||
|
az storage blob upload \
|
||||||
|
--container-name backups \
|
||||||
|
--file /backup/snapshot.db \
|
||||||
|
--name baradb/snapshot.db
|
||||||
|
```
|
||||||
+559
-32
@@ -2,39 +2,160 @@
|
|||||||
|
|
||||||
BaraQL е SQL-съвместим език за заявки с разширения за графи, вектори и документи.
|
BaraQL е SQL-съвместим език за заявки с разширения за графи, вектори и документи.
|
||||||
|
|
||||||
|
## Типове Данни
|
||||||
|
|
||||||
|
| Тип | Описание | Пример |
|
||||||
|
|------|----------|--------|
|
||||||
|
| `null` | Null стойност | `null` |
|
||||||
|
| `bool` | Булев | `true`, `false` |
|
||||||
|
| `int8` | 8-битов signed integer | `127` |
|
||||||
|
| `int16` | 16-битов signed integer | `32767` |
|
||||||
|
| `int32` | 32-битов signed integer | `2147483647` |
|
||||||
|
| `int64` | 64-битов signed integer | `9223372036854775807` |
|
||||||
|
| `float32` | 32-битов float | `3.14` |
|
||||||
|
| `float64` | 64-битов float | `3.14159265359` |
|
||||||
|
| `str` | UTF-8 низ | `'hello'` |
|
||||||
|
| `bytes` | Сурови байтове | `0xDEADBEEF` |
|
||||||
|
| `array<T>` | Хомогенен масив | `[1, 2, 3]` |
|
||||||
|
| `vector` | Float32 вектор | `[0.1, 0.2, 0.3]` |
|
||||||
|
| `vector(n)` | Float32 вектор с фиксирана размерност (SQL) | `VECTOR(768)` |
|
||||||
|
| `object` | Ключ-стойност обект | `{"a": 1}` |
|
||||||
|
| `datetime` | ISO 8601 времеви печат | `'2025-01-15T10:30:00Z'` |
|
||||||
|
| `uuid` | UUID v4 | `'550e8400-e29b-41d4-a716-446655440000'` |
|
||||||
|
| `json` | JSON документ | `{"key": "value"}` |
|
||||||
|
| `jsonb` | Бинарен JSON (валидиран) | `{"key": "value"}` |
|
||||||
|
|
||||||
## Основни Заявки
|
## Основни Заявки
|
||||||
|
|
||||||
### SELECT
|
### SELECT
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
SELECT name, age FROM users WHERE age > 18 ORDER BY name LIMIT 10;
|
-- Всички колони
|
||||||
|
SELECT * FROM users;
|
||||||
|
|
||||||
|
-- Конкретни колони
|
||||||
|
SELECT name, age FROM users;
|
||||||
|
|
||||||
|
-- Псевдоними
|
||||||
|
SELECT name AS full_name, age AS years FROM users;
|
||||||
|
|
||||||
|
-- DISTINCT
|
||||||
|
SELECT DISTINCT department FROM employees;
|
||||||
|
|
||||||
|
-- LIMIT и OFFSET
|
||||||
|
SELECT * FROM users LIMIT 10 OFFSET 20;
|
||||||
|
```
|
||||||
|
|
||||||
|
### WHERE
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Оператори за сравнение
|
||||||
|
SELECT * FROM users WHERE age > 18;
|
||||||
|
SELECT * FROM users WHERE age >= 18 AND age <= 65;
|
||||||
|
SELECT * FROM users WHERE name = 'Alice';
|
||||||
|
SELECT * FROM users WHERE name != 'Bob';
|
||||||
|
|
||||||
|
-- Диапазон
|
||||||
|
SELECT * FROM users WHERE age BETWEEN 18 AND 65;
|
||||||
|
|
||||||
|
-- Принадлежност към множество
|
||||||
|
SELECT * FROM users WHERE department IN ('Engineering', 'Sales');
|
||||||
|
|
||||||
|
-- Търсене по шаблон
|
||||||
|
SELECT * FROM users WHERE name LIKE 'A%';
|
||||||
|
SELECT * FROM users WHERE name ILIKE 'alice'; -- Case-insensitive
|
||||||
|
|
||||||
|
-- NULL проверки
|
||||||
|
SELECT * FROM users WHERE email IS NOT NULL;
|
||||||
|
|
||||||
|
-- Логически оператори
|
||||||
|
SELECT * FROM users WHERE age > 18 AND (department = 'Engineering' OR department = 'Sales');
|
||||||
|
```
|
||||||
|
|
||||||
|
### ORDER BY
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Възходящ (по подразбиране)
|
||||||
|
SELECT * FROM users ORDER BY age;
|
||||||
|
|
||||||
|
-- Низходящ
|
||||||
|
SELECT * FROM users ORDER BY age DESC;
|
||||||
|
|
||||||
|
-- Множество колони
|
||||||
|
SELECT * FROM users ORDER BY department ASC, age DESC;
|
||||||
```
|
```
|
||||||
|
|
||||||
### INSERT
|
### INSERT
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
|
-- Един ред
|
||||||
INSERT users { name := 'Alice', age := 30 };
|
INSERT users { name := 'Alice', age := 30 };
|
||||||
|
|
||||||
|
-- С явен тип
|
||||||
|
INSERT User { name := 'Alice', age := 30 };
|
||||||
|
|
||||||
|
-- Множество редове
|
||||||
|
INSERT users {
|
||||||
|
{ name := 'Alice', age := 30 },
|
||||||
|
{ name := 'Bob', age := 25 }
|
||||||
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
### UPDATE
|
### UPDATE
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
|
-- Обнови всички редове
|
||||||
|
UPDATE users SET status = 'active';
|
||||||
|
|
||||||
|
-- Условно обновяване
|
||||||
UPDATE users SET age = 31 WHERE name = 'Alice';
|
UPDATE users SET age = 31 WHERE name = 'Alice';
|
||||||
|
|
||||||
|
-- Обновяване на няколко колони
|
||||||
|
UPDATE users SET age = 32, status = 'premium' WHERE name = 'Alice';
|
||||||
```
|
```
|
||||||
|
|
||||||
### DELETE
|
### DELETE
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
DELETE FROM users WHERE name = 'Bob';
|
-- Изтрий всички редове
|
||||||
|
DELETE FROM users;
|
||||||
|
|
||||||
|
-- Условно изтриване
|
||||||
|
DELETE FROM users WHERE age < 18;
|
||||||
```
|
```
|
||||||
|
|
||||||
## Агрегати и Групиране
|
## Агрегати и Групиране
|
||||||
|
|
||||||
|
### Агрегатни Функции
|
||||||
|
|
||||||
|
| Функция | Описание |
|
||||||
|
|----------|-----------|
|
||||||
|
| `count(*)` | Брой на всички редове |
|
||||||
|
| `count(column)` | Брой на не-NULL стойности |
|
||||||
|
| `sum(column)` | Сума на стойностите |
|
||||||
|
| `avg(column)` | Средно аритметично |
|
||||||
|
| `min(column)` | Минимална стойност |
|
||||||
|
| `max(column)` | Максимална стойност |
|
||||||
|
| `stddev(column)` | Стандартно отклонение |
|
||||||
|
| `variance(column)` | Дисперсия |
|
||||||
|
|
||||||
|
### GROUP BY
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
SELECT department, count(*), avg(salary)
|
SELECT department, count(*) as emp_count, avg(salary) as avg_salary
|
||||||
|
FROM employees
|
||||||
|
GROUP BY department;
|
||||||
|
|
||||||
|
-- С HAVING
|
||||||
|
SELECT department, count(*) as emp_count
|
||||||
FROM employees
|
FROM employees
|
||||||
GROUP BY department
|
GROUP BY department
|
||||||
HAVING count(*) > 5;
|
HAVING count(*) > 5;
|
||||||
|
|
||||||
|
-- Множествено групиране
|
||||||
|
SELECT department, role, count(*), avg(salary)
|
||||||
|
FROM employees
|
||||||
|
GROUP BY department, role;
|
||||||
```
|
```
|
||||||
|
|
||||||
## JOINs
|
## JOINs
|
||||||
@@ -49,15 +170,85 @@ INNER JOIN orders o ON u.id = o.user_id;
|
|||||||
SELECT u.name, o.total
|
SELECT u.name, o.total
|
||||||
FROM users u
|
FROM users u
|
||||||
LEFT JOIN orders o ON u.id = o.user_id;
|
LEFT JOIN orders o ON u.id = o.user_id;
|
||||||
|
|
||||||
|
-- RIGHT JOIN
|
||||||
|
SELECT u.name, o.total
|
||||||
|
FROM users u
|
||||||
|
RIGHT JOIN orders o ON u.id = o.user_id;
|
||||||
|
|
||||||
|
-- FULL JOIN
|
||||||
|
SELECT u.name, o.total
|
||||||
|
FROM users u
|
||||||
|
FULL JOIN orders o ON u.id = o.user_id;
|
||||||
|
|
||||||
|
-- CROSS JOIN
|
||||||
|
SELECT u.name, p.name
|
||||||
|
FROM users u
|
||||||
|
CROSS JOIN products p;
|
||||||
|
|
||||||
|
-- Множество JOINs
|
||||||
|
SELECT u.name, o.id, p.name
|
||||||
|
FROM orders o
|
||||||
|
JOIN users u ON o.user_id = u.id
|
||||||
|
JOIN products p ON o.product_id = p.id;
|
||||||
|
|
||||||
|
-- Self JOIN
|
||||||
|
SELECT e.name, m.name as manager
|
||||||
|
FROM employees e
|
||||||
|
JOIN employees m ON e.manager_id = m.id;
|
||||||
```
|
```
|
||||||
|
|
||||||
## CTEs (Common Table Expressions)
|
## CTEs (Common Table Expressions)
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
|
-- Единичен CTE
|
||||||
WITH active_users AS (
|
WITH active_users AS (
|
||||||
SELECT * FROM users WHERE active = true
|
SELECT * FROM users WHERE active = true
|
||||||
)
|
)
|
||||||
SELECT * FROM active_users;
|
SELECT * FROM active_users;
|
||||||
|
|
||||||
|
-- Множество CTEs
|
||||||
|
WITH
|
||||||
|
recent AS (
|
||||||
|
SELECT * FROM orders WHERE date > '2025-01-01'
|
||||||
|
),
|
||||||
|
totals AS (
|
||||||
|
SELECT user_id, sum(amount) as total FROM recent GROUP BY user_id
|
||||||
|
)
|
||||||
|
SELECT u.name, t.total
|
||||||
|
FROM users u
|
||||||
|
JOIN totals t ON u.id = t.user_id;
|
||||||
|
|
||||||
|
-- Рекурсивен CTE
|
||||||
|
WITH RECURSIVE subordinates AS (
|
||||||
|
SELECT id, name, manager_id FROM employees WHERE name = 'CEO'
|
||||||
|
UNION ALL
|
||||||
|
SELECT e.id, e.name, e.manager_id
|
||||||
|
FROM employees e
|
||||||
|
JOIN subordinates s ON e.manager_id = s.id
|
||||||
|
)
|
||||||
|
SELECT * FROM subordinates;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Подзаявки
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Подзаявка в SELECT
|
||||||
|
SELECT name, (SELECT count(*) FROM orders WHERE user_id = u.id) as order_count
|
||||||
|
FROM users u;
|
||||||
|
|
||||||
|
-- Подзаявка в FROM
|
||||||
|
SELECT * FROM (SELECT id, name FROM users WHERE active = true) AS active;
|
||||||
|
|
||||||
|
-- Подзаявка в WHERE (IN)
|
||||||
|
SELECT name FROM users WHERE id IN (SELECT user_id FROM orders);
|
||||||
|
|
||||||
|
-- Подзаявка в WHERE (EXISTS)
|
||||||
|
SELECT name FROM users WHERE EXISTS (SELECT 1 FROM orders WHERE orders.user_id = users.id);
|
||||||
|
|
||||||
|
-- Корелирана подзаявка
|
||||||
|
SELECT name FROM users u
|
||||||
|
WHERE age > (SELECT avg(age) FROM users WHERE department = u.department);
|
||||||
```
|
```
|
||||||
|
|
||||||
## CASE Изрази
|
## CASE Изрази
|
||||||
@@ -65,54 +256,114 @@ SELECT * FROM active_users;
|
|||||||
```sql
|
```sql
|
||||||
SELECT name,
|
SELECT name,
|
||||||
CASE
|
CASE
|
||||||
WHEN age < 18 THEN 'minor'
|
WHEN age < 13 THEN 'child'
|
||||||
|
WHEN age < 20 THEN 'teenager'
|
||||||
WHEN age < 65 THEN 'adult'
|
WHEN age < 65 THEN 'adult'
|
||||||
ELSE 'senior'
|
ELSE 'senior'
|
||||||
END AS category
|
END AS category
|
||||||
FROM users;
|
FROM users;
|
||||||
|
|
||||||
|
-- Прост CASE
|
||||||
|
SELECT name,
|
||||||
|
CASE department
|
||||||
|
WHEN 'Engineering' THEN 'Tech'
|
||||||
|
WHEN 'Sales' THEN 'Revenue'
|
||||||
|
ELSE 'Other'
|
||||||
|
END AS division
|
||||||
|
FROM employees;
|
||||||
```
|
```
|
||||||
|
|
||||||
## Схема
|
## Set Операции
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- UNION (различни)
|
||||||
|
SELECT name FROM customers
|
||||||
|
UNION
|
||||||
|
SELECT name FROM suppliers;
|
||||||
|
|
||||||
|
-- UNION ALL (с дубликати)
|
||||||
|
SELECT name FROM customers
|
||||||
|
UNION ALL
|
||||||
|
SELECT name FROM suppliers;
|
||||||
|
|
||||||
|
-- INTERSECT
|
||||||
|
SELECT name FROM customers
|
||||||
|
INTERSECT
|
||||||
|
SELECT name FROM suppliers;
|
||||||
|
|
||||||
|
-- EXCEPT
|
||||||
|
SELECT name FROM customers
|
||||||
|
EXCEPT
|
||||||
|
SELECT name FROM suppliers;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Дефиниране на Схема
|
||||||
|
|
||||||
|
### CREATE TYPE
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
CREATE TYPE Person {
|
CREATE TYPE Person {
|
||||||
name: str,
|
name: str,
|
||||||
age: int32
|
age: int32
|
||||||
};
|
};
|
||||||
```
|
|
||||||
|
|
||||||
## Векторно Търсене
|
-- Със задължителни полета
|
||||||
|
CREATE TYPE User {
|
||||||
```sql
|
email: str REQUIRED,
|
||||||
INSERT articles {
|
name: str,
|
||||||
title := 'Nim Programming',
|
age: int32,
|
||||||
embedding := [0.1, 0.2, 0.3, ...]
|
created_at: datetime DEFAULT now()
|
||||||
};
|
};
|
||||||
|
|
||||||
SELECT title FROM articles
|
-- С връзки
|
||||||
ORDER BY cosine_distance(embedding, [0.1, 0.2, 0.3, ...])
|
CREATE TYPE Movie {
|
||||||
LIMIT 5;
|
title: str,
|
||||||
|
year: int32,
|
||||||
|
director: Person
|
||||||
|
};
|
||||||
|
|
||||||
|
-- С изчислими свойства
|
||||||
|
CREATE TYPE Employee {
|
||||||
|
name: str,
|
||||||
|
base_salary: float64,
|
||||||
|
bonus: float64,
|
||||||
|
total_compensation: float64 COMPUTED (base_salary + bonus)
|
||||||
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
## Графични Шаблони
|
### Наследяване
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
MATCH (p:Person)-[:KNOWS]->(friend:Person)
|
CREATE TYPE Animal {
|
||||||
WHERE p.name = 'Alice'
|
name: str
|
||||||
RETURN friend.name;
|
};
|
||||||
|
|
||||||
|
CREATE TYPE Dog EXTENDING Animal {
|
||||||
|
breed: str
|
||||||
|
};
|
||||||
|
|
||||||
|
CREATE TYPE Cat EXTENDING Animal {
|
||||||
|
indoor: bool
|
||||||
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
## Пълнотекстово Търсене
|
### Индекси
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
-- Създаване на FTS индекс
|
CREATE INDEX idx_users_name ON users(name);
|
||||||
CREATE INDEX idx_fts ON articles(body) USING FTS;
|
CREATE UNIQUE INDEX idx_users_email ON users(email);
|
||||||
|
CREATE INDEX idx_users_age ON users(age) USING btree;
|
||||||
-- Търсене с BM25 ранжиране
|
CREATE INDEX idx_vectors ON items(embedding) USING hnsw;
|
||||||
SELECT * FROM articles WHERE body @@ 'database programming';
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## JSON Оператори
|
### DROP
|
||||||
|
|
||||||
|
```sql
|
||||||
|
DROP TYPE User;
|
||||||
|
DROP INDEX idx_users_name;
|
||||||
|
```
|
||||||
|
|
||||||
|
### JSON Оператори за Път
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
-- Извличане на JSON поле като JSON
|
-- Извличане на JSON поле като JSON
|
||||||
@@ -122,16 +373,292 @@ SELECT data->'name' FROM users;
|
|||||||
SELECT data->>'name' FROM users;
|
SELECT data->>'name' FROM users;
|
||||||
```
|
```
|
||||||
|
|
||||||
## Set Операции
|
### Пълнотекстово Търсене (SQL)
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
SELECT name FROM customers
|
-- Създаване на FTS индекс с BM25
|
||||||
UNION ALL
|
CREATE INDEX idx_fts ON articles(body) USING FTS;
|
||||||
SELECT name FROM suppliers;
|
|
||||||
|
-- Търсене с BM25 ранжиране
|
||||||
|
SELECT * FROM articles WHERE body @@ 'machine learning';
|
||||||
```
|
```
|
||||||
|
|
||||||
## Възстановяване до Момент във Времето
|
### Възстановяване до Момент във Времето
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Векторно Търсене (SQL)
|
||||||
|
|
||||||
|
### Създаване на Векторни Колони
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE items (
|
||||||
|
id INT PRIMARY KEY,
|
||||||
|
embedding VECTOR(768)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### Вмъкване на Вектори
|
||||||
|
|
||||||
|
```sql
|
||||||
|
INSERT INTO items (id, embedding) VALUES (1, '[0.1, 0.2, 0.3, 0.4]');
|
||||||
|
```
|
||||||
|
|
||||||
|
### Функции за Разстояние
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Косинусово разстояние (0 = идентични, 2 = противоположни)
|
||||||
|
SELECT id, cosine_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Евклидово / L2 разстояние
|
||||||
|
SELECT id, euclidean_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- L2 разстояние с <-> оператор
|
||||||
|
SELECT id, embedding <-> '[0.1, 0.2, 0.3, 0.4]' AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Скаларно произведение (отрицателно dot product)
|
||||||
|
SELECT id, inner_product(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Манхатън / L1 разстояние
|
||||||
|
SELECT id, l1_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Търсене на Най-близки Съседи
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Топ-10 най-близки съседи по косинусово разстояние
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY cosine_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') ASC
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- Топ-5 най-близки съседи по евклидово разстояние
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY embedding <-> '[0.1, 0.2, 0.3, 0.4]'
|
||||||
|
LIMIT 5;
|
||||||
|
|
||||||
|
-- С филтър по метаданни
|
||||||
|
SELECT id FROM items
|
||||||
|
WHERE category = 'tech'
|
||||||
|
ORDER BY cosine_distance(embedding, '[0.1, 0.2, 0.3, 0.4]')
|
||||||
|
LIMIT 5;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Векторни Индекси
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Създаване на HNSW индекс за приблизително търсене на най-близки съседи
|
||||||
|
CREATE INDEX idx_items_vec ON items(embedding) USING hnsw;
|
||||||
|
|
||||||
|
-- Поддържани индекс методи: hnsw, ivfpq
|
||||||
|
```
|
||||||
|
|
||||||
|
## Графични Шаблони
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Намиране на приятели на Alice
|
||||||
|
MATCH (p:Person)-[:KNOWS]->(friend:Person)
|
||||||
|
WHERE p.name = 'Alice'
|
||||||
|
RETURN friend.name;
|
||||||
|
|
||||||
|
-- Намиране на най-кратък път
|
||||||
|
MATCH path = shortestPath((a:Person)-[:KNOWS*1..5]->(b:Person))
|
||||||
|
WHERE a.name = 'Alice' AND b.name = 'Bob'
|
||||||
|
RETURN path;
|
||||||
|
|
||||||
|
-- Намиране на всички връзки
|
||||||
|
MATCH (p:Person)-[r]->(other)
|
||||||
|
WHERE p.name = 'Alice'
|
||||||
|
RETURN type(r), other.name;
|
||||||
|
|
||||||
|
-- Множество преходи
|
||||||
|
MATCH (a:Person)-[:KNOWS]->(b:Person)-[:KNOWS]->(c:Person)
|
||||||
|
WHERE a.name = 'Alice'
|
||||||
|
RETURN c.name;
|
||||||
|
|
||||||
|
-- С агрегати
|
||||||
|
MATCH (p:Person)-[:KNOWS]->(friend)
|
||||||
|
RETURN p.name, count(friend) as friend_count
|
||||||
|
ORDER BY friend_count DESC;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Пълнотекстово Търсене
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Основно търсене
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('database programming');
|
||||||
|
|
||||||
|
-- С релевантност
|
||||||
|
SELECT title, relevance()
|
||||||
|
FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('Nim language')
|
||||||
|
ORDER BY relevance() DESC;
|
||||||
|
|
||||||
|
-- Булев режим
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('+Nim -Python' IN BOOLEAN MODE);
|
||||||
|
|
||||||
|
-- Fuzzy търсене
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title) AGAINST('programing' WITH FUZZINESS 2);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Транзакции
|
||||||
|
|
||||||
|
```sql
|
||||||
|
BEGIN;
|
||||||
|
INSERT users { name := 'Alice', age := 30 };
|
||||||
|
INSERT orders { user_id := last_insert_id(), total := 100 };
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
-- С savepoint
|
||||||
|
BEGIN;
|
||||||
|
INSERT users { name := 'Bob', age := 25 };
|
||||||
|
SAVEPOINT sp1;
|
||||||
|
INSERT orders { user_id := last_insert_id(), total := 200 };
|
||||||
|
-- Грешка, връщане до savepoint
|
||||||
|
ROLLBACK TO sp1;
|
||||||
|
COMMIT;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Потребителски Функции (UDF)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Регистриране на UDF
|
||||||
|
CREATE FUNCTION greet(name str) -> str {
|
||||||
|
RETURN 'Hello, ' || name || '!';
|
||||||
|
};
|
||||||
|
|
||||||
|
-- Използване
|
||||||
|
SELECT greet(name) FROM users;
|
||||||
|
|
||||||
|
-- Вградени функции
|
||||||
|
SELECT abs(-5), sqrt(16), lower('HELLO'), len('test');
|
||||||
|
```
|
||||||
|
|
||||||
|
## Подсказки за Заявки (Query Hints)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Форсиране на индекс
|
||||||
|
SELECT /*+ USE_INDEX(idx_users_age) */ * FROM users WHERE age > 18;
|
||||||
|
|
||||||
|
-- Форсиране на приблизително векторно търсене
|
||||||
|
SELECT /*+ APPROXIMATE */ * FROM vectors
|
||||||
|
ORDER BY cosine_distance(embedding, [...])
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- Паралелно изпълнение
|
||||||
|
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Window Функции
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Функции за ранжиране
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
department,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||||
|
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||||
|
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Стойностни функции
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
salary,
|
||||||
|
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||||
|
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||||
|
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||||
|
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Функции за разпределение
|
||||||
|
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Рамкови Спецификации
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ROWS рамка
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
|
||||||
|
-- RANGE рамка
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Multi-Tenant ERP
|
||||||
|
|
||||||
|
BaraDB поддържа множество компании (тенанти) в една инстанция чрез **Row-Level Security (RLS)** и **сесийни променливи**.
|
||||||
|
|
||||||
|
### Сесийни Променливи
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET app.tenant_id = 'company-123';
|
||||||
|
SELECT current_setting('app.tenant_id') AS tenant;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Текущ Потребител / Роля
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT current_user AS me, current_role AS my_role;
|
||||||
|
```
|
||||||
|
|
||||||
|
### RLS Изолация на Тенанти
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Включване на RLS за таблица
|
||||||
|
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
|
||||||
|
|
||||||
|
-- Създаване на политика за филтриране по тенант
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- Всяка сесия вижда само своите данни
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- само редове на company-a
|
||||||
|
```
|
||||||
|
|
||||||
|
### Защо Multi-Tenant?
|
||||||
|
|
||||||
|
- **Една инстанция, много тенанти** — няма нужда от 100 отделни бази данни
|
||||||
|
- **JSONB документи** — гъвкаво съхранение без схема, лесно добавяне на полета за всеки тенант
|
||||||
|
- **RLS гарантира изолация** — базата данни налага границите между тенанти, не само приложението
|
||||||
|
|
||||||
|
## Поддържани Ключови Думи
|
||||||
|
|
||||||
|
| Категория | Ключови думи |
|
||||||
|
|-----------|-------------|
|
||||||
|
| DQL | SELECT, FROM, WHERE, ORDER BY, GROUP BY, HAVING, LIMIT, OFFSET, DISTINCT |
|
||||||
|
| DML | INSERT, UPDATE, DELETE, SET, VALUES |
|
||||||
|
| DDL | CREATE TYPE, DROP TYPE, CREATE INDEX, DROP INDEX, ALTER TYPE |
|
||||||
|
| Join | INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, CROSS JOIN, ON |
|
||||||
|
| Set | UNION, UNION ALL, INTERSECT, EXCEPT |
|
||||||
|
| CTEs | WITH, RECURSIVE, AS |
|
||||||
|
| Case | CASE, WHEN, THEN, ELSE, END |
|
||||||
|
| Транзакции | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
||||||
|
| Графи | MATCH, RETURN, WHERE, shortestPath, type |
|
||||||
|
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||||
|
| Вектори | cosine_distance, euclidean_distance, inner_product, l1_distance, l2_distance, <-> |
|
||||||
|
| JSON | ->, ->> |
|
||||||
|
| FTS | @@ (BM25 съвпадение) |
|
||||||
|
| Recovery | RECOVER TO TIMESTAMP |
|
||||||
|
| Функции | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id, current_setting |
|
||||||
|
| Сесийни | SET, current_setting, current_user, current_role |
|
||||||
|
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||||
|
| Window Функции | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||||
|
|||||||
@@ -0,0 +1,223 @@
|
|||||||
|
# Списък с Промени (Changelog)
|
||||||
|
|
||||||
|
Всички забележителни промени в BaraDB са документирани в този файл.
|
||||||
|
|
||||||
|
## [Unreleased] — SQL:2023 Стабилизация
|
||||||
|
|
||||||
|
### Поправки
|
||||||
|
|
||||||
|
- **GROUPING SETS изпълнение** — `lowerSelect` вече създава `irpkGroupBy` когато `selGroupingSetsKind != gskNone`, дори ако `selGroupBy` е празен. Преди това заявки като `GROUP BY GROUPING SETS ((dept), ())` напълно заобикаляха grouping executor-a.
|
||||||
|
- **FTS CREATE INDEX docId несъответствие** — `CREATE INDEX ... USING FTS` вече изчислява `docId` като хеш на `tableName.$key`, консистентно с DML операциите (`INSERT`/`UPDATE`/`DELETE`). Преди това създаването на индекс използваше последователни ID-та (0, 1, 2...), което причиняваше `@@` заявките никога да не намират индексирани документи.
|
||||||
|
- **Тестова изолация (всички сюити)** — Всички извиквания на `newLSMTree("")` са заменени с уникални временни директории за всеки сюит. Елиминира проблеми с натрупване на WAL и нестабилни тестове от споделено състояние между тестове.
|
||||||
|
- **Window frame parser** — `parseFrameBoundary` вече не консумира `tkRow` след `tkCurrent` неправилно (използваше `tkRows`). Също така е поправен конфликт на ключовата дума `tkRow` с парсването на `ENABLE ROW LEVEL SECURITY`.
|
||||||
|
- **ORDER BY + SELECT проекция** — `lowerSelect` вече поставя `irpkSort` преди `irpkProject`, което позволява `ORDER BY` по колони, които не присъстват в `SELECT` списъка.
|
||||||
|
- **UNPIVOT изпълнение** — Проверено и поправено липсващо тестово покритие за UNPIVOT трансформация.
|
||||||
|
|
||||||
|
### Добавки
|
||||||
|
|
||||||
|
- **JSON оператори** — `@>` (съдържа), `<@` (съдържа се в), `?` (има ключ), `?|` (има някой от), `?&` (има всички) вече се поддържат в lexer, parser и executor.
|
||||||
|
- **Window frame изпълнение** — `ROWS BETWEEN X PRECEDING AND Y FOLLOWING` / `CURRENT ROW` граници на рамката вече се спазват от `FIRST_VALUE` и `LAST_VALUE`.
|
||||||
|
- **Сесийни променливи** — `SET var_name = value` и `current_setting('var_name')` за ключ/стойност съхранение на ниво връзка.
|
||||||
|
- **Текущ потребител/роля** — `current_user` и `current_role` SQL ключови думи връщат потребителя и ролята на автентикираната сесия.
|
||||||
|
- **Auth-executor мост** — Сървърът и HTTP сървърът вече попълват `ExecutionContext.currentUser` и `ExecutionContext.currentRole` след JWT/SCRAM автентикация.
|
||||||
|
- **Multi-tenant RLS** — Row-Level Security политиките вече могат да реферират `current_user`, `current_role` и `current_setting('app.tenant_id')` за изолация на данни по тенант.
|
||||||
|
|
||||||
|
## [1.1.0] — 2026-05-13
|
||||||
|
|
||||||
|
### Добавки
|
||||||
|
|
||||||
|
- **Client SDKs v1.1.0** — Пълнофункционални клиенти за всички езици:
|
||||||
|
- JavaScript: TypeScript дефиниции, package.json, примери, unit и integration тестове
|
||||||
|
- Python: Преструктуриран като пакет (`baradb/` с `__init__.py` и `core.py`), pyproject.toml, примери, тестове (query builder, wire protocol, integration)
|
||||||
|
- Nim: Примери, integration тестове, README
|
||||||
|
- Rust: Примери, integration тестове, подобрен Cargo.toml
|
||||||
|
- **SCRAM-SHA-256 Автентикация** — RFC 7677 съвместима автентикация с PBKDF2 + HMAC + SHA-256 + nonce/salt генериране
|
||||||
|
- **HTTP SCRAM Endpoints** — `/auth/scram/start` + `/auth/scram/finish` в HTTP сървъра
|
||||||
|
- **Docker Compose Тестова Конфигурация** — `docker-compose.test.yml` за тестови среди
|
||||||
|
- **CI/CD Clients Pipeline** — `.github/workflows/clients-ci.yml` за автоматизирано тестване на клиенти
|
||||||
|
|
||||||
|
### Поправки
|
||||||
|
|
||||||
|
- **Query Executor** — Унарен минус (`irNeg`) вече работи коректно в SELECT и WHERE клаузи
|
||||||
|
- **Distributed Transactions** — Rollback след commit опит вече не нарушава атомарността
|
||||||
|
- **Sharding** — Протокол за миграция на данни с TCP + `scanAll` на LSM
|
||||||
|
- **Raft** — Поправено изчисление на мнозинство за четен брой нодове
|
||||||
|
- **MVCC** — Прекъснатите транзакции вече не стават видими
|
||||||
|
- **LSM-Tree** — Поправена загуба на данни при презаписване на immutable memtable; поправено сортиране на SSTable търсене
|
||||||
|
- **Auth** — JWT подписът е променен на HMAC-SHA256 (вече не е тривиално forgeable); валидация на токен изтичане (`exp`/`nbf`/`iat`); сравнението на подписи вече е constant-time
|
||||||
|
- **Recovery** — `summary()` вече не мутира базата данни
|
||||||
|
- **Wire Protocol** — 64MB лимит + bounds проверки + max дълбочина за предотвратяване на OOM/DoS
|
||||||
|
- **SQL Injection** — `exprToSql` вече escape-ва единични кавички
|
||||||
|
- **ReDoS** — `irLike`/`irILike` вече escape-ват regex метасимволи
|
||||||
|
- **Graph** — `addEdge` вече проверява съществуването на възел
|
||||||
|
- **Vector** — Валидация на несъответствие на размерности + HNSW заключване
|
||||||
|
- **FTS** — UTF-8 токенизацията вече използва runes вместо байтове
|
||||||
|
- **Build** — `nim.cfg` добавя `-d:ssl`, така че `nimble build` работи без флагове; `--threads:on` добавен към всички CI команди
|
||||||
|
|
||||||
|
### Промени
|
||||||
|
|
||||||
|
- **Версията е вдигната до 1.1.0** във всички компоненти (сървър, Docker изображения, клиенти, CLI)
|
||||||
|
- **README** — Версионният badge е обновен; всички feature таблици вече реферират v1.1.0
|
||||||
|
- **TLA+ Формална Верификация** — Добавени `crossmodal.tla`, `backup.tla`, `recovery.tla`; symmetry reduction във всички 9 спецификации
|
||||||
|
- **Чист build** — 0 компилаторни предупреждения на Nim 2.2.10
|
||||||
|
|
||||||
|
## [0.1.0] — 2025-01-15
|
||||||
|
|
||||||
|
### Добавки
|
||||||
|
|
||||||
|
- **Ядро за Съхранение**
|
||||||
|
- LSM-Tree с MemTable, WAL, SSTables и size-tiered compaction
|
||||||
|
- B-Tree подреден индекс с range сканиране и MVCC copy-on-write
|
||||||
|
- Bloom филтри за ефективно пропускане на SSTable
|
||||||
|
- Memory-mapped I/O за SSTable четене
|
||||||
|
- LRU page cache с проследяване на hit rate
|
||||||
|
|
||||||
|
- **Query Engine (BaraQL)**
|
||||||
|
- SQL-съвместим lexer с 80+ типа токени
|
||||||
|
- Recursive descent parser генериращ AST с 25+ вида възли
|
||||||
|
- Intermediate representation (IR) за планове за изпълнение
|
||||||
|
- Code generator превеждащ IR към storage операции
|
||||||
|
- Adaptive query optimizer с cross-modal планиране
|
||||||
|
- Query executor с паралелизация
|
||||||
|
|
||||||
|
- **BaraQL Езикови Възможности**
|
||||||
|
- SELECT, INSERT, UPDATE, DELETE
|
||||||
|
- WHERE, ORDER BY, LIMIT, OFFSET
|
||||||
|
- GROUP BY, HAVING, агрегатни функции (count, sum, avg, min, max)
|
||||||
|
- INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, CROSS JOIN
|
||||||
|
- CTEs (Common Table Expressions) с WITH
|
||||||
|
- Подзаявки (EXISTS, IN, корелирани)
|
||||||
|
- CASE изрази
|
||||||
|
- UNION, INTERSECT, EXCEPT
|
||||||
|
- Дефиниране на схема: CREATE TYPE, DROP TYPE
|
||||||
|
|
||||||
|
- **Vector Engine**
|
||||||
|
- HNSW индекс за приблизително търсене на най-близки съседи
|
||||||
|
- IVF-PQ индекс за мащабно векторно търсене
|
||||||
|
- SIMD-оптимизирани функции за разстояние (cosine, L2, dot product, Manhattan)
|
||||||
|
- Квантуване: scalar 8-bit/4-bit, product quantization, binary
|
||||||
|
- Филтриране по метаданни при векторно търсене
|
||||||
|
|
||||||
|
- **Graph Engine**
|
||||||
|
- Adjacency list съхранение за насочени графи с тегла
|
||||||
|
- BFS и DFS обхождане
|
||||||
|
- Dijkstra най-кратък път
|
||||||
|
- PageRank важност на възли
|
||||||
|
- Louvain community detection
|
||||||
|
- Subgraph pattern matching
|
||||||
|
- Cypher-подобен graph query parser
|
||||||
|
|
||||||
|
- **Full-Text Search**
|
||||||
|
- Inverted index с term-document mapping
|
||||||
|
- BM25 алгоритъм за ранжиране
|
||||||
|
- TF-IDF оценяване
|
||||||
|
- Fuzzy търсене с Levenshtein разстояние
|
||||||
|
- Wildcard/regex търсене
|
||||||
|
- Многоезични токенизатори (английски, български, немски, френски, руски)
|
||||||
|
|
||||||
|
- **Columnar Storage**
|
||||||
|
- Колонково съхранение за аналитични заявки
|
||||||
|
- RLE (Run-Length Encoding) компресия
|
||||||
|
- Dictionary encoding за колони с ниска кардиналност
|
||||||
|
- SIMD-ускорени агрегати
|
||||||
|
|
||||||
|
- **Транзакции**
|
||||||
|
- MVCC (Multi-Version Concurrency Control) със snapshot изолация
|
||||||
|
- Deadlock детекция чрез wait-for граф
|
||||||
|
- Write-ahead log за устойчивост
|
||||||
|
- Savepoints и частичен rollback
|
||||||
|
|
||||||
|
- **Протоколен Слой**
|
||||||
|
- Бинарен wire протокол с 16 типа съобщения
|
||||||
|
- HTTP/REST JSON API
|
||||||
|
- WebSocket стрийминг
|
||||||
|
- Connection pooling
|
||||||
|
- JWT-базирана автентикация
|
||||||
|
- Token-bucket rate limiting
|
||||||
|
- TLS/SSL с автоматично генерирани сертификати
|
||||||
|
|
||||||
|
- **Система за Схеми**
|
||||||
|
- Силна типова система с 17 нативни типа
|
||||||
|
- Наследяване на типове с multi-base поддръжка
|
||||||
|
- Property links между типове
|
||||||
|
- Schema diffing и миграции
|
||||||
|
- Изчислими свойства
|
||||||
|
|
||||||
|
- **Разпределени Системи**
|
||||||
|
- Raft консенсус (leader election, log replication)
|
||||||
|
- Hash, range и consistent-hash шардиране
|
||||||
|
- Sync/async/semi-sync репликация
|
||||||
|
- Gossip протокол за управление на членство
|
||||||
|
- Two-phase commit за разпределени транзакции
|
||||||
|
|
||||||
|
- **Cross-Modal Заявки**
|
||||||
|
- Унифициран език за заявки през всички storage двигатели
|
||||||
|
- Cross-engine predicate pushdown
|
||||||
|
- Оптимизирани планове за изпълнение за multi-modal заявки
|
||||||
|
|
||||||
|
- **Backup & Recovery**
|
||||||
|
- Online snapshots без прекъсване
|
||||||
|
- Point-in-time recovery чрез WAL replay
|
||||||
|
- Инкрементални backups
|
||||||
|
|
||||||
|
- **Client SDKs**
|
||||||
|
- JavaScript/TypeScript клиент с бинарен протокол
|
||||||
|
- Python клиент със sync и async API
|
||||||
|
- Nim embedded режим и клиентска библиотека
|
||||||
|
- Rust клиент (async)
|
||||||
|
|
||||||
|
- **Операции**
|
||||||
|
- Интерактивен CLI shell (BaraQL REPL)
|
||||||
|
- Структурирано логване (JSON и текстови формати)
|
||||||
|
- Prometheus-съвместим metrics endpoint
|
||||||
|
- Health и readiness проби
|
||||||
|
- CPU/memory profiling endpoints
|
||||||
|
|
||||||
|
- **Docker Поддръжка**
|
||||||
|
- Multi-stage Dockerfile (Alpine Linux)
|
||||||
|
- Docker Compose конфигурация
|
||||||
|
- Health checks
|
||||||
|
|
||||||
|
### Производителност
|
||||||
|
|
||||||
|
- LSM-Tree: 580K записа/s, 720K четения/s
|
||||||
|
- B-Tree: 1.2M вмъквания/s, 1.5M търсения/s
|
||||||
|
- Vector SIMD: 850K косинусови разстояния/s (dim=768)
|
||||||
|
- FTS: 320K документи/s индексиране, 28K заявки/s BM25
|
||||||
|
- Graph: 2.5M възела/s вмъкване, 12K BFS обхождания/s
|
||||||
|
- Бинарен протокол: 380K заявки/s (100 конкурентни връзки)
|
||||||
|
|
||||||
|
### Тестове
|
||||||
|
|
||||||
|
- 262 теста в 56 тестови сюита
|
||||||
|
- 100% успеваемост
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Добавки
|
||||||
|
|
||||||
|
- **Vector SQL Integration** — Пълна поддръжка на векторно търсене на SQL ниво:
|
||||||
|
- `VECTOR(n)` тип колона в `CREATE TABLE` с валидация на размерност
|
||||||
|
- `CREATE INDEX ... USING hnsw` / `USING ivfpq` за приблизителни nearest neighbor индекси
|
||||||
|
- SQL функции за разстояние: `cosine_distance()`, `euclidean_distance()`, `inner_product()`, `l1_distance()`, `l2_distance()`
|
||||||
|
- `<->` nearest-neighbor оператор (евклидово разстояние)
|
||||||
|
- `ORDER BY` поддръжка за изрази с векторно разстояние, включително колони извън `SELECT`
|
||||||
|
- Автоматична поддръжка на HNSW индекс при `INSERT` и `UPDATE`
|
||||||
|
- **Advanced SQL Engine** — Window функции, MERGE/UPSERT, LATERAL JOIN, PIVOT/UNPIVOT, SQL/PGQ Property Graph, Разширени агрегати (ARRAY_AGG, STRING_AGG, FILTER, GROUPING SETS/ROLLUP/CUBE)
|
||||||
|
- **JavaScript Client — TCP Request Queue** — Вътрешна `_requestQueue` + `_requestLock` за безопасни конкурентни заявки. Множество паралелни извиквания на `query()` / `execute()` / `ping()` вече не размесват бинарни frame-ове по връзката.
|
||||||
|
|
||||||
|
### Поправки
|
||||||
|
|
||||||
|
- **Query Executor — Ескейпване на Стойности** — `execInsert` вече правилно ескейпва запетаи и знаци за равенство в стойностите на колоните, поправяйки корупция на съхранението за векторни литерали като `[1.0, 2.0, 3.0]`
|
||||||
|
- **Query Planner — ORDER BY Проекция** — `irpkSort` вече се поставя преди `irpkProject` в IR плана, позволявайки на `ORDER BY` да реферира колони, които не са селектирани
|
||||||
|
- **Wire Protocol — Big-Endian Float Сериализация** — `FLOAT32`/`FLOAT64` и float стойностите във вектори вече се сериализират в big-endian byte order, съвпадайки с `readFloatBE()` / `readDoubleBE()` на клиента и осигурявайки междуплатформена числова точност.
|
||||||
|
- **Gossip Protocol — Async UDP Socket** — Заменен синхронният `newSocket` + блокиращ `recvFrom` с `newAsyncSocket` + `await recvFrom`, предотвратявайки замръзване на async event loop-а до пристигане на UDP пакет.
|
||||||
|
|
||||||
|
### Планирани
|
||||||
|
|
||||||
|
- Query plan caching
|
||||||
|
- Materialized views
|
||||||
|
- Геопространствен индекс
|
||||||
|
- Time-series оптимизации
|
||||||
|
- CDC (Change Data Capture) стрийминг
|
||||||
|
- Федеративни заявки между BaraDB инстанции
|
||||||
@@ -0,0 +1,285 @@
|
|||||||
|
# Client SDKs
|
||||||
|
|
||||||
|
BaraDB предоставя официални клиентски библиотеки за JavaScript/TypeScript, Python, Nim и Rust.
|
||||||
|
|
||||||
|
## JavaScript / TypeScript
|
||||||
|
|
||||||
|
### Инсталация
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm install baradb
|
||||||
|
# или
|
||||||
|
yarn add baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Основна Употреба
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { Client } from 'baradb';
|
||||||
|
|
||||||
|
const client = new Client('localhost', 9472);
|
||||||
|
await client.connect();
|
||||||
|
|
||||||
|
// Проста заявка
|
||||||
|
const result = await client.query('SELECT name, age FROM users WHERE age > 18');
|
||||||
|
console.log(result.rows);
|
||||||
|
|
||||||
|
// Параметризирана заявка
|
||||||
|
const result2 = await client.query(
|
||||||
|
'SELECT * FROM users WHERE name = ?',
|
||||||
|
['Alice']
|
||||||
|
);
|
||||||
|
|
||||||
|
// Batch вмъкване
|
||||||
|
await client.batch([
|
||||||
|
"INSERT users { name := 'Alice', age := 30 }",
|
||||||
|
"INSERT users { name := 'Bob', age := 25 }",
|
||||||
|
]);
|
||||||
|
|
||||||
|
// Транзакции
|
||||||
|
await client.begin();
|
||||||
|
await client.query("INSERT orders { total := 100 }");
|
||||||
|
await client.query("UPDATE users SET balance = balance - 100 WHERE name = 'Alice'");
|
||||||
|
await client.commit();
|
||||||
|
|
||||||
|
await client.close();
|
||||||
|
```
|
||||||
|
|
||||||
|
### Конкурентни Заявки
|
||||||
|
|
||||||
|
JavaScript клиентът автоматично сериализира конкурентни заявки през една TCP връзка чрез вътрешна опашка. Можете безопасно да изпращате множество паралелни операции — техните бинарни frame-ове няма да се размесят:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const [users, orders, stats] = await Promise.all([
|
||||||
|
client.query('SELECT * FROM users'),
|
||||||
|
client.query('SELECT * FROM orders'),
|
||||||
|
client.query('SELECT count(*) FROM visits')
|
||||||
|
]);
|
||||||
|
```
|
||||||
|
|
||||||
|
### WebSocket Стрийминг
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { WebSocketClient } from 'baradb/ws';
|
||||||
|
|
||||||
|
const ws = new WebSocketClient('ws://localhost:9471');
|
||||||
|
ws.onMessage = (data) => console.log(data);
|
||||||
|
await ws.connect();
|
||||||
|
await ws.send('SUBSCRIBE updates');
|
||||||
|
```
|
||||||
|
|
||||||
|
## Python
|
||||||
|
|
||||||
|
### Инсталация
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Основна Употреба
|
||||||
|
|
||||||
|
```python
|
||||||
|
from baradb import Client
|
||||||
|
|
||||||
|
client = Client("localhost", 9472)
|
||||||
|
client.connect()
|
||||||
|
|
||||||
|
# Проста заявка
|
||||||
|
result = client.query("SELECT name, age FROM users WHERE age > 18")
|
||||||
|
for row in result:
|
||||||
|
print(row["name"], row["age"])
|
||||||
|
|
||||||
|
# Параметризирана заявка
|
||||||
|
result = client.query(
|
||||||
|
"SELECT * FROM users WHERE name = ?",
|
||||||
|
["Alice"]
|
||||||
|
)
|
||||||
|
|
||||||
|
# Batch операции
|
||||||
|
client.batch([
|
||||||
|
"INSERT users { name := 'Alice', age := 30 }",
|
||||||
|
"INSERT users { name := 'Bob', age := 25 }",
|
||||||
|
])
|
||||||
|
|
||||||
|
# Context manager (автоматично затваряне)
|
||||||
|
with Client("localhost", 9472) as c:
|
||||||
|
result = c.query("SELECT count(*) FROM users")
|
||||||
|
print(result[0]["count"])
|
||||||
|
```
|
||||||
|
|
||||||
|
### Async Клиент
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from baradb import AsyncClient
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
client = AsyncClient("localhost", 9472)
|
||||||
|
await client.connect()
|
||||||
|
result = await client.query("SELECT * FROM users")
|
||||||
|
print(result.rows)
|
||||||
|
await client.close()
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
## Nim (Вграден Режим)
|
||||||
|
|
||||||
|
### Добавяне на Зависимост
|
||||||
|
|
||||||
|
```nim
|
||||||
|
# Във вашия .nimble файл
|
||||||
|
requires "barabadb >= 1.1.0"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Вградена Употреба
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/lsm
|
||||||
|
import barabadb/storage/btree
|
||||||
|
import barabadb/vector/engine
|
||||||
|
import barabadb/graph/engine
|
||||||
|
|
||||||
|
# Key-Value store
|
||||||
|
var db = newLSMTree("./data")
|
||||||
|
db.put("user:1", cast[seq[byte]]("Alice"))
|
||||||
|
let (found, value) = db.get("user:1")
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
# B-Tree индекс
|
||||||
|
var btree = newBTreeIndex[string, int]()
|
||||||
|
btree.insert("Alice", 30)
|
||||||
|
let ages = btree.get("Alice")
|
||||||
|
|
||||||
|
# Векторно търсене
|
||||||
|
var idx = newHNSWIndex(dimensions = 128)
|
||||||
|
idx.insert(1, @[0.1'f32, 0.2, 0.3], {"category": "A"}.toTable)
|
||||||
|
let results = idx.search(@[0.1'f32, 0.2, 0.3], k = 10)
|
||||||
|
|
||||||
|
# Графи
|
||||||
|
var g = newGraph()
|
||||||
|
let alice = g.addNode("Person", {"name": "Alice"}.toTable)
|
||||||
|
let bob = g.addNode("Person", {"name": "Bob"}.toTable)
|
||||||
|
discard g.addEdge(alice, bob, "knows")
|
||||||
|
let path = g.shortestPath(alice, bob)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Клиентска Библиотека
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/client/client
|
||||||
|
|
||||||
|
var c = newBaraClient("localhost", 9472)
|
||||||
|
c.connect()
|
||||||
|
let result = c.query("SELECT name FROM users")
|
||||||
|
for row in result.rows:
|
||||||
|
echo row["name"]
|
||||||
|
c.close()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rust
|
||||||
|
|
||||||
|
### Добавяне на Зависимост
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[dependencies]
|
||||||
|
baradb = "0.1"
|
||||||
|
tokio = { version = "1", features = ["full"] }
|
||||||
|
```
|
||||||
|
|
||||||
|
### Основна Употреба
|
||||||
|
|
||||||
|
```rust
|
||||||
|
use baradb::Client;
|
||||||
|
|
||||||
|
#[tokio::main]
|
||||||
|
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||||
|
let mut client = Client::connect("localhost:9472").await?;
|
||||||
|
|
||||||
|
let result = client
|
||||||
|
.query("SELECT name, age FROM users WHERE age > 18")
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
for row in result.rows {
|
||||||
|
println!("{} is {} years old", row["name"], row["age"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
client.close().await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## HTTP/REST (Езиково Независим)
|
||||||
|
|
||||||
|
Всички езици могат да използват HTTP/REST API директно:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Заявка
|
||||||
|
curl -X POST http://localhost:9470/api/query \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Authorization: Bearer $TOKEN" \
|
||||||
|
-d '{"query": "SELECT * FROM users WHERE age > 18"}'
|
||||||
|
|
||||||
|
# Вмъкване
|
||||||
|
curl -X POST http://localhost:9470/api/query \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"query": "INSERT users { name := \"Alice\", age := 30 }"}'
|
||||||
|
|
||||||
|
# Схема
|
||||||
|
curl http://localhost:9470/api/schema
|
||||||
|
|
||||||
|
# Health
|
||||||
|
curl http://localhost:9470/health
|
||||||
|
|
||||||
|
# Метрики
|
||||||
|
curl http://localhost:9470/metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
## Connection Pooling
|
||||||
|
|
||||||
|
Всички официални клиенти поддържат connection pooling:
|
||||||
|
|
||||||
|
### JavaScript
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { Pool } from 'baradb';
|
||||||
|
|
||||||
|
const pool = new Pool({
|
||||||
|
host: 'localhost',
|
||||||
|
port: 9472,
|
||||||
|
min: 5,
|
||||||
|
max: 50,
|
||||||
|
idleTimeout: 30000,
|
||||||
|
});
|
||||||
|
|
||||||
|
const client = await pool.acquire();
|
||||||
|
try {
|
||||||
|
const result = await client.query('SELECT 1');
|
||||||
|
} finally {
|
||||||
|
pool.release(client);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Python
|
||||||
|
|
||||||
|
```python
|
||||||
|
from baradb import Pool
|
||||||
|
|
||||||
|
pool = Pool("localhost", 9472, min_size=5, max_size=50)
|
||||||
|
with pool.connection() as conn:
|
||||||
|
result = conn.query("SELECT 1")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Съответствие на Типове Данни
|
||||||
|
|
||||||
|
| BaraDB Тип | JavaScript | Python | Nim | Rust |
|
||||||
|
|------------|------------|--------|-----|------|
|
||||||
|
| `null` | `null` | `None` | `nil` | `Option::None` |
|
||||||
|
| `bool` | `boolean` | `bool` | `bool` | `bool` |
|
||||||
|
| `int8/16/32/64` | `number` | `int` | `int` | `i8/i16/i32/i64` |
|
||||||
|
| `float32/64` | `number` | `float` | `float32/float64` | `f32/f64` |
|
||||||
|
| `str` | `string` | `str` | `string` | `String` |
|
||||||
|
| `bytes` | `Uint8Array` | `bytes` | `seq[byte]` | `Vec<u8>` |
|
||||||
|
| `array` | `Array` | `list` | `seq` | `Vec` |
|
||||||
|
| `object` | `Object` | `dict` | `Table` | `HashMap` |
|
||||||
|
| `vector` | `Float32Array` | `list[float]` | `seq[float32]` | `Vec<f32>` |
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
# Колонково Съхранение (Columnar)
|
||||||
|
|
||||||
|
Колонково-ориентирано съхранение за аналитични заявки и агрегации.
|
||||||
|
|
||||||
|
## Употреба
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/core/columnar
|
||||||
|
|
||||||
|
var batch = newColumnBatch()
|
||||||
|
var ageCol = batch.addInt64Col("age")
|
||||||
|
var nameCol = batch.addStringCol("name")
|
||||||
|
|
||||||
|
ageCol.appendInt64(25)
|
||||||
|
nameCol.appendString("Alice")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Агрегации
|
||||||
|
|
||||||
|
```nim
|
||||||
|
echo ageCol.sumInt64()
|
||||||
|
echo ageCol.avgInt64()
|
||||||
|
echo ageCol.minInt64()
|
||||||
|
echo ageCol.maxInt64()
|
||||||
|
echo ageCol.count()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Кодиране
|
||||||
|
|
||||||
|
### RLE (Run-Length Encoding)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let rle = rleEncode(@[1'i64, 1, 1, 2, 2, 3])
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dictionary Encoding
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let dict = dictEncode(@["apple", "banana", "apple"])
|
||||||
|
```
|
||||||
|
|
||||||
|
## Типове Колони
|
||||||
|
|
||||||
|
| Тип | Описание |
|
||||||
|
|------|----------|
|
||||||
|
| `int32` | 32-битов integer |
|
||||||
|
| `int64` | 64-битов integer |
|
||||||
|
| `float32` | 32-битов float |
|
||||||
|
| `float64` | 64-битов float |
|
||||||
|
| `string` | Низ с променлива дължина |
|
||||||
|
| `bool` | Булев |
|
||||||
|
|
||||||
|
## Случаи на Употреба
|
||||||
|
|
||||||
|
- OLAP натоварвания
|
||||||
|
- Мащабни агрегации
|
||||||
|
- Data warehousing
|
||||||
|
- Анализ на времеви редове
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
# Конфигурационна Референция
|
||||||
|
|
||||||
|
BaraDB може да се конфигурира чрез **променливи на средата**, **конфигурационен файл** или **командно-редови флагове**.
|
||||||
|
|
||||||
|
## Ред на Приоритет
|
||||||
|
|
||||||
|
1. Командно-редови флагове (най-висок приоритет)
|
||||||
|
2. Променливи на средата
|
||||||
|
3. Конфигурационен файл (`baradb.conf` или `baradb.json`)
|
||||||
|
4. Вградени стойности по подразбиране (най-нисък приоритет)
|
||||||
|
|
||||||
|
## Променливи на Средата
|
||||||
|
|
||||||
|
### Мрежа
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_ADDRESS` | `127.0.0.1` | Адрес за свързване |
|
||||||
|
| `BARADB_PORT` | `9472` | TCP бинарен протокол порт |
|
||||||
|
| `BARADB_HTTP_PORT` | `9470` | HTTP/REST API порт |
|
||||||
|
| `BARADB_WS_PORT` | `9471` | WebSocket порт |
|
||||||
|
|
||||||
|
### Съхранение
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_DATA_DIR` | `./data` | Път до директория за данни |
|
||||||
|
| `BARADB_MEMTABLE_SIZE_MB` | `64` | Размер на MemTable в MB |
|
||||||
|
| `BARADB_CACHE_SIZE_MB` | `256` | Размер на page cache в MB |
|
||||||
|
| `BARADB_WAL_SYNC_INTERVAL_MS` | `0` | Интервал за WAL fsync (0 = всеки запис) |
|
||||||
|
| `BARADB_COMPACTION_INTERVAL_MS` | `60000` | Интервал за фонов compaction |
|
||||||
|
| `BARADB_BLOOM_BITS_PER_KEY` | `10` | Bloom филтър битове за ключ |
|
||||||
|
|
||||||
|
### TLS/SSL
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_TLS_ENABLED` | `false` | Включване на TLS |
|
||||||
|
| `BARADB_CERT_FILE` | — | Път до TLS сертификат |
|
||||||
|
| `BARADB_KEY_FILE` | — | Път до TLS частен ключ |
|
||||||
|
|
||||||
|
### Сигурност
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_AUTH_ENABLED` | `false` | Включване на автентикация |
|
||||||
|
| `BARADB_JWT_SECRET` | — | JWT подписващ secret |
|
||||||
|
| `BARADB_RATE_LIMIT_GLOBAL` | `10000` | Глобални заявки в секунда |
|
||||||
|
| `BARADB_RATE_LIMIT_PER_CLIENT` | `1000` | Заявки в секунда за клиент |
|
||||||
|
|
||||||
|
### Логване
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_LOG_LEVEL` | `info` | Ниво на логване: debug, info, warn, error |
|
||||||
|
| `BARADB_LOG_FILE` | — | Път до лог файл (stdout ако е празен) |
|
||||||
|
| `BARADB_LOG_FORMAT` | `json` | Формат на лога: json, text |
|
||||||
|
|
||||||
|
### Vector Engine
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_VECTOR_M` | `16` | HNSW `M` параметър |
|
||||||
|
| `BARADB_VECTOR_EF_CONSTRUCTION` | `200` | HNSW `efConstruction` |
|
||||||
|
| `BARADB_VECTOR_EF_SEARCH` | `64` | HNSW `efSearch` |
|
||||||
|
|
||||||
|
### Graph Engine
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_GRAPH_PAGE_RANK_ITERATIONS` | `20` | Брой итерации на PageRank |
|
||||||
|
| `BARADB_GRAPH_PAGE_RANK_DAMPING` | `0.85` | PageRank damping фактор |
|
||||||
|
| `BARADB_GRAPH_LOUVAIN_RESOLUTION` | `1.0` | Louvain резолюционен параметър |
|
||||||
|
|
||||||
|
### Разпределени
|
||||||
|
|
||||||
|
| Променлива | По подр. | Описание |
|
||||||
|
|------------|----------|----------|
|
||||||
|
| `BARADB_RAFT_NODE_ID` | — | Уникално ID на възел в клъстер |
|
||||||
|
| `BARADB_RAFT_PEERS` | — | Списък с адреси на peer-ове, разделени със запетая |
|
||||||
|
| `BARADB_RAFT_PORT` | `9001` | Raft вътрешен комуникационен порт |
|
||||||
|
| `BARADB_SHARD_COUNT` | `1` | Брой шардове |
|
||||||
|
| `BARADB_REPLICATION_FACTOR` | `1` | Фактор на репликация |
|
||||||
|
| `BARADB_SEED_NODES` | — | Gossip seed възли (host:port, разделени със запетая) |
|
||||||
|
|
||||||
|
## Конфигурационен Файл
|
||||||
|
|
||||||
|
### baradb.conf (INI-подобен)
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[server]
|
||||||
|
address = "0.0.0.0"
|
||||||
|
port = 9472
|
||||||
|
http_port = 9470
|
||||||
|
ws_port = 9471
|
||||||
|
|
||||||
|
[storage]
|
||||||
|
data_dir = "/var/lib/baradb"
|
||||||
|
memtable_size_mb = 256
|
||||||
|
cache_size_mb = 512
|
||||||
|
wal_sync_interval_ms = 10
|
||||||
|
compaction_interval_ms = 30000
|
||||||
|
|
||||||
|
[tls]
|
||||||
|
enabled = true
|
||||||
|
cert_file = "/etc/baradb/server.crt"
|
||||||
|
key_file = "/etc/baradb/server.key"
|
||||||
|
|
||||||
|
[auth]
|
||||||
|
enabled = true
|
||||||
|
jwt_secret = "change-me-in-production"
|
||||||
|
rate_limit_global = 10000
|
||||||
|
rate_limit_per_client = 1000
|
||||||
|
|
||||||
|
[logging]
|
||||||
|
level = "info"
|
||||||
|
format = "json"
|
||||||
|
file = "/var/log/baradb/baradb.log"
|
||||||
|
|
||||||
|
[vector]
|
||||||
|
m = 16
|
||||||
|
ef_construction = 200
|
||||||
|
ef_search = 64
|
||||||
|
|
||||||
|
[cluster]
|
||||||
|
raft_node_id = "node1"
|
||||||
|
raft_peers = "node2:9001,node3:9001"
|
||||||
|
```
|
||||||
|
|
||||||
|
### baradb.json
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"server": {
|
||||||
|
"address": "0.0.0.0",
|
||||||
|
"port": 9472,
|
||||||
|
"http_port": 9470,
|
||||||
|
"ws_port": 9471
|
||||||
|
},
|
||||||
|
"storage": {
|
||||||
|
"data_dir": "/var/lib/baradb",
|
||||||
|
"memtable_size_mb": 256,
|
||||||
|
"cache_size_mb": 512
|
||||||
|
},
|
||||||
|
"tls": {
|
||||||
|
"enabled": true,
|
||||||
|
"cert_file": "/etc/baradb/server.crt",
|
||||||
|
"key_file": "/etc/baradb/server.key"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Командно-редови Флагове
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./build/baradadb --help
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
|
||||||
|
Употреба:
|
||||||
|
baradadb [опции]
|
||||||
|
|
||||||
|
Опции:
|
||||||
|
-c, --config <файл> Път до конфигурационен файл
|
||||||
|
-p, --port <порт> TCP бинарен порт (по подр.: 9472)
|
||||||
|
--http-port <порт> HTTP порт (по подр.: 9470)
|
||||||
|
--ws-port <порт> WebSocket порт (по подр.: 9471)
|
||||||
|
-d, --data-dir <дир> Директория за данни (по подр.: ./data)
|
||||||
|
--tls-cert <файл> TLS сертификатен файл
|
||||||
|
--tls-key <файл> TLS файл с частен ключ
|
||||||
|
--log-level <ниво> Ниво на логване: debug, info, warn, error
|
||||||
|
--log-file <файл> Път до лог файл
|
||||||
|
--shell Стартиране на интерактивна обвивка
|
||||||
|
--version Показване на версия
|
||||||
|
--recover Изпълнение на WAL възстановяване
|
||||||
|
--checkpoint <файл> Checkpoint за възстановяване
|
||||||
|
-h, --help Показване на тази помощ
|
||||||
|
```
|
||||||
|
|
||||||
|
## Примерни Конфигурации
|
||||||
|
|
||||||
|
### Разработка
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./build/baradadb \
|
||||||
|
--log-level debug \
|
||||||
|
--data-dir ./dev_data
|
||||||
|
```
|
||||||
|
|
||||||
|
### Продукционен Единичен Възел
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_TLS_ENABLED=true \
|
||||||
|
BARADB_CERT_FILE=/etc/baradb/server.crt \
|
||||||
|
BARADB_KEY_FILE=/etc/baradb/server.key \
|
||||||
|
BARADB_AUTH_ENABLED=true \
|
||||||
|
BARADB_JWT_SECRET="$(openssl rand -hex 32)" \
|
||||||
|
BARADB_LOG_LEVEL=warn \
|
||||||
|
BARADB_LOG_FILE=/var/log/baradb/baradb.log \
|
||||||
|
BARADB_MEMTABLE_SIZE_MB=256 \
|
||||||
|
BARADB_CACHE_SIZE_MB=1024 \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Продукционен Клъстер (3 възела)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Възел 1
|
||||||
|
BARADB_ADDRESS=0.0.0.0 \
|
||||||
|
BARADB_PORT=9472 \
|
||||||
|
BARADB_RAFT_NODE_ID=node1 \
|
||||||
|
BARADB_RAFT_PEERS=node2:9001,node3:9001 \
|
||||||
|
BARADB_SHARD_COUNT=4 \
|
||||||
|
BARADB_REPLICATION_FACTOR=2 \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# Cross-Modal Заявки
|
||||||
|
|
||||||
|
Уникалната способност на BaraDB да изпълнява заявки, обхващащи множество storage двигатели в една унифицирана BaraQL заявка.
|
||||||
|
|
||||||
|
## Преглед
|
||||||
|
|
||||||
|
Традиционните бази данни изискват отделни заявки и join-ове на ниво приложение при работа с различни модели на данни. Cross-modal query planner-ът на BaraDB оптимизира изпълнението през:
|
||||||
|
|
||||||
|
- **Документи/KV** (LSM-Tree) — структурирани записи
|
||||||
|
- **Графи** (Adjacency List) — връзки
|
||||||
|
- **Вектори** (HNSW/IVF-PQ) — търсене на прилика
|
||||||
|
- **Пълен текст** (Inverted Index) — текстово търсене
|
||||||
|
- **Колонково** — аналитични агрегати
|
||||||
|
|
||||||
|
## Примери за Заявки
|
||||||
|
|
||||||
|
### Векторно + Пълнотекстово (Семантично + Ключово Търсене)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT title, score
|
||||||
|
FROM articles
|
||||||
|
WHERE MATCH(body) AGAINST('machine learning')
|
||||||
|
ORDER BY cosine_distance(embedding, [0.1, 0.2, 0.3, ...])
|
||||||
|
LIMIT 10;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Графово + Векторно (Социални Препоръки)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
MATCH (u:User)-[:KNOWS]->(friend:User)
|
||||||
|
WHERE u.name = 'Alice'
|
||||||
|
ORDER BY cosine_distance(friend.taste_vector, u.taste_vector)
|
||||||
|
RETURN friend.name, friend.age;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Документно + Графово (Обогатяване на Същности)
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT o.id, o.total, c.name,
|
||||||
|
(SELECT count(*) FROM orders WHERE customer_id = c.id) as order_count
|
||||||
|
FROM orders o
|
||||||
|
MATCH (c:Customer)-[:PLACED]->(o)
|
||||||
|
WHERE o.date > '2025-01-01';
|
||||||
|
```
|
||||||
|
|
||||||
|
## Cross-Modal Engine API
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/core/crossmodal
|
||||||
|
|
||||||
|
var engine = newCrossModalEngine("/tmp/baradb")
|
||||||
|
|
||||||
|
# Документни операции
|
||||||
|
engine.put("key1", cast[seq[byte]]("value1"))
|
||||||
|
let (found, val) = engine.get("key1")
|
||||||
|
|
||||||
|
# Векторни операции
|
||||||
|
engine.insertVector(1, @[1.0'f32, 0.0'f32, ...], {"cat": "A"}.toTable)
|
||||||
|
let results = engine.searchVector(@[1.0'f32, 0.1'f32, ...], 2)
|
||||||
|
|
||||||
|
# Графови операции
|
||||||
|
let n1 = engine.addNode("Person")
|
||||||
|
let n2 = engine.addNode("Person")
|
||||||
|
discard engine.addEdge(n1, n2, "knows")
|
||||||
|
let traversal = engine.traverseGraph(n1, "bfs")
|
||||||
|
|
||||||
|
# FTS операции
|
||||||
|
engine.indexText(1, "Nim programming language")
|
||||||
|
let ftsResults = engine.searchText("programming")
|
||||||
|
|
||||||
|
# Хибридно търсене
|
||||||
|
var query = newCrossModalQuery(qmHybrid)
|
||||||
|
query.vector = @[1.0'f32, 0.0'f32]
|
||||||
|
query.searchQuery = "fast"
|
||||||
|
query.vecWeight = 1.0
|
||||||
|
query.ftsWeight = 1.0
|
||||||
|
let hybridResult = engine.hybridSearch(query)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2PC Транзакции
|
||||||
|
|
||||||
|
Cross-modal engine-ът поддържа two-phase commit за атомарни операции през множество storage системи:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
var txn = newTPCTransaction(1)
|
||||||
|
txn.addParticipant("storage")
|
||||||
|
txn.addParticipant("vector")
|
||||||
|
txn.addParticipant("graph")
|
||||||
|
|
||||||
|
txn.prepare() # Всички участници потвърждават, че могат да комитнат
|
||||||
|
txn.commit() # Атомарен commit през всички участници
|
||||||
|
```
|
||||||
|
|
||||||
|
## Формална Верификация
|
||||||
|
|
||||||
|
Cross-modal консистентността е формално специфицирана в TLA+:
|
||||||
|
|
||||||
|
- **Спецификация:** `formal-verification/crossmodal.tla`
|
||||||
|
- **Проверени свойства:**
|
||||||
|
- `MetadataVectorConsistency` — insertVector обновява метаданни за филтрирано търсене
|
||||||
|
- `CrossModalAtomicity` — всички участници комитват или всички абортират
|
||||||
@@ -0,0 +1,250 @@
|
|||||||
|
# Ръководство за Внедряване (Deployment)
|
||||||
|
|
||||||
|
## Docker
|
||||||
|
|
||||||
|
За пълно ръководство за Docker deployment вижте [Docker Guide](docker.md).
|
||||||
|
|
||||||
|
### Бърз старт
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -t baradb:latest .
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Compose файлове
|
||||||
|
|
||||||
|
| Файл | Назначение |
|
||||||
|
|------|-----------|
|
||||||
|
| `docker-compose.yml` | Development |
|
||||||
|
| `docker-compose.prod.yml` | Production |
|
||||||
|
| `docker-compose.override.yml` | Dev override (автоматично) |
|
||||||
|
| `docker-compose.test.yml` | Тестова среда |
|
||||||
|
|
||||||
|
### Production
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.prod.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Swarm
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker stack deploy -c docker-compose.prod.yml baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
## systemd Услуга
|
||||||
|
|
||||||
|
Създайте `/etc/systemd/system/baradb.service`:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=BaraDB Multimodal Database
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=baradb
|
||||||
|
Group=baradb
|
||||||
|
WorkingDirectory=/var/lib/baradb
|
||||||
|
ExecStart=/usr/local/bin/baradadb
|
||||||
|
Restart=always
|
||||||
|
RestartSec=5
|
||||||
|
|
||||||
|
Environment=BARADB_PORT=9472
|
||||||
|
Environment=BARADB_HTTP_PORT=9470
|
||||||
|
Environment=BARADB_DATA_DIR=/var/lib/baradb/data
|
||||||
|
Environment=BARADB_LOG_LEVEL=info
|
||||||
|
|
||||||
|
# Подсилване на сигурността
|
||||||
|
NoNewPrivileges=true
|
||||||
|
ProtectSystem=strict
|
||||||
|
ProtectHome=true
|
||||||
|
ReadWritePaths=/var/lib/baradb/data
|
||||||
|
ProtectKernelTunables=true
|
||||||
|
ProtectKernelModules=true
|
||||||
|
ProtectControlGroups=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
```
|
||||||
|
|
||||||
|
Активиране и стартиране:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo useradd -r -s /bin/false baradb
|
||||||
|
sudo mkdir -p /var/lib/baradb/data
|
||||||
|
sudo chown -R baradb:baradb /var/lib/baradb
|
||||||
|
sudo cp build/baradadb /usr/local/bin/
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Kubernetes
|
||||||
|
|
||||||
|
### StatefulSet
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: StatefulSet
|
||||||
|
metadata:
|
||||||
|
name: baradb
|
||||||
|
spec:
|
||||||
|
serviceName: baradb
|
||||||
|
replicas: 3
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: baradb
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app: baradb
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: baradb
|
||||||
|
image: baradb:latest
|
||||||
|
ports:
|
||||||
|
- containerPort: 9472
|
||||||
|
name: binary
|
||||||
|
- containerPort: 9470
|
||||||
|
name: http
|
||||||
|
- containerPort: 9471
|
||||||
|
name: websocket
|
||||||
|
env:
|
||||||
|
- name: BARADB_DATA_DIR
|
||||||
|
value: /data
|
||||||
|
- name: BARADB_RAFT_NODE_ID
|
||||||
|
valueFrom:
|
||||||
|
fieldRef:
|
||||||
|
fieldPath: metadata.name
|
||||||
|
volumeMounts:
|
||||||
|
- name: data
|
||||||
|
mountPath: /data
|
||||||
|
volumeClaimTemplates:
|
||||||
|
- metadata:
|
||||||
|
name: data
|
||||||
|
spec:
|
||||||
|
accessModes: ["ReadWriteOnce"]
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: 100Gi
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: baradb
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app: baradb
|
||||||
|
ports:
|
||||||
|
- port: 9472
|
||||||
|
name: binary
|
||||||
|
- port: 9470
|
||||||
|
name: http
|
||||||
|
- port: 9471
|
||||||
|
name: websocket
|
||||||
|
clusterIP: None
|
||||||
|
```
|
||||||
|
|
||||||
|
## Reverse Proxy (nginx)
|
||||||
|
|
||||||
|
```nginx
|
||||||
|
upstream baradb_http {
|
||||||
|
server 127.0.0.1:9470;
|
||||||
|
}
|
||||||
|
|
||||||
|
upstream baradb_ws {
|
||||||
|
server 127.0.0.1:9471;
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
server_name db.example.com;
|
||||||
|
return 301 https://$server_name$request_uri;
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 443 ssl http2;
|
||||||
|
server_name db.example.com;
|
||||||
|
|
||||||
|
ssl_certificate /etc/letsencrypt/live/db.example.com/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/db.example.com/privkey.pem;
|
||||||
|
|
||||||
|
location /api/ {
|
||||||
|
proxy_pass http://baradb_http/;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
}
|
||||||
|
|
||||||
|
location /ws/ {
|
||||||
|
proxy_pass http://baradb_ws/;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
|
proxy_set_header Connection "upgrade";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Висока Достъпност (High Availability)
|
||||||
|
|
||||||
|
### 3-Възел Raft Клъстер
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Възел 1
|
||||||
|
BARADB_RAFT_NODE_ID=node1 \
|
||||||
|
BARADB_RAFT_PEERS=node2:9001,node3:9001 \
|
||||||
|
./build/baradadb
|
||||||
|
|
||||||
|
# Възел 2
|
||||||
|
BARADB_RAFT_NODE_ID=node2 \
|
||||||
|
BARADB_RAFT_PEERS=node1:9001,node3:9001 \
|
||||||
|
./build/baradadb
|
||||||
|
|
||||||
|
# Възел 3
|
||||||
|
BARADB_RAFT_NODE_ID=node3 \
|
||||||
|
BARADB_RAFT_PEERS=node1:9001,node2:9001 \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Облачно Внедряване
|
||||||
|
|
||||||
|
### AWS EC2
|
||||||
|
|
||||||
|
Препоръчителна инстанция: `m6i.2xlarge` (8 vCPU, 32 GB RAM)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# User data скрипт
|
||||||
|
#!/bin/bash
|
||||||
|
apt-get update
|
||||||
|
apt-get install -y nim
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-linux-amd64
|
||||||
|
chmod +x baradadb-linux-amd64
|
||||||
|
mv baradadb-linux-amd64 /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
mkdir -p /data/baradb
|
||||||
|
cat > /etc/systemd/system/baradb.service << 'EOF'
|
||||||
|
[Unit]
|
||||||
|
Description=BaraDB
|
||||||
|
After=network.target
|
||||||
|
[Service]
|
||||||
|
ExecStart=/usr/local/bin/baradadb
|
||||||
|
Environment=BARADB_DATA_DIR=/data/baradb
|
||||||
|
Restart=always
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
EOF
|
||||||
|
|
||||||
|
systemctl daemon-reload
|
||||||
|
systemctl enable --now baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
### GCP Cloud Run (само HTTP)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
gcloud run deploy baradb \
|
||||||
|
--image gcr.io/PROJECT/baradb \
|
||||||
|
--port 9470 \
|
||||||
|
--memory 4Gi \
|
||||||
|
--cpu 2 \
|
||||||
|
--max-instances 10
|
||||||
|
```
|
||||||
+58
-5
@@ -1,9 +1,11 @@
|
|||||||
# Разпределена Система
|
# Разпределена Система
|
||||||
|
|
||||||
Поддръжка за разпределено внедряване с Raft консенсус, шардиране и репликация.
|
BaraDB поддържа разпределено внедряване с Raft консенсус, шардиране, репликация и gossip протокол.
|
||||||
|
|
||||||
## Raft Консенсус
|
## Raft Консенсус
|
||||||
|
|
||||||
|
Leader election и log репликация:
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
import barabadb/core/raft
|
import barabadb/core/raft
|
||||||
|
|
||||||
@@ -13,19 +15,35 @@ cluster.addNode("node2")
|
|||||||
cluster.addNode("node3")
|
cluster.addNode("node3")
|
||||||
|
|
||||||
let n1 = cluster.nodes["n1"]
|
let n1 = cluster.nodes["n1"]
|
||||||
|
n1.becomeCandidate()
|
||||||
n1.becomeLeader()
|
n1.becomeLeader()
|
||||||
|
let entry = n1.appendLog("SET key1 value1")
|
||||||
```
|
```
|
||||||
|
|
||||||
## Шардиране
|
## Шардиране
|
||||||
|
|
||||||
|
Разпределение на данни между възли:
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
import barabadb/core/sharding
|
import barabadb/core/sharding
|
||||||
|
|
||||||
var router = newShardRouter(ShardConfig(numShards: 4, replicas: 2))
|
var router = newShardRouter(ShardConfig(
|
||||||
|
numShards: 4,
|
||||||
|
replicas: 2,
|
||||||
|
strategy: ssHash
|
||||||
|
))
|
||||||
router.rebalance(@["node1", "node2", "node3"])
|
router.rebalance(@["node1", "node2", "node3"])
|
||||||
let shard = router.getShard("user_123")
|
let shard = router.getShard("user_123")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Стратегии за Шардиране
|
||||||
|
|
||||||
|
| Стратегия | Описание |
|
||||||
|
|-----------|----------|
|
||||||
|
| `ssHash` | Хеш-базирано шардиране |
|
||||||
|
| `ssRange` | Range-базирано шардиране |
|
||||||
|
| `ssConsistent` | Consistent hashing |
|
||||||
|
|
||||||
## Репликация
|
## Репликация
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
@@ -34,11 +52,46 @@ import barabadb/core/replication
|
|||||||
var rm = newReplicationManager(rmSync)
|
var rm = newReplicationManager(rmSync)
|
||||||
rm.addReplica(newReplica("r1", "10.0.0.1", 9472))
|
rm.addReplica(newReplica("r1", "10.0.0.1", 9472))
|
||||||
rm.connectReplica("r1")
|
rm.connectReplica("r1")
|
||||||
|
let lsn = rm.writeLsn(@[1'u8, 2, 3])
|
||||||
|
rm.ackLsn("r1", lsn)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Режими на Репликация
|
||||||
|
|
||||||
|
| Режим | Описание |
|
||||||
|
|--------|----------|
|
||||||
|
| `rmSync` | Синхронна репликация |
|
||||||
|
| `rmAsync` | Асинхронна репликация |
|
||||||
|
| `rmSemiSync` | Полу-синхронна репликация |
|
||||||
|
|
||||||
|
## Gossip Протокол
|
||||||
|
|
||||||
|
Управление на членство и детекция на откази:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/core/gossip
|
||||||
|
|
||||||
|
var g = newGossipProtocol("node1", "localhost", 9472, gossipPort = 9572)
|
||||||
|
g.join(newGossipNode("node2", "10.0.0.2", 9472))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Разпределени Транзакции
|
||||||
|
|
||||||
|
Two-phase commit между възли:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/core/disttxn
|
||||||
|
|
||||||
|
var tm = newDistTxnManager()
|
||||||
|
let txn = tm.beginTransaction("node1")
|
||||||
|
txn.addParticipant("node2", "10.0.0.2", 9472)
|
||||||
|
txn.prepare()
|
||||||
|
txn.commit()
|
||||||
```
|
```
|
||||||
|
|
||||||
## Формална Верификация
|
## Формална Верификация
|
||||||
|
|
||||||
Разпределените алгоритми са формално специфицирани в TLA+ и проверени с TLC:
|
Основните разпределени алгоритми са формално специфицирани в TLA+ и проверени с TLC:
|
||||||
|
|
||||||
- **Raft Консенсус** — `formal-verification/raft.tla`
|
- **Raft Консенсус** — `formal-verification/raft.tla`
|
||||||
- Проверено: ElectionSafety, StateMachineSafety
|
- Проверено: ElectionSafety, StateMachineSafety
|
||||||
@@ -47,11 +100,11 @@ rm.connectReplica("r1")
|
|||||||
- **Репликация** — `formal-verification/replication.tla`
|
- **Репликация** — `formal-verification/replication.tla`
|
||||||
- Проверено: MonotonicLsn, AcksRemovePending
|
- Проверено: MonotonicLsn, AcksRemovePending
|
||||||
|
|
||||||
Пускане на TLC:
|
Пускане на TLC локално:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd formal-verification
|
cd formal-verification
|
||||||
java -cp tla2tools.jar tlc2.TLC -config models/raft.cfg raft.tla
|
java -cp tla2tools.jar tlc2.TLC -config models/raft.cfg raft.tla
|
||||||
java -cp tla2tools.jar tlc2.TLC -config models/twopc.cfg twopc.tla
|
java -cp tla2tools.jar tlc2.TLC -config models/twopc.cfg twopc.tla
|
||||||
java -cp tla2tools.jar tlc2.TLC -config models/replication.cfg replication.tla
|
java -cp tla2tools.jar tlc2.TLC -config models/replication.cfg replication.tla
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -0,0 +1,183 @@
|
|||||||
|
# Docker Deployment Ръководство
|
||||||
|
|
||||||
|
Това ръководство описва как да използвате BaraDB с Docker и Docker Compose.
|
||||||
|
|
||||||
|
## Бърз старт
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Клониране на репото
|
||||||
|
git clone https://codeberg.org/baraba/baradb
|
||||||
|
cd barabaDB
|
||||||
|
|
||||||
|
# Build на образа
|
||||||
|
docker build -t baradb:latest .
|
||||||
|
|
||||||
|
# Стартиране с Docker Compose
|
||||||
|
docker compose up -d
|
||||||
|
|
||||||
|
# Проверка на статуса
|
||||||
|
docker compose ps
|
||||||
|
docker compose logs -f
|
||||||
|
```
|
||||||
|
|
||||||
|
## Файлове
|
||||||
|
|
||||||
|
| Файл | Описание |
|
||||||
|
|------|----------|
|
||||||
|
| `Dockerfile` | Мулти-stage production build |
|
||||||
|
| `docker-compose.yml` | Development конфигурация |
|
||||||
|
| `docker-compose.prod.yml` | Production конфигурация |
|
||||||
|
| `docker-compose.override.yml` | Development override (автоматично се зарежда) |
|
||||||
|
| `docker-compose.test.yml` | Тестова конфигурация |
|
||||||
|
| `docker-entrypoint.sh` | Entrypoint скрипт за инициализация |
|
||||||
|
| `.dockerignore` | Файлове, които да не се копират в образа |
|
||||||
|
|
||||||
|
## Създаване на образ
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Стандартен build
|
||||||
|
docker build -t baradb:latest .
|
||||||
|
|
||||||
|
# С конкретна версия
|
||||||
|
docker build -t baradb:1.1.0 .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Стартиране
|
||||||
|
|
||||||
|
### Development (docker compose)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Стартиране на заден план
|
||||||
|
docker compose up -d
|
||||||
|
|
||||||
|
# Спиране
|
||||||
|
docker compose down
|
||||||
|
|
||||||
|
# Спиране и изтриване на volumes (ВНИМАНИЕ — изтрива данните!)
|
||||||
|
docker compose down -v
|
||||||
|
|
||||||
|
# Преглед на логове
|
||||||
|
docker compose logs -f
|
||||||
|
```
|
||||||
|
|
||||||
|
### Production (docker compose)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Стартиране с production конфигурация
|
||||||
|
docker compose -f docker-compose.prod.yml up -d
|
||||||
|
|
||||||
|
# Проверка на healthcheck
|
||||||
|
docker compose -f docker-compose.prod.yml ps
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ръчно (docker run)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d \
|
||||||
|
--name baradb \
|
||||||
|
-p 9472:9472 \
|
||||||
|
-p 9470:9470 \
|
||||||
|
-p 9471:9471 \
|
||||||
|
-v baradb_data:/data \
|
||||||
|
-e BARADB_LOG_LEVEL=info \
|
||||||
|
baradb:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
## Портове
|
||||||
|
|
||||||
|
| Порт | Описание |
|
||||||
|
|------|----------|
|
||||||
|
| `9472` | Binary wire протокол |
|
||||||
|
| `9470` | HTTP/REST API |
|
||||||
|
| `9471` | WebSocket |
|
||||||
|
|
||||||
|
## Променливи на Средата
|
||||||
|
|
||||||
|
| Променлива | Стойност по подразбиране | Описание |
|
||||||
|
|------------|--------------------------|----------|
|
||||||
|
| `BARADB_ADDRESS` | `0.0.0.0` | Адрес за слушане |
|
||||||
|
| `BARADB_PORT` | `9472` | Binary протокол порт |
|
||||||
|
| `BARADB_HTTP_PORT` | `9470` | HTTP порт |
|
||||||
|
| `BARADB_WS_PORT` | `9471` | WebSocket порт |
|
||||||
|
| `BARADB_DATA_DIR` | `/data` | Директория за данни |
|
||||||
|
| `BARADB_LOG_LEVEL` | `info` | Ниво на логове |
|
||||||
|
|
||||||
|
## Volumes
|
||||||
|
|
||||||
|
| Път в контейнера | Описание |
|
||||||
|
|------------------|----------|
|
||||||
|
| `/data` | Основна директория за базата данни |
|
||||||
|
| `/data/server/wal` | Write-ahead log |
|
||||||
|
| `/data/server/sstables` | SSTable файлове |
|
||||||
|
|
||||||
|
## Production Checklist
|
||||||
|
|
||||||
|
- [ ] Създайте TLS сертификати в `./certs/`
|
||||||
|
- [ ] Задайте силен `BARADB_JWT_SECRET`
|
||||||
|
- [ ] Настройте firewall правила
|
||||||
|
- [ ] Конфигурирайте регулярни backups
|
||||||
|
- [ ] Проверете resource limits
|
||||||
|
- [ ] Настройте мониторинг (healthcheck, logs)
|
||||||
|
|
||||||
|
## TLS в Docker
|
||||||
|
|
||||||
|
1. Създайте сертификати:
|
||||||
|
```bash
|
||||||
|
mkdir -p certs
|
||||||
|
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
|
||||||
|
-keyout certs/server.key -out certs/server.crt
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Активирайте в `docker-compose.prod.yml`:
|
||||||
|
```yaml
|
||||||
|
environment:
|
||||||
|
- BARADB_TLS_ENABLED=true
|
||||||
|
- BARADB_CERT_FILE=/certs/server.crt
|
||||||
|
- BARADB_KEY_FILE=/certs/server.key
|
||||||
|
volumes:
|
||||||
|
- ./certs:/certs:ro
|
||||||
|
```
|
||||||
|
|
||||||
|
## Backup в Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Ръчен backup
|
||||||
|
docker exec baradb baradadb --snapshot --output=/backup/snapshot.db
|
||||||
|
|
||||||
|
# Списък на backups
|
||||||
|
ls /backup/
|
||||||
|
|
||||||
|
# Възстановяване
|
||||||
|
docker exec baradb baradadb --recover --checkpoint=/backup/snapshot.db
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Контейнерът не стартира
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Проверка на логове
|
||||||
|
docker compose logs -f baradb
|
||||||
|
|
||||||
|
# Проверка на статус
|
||||||
|
docker compose ps
|
||||||
|
```
|
||||||
|
|
||||||
|
### Няма връзка с базата
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Проверка дали портовете са експозвани
|
||||||
|
docker port baradb
|
||||||
|
|
||||||
|
# Проверка отвътре
|
||||||
|
docker exec baradb wget -qO- http://localhost:9470/health
|
||||||
|
```
|
||||||
|
|
||||||
|
### Permission denied на /data
|
||||||
|
|
||||||
|
Entrypoint скриптът автоматично създава директориите и задава правилните permissions. Ако имате проблем:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec baradb ls -la /data
|
||||||
|
docker exec baradb chown -R baradb:baradb /data
|
||||||
|
```
|
||||||
+67
-15
@@ -1,6 +1,6 @@
|
|||||||
# Пълнотекстово Търсене
|
# Full-Text Search Engine
|
||||||
|
|
||||||
Инвертиран индекс с BM25 и TF-IDF ранжиране.
|
Inverted индекс с BM25 и TF-IDF ранжиране за текстово търсене.
|
||||||
|
|
||||||
## Употреба
|
## Употреба
|
||||||
|
|
||||||
@@ -8,28 +8,80 @@
|
|||||||
import barabadb/fts/engine
|
import barabadb/fts/engine
|
||||||
|
|
||||||
var idx = newInvertedIndex()
|
var idx = newInvertedIndex()
|
||||||
idx.addDocument(1, "Nim е бърз език за програмиране")
|
idx.addDocument(1, "Nim is a fast programming language")
|
||||||
idx.addDocument(2, "Python е популярен за data science")
|
idx.addDocument(2, "Python is popular for data science")
|
||||||
|
|
||||||
let results = idx.search("език програмиране")
|
# BM25 търсене
|
||||||
let tfidf = idx.searchTfidf("език")
|
let results = idx.search("programming language")
|
||||||
let fuzzy = idx.fuzzySearch("програмиране", maxDistance = 2)
|
|
||||||
|
# TF-IDF търсене
|
||||||
|
let tfidf = idx.searchTfidf("programming language")
|
||||||
|
|
||||||
|
# Fuzzy търсене (толеранс на печатни грешки)
|
||||||
|
let fuzzy = idx.fuzzySearch("programing", maxDistance = 2)
|
||||||
|
|
||||||
|
# Wildcard търсене
|
||||||
|
let wild = idx.regexSearch("prog*")
|
||||||
```
|
```
|
||||||
|
|
||||||
## Методи за Ранжиране
|
## Методи за Ранжиране
|
||||||
|
|
||||||
### BM25
|
### BM25
|
||||||
|
|
||||||
Най-добрият алгоритъм за съвпадение
|
Best matching алгоритъм за ранжиране:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
let bm25 = idx.searchBM25("query terms")
|
||||||
|
```
|
||||||
|
|
||||||
### TF-IDF
|
### TF-IDF
|
||||||
|
|
||||||
Term Frequency-Inverse Document Frequency
|
Term Frequency-Inverse Document Frequency:
|
||||||
|
|
||||||
## Търсене
|
```nim
|
||||||
|
let tfidf = idx.searchTfidf("query terms")
|
||||||
|
```
|
||||||
|
|
||||||
| Тип | Описание |
|
## Функции за Търсене
|
||||||
|-----|----------|
|
|
||||||
| Fuzzy | Толерантност към правописни грешки |
|
| Функция | Описание |
|
||||||
| Wildcard | Префикс, суфикс, и инфикс заместващи символи |
|
|---------|----------|
|
||||||
| Regex | Регулярни изрази |
|
| Fuzzy търсене | Levenshtein distance толеранс |
|
||||||
|
| Wildcard | Префиксни, суфиксни и инфиксни wildcards |
|
||||||
|
| Regex | Регулярни изрази |
|
||||||
|
| Фразово търсене | Точно съвпадение на фраза |
|
||||||
|
| Булево | AND, OR, NOT оператори |
|
||||||
|
|
||||||
|
## SQL Интерфейс
|
||||||
|
|
||||||
|
Пълнотекстовото търсене е достъпно и директно в BaraQL:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Създаване на таблица с текстова колона
|
||||||
|
CREATE TABLE articles (id INT PRIMARY KEY, title TEXT, body TEXT);
|
||||||
|
|
||||||
|
-- Създаване на FTS индекс
|
||||||
|
CREATE INDEX idx_fts ON articles(body) USING FTS;
|
||||||
|
|
||||||
|
-- Търсене с оператора @@ (BM25 ранжиране)
|
||||||
|
SELECT * FROM articles WHERE body @@ 'machine learning';
|
||||||
|
|
||||||
|
-- Търсене с множество термини
|
||||||
|
SELECT * FROM articles WHERE body @@ 'quick brown fox';
|
||||||
|
```
|
||||||
|
|
||||||
|
## Многоезична Поддръжка
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/fts/multilang
|
||||||
|
|
||||||
|
# Поддържани езици: EN, BG, DE, FR, RU
|
||||||
|
var tokenizer = newTokenizer("bg") # Български
|
||||||
|
let tokens = tokenizer.tokenize("Търсене в пълен текст")
|
||||||
|
```
|
||||||
|
|
||||||
|
Функции за всеки език:
|
||||||
|
- Токенизация
|
||||||
|
- Stop думи
|
||||||
|
- Стеминг
|
||||||
|
- Детекция на език
|
||||||
|
|||||||
+118
-21
@@ -15,6 +15,7 @@
|
|||||||
| macOS | x86_64 | ✅ Пълна поддръжка |
|
| macOS | x86_64 | ✅ Пълна поддръжка |
|
||||||
| macOS | ARM64 (Apple Silicon) | ✅ Пълна поддръжка |
|
| macOS | ARM64 (Apple Silicon) | ✅ Пълна поддръжка |
|
||||||
| Windows | x86_64 | ✅ Поддръжка |
|
| Windows | x86_64 | ✅ Поддръжка |
|
||||||
|
| FreeBSD | x86_64 | 🟡 Тествано от общността |
|
||||||
|
|
||||||
## Инсталиране на Nim
|
## Инсталиране на Nim
|
||||||
|
|
||||||
@@ -48,10 +49,13 @@ sudo port install nim
|
|||||||
### Windows
|
### Windows
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
# Winget
|
# С choosenim
|
||||||
|
curl.exe -A "MSYS2_$(uname -m)" -L https://nim-lang.org/choosenim/init.ps1 | powershell -
|
||||||
|
|
||||||
|
# С winget
|
||||||
winget install nim
|
winget install nim
|
||||||
|
|
||||||
# Scoop
|
# С scoop
|
||||||
scoop install nim
|
scoop install nim
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -79,16 +83,22 @@ sudo pacman -S openssl
|
|||||||
|
|
||||||
### macOS
|
### macOS
|
||||||
|
|
||||||
|
OpenSSL е включен в системата. Ако е необходимо:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
brew install openssl
|
brew install openssl
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
OpenSSL е включен в Nim Windows дистрибуцията. За ръчни компилации, изтеглете от [slproweb.com](https://slproweb.com/products/Win32OpenSSL.html).
|
||||||
|
|
||||||
## Компилиране на BaraDB
|
## Компилиране на BaraDB
|
||||||
|
|
||||||
### Клониране на Репозиторито
|
### Клониране на Репозиторито
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -103,26 +113,29 @@ nimble install -d -y
|
|||||||
#### Debug Компилация
|
#### Debug Компилация
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
nim c -d:ssl -o:build/baradadb src/baradadb.nim
|
nim c -d:ssl --threads:on -o:build/baradadb src/baradadb.nim
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Release Компилация (Препоръчителна)
|
#### Release Компилация (Препоръчителна)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
nim c -d:ssl -d:release --opt:speed -o:build/baradadb src/baradadb.nim
|
nim c -d:ssl --threads:on -d:release --opt:speed -o:build/baradadb src/baradadb.nim
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Използване на Nimble
|
#### Използване на Nimble Tasks
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Debug компилация
|
||||||
nimble build_debug
|
nimble build_debug
|
||||||
|
|
||||||
|
# Release компилация
|
||||||
nimble build_release
|
nimble build_release
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Минимален Размер
|
#### Минимален Размер
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
nim c -d:ssl -d:release --opt:size -o:build/baradadb src/baradadb.nim
|
nim c -d:ssl --threads:on -d:release --opt:size -o:build/baradadb src/baradadb.nim
|
||||||
strip build/baradadb
|
strip build/baradadb
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -130,17 +143,31 @@ strip build/baradadb
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/baradadb --version
|
./build/baradadb --version
|
||||||
# Очакван резултат: BaraDB v0.1.0 — Multimodal Database Engine
|
# Очакван резултат: BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
```
|
```
|
||||||
|
|
||||||
## Стартиране на Тестове
|
## Стартиране на Тестове
|
||||||
|
|
||||||
```bash
|
### Всички Тестове
|
||||||
# Всички тестове (262 теста, 56 сюита)
|
|
||||||
nim c -d:ssl -r tests/test_all.nim
|
|
||||||
|
|
||||||
# Бенчмаркове
|
```bash
|
||||||
nim c -d:ssl -d:release -r benchmarks/bench_all.nim
|
nim c --path:src -d:ssl --threads:on -r tests/test_all.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Специфични Тестови Сюити
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Storage тестове
|
||||||
|
nim c --path:src -d:ssl --threads:on -r tests/test_all.nim
|
||||||
|
|
||||||
|
# Stress тестове
|
||||||
|
nim c --path:src -d:ssl --threads:on -d:release -r tests/stress_test.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Бенчмаркове
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c --path:src -d:ssl --threads:on -d:release -r benchmarks/bench_all.nim
|
||||||
```
|
```
|
||||||
|
|
||||||
## Опции за Инсталация
|
## Опции за Инсталация
|
||||||
@@ -148,16 +175,46 @@ nim c -d:ssl -d:release -r benchmarks/bench_all.nim
|
|||||||
### Системна Инсталация
|
### Системна Инсталация
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Компилиране на release binary
|
||||||
nimble build_release
|
nimble build_release
|
||||||
|
|
||||||
|
# Инсталиране в /usr/local/bin
|
||||||
sudo cp build/baradadb /usr/local/bin/
|
sudo cp build/baradadb /usr/local/bin/
|
||||||
sudo chmod +x /usr/local/bin/baradadb
|
sudo chmod +x /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# Създаване на директория за данни
|
||||||
sudo mkdir -p /var/lib/baradb
|
sudo mkdir -p /var/lib/baradb
|
||||||
|
sudo chmod 755 /var/lib/baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Предварително Компилиран Binary
|
||||||
|
|
||||||
|
Изтеглете най-новата версия за вашата платформа:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Linux x86_64
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-linux-amd64
|
||||||
|
chmod +x baradadb-linux-amd64
|
||||||
|
mv baradadb-linux-amd64 /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# Linux ARM64
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-linux-arm64
|
||||||
|
chmod +x baradadb-linux-arm64
|
||||||
|
mv baradadb-linux-arm64 /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# macOS
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-darwin-amd64
|
||||||
|
chmod +x baradadb-darwin-amd64
|
||||||
|
mv baradadb-darwin-amd64 /usr/local/bin/baradadb
|
||||||
```
|
```
|
||||||
|
|
||||||
### Docker
|
### Docker
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Изтегляне на официален образ
|
||||||
docker pull barabadb/barabadb:latest
|
docker pull barabadb/barabadb:latest
|
||||||
|
|
||||||
|
# Стартиране
|
||||||
docker run -d \
|
docker run -d \
|
||||||
-p 9472:9472 \
|
-p 9472:9472 \
|
||||||
-p 9470:9470 \
|
-p 9470:9470 \
|
||||||
@@ -172,12 +229,16 @@ docker run -d \
|
|||||||
docker-compose up -d
|
docker-compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
### Вградено Използване
|
### Вградено Използване (Nim Проекти)
|
||||||
|
|
||||||
|
Добавете към вашия `.nimble` файл:
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
requires "barabadb >= 0.1.0"
|
requires "barabadb >= 1.1.0"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Използване в кода:
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
import barabadb/storage/lsm
|
import barabadb/storage/lsm
|
||||||
|
|
||||||
@@ -193,16 +254,52 @@ db.close()
|
|||||||
# Стартиране на сървъра
|
# Стартиране на сървъра
|
||||||
./build/baradadb
|
./build/baradadb
|
||||||
|
|
||||||
# Тестване на HTTP API
|
# Очакван изход:
|
||||||
|
# BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
# BaraDB TCP listening on 127.0.0.1:9472
|
||||||
|
|
||||||
|
# Тестване с HTTP API
|
||||||
curl http://localhost:9470/health
|
curl http://localhost:9470/health
|
||||||
|
|
||||||
# Интерактивна конзола
|
# Интерактивна обвивка
|
||||||
./build/baradadb --shell
|
./build/baradadb --shell
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Отстраняване на Проблеми с Инсталацията
|
||||||
|
|
||||||
|
### "cannot open file: hunos"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nimble install -d -y
|
||||||
|
```
|
||||||
|
|
||||||
|
### "BaraDB requires SSL support"
|
||||||
|
|
||||||
|
Винаги компилирайте с `-d:ssl`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl --threads:on -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Бавна компилация
|
||||||
|
|
||||||
|
Използвайте паралелна компилация:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl --threads:on -d:release --parallelBuild:4 -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Голям размер на binary-то
|
||||||
|
|
||||||
|
Използвайте оптимизация на размера:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl --threads:on -d:release --opt:size --passL:-s -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
## Следващи Стъпки
|
## Следващи Стъпки
|
||||||
|
|
||||||
- [Бързо Стартиране](bg/quickstart.md)
|
- [Бърз Старт](quickstart.md)
|
||||||
- [Конфигурация](en/configuration.md)
|
- [Конфигурационна Референция](configuration.md)
|
||||||
- [Архитектура](bg/architecture.md)
|
- [Преглед на Архитектурата](architecture.md)
|
||||||
- [BaraQL Заявки](bg/baraql.md)
|
- [BaraQL Език за Заявки](baraql.md)
|
||||||
|
|||||||
+48
-9
@@ -1,6 +1,29 @@
|
|||||||
# LSM-Tree Съхранение
|
# LSM-Tree Storage Engine
|
||||||
|
|
||||||
Основният двигател за съхранение използващ Log-Structured Merge-Tree архитектура.
|
Основният storage engine в BaraDB, използващ Log-Structured Merge-Tree архитектура.
|
||||||
|
|
||||||
|
## Архитектура
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ Записи │
|
||||||
|
│ (добавяне към WAL + MemTable) │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ MemTable │
|
||||||
|
│ (в-памет сортиран буфер) │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
(при запълване, flush към SSTable)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────┐
|
||||||
|
│ SSTable │
|
||||||
|
│ (сортирана string table на диска) │
|
||||||
|
└─────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
## Употреба
|
## Употреба
|
||||||
|
|
||||||
@@ -9,17 +32,33 @@ import barabadb/storage/lsm
|
|||||||
|
|
||||||
var db = newLSMTree("./data")
|
var db = newLSMTree("./data")
|
||||||
|
|
||||||
|
# Запис
|
||||||
db.put("key1", cast[seq[byte]]("value1"))
|
db.put("key1", cast[seq[byte]]("value1"))
|
||||||
|
|
||||||
|
# Четене
|
||||||
let (found, value) = db.get("key1")
|
let (found, value) = db.get("key1")
|
||||||
|
|
||||||
|
# Изтриване
|
||||||
|
db.delete("key1")
|
||||||
|
|
||||||
db.close()
|
db.close()
|
||||||
```
|
```
|
||||||
|
|
||||||
## Компоненти
|
## Възможности
|
||||||
|
|
||||||
- **MemTable**: Сортиран буфер в паметта
|
- **Write-оптимизиран**: Append-only лог структура
|
||||||
- **WAL**: Write-ahead log за трайност
|
- **Устойчивост**: Write-ahead log (WAL) осигурява crash recovery
|
||||||
- **SSTable**: Сортирани таблици на диска
|
- **Bloom Филтър**: Бързи негативни проверки
|
||||||
- **Bloom Filter**: Бързи негативни проверки
|
- **Compaction**: Size-tiered стратегия слива SSTables
|
||||||
- **Compaction**: Сливане на SSTables
|
- **Page Cache**: LRU кеш за често достъпвани страници
|
||||||
- **Page Cache**: LRU кеш
|
|
||||||
|
## Конфигурация
|
||||||
|
|
||||||
|
```nim
|
||||||
|
var db = newLSMTree(
|
||||||
|
path = "./data",
|
||||||
|
memTableSize = 64 * 1024 * 1024, # 64MB
|
||||||
|
walEnabled = true,
|
||||||
|
bloomFpRate = 0.01
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|||||||
@@ -0,0 +1,202 @@
|
|||||||
|
# Мониторинг и Наблюдаемост
|
||||||
|
|
||||||
|
## Health Checks
|
||||||
|
|
||||||
|
### HTTP Health Endpoint
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/health
|
||||||
|
```
|
||||||
|
|
||||||
|
Отговор:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "healthy",
|
||||||
|
"version": "1.1.0",
|
||||||
|
"uptime_seconds": 86400,
|
||||||
|
"checks": {
|
||||||
|
"storage": "ok",
|
||||||
|
"memory": "ok",
|
||||||
|
"connections": "ok"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Readiness Probe
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/ready
|
||||||
|
```
|
||||||
|
|
||||||
|
Връща `200 OK` когато сървърът е готов да приема трафик, `503` по време на стартиране.
|
||||||
|
|
||||||
|
## Метрики
|
||||||
|
|
||||||
|
### Prometheus-Съвместими Метрики
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
Примерен изход:
|
||||||
|
|
||||||
|
```
|
||||||
|
# HELP baradb_queries_total Общ брой изпълнени заявки
|
||||||
|
# TYPE baradb_queries_total counter
|
||||||
|
baradb_queries_total 152340
|
||||||
|
|
||||||
|
# HELP baradb_queries_duration_seconds Хистограма на времетраене на заявки
|
||||||
|
# TYPE baradb_queries_duration_seconds histogram
|
||||||
|
baradb_queries_duration_seconds_bucket{le="0.001"} 45000
|
||||||
|
baradb_queries_duration_seconds_bucket{le="0.01"} 120000
|
||||||
|
baradb_queries_duration_seconds_bucket{le="0.1"} 148000
|
||||||
|
|
||||||
|
# HELP baradb_storage_lsm_size_bytes Общ размер на LSM-Tree
|
||||||
|
# TYPE baradb_storage_lsm_size_bytes gauge
|
||||||
|
baradb_storage_lsm_size_bytes 2147483648
|
||||||
|
|
||||||
|
# HELP baradb_cache_hit_rate Page cache hit rate
|
||||||
|
# TYPE baradb_cache_hit_rate gauge
|
||||||
|
baradb_cache_hit_rate 0.94
|
||||||
|
|
||||||
|
# HELP baradb_active_connections Активни клиентски връзки
|
||||||
|
# TYPE baradb_active_connections gauge
|
||||||
|
baradb_active_connections 42
|
||||||
|
```
|
||||||
|
|
||||||
|
### JSON Метрики
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/metrics?format=json
|
||||||
|
```
|
||||||
|
|
||||||
|
## Логване
|
||||||
|
|
||||||
|
### Нива на Логване
|
||||||
|
|
||||||
|
| Ниво | Описание |
|
||||||
|
|------|----------|
|
||||||
|
| `debug` | Детайлни вътрешни операции |
|
||||||
|
| `info` | Нормални операции |
|
||||||
|
| `warn` | Възстановими проблеми |
|
||||||
|
| `error` | Грешки, изискващи внимание |
|
||||||
|
|
||||||
|
### Структурирани JSON Логове
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_LOG_LEVEL=info \
|
||||||
|
BARADB_LOG_FORMAT=json \
|
||||||
|
BARADB_LOG_FILE=/var/log/baradb/baradb.log \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
Примерен лог запис:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"timestamp": "2025-01-15T10:30:00.123Z",
|
||||||
|
"level": "info",
|
||||||
|
"component": "server",
|
||||||
|
"message": "Query executed",
|
||||||
|
"query": "SELECT * FROM users",
|
||||||
|
"duration_ms": 12,
|
||||||
|
"client_ip": "10.0.0.15"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Текстов Формат
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_LOG_FORMAT=text ./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Правила за Алармиране
|
||||||
|
|
||||||
|
### Prometheus AlertManager
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
groups:
|
||||||
|
- name: baradb
|
||||||
|
rules:
|
||||||
|
- alert: BaraDBHighErrorRate
|
||||||
|
expr: rate(baradb_errors_total[5m]) > 0.1
|
||||||
|
for: 5m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "Висок процент грешки в BaraDB"
|
||||||
|
|
||||||
|
- alert: BaraDBLowCacheHitRate
|
||||||
|
expr: baradb_cache_hit_rate < 0.8
|
||||||
|
for: 10m
|
||||||
|
labels:
|
||||||
|
severity: warning
|
||||||
|
annotations:
|
||||||
|
summary: "Cache hit rate под 80%"
|
||||||
|
|
||||||
|
- alert: BaraDBHighConnections
|
||||||
|
expr: baradb_active_connections > 800
|
||||||
|
for: 5m
|
||||||
|
labels:
|
||||||
|
severity: warning
|
||||||
|
annotations:
|
||||||
|
summary: "Голям брой връзки към BaraDB"
|
||||||
|
|
||||||
|
- alert: BaraDBDown
|
||||||
|
expr: up{job="baradb"} == 0
|
||||||
|
for: 1m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "BaraDB инстанцията не работи"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Разпределен Мониторинг
|
||||||
|
|
||||||
|
### Клъстерни Метрики
|
||||||
|
|
||||||
|
За Raft клъстери, мониторирайте:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://node1:9470/metrics/cluster
|
||||||
|
```
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"cluster_id": "baradb-cluster-1",
|
||||||
|
"nodes": [
|
||||||
|
{"id": "node1", "role": "leader", "health": "healthy"},
|
||||||
|
{"id": "node2", "role": "follower", "health": "healthy"},
|
||||||
|
{"id": "node3", "role": "follower", "health": "healthy"}
|
||||||
|
],
|
||||||
|
"raft_log_index": 15420,
|
||||||
|
"raft_commit_index": 15420,
|
||||||
|
"shards": 4,
|
||||||
|
"replication_lag_ms": 5
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Профилиране на Производителност
|
||||||
|
|
||||||
|
### Вграден CPU Profiler
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/debug/pprof/cpu?seconds=30 > cpu.prof
|
||||||
|
```
|
||||||
|
|
||||||
|
### Memory Profiler
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/debug/pprof/heap > heap.prof
|
||||||
|
```
|
||||||
|
|
||||||
|
## Отстраняване на Проблеми с Метрики
|
||||||
|
|
||||||
|
| Симптом | Метрика | Действие |
|
||||||
|
|---------|--------|----------|
|
||||||
|
| Бавни заявки | `baradb_queries_duration_seconds` | Проверете cache hit rate, добавете индекси |
|
||||||
|
| Висока памет | `process_resident_memory_bytes` | Намалете memtable/cache размери |
|
||||||
|
| Растящо съхранение | `baradb_storage_lsm_size_bytes` | Пуснете ръчен compaction |
|
||||||
|
| Грешки при връзка | `baradb_active_connections` | Увеличете connection pool или добавете възли |
|
||||||
|
| Репликационно закъснение | `baradb_replication_lag_ms` | Проверете мрежата, увеличете ресурсите |
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
# Ръководство за Производителност
|
||||||
|
|
||||||
|
## Методология на Бенчмарковете
|
||||||
|
|
||||||
|
Всички бенчмаркове са проведени с:
|
||||||
|
- **Компилатор**: Nim 2.2.0 с `-d:release --opt:speed`
|
||||||
|
- **CPU**: AMD Ryzen 9 5900X (12 ядра / 24 нишки)
|
||||||
|
- **Памет**: 64 GB DDR4-3600
|
||||||
|
- **Диск**: Samsung 980 Pro NVMe SSD
|
||||||
|
- **ОС**: Ubuntu 24.04 LTS
|
||||||
|
|
||||||
|
Пускане на пълния бенчмарк сюит:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release -r benchmarks/bench_all.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
## Storage Engine Бенчмаркове
|
||||||
|
|
||||||
|
### LSM-Tree Key-Value
|
||||||
|
|
||||||
|
| Метрика | Стойност |
|
||||||
|
|---------|----------|
|
||||||
|
| Пропускателна способност при запис | ~580,000 ops/s |
|
||||||
|
| Пропускателна способност при четене | ~720,000 ops/s |
|
||||||
|
| Средна латентност при запис | 1.7 µs |
|
||||||
|
| Средна латентност при четене | 1.4 µs |
|
||||||
|
|
||||||
|
LSM-Tree използва 64MB MemTable, WAL fsync при всеки запис и size-tiered compaction с 6 нива.
|
||||||
|
|
||||||
|
### B-Tree Индекс
|
||||||
|
|
||||||
|
| Метрика | Стойност |
|
||||||
|
|---------|----------|
|
||||||
|
| Пропускателна способност при вмъкване | ~1,200,000 ops/s |
|
||||||
|
| Пропускателна способност при точково търсене | ~1,500,000 ops/s |
|
||||||
|
| Range scan (1000 ключа) | ~0.3 ms |
|
||||||
|
|
||||||
|
## Vector Engine Бенчмаркове
|
||||||
|
|
||||||
|
### HNSW Индекс
|
||||||
|
|
||||||
|
| Метрика | Стойност |
|
||||||
|
|---------|----------|
|
||||||
|
| Вмъкване (dim=128) | ~45,000 вектора/s |
|
||||||
|
| Търсене топ-10 (dim=128, n=10K) | ~2 ms |
|
||||||
|
| Търсене топ-10 (dim=128, n=100K) | ~8 ms |
|
||||||
|
|
||||||
|
### SIMD Функции за Разстояние
|
||||||
|
|
||||||
|
| Операция | dim=128 | dim=768 | dim=1536 |
|
||||||
|
|----------|---------|---------|----------|
|
||||||
|
| Косинусово разстояние | 4.2M/s | 850K/s | 420K/s |
|
||||||
|
| L2 (Евклидово) | 4.5M/s | 920K/s | 450K/s |
|
||||||
|
| Скаларно произведение | 4.8M/s | 980K/s | 480K/s |
|
||||||
|
|
||||||
|
## Full-Text Search Бенчмаркове
|
||||||
|
|
||||||
|
| Метрика | Стойност |
|
||||||
|
|---------|----------|
|
||||||
|
| Индексиране | ~320,000 документа/s |
|
||||||
|
| BM25 търсене | ~28,000 заявки/s |
|
||||||
|
| Fuzzy търсене (distance=2) | ~850 заявки/s |
|
||||||
|
|
||||||
|
## Graph Engine Бенчмаркове
|
||||||
|
|
||||||
|
| Операция | Пропускателна способност |
|
||||||
|
|----------|--------------------------|
|
||||||
|
| Добавяне на възел | ~2.5M ops/s |
|
||||||
|
| Добавяне на ребро | ~1.8M ops/s |
|
||||||
|
| BFS (1K възела, 5K ребра) | ~12K обхождания/s |
|
||||||
|
| Dijkstra най-кратък път | ~120 µs |
|
||||||
|
| PageRank (10 итерации) | ~450 графа/s |
|
||||||
|
|
||||||
|
## Протоколни Бенчмаркове
|
||||||
|
|
||||||
|
| Протокол | Връзки | Заявки/s | Латентност p99 |
|
||||||
|
|----------|--------|----------|----------------|
|
||||||
|
| Бинарен (localhost) | 1 | 45,000 | 0.4 ms |
|
||||||
|
| Бинарен (localhost) | 100 | 380,000 | 1.2 ms |
|
||||||
|
| HTTP/REST | 1 | 12,000 | 2.1 ms |
|
||||||
|
| HTTP/REST | 100 | 95,000 | 5.8 ms |
|
||||||
|
| WebSocket | 1 | 18,000 | 1.8 ms |
|
||||||
|
|
||||||
|
## Вертикален Мащабинг
|
||||||
|
|
||||||
|
| Ядра | LSM Запис | LSM Четене | Векторно Търсене |
|
||||||
|
|------|-----------|------------|-------------------|
|
||||||
|
| 1 | 580K | 720K | 2.0 ms |
|
||||||
|
| 4 | 1.9M | 2.6M | 1.1 ms |
|
||||||
|
| 8 | 3.4M | 4.8M | 0.7 ms |
|
||||||
|
| 16 | 5.8M | 7.2M | 0.5 ms |
|
||||||
|
|
||||||
|
## Ръководство за Настройка
|
||||||
|
|
||||||
|
### За Write-Heavy Натоварвания
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export BARADB_MEMTABLE_SIZE_MB=256
|
||||||
|
export BARADB_WAL_SYNC_INTERVAL_MS=10
|
||||||
|
export BARADB_COMPACTION_INTERVAL_MS=30000
|
||||||
|
```
|
||||||
|
|
||||||
|
### За Read-Heavy Натоварвания
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export BARADB_CACHE_SIZE_MB=1024
|
||||||
|
export BARADB_BLOOM_BITS_PER_KEY=10
|
||||||
|
export BARADB_COMPACTION_INTERVAL_MS=120000
|
||||||
|
```
|
||||||
|
|
||||||
|
### За Векторно Търсене
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export BARADB_VECTOR_EF_CONSTRUCTION=200
|
||||||
|
export BARADB_VECTOR_EF_SEARCH=128
|
||||||
|
export BARADB_VECTOR_M=32
|
||||||
|
```
|
||||||
|
|
||||||
|
### За Графова Аналитика
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export BARADB_GRAPH_PAGE_RANK_ITERATIONS=20
|
||||||
|
export BARADB_GRAPH_LOUVAIN_RESOLUTION=1.0
|
||||||
|
```
|
||||||
@@ -0,0 +1,373 @@
|
|||||||
|
# Протоколна Референция
|
||||||
|
|
||||||
|
BaraDB поддържа множество протоколи за клиентска комуникация:
|
||||||
|
- **Бинарен Wire Протокол** — високопроизводителен, ниска латентност
|
||||||
|
- **HTTP/REST API** — езиково-независим, лесен за дебъгване
|
||||||
|
- **WebSocket** — стрийминг и pub/sub
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Бинарен Wire Протокол
|
||||||
|
|
||||||
|
Бинарният протокол използва big-endian кодиране за всички многобайтови стойности.
|
||||||
|
|
||||||
|
### Жизнен Цикъл на Връзката
|
||||||
|
|
||||||
|
```
|
||||||
|
Клиент Сървър
|
||||||
|
| |
|
||||||
|
|─── TCP свързване ───────────>|
|
||||||
|
|<── TLS ръкостискане (опц.) ──|
|
||||||
|
|─── Auth съобщение ──────────>|
|
||||||
|
|<── Auth_OK / Грешка ─────────|
|
||||||
|
|─── Query съобщение ─────────>|
|
||||||
|
|<── Data / Complete / Error ──|
|
||||||
|
|─── Close съобщение ─────────>|
|
||||||
|
|<── TCP затваряне ────────────|
|
||||||
|
```
|
||||||
|
|
||||||
|
### Формат на Съобщенията
|
||||||
|
|
||||||
|
Всяко съобщение започва с 12-байтов хедър:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────┬─────────────┬─────────────┬─────────────────────┐
|
||||||
|
│ Kind │ Length │ RequestId │ Payload │
|
||||||
|
│ (4 bytes) │ (4 bytes) │ (4 bytes) │ │
|
||||||
|
│ uint32 BE │ uint32 BE │ uint32 BE │ (Length bytes) │
|
||||||
|
└─────────────┴─────────────┴─────────────┴─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Типове Съобщения
|
||||||
|
|
||||||
|
| Тип | ID | Посока | Описание |
|
||||||
|
|------|----|--------|----------|
|
||||||
|
| Query | 0x01 | К→С | Изпълни заявка |
|
||||||
|
| QueryParams | 0x02 | К→С | Параметризирана заявка |
|
||||||
|
| Auth | 0x07 | К→С | Заявка за автентикация |
|
||||||
|
| Ping | 0x09 | К→С | Keepalive ping |
|
||||||
|
| Close | 0x0A | К→С | Затваряне на връзка |
|
||||||
|
| Data | 0x81 | С→К | Резултат от заявка |
|
||||||
|
| Complete | 0x82 | С→К | Заявката е завършена |
|
||||||
|
| Auth_OK | 0x83 | С→К | Успешна автентикация |
|
||||||
|
| Pong | 0x84 | С→К | Keepalive отговор |
|
||||||
|
| Error | 0x06 | С→К | Грешка |
|
||||||
|
|
||||||
|
### Query Съобщение Payload
|
||||||
|
|
||||||
|
```
|
||||||
|
┌───────────────────┬────────────────────────────┐
|
||||||
|
│ Query String │ │
|
||||||
|
│ (променлива) │ │
|
||||||
|
│ UTF-8 │ │
|
||||||
|
└───────────────────┴────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data Съобщение Payload
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────┬─────────────────────────────────────────────┐
|
||||||
|
│ Брой Колони │ Дефиниции на Колони + Данни от Редове │
|
||||||
|
│ (4 bytes) │ │
|
||||||
|
│ uint32 BE │ │
|
||||||
|
└──────────────┴─────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Типове Полета
|
||||||
|
|
||||||
|
| Тип | ID | Размер | Описание |
|
||||||
|
|------|----|--------|----------|
|
||||||
|
| NULL | 0x00 | 0 | NULL стойност |
|
||||||
|
| BOOL | 0x01 | 1 | true/false |
|
||||||
|
| INT8 | 0x02 | 1 | Signed 8-bit integer |
|
||||||
|
| INT16 | 0x03 | 2 | Signed 16-bit integer |
|
||||||
|
| INT32 | 0x04 | 4 | Signed 32-bit integer |
|
||||||
|
| INT64 | 0x05 | 8 | Signed 64-bit integer |
|
||||||
|
| FLOAT32 | 0x06 | 4 | IEEE 754 единична точност (big-endian) |
|
||||||
|
| FLOAT64 | 0x07 | 8 | IEEE 754 двойна точност (big-endian) |
|
||||||
|
| STRING | 0x08 | променлив | UTF-8 низ (4-байтов префикс за дължина) |
|
||||||
|
| BYTES | 0x09 | променлив | Сурови байтове (4-байтов префикс за дължина) |
|
||||||
|
| ARRAY | 0x0A | променлив | Масив от стойности |
|
||||||
|
| OBJECT | 0x0B | променлив | Ключ-стойност обект |
|
||||||
|
| VECTOR | 0x0C | променлив | Float32 масив (4-байтов префикс, big-endian floats) |
|
||||||
|
|
||||||
|
### Error Съобщение Payload
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────┬──────────────┬────────────────────────────┐
|
||||||
|
│ Код Грешка │ Дълж. Съобщ. │ Съобщение за Грешка │
|
||||||
|
│ (4 bytes) │ (4 bytes) │ (Дълж. Съобщ. bytes) │
|
||||||
|
│ uint32 BE │ uint32 BE │ UTF-8 │
|
||||||
|
└──────────────┴──────────────┴────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## HTTP/REST API
|
||||||
|
|
||||||
|
Базов URL: `http://localhost:9470/api/v1`
|
||||||
|
|
||||||
|
### Endpoints
|
||||||
|
|
||||||
|
#### Health
|
||||||
|
|
||||||
|
```http
|
||||||
|
GET /health
|
||||||
|
```
|
||||||
|
|
||||||
|
Отговор:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "healthy",
|
||||||
|
"version": "1.1.0",
|
||||||
|
"uptime_seconds": 86400
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Ready
|
||||||
|
|
||||||
|
```http
|
||||||
|
GET /ready
|
||||||
|
```
|
||||||
|
|
||||||
|
Връща `200` когато е готов, `503` при стартиране.
|
||||||
|
|
||||||
|
#### Query
|
||||||
|
|
||||||
|
```http
|
||||||
|
POST /query
|
||||||
|
Content-Type: application/json
|
||||||
|
Authorization: Bearer <token>
|
||||||
|
|
||||||
|
{
|
||||||
|
"query": "SELECT name, age FROM users WHERE age > 18",
|
||||||
|
"params": [],
|
||||||
|
"format": "json"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Отговор:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"columns": ["name", "age"],
|
||||||
|
"rows": [
|
||||||
|
["Alice", 30],
|
||||||
|
["Bob", 25]
|
||||||
|
],
|
||||||
|
"row_count": 2,
|
||||||
|
"duration_ms": 12
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Batch
|
||||||
|
|
||||||
|
```http
|
||||||
|
POST /batch
|
||||||
|
Content-Type: application/json
|
||||||
|
|
||||||
|
{
|
||||||
|
"queries": [
|
||||||
|
"INSERT users { name := 'Alice', age := 30 }",
|
||||||
|
"INSERT users { name := 'Bob', age := 25 }"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Отговор:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"results": [
|
||||||
|
{"status": "ok", "affected_rows": 1},
|
||||||
|
{"status": "ok", "affected_rows": 1}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Schema
|
||||||
|
|
||||||
|
```http
|
||||||
|
GET /schema
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Metrics
|
||||||
|
|
||||||
|
```http
|
||||||
|
GET /metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
Prometheus-съвместими метрики. Виж [Ръководство за Мониторинг](monitoring.md).
|
||||||
|
|
||||||
|
#### Explain
|
||||||
|
|
||||||
|
```http
|
||||||
|
POST /explain
|
||||||
|
Content-Type: application/json
|
||||||
|
|
||||||
|
{
|
||||||
|
"query": "SELECT * FROM users WHERE age > 18"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Отговор:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"plan": "IndexScan",
|
||||||
|
"index": "idx_users_age",
|
||||||
|
"estimated_rows": 42,
|
||||||
|
"cost": 120
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Backup
|
||||||
|
|
||||||
|
```http
|
||||||
|
POST /backup
|
||||||
|
Content-Type: application/json
|
||||||
|
|
||||||
|
{
|
||||||
|
"destination": "/backup/snapshot.db"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Административни Операции
|
||||||
|
|
||||||
|
```http
|
||||||
|
POST /admin/compact
|
||||||
|
POST /admin/rebalance
|
||||||
|
POST /admin/check
|
||||||
|
```
|
||||||
|
|
||||||
|
### HTTP Статус Кодове
|
||||||
|
|
||||||
|
| Код | Значение |
|
||||||
|
|------|----------|
|
||||||
|
| 200 | Успех |
|
||||||
|
| 400 | Лоша заявка (синтактична грешка) |
|
||||||
|
| 401 | Неоторизиран (изисква се auth) |
|
||||||
|
| 403 | Забранен (недостатъчни права) |
|
||||||
|
| 404 | Не е намерен (таблица/тип не съществува) |
|
||||||
|
| 429 | Твърде много заявки (rate limited) |
|
||||||
|
| 500 | Вътрешна сървърна грешка |
|
||||||
|
| 503 | Услугата е недостъпна (стартира) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## WebSocket Протокол
|
||||||
|
|
||||||
|
URL: `ws://localhost:9471`
|
||||||
|
|
||||||
|
### Формат на Frame
|
||||||
|
|
||||||
|
WebSocket текстови frame-ове съдържат JSON съобщения:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": 1,
|
||||||
|
"type": "query",
|
||||||
|
"query": "SELECT * FROM users"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Типове Съобщения
|
||||||
|
|
||||||
|
| Тип | Посока | Описание |
|
||||||
|
|------|--------|----------|
|
||||||
|
| `query` | К→С | Изпълни заявка |
|
||||||
|
| `subscribe` | К→С | Абониране за промени |
|
||||||
|
| `unsubscribe` | К→С | Отписване |
|
||||||
|
| `ping` | К→С | Keepalive |
|
||||||
|
| `result` | С→К | Резултат от заявка |
|
||||||
|
| `notification` | С→К | Известие за промяна |
|
||||||
|
| `error` | С→К | Грешка |
|
||||||
|
| `pong` | С→К | Keepalive отговор |
|
||||||
|
|
||||||
|
### Pub/Sub Пример
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
const ws = new WebSocket('ws://localhost:9471');
|
||||||
|
|
||||||
|
ws.onopen = () => {
|
||||||
|
// Абониране за промени в таблица
|
||||||
|
ws.send(JSON.stringify({
|
||||||
|
id: 1,
|
||||||
|
type: 'subscribe',
|
||||||
|
table: 'users'
|
||||||
|
}));
|
||||||
|
};
|
||||||
|
|
||||||
|
ws.onmessage = (event) => {
|
||||||
|
const msg = JSON.parse(event.data);
|
||||||
|
if (msg.type === 'notification') {
|
||||||
|
console.log('Промяна:', msg.operation, msg.data);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### Стрийминг Заявки
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
ws.send(JSON.stringify({
|
||||||
|
id: 2,
|
||||||
|
type: 'query',
|
||||||
|
query: 'SELECT * FROM logs ORDER BY timestamp',
|
||||||
|
streaming: true
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Сървърът изпраща множество result frame-ове
|
||||||
|
// Последният frame има {"complete": true}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Nim API Примери
|
||||||
|
|
||||||
|
### Бинарен Протокол
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/wire
|
||||||
|
|
||||||
|
let msg = makeQueryMessage(1, "SELECT * FROM users")
|
||||||
|
let ready = makeReadyMessage(1)
|
||||||
|
let error = makeErrorMessage(1, 42, "Syntax error")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Connection Pool
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/pool
|
||||||
|
|
||||||
|
var pool = newConnectionPool(
|
||||||
|
minConnections = 5,
|
||||||
|
maxConnections = 100,
|
||||||
|
idleTimeout = 30000
|
||||||
|
)
|
||||||
|
let conn = pool.acquire()
|
||||||
|
# Използване на връзка...
|
||||||
|
pool.release(conn)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Автентикация
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/auth
|
||||||
|
|
||||||
|
var am = newAuthManager("secret-key")
|
||||||
|
let token = am.createToken(JWTClaims(sub: "user1", role: "admin"))
|
||||||
|
let result = am.validateCredentials(
|
||||||
|
AuthCredentials(authMethod: amToken, payload: token)
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rate Limiting
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/ratelimit
|
||||||
|
|
||||||
|
var rl = newRateLimiter(
|
||||||
|
rlaTokenBucket,
|
||||||
|
globalRate = 10000,
|
||||||
|
perClientRate = 1000
|
||||||
|
)
|
||||||
|
if rl.allowRequest("client-123"):
|
||||||
|
echo "Заявката е разрешена"
|
||||||
|
```
|
||||||
@@ -0,0 +1,210 @@
|
|||||||
|
# Ръководство за Сигурност
|
||||||
|
|
||||||
|
## TLS/SSL Криптиране
|
||||||
|
|
||||||
|
BaraDB поддържа TLS 1.3 за всички протоколи (бинарен, HTTP, WebSocket). Ако не е предоставен сертификат, сървърът автоматично генерира self-signed сертификат при стартиране за криптиране без конфигурация.
|
||||||
|
|
||||||
|
### Използване на Персонализирани Сертификати
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Предоставяне на съществуващи сертификати
|
||||||
|
BARADB_TLS_ENABLED=true \
|
||||||
|
BARADB_CERT_FILE=/etc/baradb/server.crt \
|
||||||
|
BARADB_KEY_FILE=/etc/baradb/server.key \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Генериране на Self-Signed Сертификати
|
||||||
|
|
||||||
|
```bash
|
||||||
|
openssl req -x509 -newkey rsa:4096 -keyout server.key -out server.crt \
|
||||||
|
-days 365 -nodes -subj "/CN=localhost"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Let's Encrypt (Продукция)
|
||||||
|
|
||||||
|
Използвайте certbot и насочете BaraDB към генерираните файлове:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo certbot certonly --standalone -d db.example.com
|
||||||
|
|
||||||
|
BARADB_CERT_FILE=/etc/letsencrypt/live/db.example.com/fullchain.pem \
|
||||||
|
BARADB_KEY_FILE=/etc/letsencrypt/live/db.example.com/privkey.pem \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### TLS от Страна на Клиента
|
||||||
|
|
||||||
|
```python
|
||||||
|
from baradb import Client
|
||||||
|
|
||||||
|
client = Client("localhost", 9472, tls=True, tls_verify=True)
|
||||||
|
client.connect()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Автентикация
|
||||||
|
|
||||||
|
### JWT-Базирана Автентикация
|
||||||
|
|
||||||
|
BaraDB използва JWT (JSON Web Tokens) с HMAC-SHA256 подписване.
|
||||||
|
|
||||||
|
#### Включване на Автентикация
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_AUTH_ENABLED=true \
|
||||||
|
BARADB_JWT_SECRET="$(openssl rand -hex 32)" \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Създаване на Токени
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/auth
|
||||||
|
|
||||||
|
var am = newAuthManager("your-secret-key")
|
||||||
|
let token = am.createToken(JWTClaims(
|
||||||
|
sub: "user1",
|
||||||
|
role: "admin",
|
||||||
|
exp: getTime() + 24.hours
|
||||||
|
))
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Контрол на Достъп на База Роли
|
||||||
|
|
||||||
|
| Роля | Права |
|
||||||
|
|------|-------|
|
||||||
|
| `admin` | Пълен достъп |
|
||||||
|
| `write` | Четене + запис |
|
||||||
|
| `read` | Само четене |
|
||||||
|
| `monitor` | Само метрики и health |
|
||||||
|
|
||||||
|
#### Използване на Токени
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -H "Authorization: Bearer $TOKEN" \
|
||||||
|
http://localhost:9470/api/query \
|
||||||
|
-d '{"query": "SELECT * FROM users"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
from baradb import Client
|
||||||
|
|
||||||
|
client = Client("localhost", 9472)
|
||||||
|
client.connect()
|
||||||
|
client.authenticate("eyJhbGciOiJIUzI1NiIs...")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Многофакторна Автентикация (MFA)
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/auth
|
||||||
|
|
||||||
|
var am = newAuthManager("secret-key")
|
||||||
|
# TOTP-базирана MFA
|
||||||
|
let mfaCode = am.generateTOTP("user1")
|
||||||
|
let valid = am.validateTOTP("user1", mfaCode)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rate Limiting
|
||||||
|
|
||||||
|
Token-bucket rate limiting предотвратява злоупотреби:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/protocol/ratelimit
|
||||||
|
|
||||||
|
var rl = newRateLimiter(
|
||||||
|
rlaTokenBucket,
|
||||||
|
globalRate = 10000, # 10K заявки/s глобално
|
||||||
|
perClientRate = 1000, # 1K заявки/s на IP/токен
|
||||||
|
burstSize = 100 # Разрешаване на 100 заявки burst
|
||||||
|
)
|
||||||
|
|
||||||
|
if not rl.allowRequest("client-ip"):
|
||||||
|
return error("Лимитът на заявки е надвишен")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Мрежова Сигурност
|
||||||
|
|
||||||
|
### Адрес за Свързване
|
||||||
|
|
||||||
|
По подразбиране BaraDB се свързва към `127.0.0.1` (само localhost). За продукция:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Свързване към всички интерфейси (зад защитна стена или reverse proxy)
|
||||||
|
BARADB_ADDRESS=0.0.0.0 ./build/baradadb
|
||||||
|
|
||||||
|
# Свързване към конкретен вътрешен интерфейс
|
||||||
|
BARADB_ADDRESS=10.0.0.5 ./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Правила на Защитната Стена
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Разрешаване само на сървъри на приложения
|
||||||
|
sudo ufw allow from 10.0.0.0/8 to any port 9472
|
||||||
|
sudo ufw allow from 10.0.0.0/8 to any port 9470
|
||||||
|
|
||||||
|
# Блокиране на външен достъп до портове за управление
|
||||||
|
sudo ufw deny 9471 # WebSocket (само вътрешна употреба)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Криптиране на Данни в Покой
|
||||||
|
|
||||||
|
### Криптиране на Ниво ОС
|
||||||
|
|
||||||
|
Използвайте LUKS за пълно дисково криптиране:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cryptsetup luksFormat /dev/nvme0n1p2
|
||||||
|
cryptsetup open /dev/nvme0n1p2 baradb-crypt
|
||||||
|
mkfs.ext4 /dev/mapper/baradb-crypt
|
||||||
|
mount /dev/mapper/baradb-crypt /var/lib/baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Криптиране на Ниво Приложение
|
||||||
|
|
||||||
|
BaraDB поддържа прозрачно криптиране на SSTable файлове:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_STORAGE_ENCRYPTION_KEY="$(openssl rand -hex 32)" \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Одитно Логване
|
||||||
|
|
||||||
|
Всички заявки и административни действия се логват:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"timestamp": "2025-01-15T10:30:00Z",
|
||||||
|
"level": "info",
|
||||||
|
"event": "query_executed",
|
||||||
|
"client_ip": "10.0.0.15",
|
||||||
|
"user": "app_user",
|
||||||
|
"query": "SELECT * FROM users WHERE id = ?",
|
||||||
|
"duration_ms": 12,
|
||||||
|
"rows_returned": 1
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Включване на одитно логване:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_LOG_LEVEL=info \
|
||||||
|
BARADB_LOG_FORMAT=json \
|
||||||
|
BARADB_LOG_FILE=/var/log/baradb/audit.log \
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Чеклист за Сигурност
|
||||||
|
|
||||||
|
- [ ] Сменете JWT secret по подразбиране
|
||||||
|
- [ ] Включете TLS с валидни сертификати
|
||||||
|
- [ ] Свържете се към конкретни интерфейси
|
||||||
|
- [ ] Включете автентикация в продукция
|
||||||
|
- [ ] Конфигурирайте rate limiting
|
||||||
|
- [ ] Включете одитно логване
|
||||||
|
- [ ] Криптирайте данните в покой (LUKS или на ниво приложение)
|
||||||
|
- [ ] Стартирайте BaraDB като non-root потребител
|
||||||
|
- [ ] Поддържайте рестриктивни правила на защитната стена
|
||||||
|
- [ ] Ротирайте JWT secret-и редовно
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# Storage Engines
|
||||||
|
|
||||||
|
BaraDB предоставя множество storage двигатели, оптимизирани за различни модели на достъп.
|
||||||
|
|
||||||
|
## LSM-Tree (Key-Value)
|
||||||
|
|
||||||
|
Основният storage engine с write-оптимизирана append-only лог структура.
|
||||||
|
|
||||||
|
### Употреба
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/lsm
|
||||||
|
|
||||||
|
var db = newLSMTree("./data")
|
||||||
|
db.put("key1", cast[seq[byte]]("value1"))
|
||||||
|
let (found, value) = db.get("key1")
|
||||||
|
db.close()
|
||||||
|
```
|
||||||
|
|
||||||
|
### Компоненти
|
||||||
|
|
||||||
|
- **MemTable**: В-памет сортиран буфер
|
||||||
|
- **WAL**: Write-ahead log за устойчивост
|
||||||
|
- **SSTable**: Сортирани string таблици на диска
|
||||||
|
- **Bloom Filter**: Вероятностна проверка за принадлежност
|
||||||
|
- **Compaction**: Size-tiered стратегия с управление на нива
|
||||||
|
- **Page Cache**: LRU кеш с проследяване на hit rate
|
||||||
|
|
||||||
|
## B-Tree Индекс
|
||||||
|
|
||||||
|
Подреден индекс за range сканиране и точково търсене.
|
||||||
|
|
||||||
|
### Употреба
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/btree
|
||||||
|
|
||||||
|
var btree = newBTreeIndex[string, string]()
|
||||||
|
btree.insert("key1", "value1")
|
||||||
|
let values = btree.get("key1")
|
||||||
|
let range = btree.scan("key_a", "key_z")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Write-Ahead Log (WAL)
|
||||||
|
|
||||||
|
Осигурява устойчивост на операциите за запис.
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/wal
|
||||||
|
|
||||||
|
var wal = newWAL("./wal")
|
||||||
|
wal.append("txn1", "SET key1 value1")
|
||||||
|
wal.flush()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Bloom Filter
|
||||||
|
|
||||||
|
Вероятностна структура от данни за бързи негативни проверки.
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/bloom
|
||||||
|
|
||||||
|
var filter = newBloomFilter(10000, 0.01)
|
||||||
|
filter.add("key1")
|
||||||
|
if filter.mightContain("key1"):
|
||||||
|
echo "евентуално съществува"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Memory-mapped I/O
|
||||||
|
|
||||||
|
Ефективен достъп до файлове чрез mmap.
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/mmap
|
||||||
|
|
||||||
|
var mapped = mmapFile("./data/file.dat")
|
||||||
|
let data = mapped.read(0, 100)
|
||||||
|
```
|
||||||
+46
-12
@@ -1,6 +1,6 @@
|
|||||||
# Транзакции & MVCC
|
# Транзакции и MVCC
|
||||||
|
|
||||||
Multi-Version Concurrency Control със snapshot изолация.
|
MVCC (Multi-Version Concurrency Control) със snapshot изолация и deadlock детекция.
|
||||||
|
|
||||||
## Употреба
|
## Употреба
|
||||||
|
|
||||||
@@ -10,21 +10,55 @@ import barabadb/core/mvcc
|
|||||||
var tm = newTxnManager()
|
var tm = newTxnManager()
|
||||||
let txn = tm.beginTxn()
|
let txn = tm.beginTxn()
|
||||||
|
|
||||||
|
# Операции за запис
|
||||||
discard tm.write(txn, "key1", cast[seq[byte]]("value1"))
|
discard tm.write(txn, "key1", cast[seq[byte]]("value1"))
|
||||||
discard tm.write(txn, "key2", cast[seq[byte]]("value2"))
|
discard tm.write(txn, "key2", cast[seq[byte]]("value2"))
|
||||||
|
|
||||||
|
# Savepoint
|
||||||
tm.savepoint(txn)
|
tm.savepoint(txn)
|
||||||
discard tm.rollbackToSavepoint(txn)
|
discard tm.write(txn, "key3", cast[seq[byte]]("value3"))
|
||||||
|
discard tm.rollbackToSavepoint(txn) # отмяна на key3
|
||||||
|
|
||||||
|
# Commit
|
||||||
discard tm.commit(txn)
|
discard tm.commit(txn)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Изолация
|
## Изолация на Транзакции
|
||||||
|
|
||||||
BaraDB използва **snapshot isolation**:
|
BaraDB използва **snapshot изолация**:
|
||||||
- Читателите не блокират писатели
|
- Четящите не блокират пишещите
|
||||||
- Писателите не блокират читатели
|
- Пишещите не блокират четящите
|
||||||
- Всяка транзакция вижда консистентен моментна снимка
|
- Всяка транзакция вижда консистентен snapshot
|
||||||
|
|
||||||
|
## Deadlock Детекция
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/core/deadlock
|
||||||
|
|
||||||
|
var detector = newDeadlockDetector()
|
||||||
|
if detector.detectCycle(txn1, txn2):
|
||||||
|
echo "Открит deadlock!"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Write-Ahead Log
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/wal
|
||||||
|
|
||||||
|
var wal = newWAL("./wal")
|
||||||
|
wal.append(txnId, "SET key value")
|
||||||
|
wal.flush()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Savepoints
|
||||||
|
|
||||||
|
Вложени savepoints на транзакции:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
tm.savepoint(txn, "sp1")
|
||||||
|
# ... операции ...
|
||||||
|
tm.rollbackToSavepoint(txn, "sp1")
|
||||||
|
```
|
||||||
|
|
||||||
## Формална Верификация
|
## Формална Верификация
|
||||||
|
|
||||||
@@ -32,13 +66,13 @@ MVCC / Snapshot Isolation протоколът е формално специф
|
|||||||
|
|
||||||
- **Спецификация:** `formal-verification/mvcc.tla`
|
- **Спецификация:** `formal-verification/mvcc.tla`
|
||||||
- **Проверени свойства:**
|
- **Проверени свойства:**
|
||||||
- `NoDirtyReads` — транзакциите никога не четат неcommit-нати данни
|
- `NoDirtyReads` — транзакциите никога не четат некомитнати данни
|
||||||
- `ReadOwnWrites` — транзакциите винаги виждат собствените си записи
|
- `ReadOwnWrites` — транзакциите винаги виждат собствените си записи
|
||||||
- `WriteWriteConflict` — first-committer-wins
|
- `WriteWriteConflict` — first-committer-wins (няма две комитнати транзакции да пишат един и същ ключ)
|
||||||
|
|
||||||
Пускане на TLC:
|
Пускане на TLC локално:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd formal-verification
|
cd formal-verification
|
||||||
java -cp tla2tools.jar tlc2.TLC -config models/mvcc.cfg mvcc.tla
|
java -cp tla2tools.jar tlc2.TLC -config models/mvcc.cfg mvcc.tla
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -0,0 +1,164 @@
|
|||||||
|
# Ръководство за Отстраняване на Проблеми
|
||||||
|
|
||||||
|
## Проблеми с Инсталацията
|
||||||
|
|
||||||
|
### Nim не е намерен
|
||||||
|
|
||||||
|
```
|
||||||
|
nim: command not found
|
||||||
|
```
|
||||||
|
|
||||||
|
**Решение:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Linux/macOS
|
||||||
|
curl https://nim-lang.org/choosenim/init.sh -sSf | sh
|
||||||
|
|
||||||
|
# Добавяне към PATH
|
||||||
|
echo 'export PATH=$HOME/.nimble/bin:$PATH' >> ~/.bashrc
|
||||||
|
source ~/.bashrc
|
||||||
|
```
|
||||||
|
|
||||||
|
### SSL Грешка при Компилация
|
||||||
|
|
||||||
|
```
|
||||||
|
Error: BaraDB requires SSL support. Compile with -d:ssl
|
||||||
|
```
|
||||||
|
|
||||||
|
**Решение:** Винаги компилирайте с `-d:ssl`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Липсващи Зависимости
|
||||||
|
|
||||||
|
```
|
||||||
|
Error: cannot open file: hunos
|
||||||
|
```
|
||||||
|
|
||||||
|
**Решение:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nimble install -d -y
|
||||||
|
```
|
||||||
|
|
||||||
|
## Проблеми по Време на Изпълнение
|
||||||
|
|
||||||
|
### Портът е Зает
|
||||||
|
|
||||||
|
```
|
||||||
|
Error: unhandled exception: Address already in use [OSError]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Решение 1:** Сменете порта:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
BARADB_PORT=5433 ./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
**Решение 2:** Убийте съществуващия процес:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
lsof -ti:9472 | xargs kill -9
|
||||||
|
```
|
||||||
|
|
||||||
|
### Permission Denied на Директория с Данни
|
||||||
|
|
||||||
|
```
|
||||||
|
Error: Permission denied: ./data
|
||||||
|
```
|
||||||
|
|
||||||
|
**Решение:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p ./data
|
||||||
|
chmod 755 ./data
|
||||||
|
# или използвайте друга директория
|
||||||
|
BARADB_DATA_DIR=/tmp/baradb ./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
## Storage Проблеми
|
||||||
|
|
||||||
|
### Бавни Заявки
|
||||||
|
|
||||||
|
1. Проверете cache hit rate:
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/metrics | grep cache_hit_rate
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Пуснете ръчен compaction:
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/admin/compact
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Проверете броя на SSTables:
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9470/metrics | grep sstables
|
||||||
|
```
|
||||||
|
|
||||||
|
### Растящо Дисково Пространство
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Проверете размера на директорията с данни
|
||||||
|
du -sh ./data
|
||||||
|
|
||||||
|
# Проверете WAL размера
|
||||||
|
du -sh ./data/server/wal
|
||||||
|
|
||||||
|
# Пуснете ръчен compaction за освобождаване на място
|
||||||
|
curl -X POST http://localhost:9470/admin/compact
|
||||||
|
```
|
||||||
|
|
||||||
|
## Проблеми с Автентикация
|
||||||
|
|
||||||
|
### Грешка при Верификация на JWT
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"error": {"code": "AUTH_REQUIRED", "message": "Authentication required"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Решение:** Уверете се, че изпращате правилния токен:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -H "Authorization: Bearer $TOKEN" \
|
||||||
|
http://localhost:9470/api/query \
|
||||||
|
-d '{"query": "SELECT 1"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Разпределени Проблеми
|
||||||
|
|
||||||
|
### Възел не се Присъединява към Клъстер
|
||||||
|
|
||||||
|
1. Проверете мрежовата свързаност между възлите
|
||||||
|
2. Проверете gossip порта (raft порт + 100)
|
||||||
|
3. Проверете логовете за грешки при gossip
|
||||||
|
|
||||||
|
### Репликационно Закъснение
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Проверете replication lag
|
||||||
|
curl http://localhost:9470/metrics/cluster | jq .replication_lag_ms
|
||||||
|
```
|
||||||
|
|
||||||
|
## Често Задавани Въпроси
|
||||||
|
|
||||||
|
### Как да нулирам напълно базата данни?
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Спрете сървъра
|
||||||
|
# Изтрийте директорията с данни
|
||||||
|
rm -rf ./data
|
||||||
|
# Стартирайте отново — нова празна база ще бъде създадена
|
||||||
|
./build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Как да мигрирам от друга база данни?
|
||||||
|
|
||||||
|
BaraDB поддържа импорт чрез JSON и CSV:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:9470/api/import \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"table": "users", "format": "json", "data": [...]}'
|
||||||
|
```
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Потребителски Функции (UDF)
|
||||||
|
|
||||||
|
Разширете BaraQL с персонализирани функции.
|
||||||
|
|
||||||
|
## Употреба
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/query/udf
|
||||||
|
|
||||||
|
var reg = newUDFRegistry()
|
||||||
|
|
||||||
|
# Регистриране на стандартна библиотека
|
||||||
|
reg.registerStdlib() # abs, sqrt, pow, lower, upper, len, trim, substr, toString, toInt
|
||||||
|
|
||||||
|
# Персонализирана функция
|
||||||
|
reg.register("greet", @[UDFParam(name: "name", typeName: "str")],
|
||||||
|
"str", proc(args: seq[Value]): Value =
|
||||||
|
return Value(kind: vkString, strVal: "Hello, " & args[0].strVal & "!"))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Функции от Стандартната Библиотека
|
||||||
|
|
||||||
|
| Функция | Описание | Пример |
|
||||||
|
|----------|----------|--------|
|
||||||
|
| `abs(n)` | Абсолютна стойност | `abs(-5)` → 5 |
|
||||||
|
| `sqrt(n)` | Квадратен корен | `sqrt(16)` → 4 |
|
||||||
|
| `pow(n, e)` | Степенуване | `pow(2, 3)` → 8 |
|
||||||
|
| `lower(s)` | Малки букви | `lower('ABC')` → 'abc' |
|
||||||
|
| `upper(s)` | Главни букви | `upper('abc')` → 'ABC' |
|
||||||
|
| `len(s)` | Дължина | `len('hello')` → 5 |
|
||||||
|
| `trim(s)` | Премахване на интервали | `trim(' hello ')` → 'hello' |
|
||||||
|
| `substr(s, start, len)` | Подниз | `substr('hello', 0, 3)` → 'hel' |
|
||||||
|
| `toString(n)` | Конвертиране в низ | `toString(123)` → '123' |
|
||||||
|
| `toInt(s)` | Конвертиране в integer | `toInt('123')` → 123 |
|
||||||
|
|
||||||
|
## Регистриране на Функции
|
||||||
|
|
||||||
|
```nim
|
||||||
|
reg.register(
|
||||||
|
name: "my_function",
|
||||||
|
params: @[
|
||||||
|
UDFParam(name: "arg1", typeName: "str"),
|
||||||
|
UDFParam(name: "arg2", typeName: "int32")
|
||||||
|
],
|
||||||
|
returnType: "str",
|
||||||
|
body: proc(args: seq[Value]): Value =
|
||||||
|
result = Value(kind: vkString, strVal: "")
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Използване на UDF в Заявки
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT greet(name) FROM users;
|
||||||
|
```
|
||||||
+130
-14
@@ -1,8 +1,89 @@
|
|||||||
# Vector Търсене
|
# Vector Search Engine
|
||||||
|
|
||||||
HNSW и IVF-PQ индекси за търсене на прилика.
|
Нативни HNSW и IVF-PQ индекси за търсене на прилика с пълна SQL интеграция.
|
||||||
|
|
||||||
## Употреба
|
## SQL Употреба
|
||||||
|
|
||||||
|
### Създаване на Векторни Колони
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE items (
|
||||||
|
id INT PRIMARY KEY,
|
||||||
|
embedding VECTOR(768)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
Типът `VECTOR(n)` съхранява float32 масиви с фиксирана размерност `n`.
|
||||||
|
|
||||||
|
### Вмъкване на Вектори
|
||||||
|
|
||||||
|
```sql
|
||||||
|
INSERT INTO items (id, embedding) VALUES (1, '[0.1, 0.2, 0.3, ...]');
|
||||||
|
```
|
||||||
|
|
||||||
|
### Функции за Векторно Разстояние
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Косинусово разстояние (0 = идентични, 1 = ортогонални)
|
||||||
|
SELECT id, cosine_distance(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Евклидово / L2 разстояние
|
||||||
|
SELECT id, euclidean_distance(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- L2 разстояние с <-> оператор
|
||||||
|
SELECT id, embedding <-> '[0.1, 0.2, 0.3]' AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Скаларно произведение (отрицателно dot product за минимизация)
|
||||||
|
SELECT id, inner_product(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Манхатън / L1 разстояние
|
||||||
|
SELECT id, l1_distance(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Търсене на Най-близки Съседи
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Топ-10 най-близки съседи по косинусово разстояние
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY cosine_distance(embedding, '[0.1, 0.2, 0.3]') ASC
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- Топ-5 най-близки съседи по евклидово разстояние
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY embedding <-> '[0.1, 0.2, 0.3]'
|
||||||
|
LIMIT 5;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Векторни Индекси
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Създаване на HNSW индекс за приблизително търсене на най-близки съседи
|
||||||
|
CREATE INDEX idx_items_vec ON items(embedding) USING hnsw;
|
||||||
|
|
||||||
|
-- Индексът се поддържа автоматично при INSERT и UPDATE
|
||||||
|
```
|
||||||
|
|
||||||
|
Поддържани индекс методи:
|
||||||
|
- `USING hnsw` — Hierarchical Navigable Small World (по подразбиране: косинусова метрика)
|
||||||
|
- `USING ivfpq` — Inverted File с Product Quantization
|
||||||
|
|
||||||
|
### Валидация на Размерност
|
||||||
|
|
||||||
|
BaraDB валидира размерностите на векторите при вмъкване:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Това ще даде грешка: очаквани 768 размерности, но подадени 3
|
||||||
|
INSERT INTO items (id, embedding) VALUES (2, '[1.0, 2.0, 3.0]');
|
||||||
|
```
|
||||||
|
|
||||||
|
## Нативно Nim API
|
||||||
|
|
||||||
|
За вградена или високопроизводителна употреба използвайте нативното Nim API директно:
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
import barabadb/vector/engine
|
import barabadb/vector/engine
|
||||||
@@ -10,32 +91,67 @@ import barabadb/vector/engine
|
|||||||
var idx = newHNSWIndex(dimensions = 128)
|
var idx = newHNSWIndex(dimensions = 128)
|
||||||
idx.insert(1, @[1.0'f32, 0.0'f32, ...], {"category": "A"}.toTable)
|
idx.insert(1, @[1.0'f32, 0.0'f32, ...], {"category": "A"}.toTable)
|
||||||
|
|
||||||
|
# Търсене
|
||||||
let results = idx.search(queryVector, k = 10)
|
let results = idx.search(queryVector, k = 10)
|
||||||
|
|
||||||
|
# С филтриране по метаданни
|
||||||
|
let filtered = idx.searchWithFilter(queryVector, k = 10,
|
||||||
|
filter = proc(meta: Table[string, string]): bool =
|
||||||
|
return meta.getOrDefault("category") == "A")
|
||||||
```
|
```
|
||||||
|
|
||||||
## Индекс Типове
|
## Типове Индекси
|
||||||
|
|
||||||
### HNSW
|
### HNSW
|
||||||
|
|
||||||
Иерархичен навигируем малък свят:
|
Иерархичен навигируем малък свят за приблизително търсене на най-близки съседи.
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
var hnsw = newHNSWIndex(dimensions = 128, m = 16)
|
var hnsw = newHNSWIndex(
|
||||||
|
dimensions = 128,
|
||||||
|
m = 16, # връзки на слой
|
||||||
|
efConstruction = 200, # ширина на търсене при изграждане
|
||||||
|
efSearch = 100 # ширина на търсене при заявка
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
### IVF-PQ
|
### IVF-PQ
|
||||||
|
|
||||||
Инвертиран файл с продуктово квантуване:
|
Inverted File Index с продуктово квантуване за компресия.
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
var ivfpq = newIVFPQIndex(dimensions = 128, numCentroids = 256)
|
var ivfpq = newIVFPQIndex(
|
||||||
|
dimensions = 128,
|
||||||
|
numCentroids = 256,
|
||||||
|
subQuantizers = 8
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Метрики за Разстояние
|
## Метрики за Разстояние
|
||||||
|
|
||||||
| Метрика | Описание |
|
| Метрика | SQL Функция | Описание |
|
||||||
|---------|----------|
|
|---------|-------------|----------|
|
||||||
| `cosine` | Косинусова прилика |
|
| `cosine` | `cosine_distance(a, b)` | Косинусова dissimilarity (1 - similarity) |
|
||||||
| `euclidean` | L2 разстояние |
|
| `euclidean` | `euclidean_distance(a, b)` / `<->` | L2 разстояние |
|
||||||
| `dotproduct` | Скаларно произведение |
|
| `dotproduct` | `inner_product(a, b)` | Отрицателно скаларно произведение |
|
||||||
| `manhattan` | L1 разстояние |
|
| `manhattan` | `l1_distance(a, b)` | L1 разстояние |
|
||||||
|
|
||||||
|
## Квантуване
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/vector/quant
|
||||||
|
|
||||||
|
# Скаларно квантуване
|
||||||
|
let scalar = scalarQuantize(data, bits = 8)
|
||||||
|
|
||||||
|
# Продуктово квантуване
|
||||||
|
let pq = productQuantize(data, subVectors = 8, bits = 8)
|
||||||
|
```
|
||||||
|
|
||||||
|
## SIMD Ускорение
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/vector/simd
|
||||||
|
|
||||||
|
let dist = simdCosineDistance(vec1, vec2)
|
||||||
|
```
|
||||||
|
|||||||
+147
-21
@@ -18,6 +18,7 @@ BaraQL is a SQL-compatible query language with extensions for graph, vector, and
|
|||||||
| `bytes` | Raw bytes | `0xDEADBEEF` |
|
| `bytes` | Raw bytes | `0xDEADBEEF` |
|
||||||
| `array<T>` | Homogeneous array | `[1, 2, 3]` |
|
| `array<T>` | Homogeneous array | `[1, 2, 3]` |
|
||||||
| `vector` | Float32 vector | `[0.1, 0.2, 0.3]` |
|
| `vector` | Float32 vector | `[0.1, 0.2, 0.3]` |
|
||||||
|
| `vector(n)` | Fixed-dimension float32 vector (SQL) | `VECTOR(768)` |
|
||||||
| `object` | Key-value object | `{"a": 1}` |
|
| `object` | Key-value object | `{"a": 1}` |
|
||||||
| `datetime` | ISO 8601 timestamp | `'2025-01-15T10:30:00Z'` |
|
| `datetime` | ISO 8601 timestamp | `'2025-01-15T10:30:00Z'` |
|
||||||
| `uuid` | UUID v4 | `'550e8400-e29b-41d4-a716-446655440000'` |
|
| `uuid` | UUID v4 | `'550e8400-e29b-41d4-a716-446655440000'` |
|
||||||
@@ -352,6 +353,7 @@ CREATE TYPE Cat EXTENDING Animal {
|
|||||||
CREATE INDEX idx_users_name ON users(name);
|
CREATE INDEX idx_users_name ON users(name);
|
||||||
CREATE UNIQUE INDEX idx_users_email ON users(email);
|
CREATE UNIQUE INDEX idx_users_email ON users(email);
|
||||||
CREATE INDEX idx_users_age ON users(age) USING btree;
|
CREATE INDEX idx_users_age ON users(age) USING btree;
|
||||||
|
CREATE INDEX idx_vectors ON items(embedding) USING hnsw;
|
||||||
```
|
```
|
||||||
|
|
||||||
### DROP
|
### DROP
|
||||||
@@ -387,37 +389,76 @@ SELECT * FROM articles WHERE body @@ 'machine learning';
|
|||||||
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
||||||
```
|
```
|
||||||
|
|
||||||
## Vector Search
|
## Vector Search (SQL)
|
||||||
|
|
||||||
|
### Creating Vector Columns
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
-- Insert with vector
|
CREATE TABLE items (
|
||||||
INSERT articles {
|
id INT PRIMARY KEY,
|
||||||
title := 'Nim Programming',
|
embedding VECTOR(768)
|
||||||
embedding := [0.1, 0.2, 0.3, 0.4]
|
);
|
||||||
};
|
```
|
||||||
|
|
||||||
-- Similarity search (cosine distance)
|
### Inserting Vectors
|
||||||
SELECT title FROM articles
|
|
||||||
ORDER BY cosine_distance(embedding, [0.1, 0.2, 0.3, 0.4])
|
|
||||||
LIMIT 5;
|
|
||||||
|
|
||||||
-- Euclidean distance
|
```sql
|
||||||
SELECT title FROM articles
|
INSERT INTO items (id, embedding) VALUES (1, '[0.1, 0.2, 0.3, 0.4]');
|
||||||
ORDER BY l2_distance(embedding, [0.1, 0.2, 0.3, 0.4])
|
```
|
||||||
LIMIT 5;
|
|
||||||
|
|
||||||
-- Dot product
|
### Distance Functions
|
||||||
SELECT title FROM articles
|
|
||||||
ORDER BY dot_product(embedding, [0.1, 0.2, 0.3, 0.4]) DESC
|
```sql
|
||||||
|
-- Cosine distance (0 = identical, 2 = opposite)
|
||||||
|
SELECT id, cosine_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Euclidean / L2 distance
|
||||||
|
SELECT id, euclidean_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- L2 distance with <-> operator
|
||||||
|
SELECT id, embedding <-> '[0.1, 0.2, 0.3, 0.4]' AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Inner product (negative dot product)
|
||||||
|
SELECT id, inner_product(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Manhattan / L1 distance
|
||||||
|
SELECT id, l1_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') AS dist
|
||||||
|
FROM items;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Nearest Neighbor Search
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Top-10 nearest neighbors by cosine distance
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY cosine_distance(embedding, '[0.1, 0.2, 0.3, 0.4]') ASC
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- Top-5 nearest neighbors by Euclidean distance
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY embedding <-> '[0.1, 0.2, 0.3, 0.4]'
|
||||||
LIMIT 5;
|
LIMIT 5;
|
||||||
|
|
||||||
-- With metadata filter
|
-- With metadata filter
|
||||||
SELECT title FROM articles
|
SELECT id FROM items
|
||||||
WHERE category = 'tech'
|
WHERE category = 'tech'
|
||||||
ORDER BY cosine_distance(embedding, [0.1, 0.2, 0.3, 0.4])
|
ORDER BY cosine_distance(embedding, '[0.1, 0.2, 0.3, 0.4]')
|
||||||
LIMIT 5;
|
LIMIT 5;
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Vector Indexes
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Create HNSW index for approximate nearest neighbor search
|
||||||
|
CREATE INDEX idx_items_vec ON items(embedding) USING hnsw;
|
||||||
|
|
||||||
|
-- Supported index methods: hnsw, ivfpq
|
||||||
|
```
|
||||||
|
|
||||||
## Graph Patterns
|
## Graph Patterns
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
@@ -517,6 +558,88 @@ LIMIT 10;
|
|||||||
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Window Functions
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Ranking functions
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
department,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||||
|
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||||
|
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Value functions
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
salary,
|
||||||
|
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||||
|
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||||
|
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||||
|
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Distribution functions
|
||||||
|
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frame Specifications
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ROWS frame
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
|
||||||
|
-- RANGE frame
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Multi-Tenant ERP
|
||||||
|
|
||||||
|
BaraDB supports running multiple companies (tenants) in a single database instance, using **Row-Level Security (RLS)** combined with **session variables**.
|
||||||
|
|
||||||
|
### Session Variables
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET app.tenant_id = 'company-123';
|
||||||
|
SELECT current_setting('app.tenant_id') AS tenant;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Current User / Role
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT current_user AS me, current_role AS my_role;
|
||||||
|
```
|
||||||
|
|
||||||
|
### RLS Tenant Isolation
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Enable RLS on a table
|
||||||
|
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
|
||||||
|
|
||||||
|
-- Create a policy that filters by tenant
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- Each session only sees its own data
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- only company-a rows
|
||||||
|
```
|
||||||
|
|
||||||
|
### Why Multi-Tenant?
|
||||||
|
|
||||||
|
- **One instance, many tenants** — no need to run 100 separate databases
|
||||||
|
- **JSONB documents** — schema-flexible storage, easy to add fields per tenant
|
||||||
|
- **RLS guarantees isolation** — the database enforces tenant boundaries, not just the application
|
||||||
|
|
||||||
## Supported Keywords
|
## Supported Keywords
|
||||||
|
|
||||||
| Category | Keywords |
|
| Category | Keywords |
|
||||||
@@ -531,8 +654,11 @@ SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
|||||||
| Transaction | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
| Transaction | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
||||||
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
||||||
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||||
| Vector | cosine_distance, l2_distance, dot_product, manhattan_distance |
|
| Vector | cosine_distance, euclidean_distance, inner_product, l1_distance, l2_distance, <-> |
|
||||||
| JSON | ->, ->> |
|
| JSON | ->, ->> |
|
||||||
| FTS | @@ (BM25 match) |
|
| FTS | @@ (BM25 match) |
|
||||||
| Recovery | RECOVER TO TIMESTAMP |
|
| Recovery | RECOVER TO TIMESTAMP |
|
||||||
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id |
|
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id, current_setting |
|
||||||
|
| Session | SET, current_setting, current_user, current_role |
|
||||||
|
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||||
|
| Window Functions | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||||
|
|||||||
@@ -2,6 +2,26 @@
|
|||||||
|
|
||||||
All notable changes to BaraDB are documented in this file.
|
All notable changes to BaraDB are documented in this file.
|
||||||
|
|
||||||
|
## [Unreleased] — SQL:2023 Stabilization
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **GROUPING SETS execution** — `lowerSelect` now creates `irpkGroupBy` when `selGroupingSetsKind != gskNone`, even if `selGroupBy` is empty. Previously, queries like `GROUP BY GROUPING SETS ((dept), ())` bypassed the grouping executor entirely.
|
||||||
|
- **FTS CREATE INDEX docId mismatch** — `CREATE INDEX ... USING FTS` now computes `docId` as a hash of `tableName.$key`, consistent with DML operations (`INSERT`/`UPDATE`/`DELETE`). Previously, index creation used sequential IDs (0, 1, 2...), causing `@@` queries to never match indexed documents.
|
||||||
|
- **Test isolation (all suites)** — All `newLSMTree("")` calls replaced with unique temporary directories per suite. Eliminates WAL accumulation issues and flaky tests caused by shared database state between test runs.
|
||||||
|
- **Window frame parser** — `parseFrameBoundary` no longer consumes `tkRow` after `tkCurrent` incorrectly (was using `tkRows`). Also fixed `tkRow` keyword conflict with `ENABLE ROW LEVEL SECURITY` parsing.
|
||||||
|
- **ORDER BY + SELECT projection** — `lowerSelect` now places `irpkSort` before `irpkProject`, enabling `ORDER BY` on columns not present in the `SELECT` list.
|
||||||
|
- **UNPIVOT execution** — Verified and fixed missing test coverage for UNPIVOT transformation.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **JSON operators** — `@>` (contains), `<@` (contained by), `?` (has key), `?|` (has any), `?&` (has all) now supported in lexer, parser, and executor.
|
||||||
|
- **Window frame execution** — `ROWS BETWEEN X PRECEDING AND Y FOLLOWING` / `CURRENT ROW` frame boundaries now respected by `FIRST_VALUE` and `LAST_VALUE`.
|
||||||
|
- **Session variables** — `SET var_name = value` and `current_setting('var_name')` for connection-scoped key/value storage.
|
||||||
|
- **Current user/role** — `current_user` and `current_role` SQL keywords evaluate to the authenticated session's user and role.
|
||||||
|
- **Auth-executor bridge** — Wire server and HTTP server now populate `ExecutionContext.currentUser` and `ExecutionContext.currentRole` after JWT/SCRAM authentication.
|
||||||
|
- **Multi-tenant RLS** — Row-Level Security policies can now reference `current_user`, `current_role`, and `current_setting('app.tenant_id')` for per-tenant data isolation.
|
||||||
|
|
||||||
## [1.1.0] — 2026-05-13
|
## [1.1.0] — 2026-05-13
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
@@ -174,6 +194,25 @@ All notable changes to BaraDB are documented in this file.
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Vector SQL Integration** — Full SQL-level vector search support:
|
||||||
|
- `VECTOR(n)` column type in `CREATE TABLE` with dimension validation
|
||||||
|
- `CREATE INDEX ... USING hnsw` / `USING ivfpq` for approximate nearest neighbor indexes
|
||||||
|
- SQL distance functions: `cosine_distance()`, `euclidean_distance()`, `inner_product()`, `l1_distance()`, `l2_distance()`
|
||||||
|
- `<->` nearest-neighbor operator (Euclidean distance)
|
||||||
|
- `ORDER BY` support for vector distance expressions, including columns not in `SELECT`
|
||||||
|
- Automatic HNSW index maintenance on `INSERT` and `UPDATE`
|
||||||
|
- **Advanced SQL Engine** — Window functions, MERGE/UPSERT, LATERAL JOIN, PIVOT/UNPIVOT, SQL/PGQ Property Graph, Advanced Aggregates (ARRAY_AGG, STRING_AGG, FILTER, GROUPING SETS/ROLLUP/CUBE)
|
||||||
|
- **JavaScript Client — TCP Request Queue** — Internal `_requestQueue` + `_requestLock` for safe concurrent queries. Multiple parallel `query()` / `execute()` / `ping()` calls no longer interleave binary frames on the wire.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Query Executor — Row Value Escaping** — `execInsert` now properly escapes commas and equals signs in column values, fixing storage corruption for vector literals like `[1.0, 2.0, 3.0]`
|
||||||
|
- **Query Planner — ORDER BY Projection** — `irpkSort` is now placed before `irpkProject` in the IR plan, allowing `ORDER BY` to reference columns that are not selected
|
||||||
|
- **Wire Protocol — Big-Endian Float Serialization** — `FLOAT32`/`FLOAT64` and vector float values are now serialized in big-endian byte order, matching the client's `readFloatBE()` / `readDoubleBE()` and ensuring cross-platform numeric accuracy.
|
||||||
|
- **Gossip Protocol — Async UDP Socket** — Replaced synchronous `newSocket` + blocking `recvFrom` with `newAsyncSocket` + `await recvFrom`, preventing the async event loop from freezing until a UDP packet arrives.
|
||||||
|
|
||||||
### Planned
|
### Planned
|
||||||
|
|
||||||
- Query plan caching
|
- Query plan caching
|
||||||
|
|||||||
@@ -45,6 +45,18 @@ await client.commit();
|
|||||||
await client.close();
|
await client.close();
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Concurrent Queries
|
||||||
|
|
||||||
|
The JavaScript client automatically serializes concurrent requests over a single TCP connection via an internal request queue. You can safely fire multiple parallel operations — their binary frames will not interleave on the wire:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const [users, orders, stats] = await Promise.all([
|
||||||
|
client.query('SELECT * FROM users'),
|
||||||
|
client.query('SELECT * FROM orders'),
|
||||||
|
client.query('SELECT count(*) FROM visits')
|
||||||
|
]);
|
||||||
|
```
|
||||||
|
|
||||||
### WebSocket Streaming
|
### WebSocket Streaming
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
|
|||||||
@@ -156,7 +156,7 @@ raft_peers = "node2:9001,node3:9001"
|
|||||||
```
|
```
|
||||||
|
|
||||||
```
|
```
|
||||||
BaraDB v0.1.0 — Multimodal Database Engine
|
BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
baradadb [options]
|
baradadb [options]
|
||||||
|
|||||||
+1
-1
@@ -6,7 +6,7 @@
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Клониране на репото
|
# Клониране на репото
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
|
||||||
# Build на образа
|
# Build на образа
|
||||||
|
|||||||
@@ -99,7 +99,7 @@ download from [slproweb.com](https://slproweb.com/products/Win32OpenSSL.html).
|
|||||||
### Clone the Repository
|
### Clone the Repository
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -144,7 +144,7 @@ strip build/baradadb
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/baradadb --version
|
./build/baradadb --version
|
||||||
# Expected: BaraDB v0.1.0 — Multimodal Database Engine
|
# Expected: BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
```
|
```
|
||||||
|
|
||||||
## Running Tests
|
## Running Tests
|
||||||
@@ -259,7 +259,7 @@ db.close()
|
|||||||
./build/baradadb
|
./build/baradadb
|
||||||
|
|
||||||
# Expected output:
|
# Expected output:
|
||||||
# BaraDB v0.1.0 — Multimodal Database Engine
|
# BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
# BaraDB TCP listening on 127.0.0.1:9472
|
# BaraDB TCP listening on 127.0.0.1:9472
|
||||||
|
|
||||||
# Test with HTTP API
|
# Test with HTTP API
|
||||||
|
|||||||
+3
-3
@@ -98,13 +98,13 @@ Every message starts with a 8-byte header:
|
|||||||
| INT16 | 0x03 | 2 | Signed 16-bit integer |
|
| INT16 | 0x03 | 2 | Signed 16-bit integer |
|
||||||
| INT32 | 0x04 | 4 | Signed 32-bit integer |
|
| INT32 | 0x04 | 4 | Signed 32-bit integer |
|
||||||
| INT64 | 0x05 | 8 | Signed 64-bit integer |
|
| INT64 | 0x05 | 8 | Signed 64-bit integer |
|
||||||
| FLOAT32 | 0x06 | 4 | IEEE 754 single precision |
|
| FLOAT32 | 0x06 | 4 | IEEE 754 single precision (big-endian) |
|
||||||
| FLOAT64 | 0x07 | 8 | IEEE 754 double precision |
|
| FLOAT64 | 0x07 | 8 | IEEE 754 double precision (big-endian) |
|
||||||
| STRING | 0x08 | variable | UTF-8 string (4-byte length prefix) |
|
| STRING | 0x08 | variable | UTF-8 string (4-byte length prefix) |
|
||||||
| BYTES | 0x09 | variable | Raw bytes (4-byte length prefix) |
|
| BYTES | 0x09 | variable | Raw bytes (4-byte length prefix) |
|
||||||
| ARRAY | 0x0A | variable | Array of values |
|
| ARRAY | 0x0A | variable | Array of values |
|
||||||
| OBJECT | 0x0B | variable | Key-value object |
|
| OBJECT | 0x0B | variable | Key-value object |
|
||||||
| VECTOR | 0x0C | variable | Float32 array (4-byte length prefix) |
|
| VECTOR | 0x0C | variable | Float32 array (4-byte length prefix, big-endian floats) |
|
||||||
|
|
||||||
### Error Message Payload
|
### Error Message Payload
|
||||||
|
|
||||||
|
|||||||
+89
-8
@@ -1,8 +1,89 @@
|
|||||||
# Vector Search Engine
|
# Vector Search Engine
|
||||||
|
|
||||||
Native HNSW and IVF-PQ indexes for similarity search.
|
Native HNSW and IVF-PQ indexes for similarity search with full SQL integration.
|
||||||
|
|
||||||
## Usage
|
## SQL Usage
|
||||||
|
|
||||||
|
### Creating Vector Columns
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE items (
|
||||||
|
id INT PRIMARY KEY,
|
||||||
|
embedding VECTOR(768)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
The `VECTOR(n)` type stores float32 arrays of fixed dimension `n`.
|
||||||
|
|
||||||
|
### Inserting Vectors
|
||||||
|
|
||||||
|
```sql
|
||||||
|
INSERT INTO items (id, embedding) VALUES (1, '[0.1, 0.2, 0.3, ...]');
|
||||||
|
```
|
||||||
|
|
||||||
|
### Vector Distance Functions
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Cosine distance (0 = identical, 1 = orthogonal)
|
||||||
|
SELECT id, cosine_distance(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Euclidean / L2 distance
|
||||||
|
SELECT id, euclidean_distance(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- L2 distance with <-> operator
|
||||||
|
SELECT id, embedding <-> '[0.1, 0.2, 0.3]' AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Inner product (negative dot product for minimization)
|
||||||
|
SELECT id, inner_product(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
|
||||||
|
-- Manhattan / L1 distance
|
||||||
|
SELECT id, l1_distance(embedding, '[0.1, 0.2, 0.3]') AS dist
|
||||||
|
FROM items;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Nearest Neighbor Search
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Top-10 nearest neighbors by cosine distance
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY cosine_distance(embedding, '[0.1, 0.2, 0.3]') ASC
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- Top-5 nearest neighbors by Euclidean distance
|
||||||
|
SELECT id FROM items
|
||||||
|
ORDER BY embedding <-> '[0.1, 0.2, 0.3]'
|
||||||
|
LIMIT 5;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Vector Indexes
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Create HNSW index for approximate nearest neighbor search
|
||||||
|
CREATE INDEX idx_items_vec ON items(embedding) USING hnsw;
|
||||||
|
|
||||||
|
-- The index is automatically maintained on INSERT and UPDATE
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported index methods:
|
||||||
|
- `USING hnsw` — Hierarchical Navigable Small World (default: cosine metric)
|
||||||
|
- `USING ivfpq` — Inverted File with Product Quantization
|
||||||
|
|
||||||
|
### Dimension Validation
|
||||||
|
|
||||||
|
BaraDB validates vector dimensions at insert time:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- This will fail: expected 768 dimensions but got 3
|
||||||
|
INSERT INTO items (id, embedding) VALUES (2, '[1.0, 2.0, 3.0]');
|
||||||
|
```
|
||||||
|
|
||||||
|
## Native Nim API
|
||||||
|
|
||||||
|
For embedded or high-performance use, use the native Nim API directly:
|
||||||
|
|
||||||
```nim
|
```nim
|
||||||
import barabadb/vector/engine
|
import barabadb/vector/engine
|
||||||
@@ -48,12 +129,12 @@ var ivfpq = newIVFPQIndex(
|
|||||||
|
|
||||||
## Distance Metrics
|
## Distance Metrics
|
||||||
|
|
||||||
| Metric | Description |
|
| Metric | SQL Function | Description |
|
||||||
|--------|-------------|
|
|--------|--------------|-------------|
|
||||||
| `cosine` | Cosine similarity |
|
| `cosine` | `cosine_distance(a, b)` | Cosine dissimilarity (1 - similarity) |
|
||||||
| `euclidean` | L2 distance |
|
| `euclidean` | `euclidean_distance(a, b)` / `<->` | L2 distance |
|
||||||
| `dotproduct` | Dot product similarity |
|
| `dotproduct` | `inner_product(a, b)` | Negative dot product |
|
||||||
| `manhattan` | L1 distance |
|
| `manhattan` | `l1_distance(a, b)` | L1 distance |
|
||||||
|
|
||||||
## Quantization
|
## Quantization
|
||||||
|
|
||||||
|
|||||||
+106
-1
@@ -489,4 +489,109 @@ LIMIT 10;
|
|||||||
|
|
||||||
-- اجرای موازی
|
-- اجرای موازی
|
||||||
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## توابع پنجرهای
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- توابع رتبهبندی
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
department,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||||
|
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||||
|
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- توابع مقدار
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
salary,
|
||||||
|
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||||
|
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||||
|
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||||
|
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- توابع توزیع
|
||||||
|
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||||
|
```
|
||||||
|
|
||||||
|
### مشخصات فریم
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- فریم ROWS
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
|
||||||
|
-- فریم RANGE
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## ERP چند مستأجری
|
||||||
|
|
||||||
|
BaraDB از اجرای چندین شرکت (tenant) در یک نمونه پایگاه داده پشتیبانی میکند، با استفاده از **امنیت سطح سطر (RLS)** همراه با **متغیرهای جلسه**.
|
||||||
|
|
||||||
|
### متغیرهای جلسه
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET app.tenant_id = 'company-123';
|
||||||
|
SELECT current_setting('app.tenant_id') AS tenant;
|
||||||
|
```
|
||||||
|
|
||||||
|
### کاربر / نقش فعلی
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT current_user AS me, current_role AS my_role;
|
||||||
|
```
|
||||||
|
|
||||||
|
### جداسازی مستأجر با RLS
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- فعال کردن RLS روی جدول
|
||||||
|
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
|
||||||
|
|
||||||
|
-- ایجاد سیاست فیلترینگ بر اساس مستأجر
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- هر جلسه فقط دادههای خود را میبیند
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- فقط ردیفهای company-a
|
||||||
|
```
|
||||||
|
|
||||||
|
### چرا چند مستأجری؟
|
||||||
|
|
||||||
|
- **یک نمونه، مستأجران زیاد** — نیازی به اجرای ۱۰۰ پایگاه داده جداگانه نیست
|
||||||
|
- **اسناد JSONB** — ذخیرهسازی با طرح انعطافپذیر، افزودن آسان فیلدها برای هر مستأجر
|
||||||
|
- **RLS تضمین میکند** — پایگاه داده مرزهای مستأجر را اعمال میکند، نه فقط برنامه
|
||||||
|
|
||||||
|
## کلمات کلیدی پشتیبانیشده
|
||||||
|
|
||||||
|
| دسته | کلمات کلیدی |
|
||||||
|
|----------|----------|
|
||||||
|
| DQL | SELECT, FROM, WHERE, ORDER BY, GROUP BY, HAVING, LIMIT, OFFSET, DISTINCT |
|
||||||
|
| DML | INSERT, UPDATE, DELETE, SET, VALUES |
|
||||||
|
| DDL | CREATE TYPE, DROP TYPE, CREATE INDEX, DROP INDEX, ALTER TYPE |
|
||||||
|
| Join | INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, CROSS JOIN, ON |
|
||||||
|
| Set | UNION, UNION ALL, INTERSECT, EXCEPT |
|
||||||
|
| CTEs | WITH, RECURSIVE, AS |
|
||||||
|
| Case | CASE, WHEN, THEN, ELSE, END |
|
||||||
|
| Transaction | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
||||||
|
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
||||||
|
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||||
|
| Vector | cosine_distance, euclidean_distance, inner_product, l1_distance, l2_distance, <-> |
|
||||||
|
| JSON | ->, ->> |
|
||||||
|
| FTS | @@ (BM25 match) |
|
||||||
|
| Recovery | RECOVER TO TIMESTAMP |
|
||||||
|
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id, current_setting |
|
||||||
|
| Session | SET, current_setting, current_user, current_role |
|
||||||
|
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||||
|
| Window Functions | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
## شروع سریع
|
## شروع سریع
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
|
||||||
docker build -t baradb:latest .
|
docker build -t baradb:latest .
|
||||||
|
|||||||
@@ -99,7 +99,7 @@ OpenSSL همراه توزیع ویندوز Nim است. برای ساخت دست
|
|||||||
### کلون کردن مخزن
|
### کلون کردن مخزن
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -144,7 +144,7 @@ strip build/baradadb
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/baradadb --version
|
./build/baradadb --version
|
||||||
# انتظار: BaraDB v0.1.0 — Multimodal Database Engine
|
# انتظار: BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
```
|
```
|
||||||
|
|
||||||
## اجرای تستها
|
## اجرای تستها
|
||||||
@@ -259,7 +259,7 @@ db.close()
|
|||||||
./build/baradadb
|
./build/baradadb
|
||||||
|
|
||||||
# خروجی مورد انتظار:
|
# خروجی مورد انتظار:
|
||||||
# BaraDB v0.1.0 — Multimodal Database Engine
|
# BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
# BaraDB TCP listening on 127.0.0.1:9472
|
# BaraDB TCP listening on 127.0.0.1:9472
|
||||||
|
|
||||||
# تست با HTTP API
|
# تست با HTTP API
|
||||||
|
|||||||
+106
-1
@@ -489,4 +489,109 @@ LIMIT 10;
|
|||||||
|
|
||||||
-- Параллельное выполнение
|
-- Параллельное выполнение
|
||||||
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Оконные функции
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Функции ранжирования
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
department,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||||
|
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||||
|
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Функции значения
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
salary,
|
||||||
|
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||||
|
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||||
|
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||||
|
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Функции распределения
|
||||||
|
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Спецификации фрейма
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ROWS фрейм
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
|
||||||
|
-- RANGE фрейм
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Мультитенантный ERP
|
||||||
|
|
||||||
|
BaraDB поддерживает работу нескольких компаний (арендаторов) в одной базе данных, используя **безопасность на уровне строк (RLS)** в сочетании с **переменными сессии**.
|
||||||
|
|
||||||
|
### Переменные сессии
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET app.tenant_id = 'company-123';
|
||||||
|
SELECT current_setting('app.tenant_id') AS tenant;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Текущий пользователь / роль
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT current_user AS me, current_role AS my_role;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Изоляция арендаторов через RLS
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Включить RLS на таблице
|
||||||
|
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
|
||||||
|
|
||||||
|
-- Создать политику фильтрации по арендатору
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- Каждая сессия видит только свои данные
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- только строки company-a
|
||||||
|
```
|
||||||
|
|
||||||
|
### Зачем мультитенантность?
|
||||||
|
|
||||||
|
- **Один экземпляр, много арендаторов** — не нужно запускать 100 отдельных баз данных
|
||||||
|
- **JSONB документы** — гибкая схема, легко добавлять поля для каждого арендатора
|
||||||
|
- **RLS гарантирует изоляцию** — база данных обеспечивает границы арендаторов, а не только приложение
|
||||||
|
|
||||||
|
## Поддерживаемые ключевые слова
|
||||||
|
|
||||||
|
| Категория | Ключевые слова |
|
||||||
|
|----------|----------|
|
||||||
|
| DQL | SELECT, FROM, WHERE, ORDER BY, GROUP BY, HAVING, LIMIT, OFFSET, DISTINCT |
|
||||||
|
| DML | INSERT, UPDATE, DELETE, SET, VALUES |
|
||||||
|
| DDL | CREATE TYPE, DROP TYPE, CREATE INDEX, DROP INDEX, ALTER TYPE |
|
||||||
|
| Join | INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, CROSS JOIN, ON |
|
||||||
|
| Set | UNION, UNION ALL, INTERSECT, EXCEPT |
|
||||||
|
| CTEs | WITH, RECURSIVE, AS |
|
||||||
|
| Case | CASE, WHEN, THEN, ELSE, END |
|
||||||
|
| Transaction | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
||||||
|
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
||||||
|
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||||
|
| Vector | cosine_distance, euclidean_distance, inner_product, l1_distance, l2_distance, <-> |
|
||||||
|
| JSON | ->, ->> |
|
||||||
|
| FTS | @@ (BM25 match) |
|
||||||
|
| Recovery | RECOVER TO TIMESTAMP |
|
||||||
|
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id, current_setting |
|
||||||
|
| Session | SET, current_setting, current_user, current_role |
|
||||||
|
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||||
|
| Window Functions | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||||
@@ -156,7 +156,7 @@ raft_peers = "node2:9001,node3:9001"
|
|||||||
```
|
```
|
||||||
|
|
||||||
```
|
```
|
||||||
BaraDB v0.1.0 — Multimodal Database Engine
|
BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
baradadb [options]
|
baradadb [options]
|
||||||
|
|||||||
+1
-1
@@ -4,7 +4,7 @@
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Клонировать репозиторий
|
# Клонировать репозиторий
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
|
||||||
# Собрать образ
|
# Собрать образ
|
||||||
|
|||||||
@@ -99,7 +99,7 @@ OpenSSL поставляется вместе с Nim для Windows. Для ру
|
|||||||
### Клонирование репозитория
|
### Клонирование репозитория
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -144,7 +144,7 @@ strip build/baradadb
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/baradadb --version
|
./build/baradadb --version
|
||||||
# Ожидается: BaraDB v0.1.0 — Multimodal Database Engine
|
# Ожидается: BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
```
|
```
|
||||||
|
|
||||||
## Запуск тестов
|
## Запуск тестов
|
||||||
@@ -259,7 +259,7 @@ db.close()
|
|||||||
./build/baradadb
|
./build/baradadb
|
||||||
|
|
||||||
# Ожидаемый вывод:
|
# Ожидаемый вывод:
|
||||||
# BaraDB v0.1.0 — Multimodal Database Engine
|
# BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
# BaraDB TCP listening on 127.0.0.1:9472
|
# BaraDB TCP listening on 127.0.0.1:9472
|
||||||
|
|
||||||
# Проверка через HTTP API
|
# Проверка через HTTP API
|
||||||
|
|||||||
+164
-5
@@ -98,13 +98,172 @@ RETURN friend.name;
|
|||||||
```sql
|
```sql
|
||||||
BEGIN;
|
BEGIN;
|
||||||
INSERT users { name := 'Alice', age := 30 };
|
INSERT users { name := 'Alice', age := 30 };
|
||||||
|
INSERT orders { user_id := last_insert_id(), total := 100 };
|
||||||
|
COMMIT;
|
||||||
|
|
||||||
|
-- Savepoint ile
|
||||||
|
BEGIN;
|
||||||
|
INSERT users { name := 'Bob', age := 25 };
|
||||||
|
SAVEPOINT sp1;
|
||||||
|
INSERT orders { user_id := last_insert_id(), total := 200 };
|
||||||
|
-- Hata, savepoint'e geri al
|
||||||
|
ROLLBACK TO sp1;
|
||||||
COMMIT;
|
COMMIT;
|
||||||
```
|
```
|
||||||
|
|
||||||
## Grafik Kalıpları
|
## Tam Metin Arama
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
MATCH (p:Person)-[:KNOWS]->(friend:Person)
|
-- Temel arama
|
||||||
WHERE p.name = 'Alice'
|
SELECT * FROM articles
|
||||||
RETURN friend.name;
|
WHERE MATCH(title, body) AGAINST('database programming');
|
||||||
```
|
|
||||||
|
-- İlgi puanı ile
|
||||||
|
SELECT title, relevance()
|
||||||
|
FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('Nim language')
|
||||||
|
ORDER BY relevance() DESC;
|
||||||
|
|
||||||
|
-- Boolean modu
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title, body) AGAINST('+Nim -Python' IN BOOLEAN MODE);
|
||||||
|
|
||||||
|
-- Fuzzy arama
|
||||||
|
SELECT * FROM articles
|
||||||
|
WHERE MATCH(title) AGAINST('programing' WITH FUZZINESS 2);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Kullanıcı Tanımlı Fonksiyonlar
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- UDF kaydet
|
||||||
|
CREATE FUNCTION greet(name str) -> str {
|
||||||
|
RETURN 'Hello, ' || name || '!';
|
||||||
|
};
|
||||||
|
|
||||||
|
-- Kullan
|
||||||
|
SELECT greet(name) FROM users;
|
||||||
|
|
||||||
|
-- Dahili fonksiyonlar
|
||||||
|
SELECT abs(-5), sqrt(16), lower('HELLO'), len('test');
|
||||||
|
```
|
||||||
|
|
||||||
|
## Sorgu İpuçları
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- İndeks kullanımını zorla
|
||||||
|
SELECT /*+ USE_INDEX(idx_users_age) */ * FROM users WHERE age > 18;
|
||||||
|
|
||||||
|
-- Approximate vektör araması zorla
|
||||||
|
SELECT /*+ APPROXIMATE */ * FROM vectors
|
||||||
|
ORDER BY cosine_distance(embedding, [...])
|
||||||
|
LIMIT 10;
|
||||||
|
|
||||||
|
-- Paralel çalıştırma
|
||||||
|
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||||
|
```
|
||||||
|
|
||||||
|
## Pencere Fonksiyonları
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Sıralama fonksiyonları
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
department,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||||
|
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||||
|
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Değer fonksiyonları
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
salary,
|
||||||
|
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||||
|
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||||
|
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||||
|
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- Dağılım fonksiyonları
|
||||||
|
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Çerçeve Spesifikasyonları
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ROWS çerçevesi
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
|
||||||
|
-- RANGE çerçevesi
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Çok Kiracılı ERP
|
||||||
|
|
||||||
|
BaraDB, **Satır Düzeyinde Güvenlik (RLS)** ve **oturum değişkenlerini** birleştirerek tek bir veritabanı örneğinde birden fazla şirketi (kiracı) çalıştırmayı destekler.
|
||||||
|
|
||||||
|
### Oturum Değişkenleri
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET app.tenant_id = 'company-123';
|
||||||
|
SELECT current_setting('app.tenant_id') AS tenant;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Mevcut Kullanıcı / Rol
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT current_user AS me, current_role AS my_role;
|
||||||
|
```
|
||||||
|
|
||||||
|
### RLS Kiracı İzolasyonu
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Tabloda RLS etkinleştir
|
||||||
|
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
|
||||||
|
|
||||||
|
-- Kiracıya göre filtreleme ilkesi oluştur
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- Her oturum yalnızca kendi verilerini görür
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- yalnızca company-a satırları
|
||||||
|
```
|
||||||
|
|
||||||
|
### Neden Çok Kiracılı?
|
||||||
|
|
||||||
|
- **Bir örnek, çok kiracı** — 100 ayrı veritabanı çalıştırmaya gerek yok
|
||||||
|
- **JSONB belgeleri** — Esnek şema depolama, her kiracı için kolayca alan ekleme
|
||||||
|
- **RLS izolasyonu garanti eder** — Veritabanı kiracı sınırlarını uygular, yalnızca uygulama değil
|
||||||
|
|
||||||
|
## Desteklenen Anahtar Kelimeler
|
||||||
|
|
||||||
|
| Kategori | Anahtar Kelimeler |
|
||||||
|
|----------|----------|
|
||||||
|
| DQL | SELECT, FROM, WHERE, ORDER BY, GROUP BY, HAVING, LIMIT, OFFSET, DISTINCT |
|
||||||
|
| DML | INSERT, UPDATE, DELETE, SET, VALUES |
|
||||||
|
| DDL | CREATE TYPE, DROP TYPE, CREATE INDEX, DROP INDEX, ALTER TYPE |
|
||||||
|
| Join | INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, CROSS JOIN, ON |
|
||||||
|
| Set | UNION, UNION ALL, INTERSECT, EXCEPT |
|
||||||
|
| CTEs | WITH, RECURSIVE, AS |
|
||||||
|
| Case | CASE, WHEN, THEN, ELSE, END |
|
||||||
|
| Transaction | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
||||||
|
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
||||||
|
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||||
|
| Vector | cosine_distance, euclidean_distance, inner_product, l1_distance, l2_distance, <-> |
|
||||||
|
| JSON | ->, ->> |
|
||||||
|
| FTS | @@ (BM25 eşleşme) |
|
||||||
|
| Recovery | RECOVER TO TIMESTAMP |
|
||||||
|
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id, current_setting |
|
||||||
|
| Session | SET, current_setting, current_user, current_role |
|
||||||
|
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||||
|
| Window Functions | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
## Hızlı Başlangıç
|
## Hızlı Başlangıç
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
|
||||||
docker build -t baradb:latest .
|
docker build -t baradb:latest .
|
||||||
|
|||||||
+247
-1
@@ -22,38 +22,284 @@
|
|||||||
### Linux
|
### Linux
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Resmi kurulum scripti
|
||||||
curl https://nim-lang.org/choosenim/init.sh -sSf | sh
|
curl https://nim-lang.org/choosenim/init.sh -sSf | sh
|
||||||
|
|
||||||
|
# Ubuntu/Debian
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install nim
|
||||||
|
|
||||||
|
# Fedora
|
||||||
|
sudo dnf install nim
|
||||||
|
|
||||||
|
# Arch Linux
|
||||||
|
sudo pacman -S nim
|
||||||
```
|
```
|
||||||
|
|
||||||
### macOS
|
### macOS
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Homebrew
|
||||||
brew install nim
|
brew install nim
|
||||||
|
|
||||||
|
# MacPorts
|
||||||
|
sudo port install nim
|
||||||
```
|
```
|
||||||
|
|
||||||
### Windows
|
### Windows
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
|
# choosenim ile
|
||||||
|
curl.exe -A "MSYS2_$(uname -m)" -L https://nim-lang.org/choosenim/init.ps1 | powershell -
|
||||||
|
|
||||||
|
# winget ile
|
||||||
winget install nim
|
winget install nim
|
||||||
|
|
||||||
|
# scoop ile
|
||||||
|
scoop install nim
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Kurulumu Doğrulama
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim --version
|
||||||
|
# Beklenen: Nim Compiler Version 2.2.0 veya daha yeni
|
||||||
|
```
|
||||||
|
|
||||||
|
## OpenSSL Kurulumu
|
||||||
|
|
||||||
|
### Linux
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Ubuntu/Debian
|
||||||
|
sudo apt-get install libssl-dev
|
||||||
|
|
||||||
|
# Fedora
|
||||||
|
sudo dnf install openssl-devel
|
||||||
|
|
||||||
|
# Arch Linux
|
||||||
|
sudo pacman -S openssl
|
||||||
|
```
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
|
||||||
|
OpenSSL sistemle birlikte gelir. Gerekirse:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
brew install openssl
|
||||||
|
```
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
OpenSSL, Nim Windows dağıtımıyla birlikte gelir. Manuel derlemeler için [slproweb.com](https://slproweb.com/products/Win32OpenSSL.html) adresinden indirin.
|
||||||
|
|
||||||
## BaraDB Derleme
|
## BaraDB Derleme
|
||||||
|
|
||||||
|
### Depoyu Klonlama
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
```
|
||||||
|
|
||||||
|
### Bağımlılıkları Kurma
|
||||||
|
|
||||||
|
```bash
|
||||||
nimble install -d -y
|
nimble install -d -y
|
||||||
|
```
|
||||||
|
|
||||||
|
### Derleme Seçenekleri
|
||||||
|
|
||||||
|
#### Debug Derleme
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Release Derleme (Önerilen)
|
||||||
|
|
||||||
|
```bash
|
||||||
nim c -d:ssl -d:release --opt:speed -o:build/baradadb src/baradadb.nim
|
nim c -d:ssl -d:release --opt:speed -o:build/baradadb src/baradadb.nim
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Nimble Tasks Kullanma
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Debug derleme
|
||||||
|
nimble build_debug
|
||||||
|
|
||||||
|
# Release derleme
|
||||||
|
nimble build_release
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Binary Boyutunu Küçültme
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release --opt:size -o:build/baradadb src/baradadb.nim
|
||||||
|
strip build/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Derlemeyi Doğrulama
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./build/baradadb --version
|
||||||
|
# Beklenen: BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testleri Çalıştırma
|
||||||
|
|
||||||
|
### Tüm Testler
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -r tests/test_all.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Belirli Test Süitleri
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Depolama testleri
|
||||||
|
nim c -d:ssl -r tests/test_storage.nim
|
||||||
|
|
||||||
|
# Sorgu motoru testleri
|
||||||
|
nim c -d:ssl -r tests/test_query.nim
|
||||||
|
|
||||||
|
# Protokol testleri
|
||||||
|
nim c -d:ssl -r tests/test_protocol.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Benchmarklar
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release -r benchmarks/bench_all.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
## Kurulum Seçenekleri
|
||||||
|
|
||||||
|
### Sistem Geneli Kurulum
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Release binary derle
|
||||||
|
nimble build_release
|
||||||
|
|
||||||
|
# /usr/local/bin'e kur
|
||||||
|
sudo cp build/baradadb /usr/local/bin/
|
||||||
|
sudo chmod +x /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# Veri dizini oluştur
|
||||||
|
sudo mkdir -p /var/lib/baradb
|
||||||
|
sudo chmod 755 /var/lib/baradb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Ön Derlenmiş Binary
|
||||||
|
|
||||||
|
En son sürümü platformunuz için indirin:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Linux x86_64
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-linux-amd64
|
||||||
|
chmod +x baradadb-linux-amd64
|
||||||
|
mv baradadb-linux-amd64 /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# Linux ARM64
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-linux-arm64
|
||||||
|
chmod +x baradadb-linux-arm64
|
||||||
|
mv baradadb-linux-arm64 /usr/local/bin/baradadb
|
||||||
|
|
||||||
|
# macOS
|
||||||
|
wget https://github.com/katehonz/barabaDB/releases/latest/download/baradadb-darwin-amd64
|
||||||
|
chmod +x baradadb-darwin-amd64
|
||||||
|
mv baradadb-darwin-amd64 /usr/local/bin/baradadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Resmi image'i çek
|
||||||
|
docker pull barabadb/barabadb:latest
|
||||||
|
|
||||||
|
# Çalıştır
|
||||||
|
docker run -d \
|
||||||
|
-p 9472:9472 \
|
||||||
|
-p 9470:9470 \
|
||||||
|
-p 9471:9471 \
|
||||||
|
-v baradb_data:/data \
|
||||||
|
barabadb/barabadb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Compose
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker-compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Gömülü Kullanım (Nim Projeleri)
|
||||||
|
|
||||||
|
`.nimble` dosyanıza ekleyin:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
requires "barabadb >= 1.1.0"
|
||||||
|
```
|
||||||
|
|
||||||
|
Kodunuzda kullanın:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
import barabadb/storage/lsm
|
||||||
|
|
||||||
|
var db = newLSMTree("./data")
|
||||||
|
db.put("key", cast[seq[byte]]("value"))
|
||||||
|
let (found, val) = db.get("key")
|
||||||
|
db.close()
|
||||||
|
```
|
||||||
|
|
||||||
## İlk Çalıştırma
|
## İlk Çalıştırma
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Sunucuyu başlat
|
||||||
./build/baradadb
|
./build/baradadb
|
||||||
|
|
||||||
|
# Beklenen çıktı:
|
||||||
|
# BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
# BaraDB TCP listening on 127.0.0.1:9472
|
||||||
|
|
||||||
|
# HTTP API ile test et
|
||||||
curl http://localhost:9470/health
|
curl http://localhost:9470/health
|
||||||
|
|
||||||
|
# İnteraktif shell
|
||||||
./build/baradadb --shell
|
./build/baradadb --shell
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Kurulum Sorunlarını Giderme
|
||||||
|
|
||||||
|
### "cannot open file: hunos"
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nimble install -d -y
|
||||||
|
```
|
||||||
|
|
||||||
|
### "BaraDB requires SSL support"
|
||||||
|
|
||||||
|
Her zaman `-d:ssl` ile derleyin:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Yavaş derleme
|
||||||
|
|
||||||
|
Paralel derleme kullanın:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release --parallelBuild:4 -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
|
### Büyük binary boyutu
|
||||||
|
|
||||||
|
Boyut optimizasyonu kullanın:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nim c -d:ssl -d:release --opt:size --passL:-s -o:build/baradadb src/baradadb.nim
|
||||||
|
```
|
||||||
|
|
||||||
## Sonraki Adımlar
|
## Sonraki Adımlar
|
||||||
|
|
||||||
- [Hızlı Başlangıç](quickstart.md)
|
- [Hızlı Başlangıç](quickstart.md)
|
||||||
|
|||||||
+106
-1
@@ -489,4 +489,109 @@ LIMIT 10;
|
|||||||
|
|
||||||
-- 并行执行
|
-- 并行执行
|
||||||
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## 窗口函数
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- 排名函数
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
department,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||||
|
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||||
|
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- 值函数
|
||||||
|
SELECT
|
||||||
|
name,
|
||||||
|
salary,
|
||||||
|
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||||
|
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||||
|
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||||
|
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||||
|
FROM employees;
|
||||||
|
|
||||||
|
-- 分布函数
|
||||||
|
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||||
|
```
|
||||||
|
|
||||||
|
### 帧规格
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- ROWS 帧
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
|
||||||
|
-- RANGE 帧
|
||||||
|
SUM(salary) OVER (
|
||||||
|
PARTITION BY department
|
||||||
|
ORDER BY hire_date
|
||||||
|
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 多租户 ERP
|
||||||
|
|
||||||
|
BaraDB 支持在单个数据库实例中运行多个公司(租户),使用**行级安全性(RLS)**结合**会话变量**。
|
||||||
|
|
||||||
|
### 会话变量
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SET app.tenant_id = 'company-123';
|
||||||
|
SELECT current_setting('app.tenant_id') AS tenant;
|
||||||
|
```
|
||||||
|
|
||||||
|
### 当前用户 / 角色
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SELECT current_user AS me, current_role AS my_role;
|
||||||
|
```
|
||||||
|
|
||||||
|
### RLS 租户隔离
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- 在表上启用 RLS
|
||||||
|
ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;
|
||||||
|
|
||||||
|
-- 创建按租户过滤的策略
|
||||||
|
CREATE POLICY tenant_isolation ON invoices
|
||||||
|
FOR SELECT USING (tenant_id = current_setting('app.tenant_id'));
|
||||||
|
|
||||||
|
-- 每个会话只能看到自己的数据
|
||||||
|
SET app.tenant_id = 'company-a';
|
||||||
|
SELECT * FROM invoices; -- 仅限 company-a 的行
|
||||||
|
```
|
||||||
|
|
||||||
|
### 为什么选择多租户?
|
||||||
|
|
||||||
|
- **一个实例,多个租户** — 无需运行 100 个独立的数据库
|
||||||
|
- **JSONB 文档** — 模式灵活的存储,易于为每个租户添加字段
|
||||||
|
- **RLS 保证隔离** — 数据库强制执行租户边界,而不仅仅是应用程序
|
||||||
|
|
||||||
|
## 支持的关键字
|
||||||
|
|
||||||
|
| 类别 | 关键字 |
|
||||||
|
|----------|----------|
|
||||||
|
| DQL | SELECT, FROM, WHERE, ORDER BY, GROUP BY, HAVING, LIMIT, OFFSET, DISTINCT |
|
||||||
|
| DML | INSERT, UPDATE, DELETE, SET, VALUES |
|
||||||
|
| DDL | CREATE TYPE, DROP TYPE, CREATE INDEX, DROP INDEX, ALTER TYPE |
|
||||||
|
| Join | INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN, CROSS JOIN, ON |
|
||||||
|
| Set | UNION, UNION ALL, INTERSECT, EXCEPT |
|
||||||
|
| CTEs | WITH, RECURSIVE, AS |
|
||||||
|
| Case | CASE, WHEN, THEN, ELSE, END |
|
||||||
|
| Transaction | BEGIN, COMMIT, ROLLBACK, SAVEPOINT |
|
||||||
|
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
||||||
|
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||||
|
| Vector | cosine_distance, euclidean_distance, inner_product, l1_distance, l2_distance, <-> |
|
||||||
|
| JSON | ->, ->> |
|
||||||
|
| FTS | @@ (BM25 match) |
|
||||||
|
| Recovery | RECOVER TO TIMESTAMP |
|
||||||
|
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id, current_setting |
|
||||||
|
| Session | SET, current_setting, current_user, current_role |
|
||||||
|
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||||
|
| Window Functions | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||||
@@ -156,7 +156,7 @@ raft_peers = "node2:9001,node3:9001"
|
|||||||
```
|
```
|
||||||
|
|
||||||
```
|
```
|
||||||
BaraDB v0.1.0 — Multimodal Database Engine
|
BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
baradadb [options]
|
baradadb [options]
|
||||||
|
|||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
## 快速开始
|
## 快速开始
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
|
|
||||||
docker build -t baradb:latest .
|
docker build -t baradb:latest .
|
||||||
|
|||||||
@@ -99,7 +99,7 @@ OpenSSL 已捆绑在 Nim Windows 分发版中。如需手动构建,
|
|||||||
### 克隆仓库
|
### 克隆仓库
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/katehonz/barabaDB.git
|
git clone https://codeberg.org/baraba/baradb
|
||||||
cd barabaDB
|
cd barabaDB
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -144,7 +144,7 @@ strip build/baradadb
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/baradadb --version
|
./build/baradadb --version
|
||||||
# 预期输出: BaraDB v0.1.0 — Multimodal Database Engine
|
# 预期输出: BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
```
|
```
|
||||||
|
|
||||||
## 运行测试
|
## 运行测试
|
||||||
@@ -259,7 +259,7 @@ db.close()
|
|||||||
./build/baradadb
|
./build/baradadb
|
||||||
|
|
||||||
# 预期输出:
|
# 预期输出:
|
||||||
# BaraDB v0.1.0 — Multimodal Database Engine
|
# BaraDB v1.1.0 — Multimodal Database Engine
|
||||||
# BaraDB TCP listening on 127.0.0.1:9472
|
# BaraDB TCP listening on 127.0.0.1:9472
|
||||||
|
|
||||||
# 使用 HTTP API 测试
|
# 使用 HTTP API 测试
|
||||||
|
|||||||
Executable
+58
@@ -0,0 +1,58 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# BaraDB — Client Integration Test Runner
|
||||||
|
# Spins up the BaraDB server in Docker and runs all client test suites sequentially.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# ./scripts/test-clients.sh
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
COMPOSE="docker compose -f docker-compose.test.yml"
|
||||||
|
|
||||||
|
echo "=== Building BaraDB server image ==="
|
||||||
|
$COMPOSE build baradb
|
||||||
|
|
||||||
|
echo "=== Starting BaraDB server ==="
|
||||||
|
$COMPOSE up -d baradb
|
||||||
|
|
||||||
|
# Wait for server healthcheck
|
||||||
|
echo "=== Waiting for server to be healthy ==="
|
||||||
|
$COMPOSE ps baradb --format json 2>/dev/null | grep -q '"Health":"healthy"' || sleep 10
|
||||||
|
|
||||||
|
# Give a little extra time for the wire protocol port to be ready
|
||||||
|
sleep 3
|
||||||
|
|
||||||
|
run_test() {
|
||||||
|
local service=$1
|
||||||
|
echo ""
|
||||||
|
echo "========================================"
|
||||||
|
echo "=== Running $service tests ==="
|
||||||
|
echo "========================================"
|
||||||
|
if $COMPOSE run --rm "$service"; then
|
||||||
|
echo "✅ $service tests PASSED"
|
||||||
|
else
|
||||||
|
echo "❌ $service tests FAILED"
|
||||||
|
EXIT_CODE=1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
EXIT_CODE=0
|
||||||
|
|
||||||
|
run_test test-python
|
||||||
|
run_test test-javascript
|
||||||
|
run_test test-nim
|
||||||
|
run_test test-rust
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "========================================"
|
||||||
|
if [ "$EXIT_CODE" -eq 0 ]; then
|
||||||
|
echo "🎉 All client tests passed!"
|
||||||
|
else
|
||||||
|
echo "⚠️ Some client tests failed."
|
||||||
|
fi
|
||||||
|
echo "========================================"
|
||||||
|
|
||||||
|
echo "=== Stopping BaraDB server ==="
|
||||||
|
$COMPOSE down -v
|
||||||
|
|
||||||
|
exit $EXIT_CODE
|
||||||
@@ -4,6 +4,7 @@ import std/random
|
|||||||
import std/monotimes
|
import std/monotimes
|
||||||
import std/asyncdispatch
|
import std/asyncdispatch
|
||||||
import std/net
|
import std/net
|
||||||
|
import std/asyncnet
|
||||||
import std/strutils
|
import std/strutils
|
||||||
import std/streams
|
import std/streams
|
||||||
import std/os
|
import std/os
|
||||||
@@ -36,7 +37,7 @@ type
|
|||||||
fanout*: int # number of nodes to gossip to per round
|
fanout*: int # number of nodes to gossip to per round
|
||||||
gossipPort*: int
|
gossipPort*: int
|
||||||
running*: bool
|
running*: bool
|
||||||
sock*: Socket
|
sock*: AsyncSocket
|
||||||
onJoin*: proc(node: GossipNode) {.gcsafe.}
|
onJoin*: proc(node: GossipNode) {.gcsafe.}
|
||||||
onLeave*: proc(nodeId: string) {.gcsafe.}
|
onLeave*: proc(nodeId: string) {.gcsafe.}
|
||||||
onSuspect*: proc(nodeId: string) {.gcsafe.}
|
onSuspect*: proc(nodeId: string) {.gcsafe.}
|
||||||
@@ -305,25 +306,21 @@ proc startGossipRound*(gp: GossipProtocol, intervalMs: int = 2000) {.async.} =
|
|||||||
gp.broadcastGossip()
|
gp.broadcastGossip()
|
||||||
|
|
||||||
proc startGossipListener*(gp: GossipProtocol) {.async.} =
|
proc startGossipListener*(gp: GossipProtocol) {.async.} =
|
||||||
gp.sock = newSocket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
|
gp.sock = newAsyncSocket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
|
||||||
gp.sock.setSockOpt(OptReuseAddr, true)
|
gp.sock.setSockOpt(OptReuseAddr, true)
|
||||||
gp.sock.bindAddr(Port(gp.gossipPort))
|
gp.sock.bindAddr(Port(gp.gossipPort))
|
||||||
gp.running = true
|
gp.running = true
|
||||||
while gp.running:
|
while gp.running:
|
||||||
try:
|
try:
|
||||||
var data = newString(65535)
|
let (data, senderAddr, senderPort) = await gp.sock.recvFrom(65535)
|
||||||
var senderAddr = ""
|
if data.len > 0:
|
||||||
var senderPort: Port
|
|
||||||
let bytesRead = gp.sock.recvFrom(data, 65535, senderAddr, senderPort)
|
|
||||||
if bytesRead > 0:
|
|
||||||
data.setLen(bytesRead)
|
|
||||||
gp.handleIncomingGossip(data, senderAddr)
|
gp.handleIncomingGossip(data, senderAddr)
|
||||||
except:
|
except:
|
||||||
# Small sleep on error to avoid spin
|
# Small sleep on error to avoid spin
|
||||||
gp.sock.close()
|
gp.sock.close()
|
||||||
# Recreate socket for next iteration
|
# Recreate socket for next iteration
|
||||||
try:
|
try:
|
||||||
gp.sock = newSocket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
|
gp.sock = newAsyncSocket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
|
||||||
gp.sock.setSockOpt(OptReuseAddr, true)
|
gp.sock.setSockOpt(OptReuseAddr, true)
|
||||||
gp.sock.bindAddr(Port(gp.gossipPort))
|
gp.sock.bindAddr(Port(gp.gossipPort))
|
||||||
except:
|
except:
|
||||||
|
|||||||
@@ -108,6 +108,8 @@ proc queryHandler(server: HttpServer): RequestHandler =
|
|||||||
let ctx = newContext(request)
|
let ctx = newContext(request)
|
||||||
server.metrics.queriesTotal += 1
|
server.metrics.queriesTotal += 1
|
||||||
|
|
||||||
|
var userId = ""
|
||||||
|
var role = ""
|
||||||
# Auth check
|
# Auth check
|
||||||
if server.config.authEnabled:
|
if server.config.authEnabled:
|
||||||
let authHeader = request.headers["Authorization"]
|
let authHeader = request.headers["Authorization"]
|
||||||
@@ -115,10 +117,12 @@ proc queryHandler(server: HttpServer): RequestHandler =
|
|||||||
ctx.json(%*{"error": "Unauthorized"}, 401)
|
ctx.json(%*{"error": "Unauthorized"}, 401)
|
||||||
return
|
return
|
||||||
let tokenStr = authHeader[7..^1]
|
let tokenStr = authHeader[7..^1]
|
||||||
let (valid, userId, role) = server.verifyToken(tokenStr)
|
let (valid, uid, r) = server.verifyToken(tokenStr)
|
||||||
if not valid:
|
if not valid:
|
||||||
ctx.json(%*{"error": "Unauthorized"}, 401)
|
ctx.json(%*{"error": "Unauthorized"}, 401)
|
||||||
return
|
return
|
||||||
|
userId = uid
|
||||||
|
role = r
|
||||||
|
|
||||||
let body = parseJson(request.body)
|
let body = parseJson(request.body)
|
||||||
if body == nil or "query" notin body:
|
if body == nil or "query" notin body:
|
||||||
@@ -131,6 +135,8 @@ proc queryHandler(server: HttpServer): RequestHandler =
|
|||||||
return
|
return
|
||||||
|
|
||||||
var reqCtx = cloneForConnection(server.ctx)
|
var reqCtx = cloneForConnection(server.ctx)
|
||||||
|
reqCtx.currentUser = userId
|
||||||
|
reqCtx.currentRole = role
|
||||||
let tokens = tokenize(queryStr)
|
let tokens = tokenize(queryStr)
|
||||||
let astNode = parse(tokens)
|
let astNode = parse(tokens)
|
||||||
|
|
||||||
|
|||||||
@@ -436,9 +436,11 @@ proc handleClient(server: Server, client: AsyncSocket, clientId: int) {.async.}
|
|||||||
case header.kind
|
case header.kind
|
||||||
of mkAuth:
|
of mkAuth:
|
||||||
let tokenStr = parseAuthMessage(cast[seq[byte]](payload))
|
let tokenStr = parseAuthMessage(cast[seq[byte]](payload))
|
||||||
let (valid, userId, _) = verifyToken(secret, tokenStr)
|
let (valid, userId, role) = verifyToken(secret, tokenStr)
|
||||||
if valid:
|
if valid:
|
||||||
authenticated = true
|
authenticated = true
|
||||||
|
connCtx.currentUser = userId
|
||||||
|
connCtx.currentRole = role
|
||||||
let okMsg = makeAuthOkMessage(header.requestId)
|
let okMsg = makeAuthOkMessage(header.requestId)
|
||||||
await client.send(cast[string](okMsg))
|
await client.send(cast[string](okMsg))
|
||||||
info("Client " & $clientId & " authenticated as " & userId)
|
info("Client " & $clientId & " authenticated as " & userId)
|
||||||
|
|||||||
@@ -169,12 +169,14 @@ proc serializeValue*(buf: var seq[byte], val: WireValue) =
|
|||||||
of fkFloat32:
|
of fkFloat32:
|
||||||
var fl = val.float32Val
|
var fl = val.float32Val
|
||||||
var bytes: array[4, byte]
|
var bytes: array[4, byte]
|
||||||
copyMem(addr bytes, unsafeAddr fl, 4)
|
var i32 = cast[int32](fl)
|
||||||
|
bigEndian32(addr bytes, unsafeAddr i32)
|
||||||
buf.add(bytes)
|
buf.add(bytes)
|
||||||
of fkFloat64:
|
of fkFloat64:
|
||||||
var fl = val.float64Val
|
var fl = val.float64Val
|
||||||
var bytes: array[8, byte]
|
var bytes: array[8, byte]
|
||||||
copyMem(addr bytes, unsafeAddr fl, 8)
|
var i64 = cast[int64](fl)
|
||||||
|
bigEndian64(addr bytes, unsafeAddr i64)
|
||||||
buf.add(bytes)
|
buf.add(bytes)
|
||||||
of fkString: buf.writeString(val.strVal)
|
of fkString: buf.writeString(val.strVal)
|
||||||
of fkBytes: buf.writeBytes(val.bytesVal)
|
of fkBytes: buf.writeBytes(val.bytesVal)
|
||||||
@@ -192,7 +194,8 @@ proc serializeValue*(buf: var seq[byte], val: WireValue) =
|
|||||||
for f in val.vecVal:
|
for f in val.vecVal:
|
||||||
var fl = f
|
var fl = f
|
||||||
var bytes: array[4, byte]
|
var bytes: array[4, byte]
|
||||||
copyMem(addr bytes, unsafeAddr fl, 4)
|
var i32 = cast[int32](fl)
|
||||||
|
bigEndian32(addr bytes, unsafeAddr i32)
|
||||||
buf.add(bytes)
|
buf.add(bytes)
|
||||||
of fkJson:
|
of fkJson:
|
||||||
buf.writeString(val.jsonVal)
|
buf.writeString(val.jsonVal)
|
||||||
@@ -227,14 +230,18 @@ proc deserializeValue*(buf: openArray[byte], pos: var int, depth: int = 0): Wire
|
|||||||
var fl: float32
|
var fl: float32
|
||||||
var bytes: array[4, byte]
|
var bytes: array[4, byte]
|
||||||
for i in 0..3: bytes[i] = buf[pos + i]
|
for i in 0..3: bytes[i] = buf[pos + i]
|
||||||
copyMem(addr fl, unsafeAddr bytes, 4)
|
var i32: int32
|
||||||
|
bigEndian32(addr i32, unsafeAddr bytes)
|
||||||
|
fl = cast[float32](i32)
|
||||||
pos += 4
|
pos += 4
|
||||||
result = WireValue(kind: fkFloat32, float32Val: fl)
|
result = WireValue(kind: fkFloat32, float32Val: fl)
|
||||||
of fkFloat64:
|
of fkFloat64:
|
||||||
var fl: float64
|
var fl: float64
|
||||||
var bytes: array[8, byte]
|
var bytes: array[8, byte]
|
||||||
for i in 0..7: bytes[i] = buf[pos + i]
|
for i in 0..7: bytes[i] = buf[pos + i]
|
||||||
copyMem(addr fl, unsafeAddr bytes, 8)
|
var i64: int64
|
||||||
|
bigEndian64(addr i64, unsafeAddr bytes)
|
||||||
|
fl = cast[float64](i64)
|
||||||
pos += 8
|
pos += 8
|
||||||
result = WireValue(kind: fkFloat64, float64Val: fl)
|
result = WireValue(kind: fkFloat64, float64Val: fl)
|
||||||
of fkString:
|
of fkString:
|
||||||
@@ -270,7 +277,9 @@ proc deserializeValue*(buf: openArray[byte], pos: var int, depth: int = 0): Wire
|
|||||||
var fl: float32
|
var fl: float32
|
||||||
var bytes: array[4, byte]
|
var bytes: array[4, byte]
|
||||||
for j in 0..3: bytes[j] = buf[pos + j]
|
for j in 0..3: bytes[j] = buf[pos + j]
|
||||||
copyMem(addr fl, unsafeAddr bytes, 4)
|
var i32: int32
|
||||||
|
bigEndian32(addr i32, unsafeAddr bytes)
|
||||||
|
fl = cast[float32](i32)
|
||||||
pos += 4
|
pos += 4
|
||||||
vec.add(fl)
|
vec.add(fl)
|
||||||
result = WireValue(kind: fkVector, vecVal: vec)
|
result = WireValue(kind: fkVector, vecVal: vec)
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ type
|
|||||||
nkInsert
|
nkInsert
|
||||||
nkUpdate
|
nkUpdate
|
||||||
nkDelete
|
nkDelete
|
||||||
|
nkMerge
|
||||||
nkCreateType
|
nkCreateType
|
||||||
nkDropType
|
nkDropType
|
||||||
nkAlterType
|
nkAlterType
|
||||||
@@ -39,6 +40,7 @@ type
|
|||||||
nkDisableRLS
|
nkDisableRLS
|
||||||
nkGrant
|
nkGrant
|
||||||
nkRevoke
|
nkRevoke
|
||||||
|
nkSetVar
|
||||||
|
|
||||||
# Clauses
|
# Clauses
|
||||||
nkFrom
|
nkFrom
|
||||||
@@ -77,6 +79,8 @@ type
|
|||||||
nkIsExpr
|
nkIsExpr
|
||||||
nkStar
|
nkStar
|
||||||
nkPlaceholder
|
nkPlaceholder
|
||||||
|
nkCurrentUser
|
||||||
|
nkCurrentRole
|
||||||
|
|
||||||
# Graph-specific
|
# Graph-specific
|
||||||
nkGraphTraversal
|
nkGraphTraversal
|
||||||
@@ -89,6 +93,10 @@ type
|
|||||||
nkVectorSimilar
|
nkVectorSimilar
|
||||||
nkVectorNearest
|
nkVectorNearest
|
||||||
|
|
||||||
|
# Pivot
|
||||||
|
nkPivot
|
||||||
|
nkUnpivot
|
||||||
|
|
||||||
# Set operations
|
# Set operations
|
||||||
nkSetOp
|
nkSetOp
|
||||||
|
|
||||||
@@ -102,6 +110,12 @@ type
|
|||||||
nkColumnDef
|
nkColumnDef
|
||||||
nkConstraintDef
|
nkConstraintDef
|
||||||
|
|
||||||
|
# Window functions
|
||||||
|
nkWindowExpr
|
||||||
|
nkOverClause
|
||||||
|
nkFrameSpec
|
||||||
|
nkFrameBoundary
|
||||||
|
|
||||||
# Top-level
|
# Top-level
|
||||||
nkStatementList
|
nkStatementList
|
||||||
|
|
||||||
@@ -132,6 +146,11 @@ type
|
|||||||
bkJsonPath = "->"
|
bkJsonPath = "->"
|
||||||
bkJsonPathText = "->>"
|
bkJsonPathText = "->>"
|
||||||
bkFtsMatch = "@@"
|
bkFtsMatch = "@@"
|
||||||
|
bkDistance = "<->"
|
||||||
|
bkJsonContains = "@>"
|
||||||
|
bkJsonContainedBy = "<@"
|
||||||
|
bkJsonHasAny = "?|"
|
||||||
|
bkJsonHasAll = "?&"
|
||||||
|
|
||||||
UnaryOpKind* = enum
|
UnaryOpKind* = enum
|
||||||
ukNeg = "-"
|
ukNeg = "-"
|
||||||
@@ -151,6 +170,12 @@ type
|
|||||||
sdkIntersect
|
sdkIntersect
|
||||||
sdkExcept
|
sdkExcept
|
||||||
|
|
||||||
|
GroupingSetsKind* = enum
|
||||||
|
gskNone
|
||||||
|
gskGroupingSets
|
||||||
|
gskRollup
|
||||||
|
gskCube
|
||||||
|
|
||||||
SortDir* = enum
|
SortDir* = enum
|
||||||
sdAsc
|
sdAsc
|
||||||
sdDesc
|
sdDesc
|
||||||
@@ -168,6 +193,8 @@ type
|
|||||||
selJoins*: seq[Node]
|
selJoins*: seq[Node]
|
||||||
selWhere*: Node
|
selWhere*: Node
|
||||||
selGroupBy*: seq[Node]
|
selGroupBy*: seq[Node]
|
||||||
|
selGroupingSetsKind*: GroupingSetsKind
|
||||||
|
selGroupingSets*: seq[seq[Node]]
|
||||||
selHaving*: Node
|
selHaving*: Node
|
||||||
selOrderBy*: seq[Node]
|
selOrderBy*: seq[Node]
|
||||||
selLimit*: Node
|
selLimit*: Node
|
||||||
@@ -189,6 +216,15 @@ type
|
|||||||
delAlias*: string
|
delAlias*: string
|
||||||
delWhere*: Node
|
delWhere*: Node
|
||||||
delReturning*: seq[Node]
|
delReturning*: seq[Node]
|
||||||
|
of nkMerge:
|
||||||
|
mergeTarget*: string
|
||||||
|
mergeTargetAlias*: string
|
||||||
|
mergeSource*: Node
|
||||||
|
mergeSourceAlias*: string
|
||||||
|
mergeOn*: Node
|
||||||
|
mergeMatchedUpdate*: seq[Node] # SET assignments, empty if no UPDATE
|
||||||
|
mergeNotMatchedInsert*: seq[Node] # column list for INSERT
|
||||||
|
mergeNotMatchedValues*: seq[Node] # value expressions for INSERT
|
||||||
of nkCreateType:
|
of nkCreateType:
|
||||||
ctName*: string
|
ctName*: string
|
||||||
ctBases*: seq[string]
|
ctBases*: seq[string]
|
||||||
@@ -288,6 +324,9 @@ type
|
|||||||
rvPrivilege*: string
|
rvPrivilege*: string
|
||||||
rvTable*: string
|
rvTable*: string
|
||||||
rvGrantee*: string
|
rvGrantee*: string
|
||||||
|
of nkSetVar:
|
||||||
|
svName*: string
|
||||||
|
svValue*: string
|
||||||
of nkApplyMigration:
|
of nkApplyMigration:
|
||||||
amName*: string
|
amName*: string
|
||||||
of nkMigrationStatus:
|
of nkMigrationStatus:
|
||||||
@@ -309,6 +348,7 @@ type
|
|||||||
of nkFrom:
|
of nkFrom:
|
||||||
fromTable*: string
|
fromTable*: string
|
||||||
fromAlias*: string
|
fromAlias*: string
|
||||||
|
fromSubquery*: Node
|
||||||
of nkWhere:
|
of nkWhere:
|
||||||
whereExpr*: Node
|
whereExpr*: Node
|
||||||
of nkOrderBy:
|
of nkOrderBy:
|
||||||
@@ -340,6 +380,7 @@ type
|
|||||||
of nkFuncCall:
|
of nkFuncCall:
|
||||||
funcName*: string
|
funcName*: string
|
||||||
funcArgs*: seq[Node]
|
funcArgs*: seq[Node]
|
||||||
|
funcFilter*: Node
|
||||||
of nkTypeCast:
|
of nkTypeCast:
|
||||||
castType*: string
|
castType*: string
|
||||||
castExpr*: Node
|
castExpr*: Node
|
||||||
@@ -391,11 +432,13 @@ type
|
|||||||
isType*: string
|
isType*: string
|
||||||
isNegated*: bool
|
isNegated*: bool
|
||||||
of nkGraphTraversal:
|
of nkGraphTraversal:
|
||||||
|
gtGraphName*: string
|
||||||
gtStart*: Node
|
gtStart*: Node
|
||||||
gtEdge*: string
|
gtEdge*: string
|
||||||
gtDirection*: string
|
gtDirection*: string
|
||||||
gtEnd*: Node
|
gtEnd*: Node
|
||||||
gtMaxDepth*: int
|
gtMaxDepth*: int
|
||||||
|
gtReturnCols*: seq[string]
|
||||||
of nkBfsQuery:
|
of nkBfsQuery:
|
||||||
bfsStart*: Node
|
bfsStart*: Node
|
||||||
bfsTarget*: Node
|
bfsTarget*: Node
|
||||||
@@ -426,11 +469,22 @@ type
|
|||||||
vnVector*: Node
|
vnVector*: Node
|
||||||
vnLimit*: int
|
vnLimit*: int
|
||||||
vnMetric*: string
|
vnMetric*: string
|
||||||
|
of nkPivot:
|
||||||
|
pivotSource*: Node
|
||||||
|
pivotAgg*: Node
|
||||||
|
pivotForCol*: string
|
||||||
|
pivotInValues*: seq[string]
|
||||||
|
of nkUnpivot:
|
||||||
|
unpivotSource*: Node
|
||||||
|
unpivotValueCol*: string
|
||||||
|
unpivotForCol*: string
|
||||||
|
unpivotInCols*: seq[string]
|
||||||
of nkJoin:
|
of nkJoin:
|
||||||
joinKind*: JoinKind
|
joinKind*: JoinKind
|
||||||
joinTarget*: Node
|
joinTarget*: Node
|
||||||
joinOn*: Node
|
joinOn*: Node
|
||||||
joinAlias*: string
|
joinAlias*: string
|
||||||
|
joinLateral*: bool
|
||||||
of nkSetOp:
|
of nkSetOp:
|
||||||
setOpKind*: SetOpKind
|
setOpKind*: SetOpKind
|
||||||
setOpAll*: bool
|
setOpAll*: bool
|
||||||
@@ -440,6 +494,10 @@ type
|
|||||||
discard
|
discard
|
||||||
of nkPlaceholder:
|
of nkPlaceholder:
|
||||||
discard
|
discard
|
||||||
|
of nkCurrentUser:
|
||||||
|
discard
|
||||||
|
of nkCurrentRole:
|
||||||
|
discard
|
||||||
of nkPropertyDef:
|
of nkPropertyDef:
|
||||||
pdName*: string
|
pdName*: string
|
||||||
pdType*: string
|
pdType*: string
|
||||||
@@ -456,6 +514,24 @@ type
|
|||||||
idName*: string
|
idName*: string
|
||||||
idExpr*: Node
|
idExpr*: Node
|
||||||
idKind*: IndexKind
|
idKind*: IndexKind
|
||||||
|
of nkWindowExpr:
|
||||||
|
winFunc*: string
|
||||||
|
winArgs*: seq[Node]
|
||||||
|
winOver*: Node
|
||||||
|
of nkOverClause:
|
||||||
|
overPartition*: seq[Node]
|
||||||
|
overOrderBy*: seq[Node]
|
||||||
|
overFrame*: Node
|
||||||
|
of nkFrameSpec:
|
||||||
|
frameMode*: string # "ROWS" or "RANGE"
|
||||||
|
frameStartType*: string # "UNBOUNDED PRECEDING", "CURRENT ROW", "EXPR PRECEDING", "EXPR FOLLOWING"
|
||||||
|
frameStartExpr*: Node
|
||||||
|
frameEndType*: string
|
||||||
|
frameEndExpr*: Node
|
||||||
|
frameExclude*: string # "NO OTHERS", "CURRENT ROW", "GROUP", "TIES"
|
||||||
|
of nkFrameBoundary:
|
||||||
|
fbType*: string # "UNBOUNDED PRECEDING", "CURRENT ROW", "EXPR PRECEDING", "EXPR FOLLOWING", "UNBOUNDED FOLLOWING"
|
||||||
|
fbExpr*: Node
|
||||||
of nkStatementList:
|
of nkStatementList:
|
||||||
stmts*: seq[Node]
|
stmts*: seq[Node]
|
||||||
|
|
||||||
@@ -470,5 +546,11 @@ proc newNode*(kind: NodeKind, line: int = 0, col: int = 0): Node =
|
|||||||
of nkAlterTable: result.altOps = @[]
|
of nkAlterTable: result.altOps = @[]
|
||||||
of nkColumnDef: result.cdConstraints = @[]
|
of nkColumnDef: result.cdConstraints = @[]
|
||||||
of nkConstraintDef: result.cstColumns = @[]; result.cstRefColumns = @[]
|
of nkConstraintDef: result.cstColumns = @[]; result.cstRefColumns = @[]
|
||||||
|
of nkWindowExpr: result.winArgs = @[]
|
||||||
|
of nkOverClause: result.overPartition = @[]; result.overOrderBy = @[]
|
||||||
|
of nkMerge:
|
||||||
|
result.mergeMatchedUpdate = @[]
|
||||||
|
result.mergeNotMatchedInsert = @[]
|
||||||
|
result.mergeNotMatchedValues = @[]
|
||||||
of nkStatementList: result.stmts = @[]
|
of nkStatementList: result.stmts = @[]
|
||||||
else: discard
|
else: discard
|
||||||
|
|||||||
@@ -194,6 +194,28 @@ proc codegenPlan*(plan: IRPlan): StorageOp =
|
|||||||
|
|
||||||
of irpkExplain:
|
of irpkExplain:
|
||||||
return codegenPlan(plan.explainPlan)
|
return codegenPlan(plan.explainPlan)
|
||||||
|
of irpkWindow:
|
||||||
|
let sourceOp = codegenPlan(plan.winSource)
|
||||||
|
let op = newStorageOp(sokProject)
|
||||||
|
for wf in plan.winFuncs:
|
||||||
|
op.columns.add(wf.wfName)
|
||||||
|
if sourceOp != nil:
|
||||||
|
op.children.add(sourceOp)
|
||||||
|
return op
|
||||||
|
of irpkMerge:
|
||||||
|
return newStorageOp(sokScan)
|
||||||
|
of irpkPivot:
|
||||||
|
let sourceOp = codegenPlan(plan.pivotSource)
|
||||||
|
let op = newStorageOp(sokScan)
|
||||||
|
if sourceOp != nil: op.children.add(sourceOp)
|
||||||
|
return op
|
||||||
|
of irpkUnpivot:
|
||||||
|
let sourceOp = codegenPlan(plan.unpivotSource)
|
||||||
|
let op = newStorageOp(sokScan)
|
||||||
|
if sourceOp != nil: op.children.add(sourceOp)
|
||||||
|
return op
|
||||||
|
of irpkGraphTraversal:
|
||||||
|
return newStorageOp(sokScan)
|
||||||
|
|
||||||
proc estimateCost*(op: StorageOp): float64 =
|
proc estimateCost*(op: StorageOp): float64 =
|
||||||
if op == nil:
|
if op == nil:
|
||||||
|
|||||||
+1194
-124
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,6 @@
|
|||||||
## BaraQL IR — Intermediate Representation for compilation
|
## BaraQL IR — Intermediate Representation for compilation
|
||||||
import std/tables
|
import std/tables
|
||||||
|
import std/strutils
|
||||||
import ../core/types
|
import ../core/types
|
||||||
|
|
||||||
type
|
type
|
||||||
@@ -27,9 +28,15 @@ type
|
|||||||
irBetween
|
irBetween
|
||||||
irIsNull, irIsNotNull
|
irIsNull, irIsNotNull
|
||||||
irFtsMatch
|
irFtsMatch
|
||||||
|
irDistance
|
||||||
|
irJsonContains
|
||||||
|
irJsonContainedBy
|
||||||
|
irJsonHasAny
|
||||||
|
irJsonHasAll
|
||||||
|
|
||||||
IRAggregate* = enum
|
IRAggregate* = enum
|
||||||
irCount, irSum, irAvg, irMin, irMax
|
irCount, irSum, irAvg, irMin, irMax
|
||||||
|
irArrayAgg, irStringAgg
|
||||||
|
|
||||||
IRLiteral* = object
|
IRLiteral* = object
|
||||||
case kind*: ValueKind
|
case kind*: ValueKind
|
||||||
@@ -50,8 +57,10 @@ type
|
|||||||
irekCast
|
irekCast
|
||||||
irekConditional
|
irekConditional
|
||||||
irekExists
|
irekExists
|
||||||
|
irekSubquery
|
||||||
irekStar
|
irekStar
|
||||||
irekJsonPath
|
irekJsonPath
|
||||||
|
irekWindowFunc
|
||||||
|
|
||||||
IRJoinKind* = enum
|
IRJoinKind* = enum
|
||||||
irjkInner
|
irjkInner
|
||||||
@@ -71,11 +80,22 @@ type
|
|||||||
irpkInsert
|
irpkInsert
|
||||||
irpkUpdate
|
irpkUpdate
|
||||||
irpkDelete
|
irpkDelete
|
||||||
|
irpkMerge
|
||||||
irpkCreateType
|
irpkCreateType
|
||||||
irpkUnion
|
irpkUnion
|
||||||
irpkCTE
|
irpkCTE
|
||||||
irpkValues
|
irpkValues
|
||||||
irpkExplain
|
irpkExplain
|
||||||
|
irpkWindow
|
||||||
|
irpkPivot
|
||||||
|
irpkUnpivot
|
||||||
|
irpkGraphTraversal
|
||||||
|
|
||||||
|
IRGroupingSetsKind* = enum
|
||||||
|
irgskNone
|
||||||
|
irgskGroupingSets
|
||||||
|
irgskRollup
|
||||||
|
irgskCube
|
||||||
|
|
||||||
IRPlan* = ref object
|
IRPlan* = ref object
|
||||||
case kind*: IRPlanKind
|
case kind*: IRPlanKind
|
||||||
@@ -94,12 +114,15 @@ type
|
|||||||
groupKeys*: seq[IRExpr]
|
groupKeys*: seq[IRExpr]
|
||||||
groupAggs*: seq[IRExpr]
|
groupAggs*: seq[IRExpr]
|
||||||
groupHaving*: IRExpr
|
groupHaving*: IRExpr
|
||||||
|
groupingSetsKind*: IRGroupingSetsKind
|
||||||
|
groupingSets*: seq[seq[IRExpr]]
|
||||||
of irpkJoin:
|
of irpkJoin:
|
||||||
joinKind*: IRJoinKind
|
joinKind*: IRJoinKind
|
||||||
joinLeft*: IRPlan
|
joinLeft*: IRPlan
|
||||||
joinRight*: IRPlan
|
joinRight*: IRPlan
|
||||||
joinCond*: IRExpr
|
joinCond*: IRExpr
|
||||||
joinAlias*: string
|
joinAlias*: string
|
||||||
|
joinLateral*: bool
|
||||||
of irpkSort:
|
of irpkSort:
|
||||||
sortSource*: IRPlan
|
sortSource*: IRPlan
|
||||||
sortExprs*: seq[IRExpr]
|
sortExprs*: seq[IRExpr]
|
||||||
@@ -121,6 +144,14 @@ type
|
|||||||
deleteTable*: string
|
deleteTable*: string
|
||||||
deleteAlias*: string
|
deleteAlias*: string
|
||||||
deleteSource*: IRPlan
|
deleteSource*: IRPlan
|
||||||
|
of irpkMerge:
|
||||||
|
mergeTarget*: string
|
||||||
|
mergeTargetAlias*: string
|
||||||
|
mergeSourcePlan*: IRPlan
|
||||||
|
mergeOnCond*: IRExpr
|
||||||
|
mergeUpdateSets*: seq[(string, IRExpr)]
|
||||||
|
mergeInsertFields*: seq[string]
|
||||||
|
mergeInsertValues*: seq[seq[IRExpr]]
|
||||||
of irpkCreateType:
|
of irpkCreateType:
|
||||||
createTypeName*: string
|
createTypeName*: string
|
||||||
createTypeDef*: IRType
|
createTypeDef*: IRType
|
||||||
@@ -136,6 +167,34 @@ type
|
|||||||
valuesRows*: seq[seq[IRExpr]]
|
valuesRows*: seq[seq[IRExpr]]
|
||||||
of irpkExplain:
|
of irpkExplain:
|
||||||
explainPlan*: IRPlan
|
explainPlan*: IRPlan
|
||||||
|
of irpkWindow:
|
||||||
|
winSource*: IRPlan
|
||||||
|
winFuncs*: seq[IRExpr]
|
||||||
|
winPartition*: seq[IRExpr]
|
||||||
|
winOrderBy*: seq[IRExpr]
|
||||||
|
winOrderDirs*: seq[bool]
|
||||||
|
winFrameMode*: string
|
||||||
|
winFrameStart*: string
|
||||||
|
winFrameEnd*: string
|
||||||
|
of irpkPivot:
|
||||||
|
pivotSource*: IRPlan
|
||||||
|
pivotAgg*: IRExpr
|
||||||
|
pivotForCol*: string
|
||||||
|
pivotInValues*: seq[string]
|
||||||
|
of irpkUnpivot:
|
||||||
|
unpivotSource*: IRPlan
|
||||||
|
unpivotValueCol*: string
|
||||||
|
unpivotForCol*: string
|
||||||
|
unpivotInCols*: seq[string]
|
||||||
|
of irpkGraphTraversal:
|
||||||
|
graphName*: string
|
||||||
|
graphAlgo*: string # "bfs", "dfs", "shortest", "pagerank"
|
||||||
|
graphStartNode*: string
|
||||||
|
graphEndNode*: string
|
||||||
|
graphEdgeLabel*: string
|
||||||
|
graphMaxDepth*: int
|
||||||
|
graphFilter*: IRExpr
|
||||||
|
graphReturnCols*: seq[string]
|
||||||
|
|
||||||
IRExpr* = ref object
|
IRExpr* = ref object
|
||||||
case kind*: IRExprKind
|
case kind*: IRExprKind
|
||||||
@@ -154,6 +213,7 @@ type
|
|||||||
aggOp*: IRAggregate
|
aggOp*: IRAggregate
|
||||||
aggArgs*: seq[IRExpr]
|
aggArgs*: seq[IRExpr]
|
||||||
aggDistinct*: bool
|
aggDistinct*: bool
|
||||||
|
aggFilter*: IRExpr
|
||||||
of irekFuncCall:
|
of irekFuncCall:
|
||||||
irFunc*: string
|
irFunc*: string
|
||||||
irFuncArgs*: seq[IRExpr]
|
irFuncArgs*: seq[IRExpr]
|
||||||
@@ -166,12 +226,23 @@ type
|
|||||||
elseExpr*: IRExpr
|
elseExpr*: IRExpr
|
||||||
of irekExists:
|
of irekExists:
|
||||||
existsSubquery*: IRPlan
|
existsSubquery*: IRPlan
|
||||||
|
of irekSubquery:
|
||||||
|
subqueryPlan*: IRPlan
|
||||||
of irekStar:
|
of irekStar:
|
||||||
discard
|
discard
|
||||||
of irekJsonPath:
|
of irekJsonPath:
|
||||||
jpExpr*: IRExpr
|
jpExpr*: IRExpr
|
||||||
jpKey*: string
|
jpKey*: string
|
||||||
jpAsText*: bool
|
jpAsText*: bool
|
||||||
|
of irekWindowFunc:
|
||||||
|
wfName*: string
|
||||||
|
wfArgs*: seq[IRExpr]
|
||||||
|
wfPartition*: seq[IRExpr]
|
||||||
|
wfOrderBy*: seq[IRExpr]
|
||||||
|
wfOrderDirs*: seq[bool]
|
||||||
|
wfFrameMode*: string
|
||||||
|
wfFrameStart*: string
|
||||||
|
wfFrameEnd*: string
|
||||||
|
|
||||||
type
|
type
|
||||||
TypeChecker* = ref object
|
TypeChecker* = ref object
|
||||||
@@ -241,6 +312,10 @@ proc inferExpr*(tc: TypeChecker, expr: IRExpr, context: Table[string, IRType]):
|
|||||||
if expr.aggArgs.len > 0:
|
if expr.aggArgs.len > 0:
|
||||||
return tc.inferExpr(expr.aggArgs[0], context)
|
return tc.inferExpr(expr.aggArgs[0], context)
|
||||||
return nil
|
return nil
|
||||||
|
of irArrayAgg:
|
||||||
|
return IRType(name: "array", kind: itkArray)
|
||||||
|
of irStringAgg:
|
||||||
|
return IRType(name: "text", kind: itkScalar)
|
||||||
of irekFuncCall:
|
of irekFuncCall:
|
||||||
return IRType(name: "unknown", kind: itkScalar)
|
return IRType(name: "unknown", kind: itkScalar)
|
||||||
of irekCast:
|
of irekCast:
|
||||||
@@ -250,8 +325,21 @@ proc inferExpr*(tc: TypeChecker, expr: IRExpr, context: Table[string, IRType]):
|
|||||||
return thenType
|
return thenType
|
||||||
of irekExists:
|
of irekExists:
|
||||||
return IRType(name: "bool", kind: itkScalar)
|
return IRType(name: "bool", kind: itkScalar)
|
||||||
|
of irekSubquery:
|
||||||
|
return IRType(name: "set", kind: itkSet)
|
||||||
of irekStar:
|
of irekStar:
|
||||||
return IRType(name: "star", kind: itkScalar)
|
return IRType(name: "star", kind: itkScalar)
|
||||||
of irekJsonPath:
|
of irekJsonPath:
|
||||||
if expr.jpAsText: return IRType(name: "str", kind: itkScalar)
|
if expr.jpAsText: return IRType(name: "str", kind: itkScalar)
|
||||||
return IRType(name: "json", kind: itkScalar)
|
return IRType(name: "json", kind: itkScalar)
|
||||||
|
of irekWindowFunc:
|
||||||
|
# Window functions return int64 for ranking, or the type of the argument for value functions
|
||||||
|
case expr.wfName.toLower()
|
||||||
|
of "row_number", "rank", "dense_rank", "ntile":
|
||||||
|
return IRType(name: "int64", kind: itkScalar)
|
||||||
|
of "lead", "lag", "first_value", "last_value":
|
||||||
|
if expr.wfArgs.len > 0:
|
||||||
|
return tc.inferExpr(expr.wfArgs[0], context)
|
||||||
|
return nil
|
||||||
|
else:
|
||||||
|
return IRType(name: "unknown", kind: itkScalar)
|
||||||
|
|||||||
@@ -35,6 +35,7 @@ type
|
|||||||
tkOuter
|
tkOuter
|
||||||
tkFull
|
tkFull
|
||||||
tkCross
|
tkCross
|
||||||
|
tkLateral
|
||||||
tkOrder
|
tkOrder
|
||||||
tkBy
|
tkBy
|
||||||
tkAsc
|
tkAsc
|
||||||
@@ -77,6 +78,7 @@ type
|
|||||||
tkBetween
|
tkBetween
|
||||||
tkLike
|
tkLike
|
||||||
tkILike
|
tkILike
|
||||||
|
tkFilter
|
||||||
tkReturning
|
tkReturning
|
||||||
tkPrimary
|
tkPrimary
|
||||||
tkKey
|
tkKey
|
||||||
@@ -120,6 +122,24 @@ type
|
|||||||
tkAvg
|
tkAvg
|
||||||
tkMin
|
tkMin
|
||||||
tkMax
|
tkMax
|
||||||
|
tkArrayAgg
|
||||||
|
tkStringAgg
|
||||||
|
tkGrouping
|
||||||
|
tkSets
|
||||||
|
tkRollup
|
||||||
|
tkCube
|
||||||
|
tkPivot
|
||||||
|
tkUnpivot
|
||||||
|
tkVertex
|
||||||
|
tkEdge
|
||||||
|
tkLabels
|
||||||
|
tkGraphTable
|
||||||
|
tkMatch
|
||||||
|
tkColumns
|
||||||
|
tkSrc
|
||||||
|
tkDst
|
||||||
|
tkMerge
|
||||||
|
tkMatched
|
||||||
tkArray
|
tkArray
|
||||||
tkVector
|
tkVector
|
||||||
tkGraph
|
tkGraph
|
||||||
@@ -127,6 +147,10 @@ type
|
|||||||
tkArrowR # ->
|
tkArrowR # ->
|
||||||
tkArrowRR # ->>
|
tkArrowRR # ->>
|
||||||
tkFtsMatch # @@
|
tkFtsMatch # @@
|
||||||
|
tkJsonContains # @>
|
||||||
|
tkJsonContainedBy # <@
|
||||||
|
tkJsonHasAny # ?|
|
||||||
|
tkJsonHasAll # ?&
|
||||||
tkSimilar
|
tkSimilar
|
||||||
tkNearest
|
tkNearest
|
||||||
tkTo
|
tkTo
|
||||||
@@ -134,6 +158,27 @@ type
|
|||||||
tkDfs
|
tkDfs
|
||||||
tkPath
|
tkPath
|
||||||
|
|
||||||
|
# Window functions
|
||||||
|
tkOver
|
||||||
|
tkPartition
|
||||||
|
tkRow
|
||||||
|
tkRows
|
||||||
|
tkRange
|
||||||
|
tkUnbounded
|
||||||
|
tkPreceding
|
||||||
|
tkFollowing
|
||||||
|
tkCurrent
|
||||||
|
tkCurrentUser
|
||||||
|
tkCurrentRole
|
||||||
|
tkRowNumber
|
||||||
|
tkRank
|
||||||
|
tkDenseRank
|
||||||
|
tkLead
|
||||||
|
tkLag
|
||||||
|
tkFirstValue
|
||||||
|
tkLastValue
|
||||||
|
tkNtile
|
||||||
|
|
||||||
# Operators
|
# Operators
|
||||||
tkPlus
|
tkPlus
|
||||||
tkMinus
|
tkMinus
|
||||||
@@ -166,6 +211,7 @@ type
|
|||||||
tkConcat
|
tkConcat
|
||||||
tkCoalesce
|
tkCoalesce
|
||||||
tkFloorDiv
|
tkFloorDiv
|
||||||
|
tkDistanceOp # <->
|
||||||
tkPlaceholder
|
tkPlaceholder
|
||||||
|
|
||||||
# Special
|
# Special
|
||||||
@@ -206,6 +252,7 @@ const keywords*: Table[string, TokenKind] = {
|
|||||||
"outer": tkOuter,
|
"outer": tkOuter,
|
||||||
"full": tkFull,
|
"full": tkFull,
|
||||||
"cross": tkCross,
|
"cross": tkCross,
|
||||||
|
"lateral": tkLateral,
|
||||||
"order": tkOrder,
|
"order": tkOrder,
|
||||||
"by": tkBy,
|
"by": tkBy,
|
||||||
"asc": tkAsc,
|
"asc": tkAsc,
|
||||||
@@ -248,6 +295,7 @@ const keywords*: Table[string, TokenKind] = {
|
|||||||
"between": tkBetween,
|
"between": tkBetween,
|
||||||
"like": tkLike,
|
"like": tkLike,
|
||||||
"ilike": tkILike,
|
"ilike": tkILike,
|
||||||
|
"filter": tkFilter,
|
||||||
"returning": tkReturning,
|
"returning": tkReturning,
|
||||||
"primary": tkPrimary,
|
"primary": tkPrimary,
|
||||||
"key": tkKey,
|
"key": tkKey,
|
||||||
@@ -291,6 +339,27 @@ const keywords*: Table[string, TokenKind] = {
|
|||||||
"avg": tkAvg,
|
"avg": tkAvg,
|
||||||
"min": tkMin,
|
"min": tkMin,
|
||||||
"max": tkMax,
|
"max": tkMax,
|
||||||
|
"array_agg": tkArrayAgg,
|
||||||
|
"string_agg": tkStringAgg,
|
||||||
|
"grouping": tkGrouping,
|
||||||
|
"sets": tkSets,
|
||||||
|
"rollup": tkRollup,
|
||||||
|
"cube": tkCube,
|
||||||
|
"pivot": tkPivot,
|
||||||
|
"unpivot": tkUnpivot,
|
||||||
|
"vertex": tkVertex,
|
||||||
|
"vertices": tkVertex,
|
||||||
|
"edge": tkEdge,
|
||||||
|
"edges": tkEdge,
|
||||||
|
"label": tkLabels,
|
||||||
|
"labels": tkLabels,
|
||||||
|
"graph_table": tkGraphTable,
|
||||||
|
"match": tkMatch,
|
||||||
|
"columns": tkColumns,
|
||||||
|
"src": tkSrc,
|
||||||
|
"dst": tkDst,
|
||||||
|
"merge": tkMerge,
|
||||||
|
"matched": tkMatched,
|
||||||
"array": tkArray,
|
"array": tkArray,
|
||||||
"vector": tkVector,
|
"vector": tkVector,
|
||||||
"graph": tkGraph,
|
"graph": tkGraph,
|
||||||
@@ -301,6 +370,25 @@ const keywords*: Table[string, TokenKind] = {
|
|||||||
"bfs": tkBfs,
|
"bfs": tkBfs,
|
||||||
"dfs": tkDfs,
|
"dfs": tkDfs,
|
||||||
"path": tkPath,
|
"path": tkPath,
|
||||||
|
"over": tkOver,
|
||||||
|
"partition": tkPartition,
|
||||||
|
"row": tkRow,
|
||||||
|
"rows": tkRows,
|
||||||
|
"range": tkRange,
|
||||||
|
"unbounded": tkUnbounded,
|
||||||
|
"preceding": tkPreceding,
|
||||||
|
"following": tkFollowing,
|
||||||
|
"current": tkCurrent,
|
||||||
|
"current_user": tkCurrentUser,
|
||||||
|
"current_role": tkCurrentRole,
|
||||||
|
"row_number": tkRowNumber,
|
||||||
|
"rank": tkRank,
|
||||||
|
"dense_rank": tkDenseRank,
|
||||||
|
"lead": tkLead,
|
||||||
|
"lag": tkLag,
|
||||||
|
"first_value": tkFirstValue,
|
||||||
|
"last_value": tkLastValue,
|
||||||
|
"ntile": tkNtile,
|
||||||
}.toTable
|
}.toTable
|
||||||
|
|
||||||
proc newLexer*(input: string): Lexer =
|
proc newLexer*(input: string): Lexer =
|
||||||
@@ -495,6 +583,11 @@ proc nextToken*(l: var Lexer): Token =
|
|||||||
discard l.advance()
|
discard l.advance()
|
||||||
return Token(kind: tkInvalid, value: "!", line: startLine, col: startCol)
|
return Token(kind: tkInvalid, value: "!", line: startLine, col: startCol)
|
||||||
of '<':
|
of '<':
|
||||||
|
if l.pos + 2 < l.input.len and l.input[l.pos + 1] == '-' and l.input[l.pos + 2] == '>':
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
return Token(kind: tkDistanceOp, value: "<->", line: startLine, col: startCol)
|
||||||
if l.pos + 1 < l.input.len and l.input[l.pos + 1] == '=':
|
if l.pos + 1 < l.input.len and l.input[l.pos + 1] == '=':
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
@@ -503,6 +596,10 @@ proc nextToken*(l: var Lexer): Token =
|
|||||||
discard l.advance()
|
discard l.advance()
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
return Token(kind: tkNotEq, value: "<>", line: startLine, col: startCol)
|
return Token(kind: tkNotEq, value: "<>", line: startLine, col: startCol)
|
||||||
|
if l.pos + 1 < l.input.len and l.input[l.pos + 1] == '@':
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
return Token(kind: tkJsonContainedBy, value: "<@", line: startLine, col: startCol)
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
return Token(kind: tkLt, value: "<", line: startLine, col: startCol)
|
return Token(kind: tkLt, value: "<", line: startLine, col: startCol)
|
||||||
of '>':
|
of '>':
|
||||||
@@ -513,6 +610,16 @@ proc nextToken*(l: var Lexer): Token =
|
|||||||
discard l.advance()
|
discard l.advance()
|
||||||
return Token(kind: tkGt, value: ">", line: startLine, col: startCol)
|
return Token(kind: tkGt, value: ">", line: startLine, col: startCol)
|
||||||
of '?':
|
of '?':
|
||||||
|
if l.pos + 2 < l.input.len and l.input[l.pos + 1] == '|':
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
return Token(kind: tkJsonHasAny, value: "?|", line: startLine, col: startCol)
|
||||||
|
if l.pos + 2 < l.input.len and l.input[l.pos + 1] == '&':
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
return Token(kind: tkJsonHasAll, value: "?&", line: startLine, col: startCol)
|
||||||
if l.pos + 1 < l.input.len and l.input[l.pos + 1] == '?':
|
if l.pos + 1 < l.input.len and l.input[l.pos + 1] == '?':
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
@@ -524,6 +631,10 @@ proc nextToken*(l: var Lexer): Token =
|
|||||||
discard l.advance()
|
discard l.advance()
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
return Token(kind: tkFtsMatch, value: "@@", line: startLine, col: startCol)
|
return Token(kind: tkFtsMatch, value: "@@", line: startLine, col: startCol)
|
||||||
|
if l.pos + 1 < l.input.len and l.input[l.pos + 1] == '>':
|
||||||
|
discard l.advance()
|
||||||
|
discard l.advance()
|
||||||
|
return Token(kind: tkJsonContains, value: "@>", line: startLine, col: startCol)
|
||||||
discard l.advance()
|
discard l.advance()
|
||||||
return Token(kind: tkInvalid, value: "@", line: startLine, col: startCol)
|
return Token(kind: tkInvalid, value: "@", line: startLine, col: startCol)
|
||||||
of '.':
|
of '.':
|
||||||
|
|||||||
+390
-24
@@ -38,6 +38,10 @@ proc parseExpr(p: var Parser): Node
|
|||||||
proc parseSelect(p: var Parser): Node
|
proc parseSelect(p: var Parser): Node
|
||||||
proc parseStatement*(p: var Parser): Node
|
proc parseStatement*(p: var Parser): Node
|
||||||
proc parseCreateType(p: var Parser): Node
|
proc parseCreateType(p: var Parser): Node
|
||||||
|
proc parseOverClause(p: var Parser): Node
|
||||||
|
proc parseFrameSpec(p: var Parser): Node
|
||||||
|
proc parseFrameBoundary(p: var Parser): string
|
||||||
|
proc parseSetVar(p: var Parser): Node
|
||||||
|
|
||||||
proc parsePrimary(p: var Parser): Node =
|
proc parsePrimary(p: var Parser): Node =
|
||||||
let tok = p.peek()
|
let tok = p.peek()
|
||||||
@@ -57,8 +61,15 @@ proc parsePrimary(p: var Parser): Node =
|
|||||||
of tkNull:
|
of tkNull:
|
||||||
discard p.advance()
|
discard p.advance()
|
||||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||||
of tkIdent:
|
of tkCurrentUser:
|
||||||
discard p.advance()
|
discard p.advance()
|
||||||
|
Node(kind: nkCurrentUser, line: tok.line, col: tok.col)
|
||||||
|
of tkCurrentRole:
|
||||||
|
discard p.advance()
|
||||||
|
Node(kind: nkCurrentRole, line: tok.line, col: tok.col)
|
||||||
|
of tkIdent, tkRowNumber, tkRank, tkDenseRank, tkLead, tkLag, tkFirstValue, tkLastValue, tkNtile:
|
||||||
|
discard p.advance()
|
||||||
|
let funcName = tok.value
|
||||||
# Check for function call: ident(...)
|
# Check for function call: ident(...)
|
||||||
if p.peek().kind == tkLParen:
|
if p.peek().kind == tkLParen:
|
||||||
discard p.advance() # consume (
|
discard p.advance() # consume (
|
||||||
@@ -68,15 +79,21 @@ proc parsePrimary(p: var Parser): Node =
|
|||||||
while p.match(tkComma):
|
while p.match(tkComma):
|
||||||
args.add(p.parseExpr())
|
args.add(p.parseExpr())
|
||||||
discard p.expect(tkRParen)
|
discard p.expect(tkRParen)
|
||||||
return Node(kind: nkFuncCall, funcName: tok.value, funcArgs: args,
|
var funcNode = Node(kind: nkFuncCall, funcName: funcName, funcArgs: args,
|
||||||
line: tok.line, col: tok.col)
|
line: tok.line, col: tok.col)
|
||||||
|
# Check for window function: func(...) OVER (...)
|
||||||
|
if p.peek().kind == tkOver:
|
||||||
|
let overClause = p.parseOverClause()
|
||||||
|
return Node(kind: nkWindowExpr, winFunc: funcName, winArgs: args,
|
||||||
|
winOver: overClause, line: tok.line, col: tok.col)
|
||||||
|
return funcNode
|
||||||
# Check for dotted path: ident.ident.ident
|
# Check for dotted path: ident.ident.ident
|
||||||
var parts = @[tok.value]
|
var parts = @[funcName]
|
||||||
while p.peek().kind == tkDot:
|
while p.peek().kind == tkDot:
|
||||||
discard p.advance() # consume .
|
discard p.advance() # consume .
|
||||||
parts.add(p.expect(tkIdent).value)
|
parts.add(p.expect(tkIdent).value)
|
||||||
if parts.len == 1:
|
if parts.len == 1:
|
||||||
return Node(kind: nkIdent, identName: tok.value, line: tok.line, col: tok.col)
|
return Node(kind: nkIdent, identName: funcName, line: tok.line, col: tok.col)
|
||||||
return Node(kind: nkPath, pathParts: parts, line: tok.line, col: tok.col)
|
return Node(kind: nkPath, pathParts: parts, line: tok.line, col: tok.col)
|
||||||
of tkLParen:
|
of tkLParen:
|
||||||
discard p.advance()
|
discard p.advance()
|
||||||
@@ -119,7 +136,7 @@ proc parsePrimary(p: var Parser): Node =
|
|||||||
discard p.advance()
|
discard p.advance()
|
||||||
let operand = p.parsePrimary()
|
let operand = p.parsePrimary()
|
||||||
Node(kind: nkUnaryOp, unOp: ukNeg, unOperand: operand, line: tok.line, col: tok.col)
|
Node(kind: nkUnaryOp, unOp: ukNeg, unOperand: operand, line: tok.line, col: tok.col)
|
||||||
of tkCount, tkSum, tkAvg, tkMin, tkMax:
|
of tkCount, tkSum, tkAvg, tkMin, tkMax, tkArrayAgg, tkStringAgg:
|
||||||
let funcName = tok.value
|
let funcName = tok.value
|
||||||
discard p.advance()
|
discard p.advance()
|
||||||
discard p.expect(tkLParen)
|
discard p.expect(tkLParen)
|
||||||
@@ -131,9 +148,17 @@ proc parsePrimary(p: var Parser): Node =
|
|||||||
hasDistinct = true
|
hasDistinct = true
|
||||||
if p.peek().kind != tkRParen:
|
if p.peek().kind != tkRParen:
|
||||||
args.add(p.parseExpr())
|
args.add(p.parseExpr())
|
||||||
|
while p.match(tkComma):
|
||||||
|
args.add(p.parseExpr())
|
||||||
discard p.expect(tkRParen)
|
discard p.expect(tkRParen)
|
||||||
var node = Node(kind: nkFuncCall, funcName: funcName.toLower(), funcArgs: args,
|
var node = Node(kind: nkFuncCall, funcName: funcName.toLower(), funcArgs: args,
|
||||||
line: tok.line, col: tok.col)
|
line: tok.line, col: tok.col)
|
||||||
|
# Handle FILTER (WHERE ...)
|
||||||
|
if p.match(tkFilter):
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
discard p.expect(tkWhere)
|
||||||
|
node.funcFilter = p.parseExpr()
|
||||||
|
discard p.expect(tkRParen)
|
||||||
return node
|
return node
|
||||||
of tkCase:
|
of tkCase:
|
||||||
discard p.advance()
|
discard p.advance()
|
||||||
@@ -157,6 +182,80 @@ proc parsePrimary(p: var Parser): Node =
|
|||||||
discard p.advance()
|
discard p.advance()
|
||||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||||
|
|
||||||
|
proc parseOverClause(p: var Parser): Node =
|
||||||
|
## Parse OVER ( [PARTITION BY expr, ...] [ORDER BY expr [ASC|DESC], ...] [frame] )
|
||||||
|
discard p.expect(tkOver)
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
result = Node(kind: nkOverClause)
|
||||||
|
# PARTITION BY
|
||||||
|
if p.match(tkPartition):
|
||||||
|
discard p.expect(tkBy)
|
||||||
|
result.overPartition.add(p.parseExpr())
|
||||||
|
while p.match(tkComma):
|
||||||
|
result.overPartition.add(p.parseExpr())
|
||||||
|
# ORDER BY
|
||||||
|
if p.match(tkOrder):
|
||||||
|
discard p.expect(tkBy)
|
||||||
|
var expr = p.parseExpr()
|
||||||
|
var dir = sdAsc
|
||||||
|
if p.match(tkDesc):
|
||||||
|
dir = sdDesc
|
||||||
|
elif p.match(tkAsc):
|
||||||
|
dir = sdAsc
|
||||||
|
result.overOrderBy.add(Node(kind: nkOrderBy, orderByExpr: expr, orderByDir: dir))
|
||||||
|
while p.match(tkComma):
|
||||||
|
expr = p.parseExpr()
|
||||||
|
dir = sdAsc
|
||||||
|
if p.match(tkDesc):
|
||||||
|
dir = sdDesc
|
||||||
|
elif p.match(tkAsc):
|
||||||
|
dir = sdAsc
|
||||||
|
result.overOrderBy.add(Node(kind: nkOrderBy, orderByExpr: expr, orderByDir: dir))
|
||||||
|
# Frame specification
|
||||||
|
if p.peek().kind in {tkRows, tkRange}:
|
||||||
|
result.overFrame = p.parseFrameSpec()
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
|
||||||
|
proc parseFrameSpec(p: var Parser): Node =
|
||||||
|
## Parse ROWS|RANGE frame
|
||||||
|
let modeTok = p.advance() # tkRows or tkRange
|
||||||
|
result = Node(kind: nkFrameSpec, frameMode: modeTok.value)
|
||||||
|
if p.match(tkBetween):
|
||||||
|
result.frameStartType = p.parseFrameBoundary()
|
||||||
|
discard p.expect(tkAnd)
|
||||||
|
result.frameEndType = p.parseFrameBoundary()
|
||||||
|
else:
|
||||||
|
result.frameStartType = p.parseFrameBoundary()
|
||||||
|
result.frameEndType = "CURRENT ROW"
|
||||||
|
|
||||||
|
proc parseFrameBoundary(p: var Parser): string =
|
||||||
|
## Returns boundary type string like "UNBOUNDED PRECEDING", "CURRENT ROW", "1 PRECEDING"
|
||||||
|
if p.match(tkUnbounded):
|
||||||
|
if p.match(tkPreceding):
|
||||||
|
return "UNBOUNDED PRECEDING"
|
||||||
|
elif p.match(tkFollowing):
|
||||||
|
return "UNBOUNDED FOLLOWING"
|
||||||
|
else:
|
||||||
|
raise newException(ValueError, "Expected PRECEDING or FOLLOWING after UNBOUNDED")
|
||||||
|
elif p.match(tkCurrent):
|
||||||
|
discard p.expect(tkRow)
|
||||||
|
return "CURRENT ROW"
|
||||||
|
else:
|
||||||
|
# Expect a simple integer literal for offset boundaries
|
||||||
|
let offsetExpr = p.parsePrimary()
|
||||||
|
var offsetStr = ""
|
||||||
|
case offsetExpr.kind:
|
||||||
|
of nkIntLit: offsetStr = $offsetExpr.intVal
|
||||||
|
of nkIdent: offsetStr = offsetExpr.identName
|
||||||
|
else:
|
||||||
|
raise newException(ValueError, "Expected integer literal or identifier in frame boundary, got " & $offsetExpr.kind)
|
||||||
|
if p.match(tkPreceding):
|
||||||
|
return offsetStr & " PRECEDING"
|
||||||
|
elif p.match(tkFollowing):
|
||||||
|
return offsetStr & " FOLLOWING"
|
||||||
|
else:
|
||||||
|
raise newException(ValueError, "Expected PRECEDING or FOLLOWING after frame offset")
|
||||||
|
|
||||||
proc parsePostfix(p: var Parser): Node =
|
proc parsePostfix(p: var Parser): Node =
|
||||||
result = p.parsePrimary()
|
result = p.parsePrimary()
|
||||||
while p.peek().kind in {tkArrowR, tkArrowRR}:
|
while p.peek().kind in {tkArrowR, tkArrowRR}:
|
||||||
@@ -226,7 +325,7 @@ proc parseComparison(p: var Parser): Node =
|
|||||||
discard p.advance() # consume NULL token (assumed)
|
discard p.advance() # consume NULL token (assumed)
|
||||||
return Node(kind: nkIsExpr, isExpr: result, isNegated: negated,
|
return Node(kind: nkIsExpr, isExpr: result, isNegated: negated,
|
||||||
line: tok.line, col: tok.col)
|
line: tok.line, col: tok.col)
|
||||||
while p.peek().kind in {tkEq, tkNotEq, tkLt, tkLtEq, tkGt, tkGtEq, tkFtsMatch}:
|
while p.peek().kind in {tkEq, tkNotEq, tkLt, tkLtEq, tkGt, tkGtEq, tkFtsMatch, tkDistanceOp, tkJsonContains, tkJsonContainedBy, tkJsonHasAny, tkJsonHasAll}:
|
||||||
let op = case p.peek().kind
|
let op = case p.peek().kind
|
||||||
of tkEq: bkEq
|
of tkEq: bkEq
|
||||||
of tkNotEq: bkNotEq
|
of tkNotEq: bkNotEq
|
||||||
@@ -235,6 +334,11 @@ proc parseComparison(p: var Parser): Node =
|
|||||||
of tkGt: bkGt
|
of tkGt: bkGt
|
||||||
of tkGtEq: bkGtEq
|
of tkGtEq: bkGtEq
|
||||||
of tkFtsMatch: bkFtsMatch
|
of tkFtsMatch: bkFtsMatch
|
||||||
|
of tkDistanceOp: bkDistance
|
||||||
|
of tkJsonContains: bkJsonContains
|
||||||
|
of tkJsonContainedBy: bkJsonContainedBy
|
||||||
|
of tkJsonHasAny: bkJsonHasAny
|
||||||
|
of tkJsonHasAll: bkJsonHasAll
|
||||||
else: bkEq
|
else: bkEq
|
||||||
let tok = p.advance()
|
let tok = p.advance()
|
||||||
let right = p.parseAddSub()
|
let right = p.parseAddSub()
|
||||||
@@ -354,7 +458,66 @@ proc parseSelect(p: var Parser): Node =
|
|||||||
elif p.peek().kind == tkIdent:
|
elif p.peek().kind == tkIdent:
|
||||||
alias = p.advance().value
|
alias = p.advance().value
|
||||||
result.selFrom = Node(kind: nkFrom, fromTable: "(subquery)",
|
result.selFrom = Node(kind: nkFrom, fromTable: "(subquery)",
|
||||||
fromAlias: alias, line: tok.line, col: tok.col)
|
fromAlias: alias, fromSubquery: subquery, line: tok.line, col: tok.col)
|
||||||
|
elif p.peek().kind == tkGraphTable:
|
||||||
|
# GRAPH_TABLE(name MATCH (pattern) COLUMNS (cols))
|
||||||
|
discard p.advance()
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
let graphName = p.expect(tkIdent).value
|
||||||
|
discard p.expect(tkMatch)
|
||||||
|
# Parse pattern: (node)-[edge]->(node)
|
||||||
|
var patternNodes: seq[string]
|
||||||
|
var patternEdges: seq[string]
|
||||||
|
# First node
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
if p.peek().kind == tkIdent:
|
||||||
|
patternNodes.add(p.advance().value)
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
# Edge and next node(s)
|
||||||
|
while p.peek().kind == tkMinus or p.peek().kind == tkArrowR:
|
||||||
|
if p.match(tkArrowR):
|
||||||
|
discard
|
||||||
|
elif p.match(tkMinus):
|
||||||
|
if p.peek().kind == tkLBracket:
|
||||||
|
discard p.advance()
|
||||||
|
if p.peek().kind == tkIdent:
|
||||||
|
patternEdges.add(p.advance().value)
|
||||||
|
discard p.expect(tkRBracket)
|
||||||
|
discard p.expect(tkArrowR)
|
||||||
|
else:
|
||||||
|
discard
|
||||||
|
if p.peek().kind == tkLParen:
|
||||||
|
discard p.advance()
|
||||||
|
if p.peek().kind == tkIdent:
|
||||||
|
patternNodes.add(p.advance().value)
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
# COLUMNS (col1, col2, ...)
|
||||||
|
var returnCols: seq[string]
|
||||||
|
if p.match(tkColumns):
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
if p.peek().kind == tkIdent:
|
||||||
|
var colName = p.advance().value
|
||||||
|
# Handle dotted names: e.name
|
||||||
|
while p.peek().kind == tkDot:
|
||||||
|
discard p.advance() # skip dot
|
||||||
|
colName &= "." & p.expect(tkIdent).value
|
||||||
|
returnCols.add(colName)
|
||||||
|
while p.match(tkComma):
|
||||||
|
if p.peek().kind == tkIdent:
|
||||||
|
colName = p.advance().value
|
||||||
|
while p.peek().kind == tkDot:
|
||||||
|
discard p.advance()
|
||||||
|
colName &= "." & p.expect(tkIdent).value
|
||||||
|
returnCols.add(colName)
|
||||||
|
if p.match(tkAs):
|
||||||
|
discard p.advance() # skip alias
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
# Create a graph traversal node
|
||||||
|
result.selFrom = Node(kind: nkGraphTraversal, gtGraphName: graphName,
|
||||||
|
gtStart: nil, gtEdge: if patternEdges.len > 0: patternEdges[0] else: "",
|
||||||
|
gtDirection: "out", gtEnd: nil, gtMaxDepth: -1,
|
||||||
|
gtReturnCols: returnCols, line: tok.line, col: tok.col)
|
||||||
else:
|
else:
|
||||||
let tableTok = p.expect(tkIdent)
|
let tableTok = p.expect(tkIdent)
|
||||||
var alias = ""
|
var alias = ""
|
||||||
@@ -365,26 +528,89 @@ proc parseSelect(p: var Parser): Node =
|
|||||||
result.selFrom = Node(kind: nkFrom, fromTable: tableTok.value,
|
result.selFrom = Node(kind: nkFrom, fromTable: tableTok.value,
|
||||||
fromAlias: alias, line: tableTok.line, col: tableTok.col)
|
fromAlias: alias, line: tableTok.line, col: tableTok.col)
|
||||||
|
|
||||||
|
# Parse PIVOT / UNPIVOT after FROM source
|
||||||
|
if p.peek().kind == tkPivot:
|
||||||
|
discard p.advance()
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
let aggFunc = p.parseExpr() # e.g. SUM(salary)
|
||||||
|
discard p.expect(tkFor)
|
||||||
|
let forCol = p.expect(tkIdent).value
|
||||||
|
discard p.expect(tkIn)
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
var inValues: seq[string] = @[]
|
||||||
|
inValues.add(p.expect(tkStringLit).value)
|
||||||
|
while p.match(tkComma):
|
||||||
|
inValues.add(p.expect(tkStringLit).value)
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
result.selFrom = Node(kind: nkPivot, pivotSource: result.selFrom,
|
||||||
|
pivotAgg: aggFunc, pivotForCol: forCol,
|
||||||
|
pivotInValues: inValues, line: tok.line, col: tok.col)
|
||||||
|
elif p.peek().kind == tkUnpivot:
|
||||||
|
discard p.advance()
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
let valCol = p.expect(tkIdent).value
|
||||||
|
discard p.expect(tkFor)
|
||||||
|
let forCol = p.expect(tkIdent).value
|
||||||
|
discard p.expect(tkIn)
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
var inCols: seq[string] = @[]
|
||||||
|
inCols.add(p.expect(tkIdent).value)
|
||||||
|
while p.match(tkComma):
|
||||||
|
inCols.add(p.expect(tkIdent).value)
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
result.selFrom = Node(kind: nkUnpivot, unpivotSource: result.selFrom,
|
||||||
|
unpivotValueCol: valCol, unpivotForCol: forCol,
|
||||||
|
unpivotInCols: inCols, line: tok.line, col: tok.col)
|
||||||
|
|
||||||
# Parse JOINs
|
# Parse JOINs
|
||||||
while p.peek().kind == tkJoin or
|
while p.peek().kind == tkJoin or
|
||||||
|
p.peek().kind == tkLateral or
|
||||||
(p.peek().kind in {tkInner, tkLeft, tkRight, tkFull, tkCross} and
|
(p.peek().kind in {tkInner, tkLeft, tkRight, tkFull, tkCross} and
|
||||||
p.pos + 1 < p.tokens.len and p.tokens[p.pos + 1].kind == tkJoin):
|
p.pos + 1 < p.tokens.len and p.tokens[p.pos + 1].kind in {tkJoin, tkLateral}):
|
||||||
let jk = p.parseJoinType()
|
var isLateral = false
|
||||||
discard p.expect(tkJoin)
|
var jk = jkInner
|
||||||
let joinTable = p.expect(tkIdent)
|
# Check for LATERAL before join type
|
||||||
|
if p.match(tkLateral):
|
||||||
|
isLateral = true
|
||||||
|
# Could be standalone LATERAL or LATERAL JOIN
|
||||||
|
if p.peek().kind == tkJoin:
|
||||||
|
discard p.advance()
|
||||||
|
else:
|
||||||
|
jk = p.parseJoinType()
|
||||||
|
discard p.expect(tkJoin)
|
||||||
|
# Check for LATERAL after JOIN keyword
|
||||||
|
if p.match(tkLateral):
|
||||||
|
isLateral = true
|
||||||
|
|
||||||
|
var joinTarget: Node
|
||||||
var joinAlias = ""
|
var joinAlias = ""
|
||||||
if p.match(tkAs):
|
if isLateral:
|
||||||
joinAlias = p.expect(tkIdent).value
|
# LATERAL (subquery) AS alias
|
||||||
elif p.peek().kind == tkIdent:
|
discard p.expect(tkLParen)
|
||||||
joinAlias = p.advance().value
|
let subquery = p.parseSelect()
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
if p.match(tkAs):
|
||||||
|
joinAlias = p.expect(tkIdent).value
|
||||||
|
elif p.peek().kind == tkIdent:
|
||||||
|
joinAlias = p.advance().value
|
||||||
|
joinTarget = Node(kind: nkSubquery, subQuery: subquery,
|
||||||
|
line: tok.line, col: tok.col)
|
||||||
|
else:
|
||||||
|
let joinTable = p.expect(tkIdent)
|
||||||
|
if p.match(tkAs):
|
||||||
|
joinAlias = p.expect(tkIdent).value
|
||||||
|
elif p.peek().kind == tkIdent:
|
||||||
|
joinAlias = p.advance().value
|
||||||
|
joinTarget = Node(kind: nkFrom, fromTable: joinTable.value,
|
||||||
|
fromAlias: joinAlias, line: joinTable.line, col: joinTable.col)
|
||||||
var joinCond: Node = nil
|
var joinCond: Node = nil
|
||||||
if p.match(tkOn):
|
if p.match(tkOn):
|
||||||
joinCond = p.parseExpr()
|
joinCond = p.parseExpr()
|
||||||
let joinTarget = Node(kind: nkFrom, fromTable: joinTable.value,
|
|
||||||
fromAlias: joinAlias, line: joinTable.line, col: joinTable.col)
|
|
||||||
result.selJoins.add(Node(kind: nkJoin, joinKind: jk, joinTarget: joinTarget,
|
result.selJoins.add(Node(kind: nkJoin, joinKind: jk, joinTarget: joinTarget,
|
||||||
joinOn: joinCond, joinAlias: joinAlias,
|
joinOn: joinCond, joinAlias: joinAlias, joinLateral: isLateral,
|
||||||
line: joinTable.line, col: joinTable.col))
|
line: tok.line, col: tok.col))
|
||||||
|
|
||||||
# Parse WHERE
|
# Parse WHERE
|
||||||
if p.match(tkWhere):
|
if p.match(tkWhere):
|
||||||
@@ -394,9 +620,47 @@ proc parseSelect(p: var Parser): Node =
|
|||||||
if p.match(tkGroup):
|
if p.match(tkGroup):
|
||||||
discard p.expect(tkBy)
|
discard p.expect(tkBy)
|
||||||
result.selGroupBy = @[]
|
result.selGroupBy = @[]
|
||||||
result.selGroupBy.add(p.parseExpr())
|
result.selGroupingSetsKind = gskNone
|
||||||
while p.match(tkComma):
|
result.selGroupingSets = @[]
|
||||||
|
# Check for GROUPING SETS, ROLLUP, CUBE
|
||||||
|
if p.peek().kind == tkGrouping:
|
||||||
|
discard p.advance()
|
||||||
|
discard p.expect(tkSets)
|
||||||
|
result.selGroupingSetsKind = gskGroupingSets
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
# Parse each set: (col1, col2) or ()
|
||||||
|
while true:
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
var setExprs: seq[Node] = @[]
|
||||||
|
if p.peek().kind != tkRParen:
|
||||||
|
setExprs.add(p.parseExpr())
|
||||||
|
while p.match(tkComma):
|
||||||
|
setExprs.add(p.parseExpr())
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
result.selGroupingSets.add(setExprs)
|
||||||
|
if not p.match(tkComma): break
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
elif p.peek().kind == tkRollup:
|
||||||
|
discard p.advance()
|
||||||
|
result.selGroupingSetsKind = gskRollup
|
||||||
|
discard p.expect(tkLParen)
|
||||||
result.selGroupBy.add(p.parseExpr())
|
result.selGroupBy.add(p.parseExpr())
|
||||||
|
while p.match(tkComma):
|
||||||
|
result.selGroupBy.add(p.parseExpr())
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
elif p.peek().kind == tkCube:
|
||||||
|
discard p.advance()
|
||||||
|
result.selGroupingSetsKind = gskCube
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
result.selGroupBy.add(p.parseExpr())
|
||||||
|
while p.match(tkComma):
|
||||||
|
result.selGroupBy.add(p.parseExpr())
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
else:
|
||||||
|
# Regular GROUP BY
|
||||||
|
result.selGroupBy.add(p.parseExpr())
|
||||||
|
while p.match(tkComma):
|
||||||
|
result.selGroupBy.add(p.parseExpr())
|
||||||
|
|
||||||
# Parse HAVING
|
# Parse HAVING
|
||||||
if p.match(tkHaving):
|
if p.match(tkHaving):
|
||||||
@@ -540,6 +804,64 @@ proc parseDelete(p: var Parser): Node =
|
|||||||
while p.match(tkComma):
|
while p.match(tkComma):
|
||||||
result.delReturning.add(p.parseExpr())
|
result.delReturning.add(p.parseExpr())
|
||||||
|
|
||||||
|
proc parseMerge(p: var Parser): Node =
|
||||||
|
let tok = p.expect(tkMerge)
|
||||||
|
discard p.match(tkInto) # optional INTO
|
||||||
|
result = Node(kind: nkMerge, line: tok.line, col: tok.col)
|
||||||
|
result.mergeTarget = p.expect(tkIdent).value
|
||||||
|
if p.match(tkAs):
|
||||||
|
result.mergeTargetAlias = p.expect(tkIdent).value
|
||||||
|
elif p.peek().kind == tkIdent:
|
||||||
|
result.mergeTargetAlias = p.advance().value
|
||||||
|
discard p.expect(tkUsing)
|
||||||
|
# Source can be a table name or a subquery
|
||||||
|
if p.peek().kind == tkLParen:
|
||||||
|
discard p.advance() # consume (
|
||||||
|
result.mergeSource = p.parseSelect()
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
else:
|
||||||
|
let srcTable = p.expect(tkIdent).value
|
||||||
|
result.mergeSource = Node(kind: nkIdent, identName: srcTable, line: tok.line, col: tok.col)
|
||||||
|
if p.match(tkAs):
|
||||||
|
result.mergeSourceAlias = p.expect(tkIdent).value
|
||||||
|
elif p.peek().kind == tkIdent:
|
||||||
|
result.mergeSourceAlias = p.advance().value
|
||||||
|
discard p.expect(tkOn)
|
||||||
|
result.mergeOn = p.parseExpr()
|
||||||
|
# WHEN MATCHED THEN UPDATE SET ...
|
||||||
|
if p.match(tkWhen):
|
||||||
|
if p.match(tkNot):
|
||||||
|
discard p.expect(tkMatched)
|
||||||
|
discard p.expect(tkThen)
|
||||||
|
discard p.expect(tkInsert)
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
result.mergeNotMatchedInsert.add(Node(kind: nkIdent, identName: p.expect(tkIdent).value))
|
||||||
|
while p.match(tkComma):
|
||||||
|
result.mergeNotMatchedInsert.add(Node(kind: nkIdent, identName: p.expect(tkIdent).value))
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
discard p.expect(tkValues)
|
||||||
|
discard p.expect(tkLParen)
|
||||||
|
result.mergeNotMatchedValues.add(p.parseExpr())
|
||||||
|
while p.match(tkComma):
|
||||||
|
result.mergeNotMatchedValues.add(p.parseExpr())
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
else:
|
||||||
|
discard p.expect(tkMatched)
|
||||||
|
discard p.expect(tkThen)
|
||||||
|
discard p.expect(tkUpdate)
|
||||||
|
discard p.expect(tkSet)
|
||||||
|
let col = p.expect(tkIdent).value
|
||||||
|
discard p.expect(tkEq)
|
||||||
|
result.mergeMatchedUpdate.add(Node(kind: nkBinOp, binOp: bkAssign,
|
||||||
|
binLeft: Node(kind: nkIdent, identName: col),
|
||||||
|
binRight: p.parseExpr()))
|
||||||
|
while p.match(tkComma):
|
||||||
|
let col2 = p.expect(tkIdent).value
|
||||||
|
discard p.expect(tkEq)
|
||||||
|
result.mergeMatchedUpdate.add(Node(kind: nkBinOp, binOp: bkAssign,
|
||||||
|
binLeft: Node(kind: nkIdent, identName: col2),
|
||||||
|
binRight: p.parseExpr()))
|
||||||
|
|
||||||
proc parseCreateType(p: var Parser): Node =
|
proc parseCreateType(p: var Parser): Node =
|
||||||
let tok = p.expect(tkCreate)
|
let tok = p.expect(tkCreate)
|
||||||
discard p.expect(tkType)
|
discard p.expect(tkType)
|
||||||
@@ -672,6 +994,14 @@ proc parseCreateTable(p: var Parser): Node =
|
|||||||
let size = p.expect(tkIntLit).value
|
let size = p.expect(tkIntLit).value
|
||||||
colType &= "(" & size & ")"
|
colType &= "(" & size & ")"
|
||||||
discard p.expect(tkRParen)
|
discard p.expect(tkRParen)
|
||||||
|
elif p.peek().kind == tkVector:
|
||||||
|
discard p.advance()
|
||||||
|
colType = "VECTOR"
|
||||||
|
if p.peek().kind == tkLParen:
|
||||||
|
discard p.advance()
|
||||||
|
let size = p.expect(tkIntLit).value
|
||||||
|
colType &= "(" & size & ")"
|
||||||
|
discard p.expect(tkRParen)
|
||||||
|
|
||||||
let colDef = Node(kind: nkColumnDef, cdName: colName, cdType: colType)
|
let colDef = Node(kind: nkColumnDef, cdName: colName, cdType: colType)
|
||||||
colDef.cdConstraints = @[]
|
colDef.cdConstraints = @[]
|
||||||
@@ -737,13 +1067,13 @@ proc parseAlterTable(p: var Parser): Node =
|
|||||||
# Check for ENABLE/DISABLE ROW LEVEL SECURITY
|
# Check for ENABLE/DISABLE ROW LEVEL SECURITY
|
||||||
if p.peek().kind == tkEnable:
|
if p.peek().kind == tkEnable:
|
||||||
discard p.advance()
|
discard p.advance()
|
||||||
discard p.expect(tkIdent) # ROW
|
discard p.expect(tkRow) # ROW
|
||||||
discard p.expect(tkIdent) # LEVEL
|
discard p.expect(tkIdent) # LEVEL
|
||||||
discard p.expect(tkIdent) # SECURITY
|
discard p.expect(tkIdent) # SECURITY
|
||||||
return Node(kind: nkEnableRLS, erlsTable: tableName, line: tok.line, col: tok.col)
|
return Node(kind: nkEnableRLS, erlsTable: tableName, line: tok.line, col: tok.col)
|
||||||
elif p.peek().kind == tkDisable:
|
elif p.peek().kind == tkDisable:
|
||||||
discard p.advance()
|
discard p.advance()
|
||||||
discard p.expect(tkIdent) # ROW
|
discard p.expect(tkRow) # ROW
|
||||||
discard p.expect(tkIdent) # LEVEL
|
discard p.expect(tkIdent) # LEVEL
|
||||||
discard p.expect(tkIdent) # SECURITY
|
discard p.expect(tkIdent) # SECURITY
|
||||||
return Node(kind: nkDisableRLS, drlsTable: tableName, line: tok.line, col: tok.col)
|
return Node(kind: nkDisableRLS, drlsTable: tableName, line: tok.line, col: tok.col)
|
||||||
@@ -781,6 +1111,10 @@ proc parseCreateIndex(p: var Parser): Node =
|
|||||||
let idxMethod = p.expect(tkIdent).value.toLower()
|
let idxMethod = p.expect(tkIdent).value.toLower()
|
||||||
if idxMethod == "fts" or idxMethod == "fulltext":
|
if idxMethod == "fts" or idxMethod == "fulltext":
|
||||||
idxKind = ikFullText
|
idxKind = ikFullText
|
||||||
|
elif idxMethod == "hnsw":
|
||||||
|
idxKind = ikHNSW
|
||||||
|
elif idxMethod == "ivfpq":
|
||||||
|
idxKind = ikIVFPQ
|
||||||
result = Node(kind: nkCreateIndex, ciName: idxName, ciTarget: tableName,
|
result = Node(kind: nkCreateIndex, ciName: idxName, ciTarget: tableName,
|
||||||
ciColumns: colNames, ciKind: idxKind, line: tok.line, col: tok.col)
|
ciColumns: colNames, ciKind: idxKind, line: tok.line, col: tok.col)
|
||||||
|
|
||||||
@@ -1099,12 +1433,42 @@ proc parseRevoke(p: var Parser): Node =
|
|||||||
result = Node(kind: nkRevoke, rvPrivilege: priv, rvTable: tableName,
|
result = Node(kind: nkRevoke, rvPrivilege: priv, rvTable: tableName,
|
||||||
rvGrantee: grantee, line: tok.line, col: tok.col)
|
rvGrantee: grantee, line: tok.line, col: tok.col)
|
||||||
|
|
||||||
|
proc parseSetVar(p: var Parser): Node =
|
||||||
|
let tok = p.expect(tkSet)
|
||||||
|
var varName = p.expect(tkIdent).value
|
||||||
|
while p.peek().kind == tkDot:
|
||||||
|
discard p.advance()
|
||||||
|
varName.add(".")
|
||||||
|
varName.add(p.expect(tkIdent).value)
|
||||||
|
if p.match(tkEq) or p.match(tkTo):
|
||||||
|
discard
|
||||||
|
let valTok = p.peek()
|
||||||
|
var valStr = ""
|
||||||
|
case valTok.kind
|
||||||
|
of tkStringLit:
|
||||||
|
valStr = valTok.value
|
||||||
|
discard p.advance()
|
||||||
|
of tkIntLit:
|
||||||
|
valStr = valTok.value
|
||||||
|
discard p.advance()
|
||||||
|
of tkFloatLit:
|
||||||
|
valStr = valTok.value
|
||||||
|
discard p.advance()
|
||||||
|
of tkIdent:
|
||||||
|
valStr = valTok.value
|
||||||
|
discard p.advance()
|
||||||
|
else:
|
||||||
|
raise newException(ValueError, "Expected value after SET " & varName)
|
||||||
|
result = Node(kind: nkSetVar, svName: varName, svValue: valStr,
|
||||||
|
line: tok.line, col: tok.col)
|
||||||
|
|
||||||
proc parseStatement*(p: var Parser): Node =
|
proc parseStatement*(p: var Parser): Node =
|
||||||
case p.peek().kind
|
case p.peek().kind
|
||||||
of tkWith, tkSelect: p.parseSelect()
|
of tkWith, tkSelect: p.parseSelect()
|
||||||
of tkInsert: p.parseInsert()
|
of tkInsert: p.parseInsert()
|
||||||
of tkUpdate: p.parseUpdate()
|
of tkUpdate: p.parseUpdate()
|
||||||
of tkDelete: p.parseDelete()
|
of tkDelete: p.parseDelete()
|
||||||
|
of tkMerge: p.parseMerge()
|
||||||
of tkCreate:
|
of tkCreate:
|
||||||
if p.pos + 1 < p.tokens.len:
|
if p.pos + 1 < p.tokens.len:
|
||||||
let next = p.tokens[p.pos + 1]
|
let next = p.tokens[p.pos + 1]
|
||||||
@@ -1159,6 +1523,8 @@ proc parseStatement*(p: var Parser): Node =
|
|||||||
p.parseGrant()
|
p.parseGrant()
|
||||||
of tkRevoke:
|
of tkRevoke:
|
||||||
p.parseRevoke()
|
p.parseRevoke()
|
||||||
|
of tkSet:
|
||||||
|
p.parseSetVar()
|
||||||
of tkApply:
|
of tkApply:
|
||||||
if p.pos + 1 < p.tokens.len and p.tokens[p.pos + 1].kind == tkMigration:
|
if p.pos + 1 < p.tokens.len and p.tokens[p.pos + 1].kind == tkMigration:
|
||||||
p.parseApplyMigration()
|
p.parseApplyMigration()
|
||||||
|
|||||||
@@ -79,3 +79,32 @@ suite "JOIN execution":
|
|||||||
test "count after CROSS JOIN":
|
test "count after CROSS JOIN":
|
||||||
let r = execSql(ctx, "SELECT COUNT(*) AS cnt FROM users u CROSS JOIN orders o")
|
let r = execSql(ctx, "SELECT COUNT(*) AS cnt FROM users u CROSS JOIN orders o")
|
||||||
check r.rows[0]["cnt"] == "6"
|
check r.rows[0]["cnt"] == "6"
|
||||||
|
|
||||||
|
test "LATERAL JOIN with correlated subquery":
|
||||||
|
let r = execSql(ctx,
|
||||||
|
"SELECT u.name, recent.total FROM users u JOIN LATERAL (SELECT o.total FROM orders o WHERE o.user_id = u.id ORDER BY o.total DESC LIMIT 1) AS recent ON 1=1")
|
||||||
|
check r.rows.len == 1
|
||||||
|
check r.rows[0]["name"] == "Alice"
|
||||||
|
check r.rows[0]["total"] == "99.5"
|
||||||
|
|
||||||
|
test "LATERAL JOIN returns no rows when subquery empty":
|
||||||
|
let r = execSql(ctx,
|
||||||
|
"SELECT u.name, x.total FROM users u JOIN LATERAL (SELECT o.total FROM orders o WHERE o.user_id = u.id AND o.total > 1000) AS x ON 1=1")
|
||||||
|
check r.rows.len == 0
|
||||||
|
|
||||||
|
test "LEFT LATERAL JOIN keeps unmatched rows":
|
||||||
|
let r = execSql(ctx,
|
||||||
|
"SELECT u.name, x.total FROM users u LEFT JOIN LATERAL (SELECT o.total FROM orders o WHERE o.user_id = u.id ORDER BY o.total DESC LIMIT 1) AS x ON 1=1")
|
||||||
|
check r.rows.len == 2
|
||||||
|
# Alice has match (99.5), Bob has no orders -> NULL
|
||||||
|
var foundBob = false
|
||||||
|
for row in r.rows:
|
||||||
|
if row["name"] == "Bob":
|
||||||
|
check row["total"] == ""
|
||||||
|
foundBob = true
|
||||||
|
check foundBob
|
||||||
|
|
||||||
|
test "CROSS JOIN LATERAL":
|
||||||
|
let r = execSql(ctx,
|
||||||
|
"SELECT u.name, x.total FROM users u CROSS JOIN LATERAL (SELECT o.total FROM orders o WHERE o.user_id = u.id) AS x")
|
||||||
|
check r.rows.len == 2 # Alice has 2 orders, Bob has 0
|
||||||
|
|||||||
+526
-9
@@ -969,8 +969,40 @@ suite "BaraQL Parser — Extended":
|
|||||||
|
|
||||||
test "Parse GROUP BY with HAVING":
|
test "Parse GROUP BY with HAVING":
|
||||||
let ast = parse("SELECT dept, count(*) FROM employees GROUP BY dept HAVING count(*) > 5")
|
let ast = parse("SELECT dept, count(*) FROM employees GROUP BY dept HAVING count(*) > 5")
|
||||||
|
check ast.stmts[0].selGroupBy.len == 1
|
||||||
check ast.stmts[0].selHaving != nil
|
check ast.stmts[0].selHaving != nil
|
||||||
|
|
||||||
|
test "Parse FILTER clause":
|
||||||
|
let ast = parse("SELECT COUNT(*) FILTER (WHERE active = true) FROM users")
|
||||||
|
check ast.stmts[0].selResult.len == 1
|
||||||
|
check ast.stmts[0].selResult[0].funcFilter != nil
|
||||||
|
|
||||||
|
test "Parse ROLLUP":
|
||||||
|
let ast = parse("SELECT dept, SUM(amount) FROM sales GROUP BY ROLLUP (dept)")
|
||||||
|
check ast.stmts[0].selGroupingSetsKind == gskRollup
|
||||||
|
check ast.stmts[0].selGroupBy.len == 1
|
||||||
|
|
||||||
|
test "Parse CUBE":
|
||||||
|
let ast = parse("SELECT dept, job, SUM(amount) FROM sales GROUP BY CUBE (dept, job)")
|
||||||
|
check ast.stmts[0].selGroupingSetsKind == gskCube
|
||||||
|
check ast.stmts[0].selGroupBy.len == 2
|
||||||
|
|
||||||
|
test "Parse GROUPING SETS":
|
||||||
|
let ast = parse("SELECT dept, job, SUM(amount) FROM sales GROUP BY GROUPING SETS ((dept), (job), ())")
|
||||||
|
check ast.stmts[0].selGroupingSetsKind == gskGroupingSets
|
||||||
|
check ast.stmts[0].selGroupingSets.len == 3
|
||||||
|
|
||||||
|
test "Parse PIVOT":
|
||||||
|
let ast = parse("SELECT * FROM (SELECT dept, salary FROM emp) PIVOT (SUM(salary) FOR dept IN ('Eng', 'Sales'))")
|
||||||
|
check ast.stmts[0].selFrom.kind == nkPivot
|
||||||
|
check ast.stmts[0].selFrom.pivotForCol == "dept"
|
||||||
|
check ast.stmts[0].selFrom.pivotInValues.len == 2
|
||||||
|
|
||||||
|
test "Parse GRAPH_TABLE":
|
||||||
|
let ast = parse("SELECT * FROM GRAPH_TABLE(org_chart MATCH (e)-[r]->(d) COLUMNS (e.name, d.name))")
|
||||||
|
check ast.stmts[0].selFrom.kind == nkGraphTraversal
|
||||||
|
check ast.stmts[0].selFrom.gtGraphName == "org_chart"
|
||||||
|
|
||||||
test "Parse ORDER BY with direction":
|
test "Parse ORDER BY with direction":
|
||||||
let ast = parse("SELECT name FROM users ORDER BY age DESC")
|
let ast = parse("SELECT name FROM users ORDER BY age DESC")
|
||||||
check ast.stmts[0].selOrderBy.len == 1
|
check ast.stmts[0].selOrderBy.len == 1
|
||||||
@@ -994,6 +1026,25 @@ suite "BaraQL Parser — Extended":
|
|||||||
let ast = parse("SELECT * FROM a JOIN b ON a.id = b.aid JOIN c ON b.id = c.bid")
|
let ast = parse("SELECT * FROM a JOIN b ON a.id = b.aid JOIN c ON b.id = c.bid")
|
||||||
check ast.stmts[0].selJoins.len == 2
|
check ast.stmts[0].selJoins.len == 2
|
||||||
|
|
||||||
|
test "Parse LATERAL JOIN":
|
||||||
|
let ast = parse("SELECT u.name, x.total FROM users u JOIN LATERAL (SELECT o.total FROM orders o WHERE o.user_id = u.id) AS x ON 1=1")
|
||||||
|
check ast.stmts[0].selJoins.len == 1
|
||||||
|
check ast.stmts[0].selJoins[0].joinLateral == true
|
||||||
|
check ast.stmts[0].selJoins[0].joinTarget.kind == nkSubquery
|
||||||
|
check ast.stmts[0].selJoins[0].joinAlias == "x"
|
||||||
|
|
||||||
|
test "Parse LEFT LATERAL JOIN":
|
||||||
|
let ast = parse("SELECT u.name FROM users u LEFT JOIN LATERAL (SELECT 1) AS x ON 1=1")
|
||||||
|
check ast.stmts[0].selJoins.len == 1
|
||||||
|
check ast.stmts[0].selJoins[0].joinKind == jkLeft
|
||||||
|
check ast.stmts[0].selJoins[0].joinLateral == true
|
||||||
|
|
||||||
|
test "Parse CROSS JOIN LATERAL":
|
||||||
|
let ast = parse("SELECT * FROM users u CROSS JOIN LATERAL (SELECT 1) AS x")
|
||||||
|
check ast.stmts[0].selJoins.len == 1
|
||||||
|
check ast.stmts[0].selJoins[0].joinKind == jkCross
|
||||||
|
check ast.stmts[0].selJoins[0].joinLateral == true
|
||||||
|
|
||||||
test "Parse CTE (WITH)":
|
test "Parse CTE (WITH)":
|
||||||
let ast = parse("WITH active AS (SELECT * FROM users WHERE active = true) SELECT * FROM active")
|
let ast = parse("WITH active AS (SELECT * FROM users WHERE active = true) SELECT * FROM active")
|
||||||
check ast.stmts[0].selWith.len == 1
|
check ast.stmts[0].selWith.len == 1
|
||||||
@@ -2236,6 +2287,69 @@ suite "Row-Level Security":
|
|||||||
check res.success
|
check res.success
|
||||||
check res.rows.len == 1 # superuser sees all (only 1 row exists)
|
check res.rows.len == 1 # superuser sees all (only 1 row exists)
|
||||||
|
|
||||||
|
suite "Session Variables and Multi-Tenant":
|
||||||
|
test "SET statement parse":
|
||||||
|
let ast = parse("SET app.tenant_id = 'company-123'")
|
||||||
|
check ast.stmts.len == 1
|
||||||
|
check ast.stmts[0].kind == nkSetVar
|
||||||
|
check ast.stmts[0].svName == "app.tenant_id"
|
||||||
|
check ast.stmts[0].svValue == "company-123"
|
||||||
|
|
||||||
|
test "SET and current_setting":
|
||||||
|
let tmpDir = getTempDir() / "baradb_session_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
|
var ctx = qexec.newExecutionContext(db)
|
||||||
|
discard qexec.executeQuery(ctx, parse("SET app.tenant_id = 'company-123'"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT current_setting('app.tenant_id') AS tenant"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
check r.rows[0]["tenant"] == "company-123"
|
||||||
|
|
||||||
|
test "current_user in SELECT":
|
||||||
|
let tmpDir = getTempDir() / "baradb_user_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
|
var ctx = qexec.newExecutionContext(db)
|
||||||
|
ctx.currentUser = "alice"
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT current_user AS me"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
check r.rows[0]["me"] == "alice"
|
||||||
|
|
||||||
|
test "current_role in SELECT":
|
||||||
|
let tmpDir = getTempDir() / "baradb_role_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
|
var ctx = qexec.newExecutionContext(db)
|
||||||
|
ctx.currentRole = "admin"
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT current_role AS role"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
check r.rows[0]["role"] == "admin"
|
||||||
|
|
||||||
|
test "Multi-tenant RLS with current_setting":
|
||||||
|
let tmpDir = getTempDir() / "baradb_multitenant_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
|
var ctx = qexec.newExecutionContext(db)
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE invoices (id INTEGER, tenant_id TEXT, amount INT)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO invoices (id, tenant_id, amount) VALUES (1, 'company-a', 100)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO invoices (id, tenant_id, amount) VALUES (2, 'company-b', 200)"))
|
||||||
|
# Set session variable for tenant
|
||||||
|
discard qexec.executeQuery(ctx, parse("SET app.tenant_id = 'company-a'"))
|
||||||
|
# Create policy that uses current_setting
|
||||||
|
ctx.users["app"] = qexec.UserDef(name: "app", passwordHash: "", isSuperuser: false, roles: @[])
|
||||||
|
ctx.currentUser = "app"
|
||||||
|
ctx.policies["invoices"] = @[
|
||||||
|
qexec.PolicyDef(name: "tenant_isolation", tableName: "invoices", command: "SELECT",
|
||||||
|
usingExpr: Node(kind: nkBinOp, binOp: bkEq,
|
||||||
|
binLeft: Node(kind: nkIdent, identName: "tenant_id"),
|
||||||
|
binRight: Node(kind: nkFuncCall, funcName: "current_setting",
|
||||||
|
funcArgs: @[Node(kind: nkStringLit, strVal: "app.tenant_id")])),
|
||||||
|
withCheckExpr: nil)
|
||||||
|
]
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id, tenant_id FROM invoices"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
check r.rows[0]["tenant_id"] == "company-a"
|
||||||
|
|
||||||
suite "UTF-8 Support":
|
suite "UTF-8 Support":
|
||||||
test "Tokenize UTF-8 identifiers":
|
test "Tokenize UTF-8 identifiers":
|
||||||
let tokens = lex.tokenize("SELECT имя FROM потребители")
|
let tokens = lex.tokenize("SELECT имя FROM потребители")
|
||||||
@@ -2252,7 +2366,8 @@ suite "UTF-8 Support":
|
|||||||
check ast.stmts[0].selWhere.whereExpr.binRight.strVal == "София"
|
check ast.stmts[0].selWhere.whereExpr.binRight.strVal == "София"
|
||||||
|
|
||||||
test "Execute query with UTF-8 data":
|
test "Execute query with UTF-8 data":
|
||||||
var db = newLSMTree("")
|
let tmpDir = getTempDir() / "baradb_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
var ctx = qexec.newExecutionContext(db)
|
var ctx = qexec.newExecutionContext(db)
|
||||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE потребители (имя TEXT, град TEXT)"))
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE потребители (имя TEXT, град TEXT)"))
|
||||||
discard qexec.executeQuery(ctx, parse("INSERT INTO потребители (имя, град) VALUES ('Иван', 'София'), ('Мария', 'Пловдив')"))
|
discard qexec.executeQuery(ctx, parse("INSERT INTO потребители (имя, град) VALUES ('Иван', 'София'), ('Мария', 'Пловдив')"))
|
||||||
@@ -2264,7 +2379,8 @@ suite "UTF-8 Support":
|
|||||||
|
|
||||||
suite "B-Tree Range Scan":
|
suite "B-Tree Range Scan":
|
||||||
test "BETWEEN uses index range scan":
|
test "BETWEEN uses index range scan":
|
||||||
var db = newLSMTree("")
|
let tmpDir = getTempDir() / "baradb_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
var ctx = qexec.newExecutionContext(db)
|
var ctx = qexec.newExecutionContext(db)
|
||||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE products (id INTEGER, name TEXT)"))
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE products (id INTEGER, name TEXT)"))
|
||||||
discard qexec.executeQuery(ctx, parse("INSERT INTO products (id, name) VALUES (1, 'apple'), (2, 'banana'), (3, 'cherry'), (4, 'date'), (5, 'elderberry')"))
|
discard qexec.executeQuery(ctx, parse("INSERT INTO products (id, name) VALUES (1, 'apple'), (2, 'banana'), (3, 'cherry'), (4, 'date'), (5, 'elderberry')"))
|
||||||
@@ -2274,7 +2390,8 @@ suite "B-Tree Range Scan":
|
|||||||
check res.rows.len == 3
|
check res.rows.len == 3
|
||||||
|
|
||||||
test "Greater than uses index range scan":
|
test "Greater than uses index range scan":
|
||||||
var db = newLSMTree("")
|
let tmpDir = getTempDir() / "baradb_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
var ctx = qexec.newExecutionContext(db)
|
var ctx = qexec.newExecutionContext(db)
|
||||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE nums (id INTEGER, val TEXT)"))
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE nums (id INTEGER, val TEXT)"))
|
||||||
discard qexec.executeQuery(ctx, parse("INSERT INTO nums (id, val) VALUES (1, '10'), (2, '20'), (3, '30'), (4, '40'), (5, '50')"))
|
discard qexec.executeQuery(ctx, parse("INSERT INTO nums (id, val) VALUES (1, '10'), (2, '20'), (3, '30'), (4, '40'), (5, '50')"))
|
||||||
@@ -2284,7 +2401,8 @@ suite "B-Tree Range Scan":
|
|||||||
check res.rows.len == 3
|
check res.rows.len == 3
|
||||||
|
|
||||||
test "Less than or equal uses index range scan":
|
test "Less than or equal uses index range scan":
|
||||||
var db = newLSMTree("")
|
let tmpDir = getTempDir() / "baradb_test_" & $getMonoTime().ticks
|
||||||
|
var db = newLSMTree(tmpDir)
|
||||||
var ctx = qexec.newExecutionContext(db)
|
var ctx = qexec.newExecutionContext(db)
|
||||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE nums2 (id INTEGER, val TEXT)"))
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE nums2 (id INTEGER, val TEXT)"))
|
||||||
discard qexec.executeQuery(ctx, parse("INSERT INTO nums2 (id, val) VALUES (1, '10'), (2, '20'), (3, '30'), (4, '40'), (5, '50')"))
|
discard qexec.executeQuery(ctx, parse("INSERT INTO nums2 (id, val) VALUES (1, '10'), (2, '20'), (3, '30'), (4, '40'), (5, '50')"))
|
||||||
@@ -2407,13 +2525,18 @@ suite "Enhanced Migrations":
|
|||||||
suite "Parameterized queries":
|
suite "Parameterized queries":
|
||||||
var db: LSMTree
|
var db: LSMTree
|
||||||
var ctx: qexec.ExecutionContext
|
var ctx: qexec.ExecutionContext
|
||||||
|
var tmpDir: string
|
||||||
|
var paramInitialized = false
|
||||||
|
|
||||||
setup:
|
setup:
|
||||||
db = newLSMTree("")
|
if not paramInitialized:
|
||||||
ctx = qexec.newExecutionContext(db)
|
tmpDir = getTempDir() / "baradb_param_test_" & $getMonoTime().ticks
|
||||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE users (id INT, name TEXT, age INT)"))
|
db = newLSMTree(tmpDir)
|
||||||
discard qexec.executeQuery(ctx, parse("INSERT INTO users (id, name, age) VALUES (1, 'Alice', 30)"))
|
ctx = qexec.newExecutionContext(db)
|
||||||
discard qexec.executeQuery(ctx, parse("INSERT INTO users (id, name, age) VALUES (2, 'Bob', 25)"))
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE users (id INT, name TEXT, age INT, active BOOLEAN)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO users (id, name, age, active) VALUES (1, 'Alice', 30, true)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO users (id, name, age, active) VALUES (2, 'Bob', 25, false)"))
|
||||||
|
paramInitialized = true
|
||||||
|
|
||||||
test "SELECT with placeholder params":
|
test "SELECT with placeholder params":
|
||||||
let sql = "SELECT * FROM users WHERE id = ?"
|
let sql = "SELECT * FROM users WHERE id = ?"
|
||||||
@@ -2532,6 +2655,34 @@ suite "Parameterized queries":
|
|||||||
check r.success
|
check r.success
|
||||||
check r.rows.len >= 1
|
check r.rows.len >= 1
|
||||||
|
|
||||||
|
test "JSON contains @>":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE IF NOT EXISTS jsontest2 (id INT PRIMARY KEY, data JSON)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO jsontest2 (id, data) VALUES (1, '{\"name\": \"Alice\", \"age\": 30}')"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id FROM jsontest2 WHERE data @> '{\"name\": \"Alice\"}'"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
check r.rows[0]["id"] == "1"
|
||||||
|
|
||||||
|
test "JSON contained by <@":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id FROM jsontest2 WHERE '{\"name\": \"Alice\"}' <@ data"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
|
||||||
|
test "JSON has key json_has_key":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id FROM jsontest2 WHERE json_has_key(data, 'name') = 'true'"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
|
||||||
|
test "JSON has any key ?|":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id FROM jsontest2 WHERE data ?| '[\"name\", \"missing\"]'"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
|
||||||
|
test "JSON has all keys ?&":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id FROM jsontest2 WHERE data ?& '[\"name\", \"age\"]'"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
|
||||||
test "FTS match operator @@ parse":
|
test "FTS match operator @@ parse":
|
||||||
let ast = parse("SELECT * FROM docs WHERE content @@ 'hello'")
|
let ast = parse("SELECT * FROM docs WHERE content @@ 'hello'")
|
||||||
check ast.stmts[0].kind == nkSelect
|
check ast.stmts[0].kind == nkSelect
|
||||||
@@ -2576,8 +2727,374 @@ suite "Parameterized queries":
|
|||||||
check r.rows.len == 1
|
check r.rows.len == 1
|
||||||
check r.rows[0]["name"] == "Bob"
|
check r.rows[0]["name"] == "Bob"
|
||||||
|
|
||||||
|
suite "Window Functions":
|
||||||
|
var db: LSMTree
|
||||||
|
var ctx: qexec.ExecutionContext
|
||||||
|
var tmpDir: string
|
||||||
|
|
||||||
|
setup:
|
||||||
|
tmpDir = getTempDir() / "baradb_window_test_" & $getMonoTime().ticks
|
||||||
|
db = newLSMTree(tmpDir)
|
||||||
|
ctx = qexec.newExecutionContext(db)
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE employees (id INT, name TEXT, department TEXT, salary INT)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (1, 'Alice', 'Engineering', 90000)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (2, 'Bob', 'Engineering', 80000)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (3, 'Charlie', 'Sales', 70000)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (4, 'Diana', 'Sales', 75000)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (5, 'Eve', 'Engineering', 95000)"))
|
||||||
|
|
||||||
|
teardown:
|
||||||
|
removeDir(tmpDir)
|
||||||
|
|
||||||
|
test "ROW_NUMBER with PARTITION BY and ORDER BY":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT name, department, ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn FROM employees"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 5
|
||||||
|
# Engineering: Eve(95000)=1, Alice(90000)=2, Bob(80000)=3
|
||||||
|
# Sales: Diana(75000)=1, Charlie(70000)=2
|
||||||
|
var found = initTable[string, string]()
|
||||||
|
for row in r.rows:
|
||||||
|
found[row["name"]] = row["rn"]
|
||||||
|
check found["Eve"] == "1"
|
||||||
|
check found["Alice"] == "2"
|
||||||
|
check found["Bob"] == "3"
|
||||||
|
check found["Diana"] == "1"
|
||||||
|
check found["Charlie"] == "2"
|
||||||
|
|
||||||
|
test "RANK and DENSE_RANK":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT name, salary, RANK() OVER (ORDER BY salary DESC) AS r, DENSE_RANK() OVER (ORDER BY salary DESC) AS dr FROM employees"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 5
|
||||||
|
for row in r.rows:
|
||||||
|
if row["name"] == "Eve":
|
||||||
|
check row["r"] == "1"
|
||||||
|
check row["dr"] == "1"
|
||||||
|
if row["name"] == "Alice":
|
||||||
|
check row["r"] == "2"
|
||||||
|
check row["dr"] == "2"
|
||||||
|
|
||||||
|
test "LEAD and LAG":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT name, salary, LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev, LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next FROM employees"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 5
|
||||||
|
for row in r.rows:
|
||||||
|
if row["name"] == "Charlie":
|
||||||
|
check row["prev"] == "0"
|
||||||
|
check row["next"] == "75000"
|
||||||
|
if row["name"] == "Diana":
|
||||||
|
check row["prev"] == "70000"
|
||||||
|
check row["next"] == "80000"
|
||||||
|
|
||||||
|
test "FIRST_VALUE and LAST_VALUE":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT department, FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS first, LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS last FROM employees"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 5
|
||||||
|
|
||||||
|
test "NTILE":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT name, NTILE(2) OVER (ORDER BY salary) AS bucket FROM employees"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 5
|
||||||
|
for row in r.rows:
|
||||||
|
check row["bucket"] in @["1", "2"]
|
||||||
|
|
||||||
|
test "FIRST_VALUE with frame":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT name, salary, FIRST_VALUE(salary) OVER (ORDER BY salary ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS first_sal FROM employees"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 5
|
||||||
|
for row in r.rows:
|
||||||
|
if row["name"] == "Charlie":
|
||||||
|
check row["first_sal"] == "70000"
|
||||||
|
if row["name"] == "Diana":
|
||||||
|
check row["first_sal"] == "70000"
|
||||||
|
if row["name"] == "Bob":
|
||||||
|
check row["first_sal"] == "75000"
|
||||||
|
|
||||||
|
test "LAST_VALUE with frame":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT name, salary, LAST_VALUE(salary) OVER (ORDER BY salary ROWS BETWEEN CURRENT ROW AND 1 FOLLOWING) AS last_sal FROM employees"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 5
|
||||||
|
for row in r.rows:
|
||||||
|
if row["name"] == "Charlie":
|
||||||
|
check row["last_sal"] == "75000"
|
||||||
|
if row["name"] == "Diana":
|
||||||
|
check row["last_sal"] == "80000"
|
||||||
|
if row["name"] == "Bob":
|
||||||
|
check row["last_sal"] == "90000"
|
||||||
|
|
||||||
|
suite "GROUP BY Aggregates":
|
||||||
|
var db: LSMTree
|
||||||
|
var ctx: qexec.ExecutionContext
|
||||||
|
var tmpDir: string
|
||||||
|
|
||||||
|
setup:
|
||||||
|
tmpDir = getTempDir() / "baradb_group_test_" & $getMonoTime().ticks
|
||||||
|
db = newLSMTree(tmpDir)
|
||||||
|
ctx = qexec.newExecutionContext(db)
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE sales (id INT, dept TEXT, amount INT)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO sales (id, dept, amount) VALUES (1, 'A', 100)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO sales (id, dept, amount) VALUES (2, 'A', 200)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO sales (id, dept, amount) VALUES (3, 'B', 150)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO sales (id, dept, amount) VALUES (4, 'B', 50)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO sales (id, dept, amount) VALUES (5, 'C', 300)"))
|
||||||
|
|
||||||
|
teardown:
|
||||||
|
removeDir(tmpDir)
|
||||||
|
|
||||||
|
test "GROUP BY with COUNT(*)":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, COUNT(*) AS cnt FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
|
||||||
|
test "GROUP BY with SUM":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, SUM(amount) AS total FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
var foundA = false
|
||||||
|
for row in r.rows:
|
||||||
|
if row["dept"] == "A":
|
||||||
|
check row["total"] == "300.0"
|
||||||
|
foundA = true
|
||||||
|
check foundA
|
||||||
|
|
||||||
|
test "GROUP BY with AVG":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, AVG(amount) AS avg_amt FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
var foundB = false
|
||||||
|
for row in r.rows:
|
||||||
|
if row["dept"] == "B":
|
||||||
|
check row["avg_amt"] == "100.0"
|
||||||
|
foundB = true
|
||||||
|
check foundB
|
||||||
|
|
||||||
|
test "GROUP BY with MIN and MAX":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, MIN(amount) AS lo, MAX(amount) AS hi FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
var foundA = false
|
||||||
|
for row in r.rows:
|
||||||
|
if row["dept"] == "A":
|
||||||
|
check row["lo"] == "100"
|
||||||
|
check row["hi"] == "200"
|
||||||
|
foundA = true
|
||||||
|
check foundA
|
||||||
|
|
||||||
|
test "GROUP BY with HAVING":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, SUM(amount) AS total FROM sales GROUP BY dept HAVING SUM(amount) > 200"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 2 # A (300) and C (300)
|
||||||
|
for row in r.rows:
|
||||||
|
check row["dept"] in @["A", "C"]
|
||||||
|
|
||||||
|
test "GROUP BY with multiple aggregates":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, COUNT(*) AS cnt, SUM(amount) AS total, AVG(amount) AS avg_amt FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
|
||||||
|
test "COUNT with FILTER (WHERE ...)":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT COUNT(*) AS total, COUNT(*) FILTER (WHERE amount > 100) AS big FROM sales"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 1
|
||||||
|
check r.rows[0]["total"] == "5"
|
||||||
|
check r.rows[0]["big"] == "3"
|
||||||
|
|
||||||
|
test "SUM with FILTER (WHERE ...)":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, SUM(amount) FILTER (WHERE amount > 100) AS big_total FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
var foundA = false
|
||||||
|
for row in r.rows:
|
||||||
|
if row["dept"] == "A":
|
||||||
|
check row["big_total"] == "200.0"
|
||||||
|
foundA = true
|
||||||
|
check foundA
|
||||||
|
|
||||||
|
test "ARRAY_AGG":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, ARRAY_AGG(amount) AS amounts FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
var foundA = false
|
||||||
|
for row in r.rows:
|
||||||
|
if row["dept"] == "A":
|
||||||
|
check "100" in row["amounts"]
|
||||||
|
check "200" in row["amounts"]
|
||||||
|
foundA = true
|
||||||
|
check foundA
|
||||||
|
|
||||||
|
test "STRING_AGG":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, STRING_AGG(amount, ',') AS vals FROM sales GROUP BY dept"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
|
||||||
|
test "ROLLUP":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, SUM(amount) AS total FROM sales GROUP BY ROLLUP (dept)"))
|
||||||
|
check r.success
|
||||||
|
# 3 dept groups + 1 grand total = 4 rows
|
||||||
|
check r.rows.len == 4
|
||||||
|
|
||||||
|
test "PIVOT":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE emp (name TEXT, dept TEXT, salary INT)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO emp (name, dept, salary) VALUES ('Alice', 'Eng', 90000)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO emp (name, dept, salary) VALUES ('Bob', 'Eng', 80000)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO emp (name, dept, salary) VALUES ('Charlie', 'Sales', 70000)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO emp (name, dept, salary) VALUES ('Diana', 'Sales', 75000)"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT * FROM (SELECT name, dept, salary FROM emp) PIVOT (SUM(salary) FOR dept IN ('Eng', 'Sales'))"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 4 # one row per employee
|
||||||
|
|
||||||
|
test "CUBE execution":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, SUM(amount) AS total FROM sales GROUP BY CUBE (dept)"))
|
||||||
|
check r.success
|
||||||
|
# 3 dept groups + 1 grand total = 4 rows
|
||||||
|
check r.rows.len == 4
|
||||||
|
|
||||||
|
test "GROUPING SETS execution":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT dept, SUM(amount) AS total FROM sales GROUP BY GROUPING SETS ((dept), ())"))
|
||||||
|
check r.success
|
||||||
|
# 3 dept groups + 1 grand total = 4 rows
|
||||||
|
check r.rows.len == 4
|
||||||
|
|
||||||
|
test "UNPIVOT execution":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE pivoted (name TEXT, eng_salary INT, sales_salary INT)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO pivoted (name, eng_salary, sales_salary) VALUES ('Alice', 90000, 0)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO pivoted (name, eng_salary, sales_salary) VALUES ('Bob', 0, 70000)"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT * FROM pivoted UNPIVOT (salary FOR dept IN (eng_salary, sales_salary))"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 4
|
||||||
|
|
||||||
# JOIN tests
|
# JOIN tests
|
||||||
include "join_tests"
|
include "join_tests"
|
||||||
|
|
||||||
# TLA+ faithfulness tests
|
# TLA+ faithfulness tests
|
||||||
include "tla_faithfulness"
|
include "tla_faithfulness"
|
||||||
|
|
||||||
|
suite "MERGE Statement":
|
||||||
|
var db: LSMTree
|
||||||
|
var ctx: qexec.ExecutionContext
|
||||||
|
var tmpDir: string
|
||||||
|
|
||||||
|
setup:
|
||||||
|
tmpDir = getTempDir() / "baradb_merge_test_" & $getMonoTime().ticks
|
||||||
|
db = newLSMTree(tmpDir)
|
||||||
|
ctx = qexec.newExecutionContext(db)
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE inventory (id INT PRIMARY KEY, sku TEXT, qty INT)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO inventory (id, sku, qty) VALUES (1, 'SKU001', 100)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO inventory (id, sku, qty) VALUES (2, 'SKU002', 200)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE updates (sku TEXT, delta INT)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO updates (sku, delta) VALUES ('SKU001', 50)"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO updates (sku, delta) VALUES ('SKU003', 300)"))
|
||||||
|
|
||||||
|
teardown:
|
||||||
|
removeDir(tmpDir)
|
||||||
|
|
||||||
|
test "MERGE WHEN MATCHED UPDATE":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("""
|
||||||
|
MERGE INTO inventory AS target
|
||||||
|
USING updates AS source
|
||||||
|
ON target.sku = source.sku
|
||||||
|
WHEN MATCHED THEN UPDATE SET qty = target.qty + source.delta
|
||||||
|
"""))
|
||||||
|
check r.success
|
||||||
|
check r.affectedRows == 1
|
||||||
|
let verify = qexec.executeQuery(ctx, parse("SELECT * FROM inventory WHERE sku = 'SKU001'"))
|
||||||
|
check verify.rows[0]["qty"] == "150.0"
|
||||||
|
|
||||||
|
test "MERGE WHEN NOT MATCHED INSERT":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("""
|
||||||
|
MERGE INTO inventory AS target
|
||||||
|
USING updates AS source
|
||||||
|
ON target.sku = source.sku
|
||||||
|
WHEN NOT MATCHED THEN INSERT (id, sku, qty) VALUES (3, source.sku, source.delta)
|
||||||
|
"""))
|
||||||
|
check r.success
|
||||||
|
check r.affectedRows == 1
|
||||||
|
let verify = qexec.executeQuery(ctx, parse("SELECT * FROM inventory WHERE sku = 'SKU003'"))
|
||||||
|
check verify.rows.len == 1
|
||||||
|
check verify.rows[0]["qty"] == "300"
|
||||||
|
|
||||||
|
|
||||||
|
suite "Vector SQL Integration":
|
||||||
|
var db: LSMTree
|
||||||
|
var ctx: qexec.ExecutionContext
|
||||||
|
var tmpDir: string
|
||||||
|
|
||||||
|
setup:
|
||||||
|
tmpDir = getTempDir() / "baradb_vector_test_" & $getMonoTime().ticks
|
||||||
|
db = newLSMTree(tmpDir)
|
||||||
|
ctx = qexec.newExecutionContext(db)
|
||||||
|
|
||||||
|
teardown:
|
||||||
|
removeDir(tmpDir)
|
||||||
|
|
||||||
|
test "CREATE TABLE with VECTOR column":
|
||||||
|
let r = qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
check r.success
|
||||||
|
let tbl = ctx.tables["items"]
|
||||||
|
check tbl.columns.len == 2
|
||||||
|
check tbl.columns[1].colType == "VECTOR(3)"
|
||||||
|
|
||||||
|
test "INSERT vector values":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (1, '[1.0, 0.0, 0.0]')"))
|
||||||
|
check r.success
|
||||||
|
check r.affectedRows == 1
|
||||||
|
let r2 = qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (2, '[0.0, 1.0, 0.0]')"))
|
||||||
|
check r2.success
|
||||||
|
let sel = qexec.executeQuery(ctx, parse("SELECT * FROM items"))
|
||||||
|
check sel.rows.len == 2
|
||||||
|
|
||||||
|
test "SELECT with cosine_distance":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (1, '[1.0, 0.0, 0.0]')"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (2, '[0.0, 1.0, 0.0]')"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id, cosine_distance(embedding, '[1.0, 0.0, 0.0]') AS dist FROM items"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 2
|
||||||
|
check r.rows[0]["dist"] == "0.0"
|
||||||
|
check r.rows[1]["dist"] == "1.0"
|
||||||
|
|
||||||
|
test "SELECT with <-> operator":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (1, '[1.0, 0.0, 0.0]')"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (2, '[0.0, 1.0, 0.0]')"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id, embedding <-> '[1.0, 0.0, 0.0]' AS dist FROM items"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 2
|
||||||
|
check r.rows[0]["dist"] == "0.0"
|
||||||
|
check r.rows[1]["dist"] == "1.4142135623730951"
|
||||||
|
|
||||||
|
test "ORDER BY cosine_distance":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (1, '[1.0, 0.0, 0.0]')"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (2, '[0.0, 1.0, 0.0]')"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (3, '[0.5, 0.5, 0.0]')"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id FROM items ORDER BY cosine_distance(embedding, '[1.0, 0.0, 0.0]') ASC"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 3
|
||||||
|
check r.rows[0]["id"] == "1"
|
||||||
|
check r.rows[1]["id"] == "3"
|
||||||
|
check r.rows[2]["id"] == "2"
|
||||||
|
|
||||||
|
test "CREATE VECTOR INDEX":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (1, '[1.0, 0.0, 0.0]')"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (2, '[0.0, 1.0, 0.0]')"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("CREATE INDEX idx_items_vec ON items(embedding) USING hnsw"))
|
||||||
|
check r.success
|
||||||
|
check r.message.contains("HNSW")
|
||||||
|
check ctx.vectorIndexes.hasKey("items.embedding")
|
||||||
|
|
||||||
|
test "Vector dimension validation":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (1, '[1.0, 0.0]')"))
|
||||||
|
check not r.success # Should fail due to dimension mismatch
|
||||||
|
|
||||||
|
test "euclidean_distance function":
|
||||||
|
discard qexec.executeQuery(ctx, parse("CREATE TABLE items (id INT PRIMARY KEY, embedding VECTOR(3))"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (1, '[0.0, 0.0, 0.0]')"))
|
||||||
|
discard qexec.executeQuery(ctx, parse("INSERT INTO items (id, embedding) VALUES (2, '[1.0, 1.0, 1.0]')"))
|
||||||
|
let r = qexec.executeQuery(ctx, parse("SELECT id, euclidean_distance(embedding, '[0.0, 0.0, 0.0]') AS dist FROM items"))
|
||||||
|
check r.success
|
||||||
|
check r.rows.len == 2
|
||||||
|
check r.rows[0]["dist"] == "0.0"
|
||||||
|
check r.rows[1]["dist"] == "1.7320508075688772"
|
||||||
|
|||||||
Reference in New Issue
Block a user