docs: update README and all docs with formal verification, new BaraQL features, OpenTelemetry tracing
- Add Formal Verification section to README and architecture docs - Document TLA+ specs for Raft, 2PC, MVCC, Replication - Add new BaraQL features: JSON path (->, ->>), FTS @@, CREATE INDEX USING FTS, RECOVER TO TIMESTAMP, UNION/INTERSECT/EXCEPT - Add OpenTelemetry tracing example to README - Update Quick Start to use nimble test/bench - Update EN and BG documentation
This commit is contained in:
@@ -123,6 +123,17 @@ Client → Protocol → Auth → Parser → AST → IR → Codegen
|
||||
- **Gossip Protocol** (`core/gossip.nim`): SWIM-подобно управление на членство
|
||||
- **Distributed Transactions** (`core/disttxn.nim`): Two-phase commit
|
||||
|
||||
## Ключови Дизайн Решения
|
||||
|
||||
1. **Чист Nim**: Без Cython, Python или Rust зависимости
|
||||
2. **Обединено Съхранение**: Един двигател за KV, граф, вектор, FTS и колонно
|
||||
3. **Вграден Режим**: Работи като библиотека или сървър
|
||||
4. **Бинарен Протокол**: Ефективен wire протокол
|
||||
5. **MVCC**: Multi-version concurrency control
|
||||
6. **Schema-First**: Строго типизирана система с наследяване
|
||||
7. **Крос-Модал**: Един език за заявки през всички модели данни
|
||||
8. **Формално Верифициран**: Разпределените алгоритми са специфицирани в TLA+ и проверени с TLC
|
||||
|
||||
## Статистика на Модулите
|
||||
|
||||
| Категория | Модули | Редове Код | Предназначение |
|
||||
|
||||
+29
-2
@@ -105,6 +105,33 @@ RETURN friend.name;
|
||||
## Пълнотекстово Търсене
|
||||
|
||||
```sql
|
||||
SELECT * FROM articles
|
||||
WHERE MATCH(title, body) AGAINST('database programming');
|
||||
-- Създаване на FTS индекс
|
||||
CREATE INDEX idx_fts ON articles(body) USING FTS;
|
||||
|
||||
-- Търсене с BM25 ранжиране
|
||||
SELECT * FROM articles WHERE body @@ 'database programming';
|
||||
```
|
||||
|
||||
## JSON Оператори
|
||||
|
||||
```sql
|
||||
-- Извличане на JSON поле като JSON
|
||||
SELECT data->'name' FROM users;
|
||||
|
||||
-- Извличане на JSON поле като текст
|
||||
SELECT data->>'name' FROM users;
|
||||
```
|
||||
|
||||
## Set Операции
|
||||
|
||||
```sql
|
||||
SELECT name FROM customers
|
||||
UNION ALL
|
||||
SELECT name FROM suppliers;
|
||||
```
|
||||
|
||||
## Възстановяване до Момент във Времето
|
||||
|
||||
```sql
|
||||
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
||||
```
|
||||
@@ -34,4 +34,24 @@ import barabadb/core/replication
|
||||
var rm = newReplicationManager(rmSync)
|
||||
rm.addReplica(newReplica("r1", "10.0.0.1", 9472))
|
||||
rm.connectReplica("r1")
|
||||
```
|
||||
|
||||
## Формална Верификация
|
||||
|
||||
Разпределените алгоритми са формално специфицирани в TLA+ и проверени с TLC:
|
||||
|
||||
- **Raft Консенсус** — `formal-verification/raft.tla`
|
||||
- Проверено: ElectionSafety, StateMachineSafety
|
||||
- **Two-Phase Commit** — `formal-verification/twopc.tla`
|
||||
- Проверено: Atomicity, NoOrphanBlocks
|
||||
- **Репликация** — `formal-verification/replication.tla`
|
||||
- Проверено: MonotonicLsn, AcksRemovePending
|
||||
|
||||
Пускане на TLC:
|
||||
|
||||
```bash
|
||||
cd formal-verification
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/raft.cfg raft.tla
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/twopc.cfg twopc.tla
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/replication.cfg replication.tla
|
||||
```
|
||||
+18
-1
@@ -24,4 +24,21 @@ discard tm.commit(txn)
|
||||
BaraDB използва **snapshot isolation**:
|
||||
- Читателите не блокират писатели
|
||||
- Писателите не блокират читатели
|
||||
- Всяка транзакция вижда консистентен моментна снимка
|
||||
- Всяка транзакция вижда консистентен моментна снимка
|
||||
|
||||
## Формална Верификация
|
||||
|
||||
MVCC / Snapshot Isolation протоколът е формално специфициран в TLA+:
|
||||
|
||||
- **Спецификация:** `formal-verification/mvcc.tla`
|
||||
- **Проверени свойства:**
|
||||
- `NoDirtyReads` — транзакциите никога не четат неcommit-нати данни
|
||||
- `ReadOwnWrites` — транзакциите винаги виждат собствените си записи
|
||||
- `WriteWriteConflict` — first-committer-wins
|
||||
|
||||
Пускане на TLC:
|
||||
|
||||
```bash
|
||||
cd formal-verification
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/mvcc.cfg mvcc.tla
|
||||
```
|
||||
@@ -133,6 +133,7 @@ Client → Protocol → Auth → Parser → AST → IR → Codegen
|
||||
5. **MVCC**: Multi-version concurrency control
|
||||
6. **Schema-First**: Strongly typed schema system with inheritance
|
||||
7. **Cross-Modal**: Single query language across all data models
|
||||
8. **Formally Verified**: Core distributed algorithms specified in TLA+ and model-checked with TLC
|
||||
|
||||
## Module Statistics
|
||||
|
||||
|
||||
@@ -75,6 +75,14 @@ BARADB_WAL_ARCHIVE_INTERVAL_MS=60000 \
|
||||
--target-time="2025-01-15T10:30:00Z"
|
||||
```
|
||||
|
||||
### Recovery via SQL
|
||||
|
||||
You can also recover directly via BaraQL:
|
||||
|
||||
```sql
|
||||
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
||||
```
|
||||
|
||||
### Incremental Backups
|
||||
|
||||
Incremental backups only copy changed SSTables:
|
||||
|
||||
@@ -22,6 +22,7 @@ BaraQL is a SQL-compatible query language with extensions for graph, vector, and
|
||||
| `datetime` | ISO 8601 timestamp | `'2025-01-15T10:30:00Z'` |
|
||||
| `uuid` | UUID v4 | `'550e8400-e29b-41d4-a716-446655440000'` |
|
||||
| `json` | JSON document | `{"key": "value"}` |
|
||||
| `jsonb` | Binary JSON (validated) | `{"key": "value"}` |
|
||||
|
||||
## Basic Queries
|
||||
|
||||
@@ -360,6 +361,32 @@ DROP TYPE User;
|
||||
DROP INDEX idx_users_name;
|
||||
```
|
||||
|
||||
### JSON Path Operators
|
||||
|
||||
```sql
|
||||
-- Extract JSON field as JSON
|
||||
SELECT data->'name' FROM users;
|
||||
|
||||
-- Extract JSON field as text
|
||||
SELECT data->>'name' FROM users;
|
||||
```
|
||||
|
||||
### Full-Text Search (SQL)
|
||||
|
||||
```sql
|
||||
-- Create FTS index with BM25
|
||||
CREATE INDEX idx_fts ON articles(body) USING FTS;
|
||||
|
||||
-- Search with BM25 ranking
|
||||
SELECT * FROM articles WHERE body @@ 'machine learning';
|
||||
```
|
||||
|
||||
### Point-in-Time Recovery
|
||||
|
||||
```sql
|
||||
RECOVER TO TIMESTAMP '2026-05-07T12:00:00';
|
||||
```
|
||||
|
||||
## Vector Search
|
||||
|
||||
```sql
|
||||
@@ -505,4 +532,7 @@ SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||
| Graph | MATCH, RETURN, WHERE, shortestPath, type |
|
||||
| FTS | MATCH, AGAINST, relevance, IN BOOLEAN MODE, WITH FUZZINESS |
|
||||
| Vector | cosine_distance, l2_distance, dot_product, manhattan_distance |
|
||||
| JSON | ->, ->> |
|
||||
| FTS | @@ (BM25 match) |
|
||||
| Recovery | RECOVER TO TIMESTAMP |
|
||||
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id |
|
||||
|
||||
@@ -87,4 +87,24 @@ import barabadb/core/disttxn
|
||||
var dt = newDistributedTxn()
|
||||
dt.prepare(@["node1", "node2"])
|
||||
dt.commit()
|
||||
```
|
||||
|
||||
## Formal Verification
|
||||
|
||||
Core distributed algorithms are formally specified in TLA+ and model-checked:
|
||||
|
||||
- **Raft Consensus** — `formal-verification/raft.tla`
|
||||
- Verified: ElectionSafety, StateMachineSafety
|
||||
- **Two-Phase Commit** — `formal-verification/twopc.tla`
|
||||
- Verified: Atomicity, NoOrphanBlocks
|
||||
- **Replication** — `formal-verification/replication.tla`
|
||||
- Verified: MonotonicLsn, AcksRemovePending
|
||||
|
||||
Run TLC locally:
|
||||
|
||||
```bash
|
||||
cd formal-verification
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/raft.cfg raft.tla
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/twopc.cfg twopc.tla
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/replication.cfg replication.tla
|
||||
```
|
||||
@@ -52,6 +52,24 @@ let tfidf = idx.searchTfidf("query terms")
|
||||
| Phrase search | Exact phrase matching |
|
||||
| Boolean | AND, OR, NOT operators |
|
||||
|
||||
## SQL Interface
|
||||
|
||||
Full-text search is also available directly in BaraQL:
|
||||
|
||||
```sql
|
||||
-- Create a table with text column
|
||||
CREATE TABLE articles (id INT PRIMARY KEY, title TEXT, body TEXT);
|
||||
|
||||
-- Create an FTS index
|
||||
CREATE INDEX idx_fts ON articles(body) USING FTS;
|
||||
|
||||
-- Search with the @@ operator (BM25 ranking)
|
||||
SELECT * FROM articles WHERE body @@ 'machine learning';
|
||||
|
||||
-- Search with multiple terms
|
||||
SELECT * FROM articles WHERE body @@ 'quick brown fox';
|
||||
```
|
||||
|
||||
## Multi-Language Support
|
||||
|
||||
```nim
|
||||
|
||||
@@ -58,4 +58,21 @@ Nested transaction savepoints:
|
||||
tm.savepoint(txn, "sp1")
|
||||
# ... operations ...
|
||||
tm.rollbackToSavepoint(txn, "sp1")
|
||||
```
|
||||
|
||||
## Formal Verification
|
||||
|
||||
The MVCC / Snapshot Isolation protocol is formally specified in TLA+:
|
||||
|
||||
- **Spec:** `formal-verification/mvcc.tla`
|
||||
- **Verified properties:**
|
||||
- `NoDirtyReads` — transactions never read uncommitted data
|
||||
- `ReadOwnWrites` — transactions always see their own writes
|
||||
- `WriteWriteConflict` — first-committer-wins (no two committed transactions write the same key)
|
||||
|
||||
Run TLC locally:
|
||||
|
||||
```bash
|
||||
cd formal-verification
|
||||
java -cp tla2tools.jar tlc2.TLC -config models/mvcc.cfg mvcc.tla
|
||||
```
|
||||
Reference in New Issue
Block a user