Add comprehensive documentation with i18n support (EN/BG)
- Add docs/ folder with English (en/) and Bulgarian (bg/) documentation - Create index.md with language switching and links - English docs: installation, quickstart, architecture, baraql, storage, schema, lsm, btree, vector, graph, fts, columnar, transactions, distributed, protocol, udf, api-binary, api-http, api-websocket - Bulgarian docs: installation, quickstart, architecture, baraql, schema, lsm, btree, vector, graph, fts, transactions, distributed - Update README license to BSD 3-Clause - Add LICENSE file with BSD 3-Clause text
This commit is contained in:
@@ -0,0 +1,119 @@
|
||||
# BaraDB Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
BaraDB is a **multimodal database engine** written in Nim that combines document (KV), graph, vector, columnar, and full-text search storage in a single engine with a unified query language called **BaraQL**.
|
||||
|
||||
## Layer Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 1. CLIENT LAYER │
|
||||
│ Binary Protocol │ HTTP/REST │ WebSocket │ Embedded │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 2. QUERY LAYER (BaraQL) │
|
||||
│ Lexer → Parser → AST → IR → Optimizer → Codegen │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 3. EXECUTION ENGINE │
|
||||
│ Document │ Graph │ Vector │ Columnar │ FTS │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 4. STORAGE │
|
||||
│ LSM-Tree │ B-Tree │ WAL │ Bloom │ Compaction │ Cache │
|
||||
├─────────────────────────────────────────────────────────┤
|
||||
│ 5. DISTRIBUTED │
|
||||
│ Raft Consensus │ Sharding │ Replication │ Gossip │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Layer 1: Client Layer
|
||||
|
||||
Multiple communication protocols:
|
||||
|
||||
- **Binary Protocol** (`protocol/wire.nim`): Efficient big-endian binary protocol with 16 message types
|
||||
- **HTTP/REST** (`protocol/http.nim`): JSON-based REST API
|
||||
- **WebSocket** (`protocol/websocket.nim`): Full-duplex streaming
|
||||
- **Embedded** (`storage/lsm.nim`): Direct in-process access
|
||||
|
||||
## Layer 2: Query Layer (BaraQL)
|
||||
|
||||
The BaraQL pipeline:
|
||||
|
||||
1. **Lexer** (`query/lexer.nim`): Tokenizes input into 80+ token types
|
||||
2. **Parser** (`query/parser.nim`): Recursive descent parser producing AST
|
||||
3. **AST** (`query/ast.nim`): 300+ lines covering 25+ node kinds
|
||||
4. **IR** (`query/ir.nim`): Intermediate representation for execution plans
|
||||
5. **Optimizer/Codegen** (`query/codegen.nim`): Translates IR to storage operations
|
||||
|
||||
## Layer 3: Execution Engine
|
||||
|
||||
### Document/KV Engine
|
||||
- **LSM-Tree** (`storage/lsm.nim`): Write-optimized storage
|
||||
- **B-Tree Index** (`storage/btree.nim`): Ordered index for range scans
|
||||
|
||||
### Vector Engine (`vector/`)
|
||||
- **HNSW Index**: Hierarchical Navigable Small World graph
|
||||
- **IVF-PQ Index**: Inverted File Index with Product Quantization
|
||||
- **SIMD Operations**: Unrolled distance computations
|
||||
|
||||
### Graph Engine (`graph/`)
|
||||
- **Adjacency List**: Edge-weighted directed graph
|
||||
- **Algorithms**: BFS, DFS, Dijkstra, PageRank, Louvain
|
||||
|
||||
### Full-Text Search (`fts/`)
|
||||
- **Inverted Index**: Term-document index
|
||||
- **Ranking**: BM25 and TF-IDF scoring
|
||||
- **Multi-Language**: Tokenizers for EN, BG, DE, FR, RU
|
||||
|
||||
### Columnar Engine (`core/columnar.nim`)
|
||||
- Per-column storage for analytical queries
|
||||
- RLE and dictionary encoding
|
||||
|
||||
## Layer 4: Storage
|
||||
|
||||
- **LSM-Tree**: MemTable, WAL, SSTable, Bloom Filter, Compaction
|
||||
- **Page Cache**: LRU cache with hit rate tracking
|
||||
- **Memory-mapped I/O**: mmap-based file access
|
||||
|
||||
## Layer 5: Distributed
|
||||
|
||||
- **Raft Consensus**: Leader election, log replication
|
||||
- **Sharding**: Hash, range, and consistent hashing
|
||||
- **Replication**: Sync, async, semi-sync modes
|
||||
- **Gossip Protocol**: Membership management
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Write Path
|
||||
```
|
||||
Client → Protocol → Auth → Parser → AST → IR → Codegen
|
||||
→ StorageOp → MVCC Txn → WAL Write → MemTable → Commit
|
||||
```
|
||||
|
||||
### Read Path
|
||||
```
|
||||
Client → Protocol → Auth → Parser → AST → IR → Codegen
|
||||
→ StorageOp → MVCC Snapshot → MemTable → SSTable → Result
|
||||
```
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
1. **Pure Nim**: No Cython, Python, or Rust dependencies
|
||||
2. **Unified Storage**: One engine handles KV, graph, vector, FTS, and columnar
|
||||
3. **Embedded Mode**: Can run as library or server
|
||||
4. **Binary Protocol**: Custom efficient wire protocol
|
||||
5. **MVCC**: Multi-version concurrency control
|
||||
6. **Schema-First**: Strongly typed schema system with inheritance
|
||||
|
||||
## Module Statistics
|
||||
|
||||
| Category | Modules |
|
||||
|----------|---------|
|
||||
| Core | 10 |
|
||||
| Storage | 7 |
|
||||
| Query | 7 |
|
||||
| Vector | 3 |
|
||||
| Graph | 3 |
|
||||
| FTS | 2 |
|
||||
| Protocol | 7 |
|
||||
| Distributed | 5 |
|
||||
| **Total** | **48** |
|
||||
Reference in New Issue
Block a user