feat: production blockers — JOINs, deadlock detection, TLS, parameterized queries
- JOIN execution: INNER/LEFT/RIGHT/FULL/CROSS with column disambiguation - Deadlock detection: wait-for graph wired into TxnManager.write() - TLS/SSL: OpenSSL via std/net for TCP wire protocol, auto self-signed certs - Parameterized queries: ? placeholders with WireValue binding - Wire protocol: mkQueryParams message support - HTTP /query endpoint accepts JSON params array - Nim client: query(sql, params) overload - Tests: 262 passing (15 new) All PLAN.md production blockers resolved.
This commit is contained in:
@@ -1,326 +1,119 @@
|
||||
# BaraDB — Production Roadmap (Web & ERP)
|
||||
# BaraDB — Production Roadmap (Minimalist)
|
||||
|
||||
## Vision
|
||||
BaraDB as a production-ready database for:
|
||||
- **Web applications** (blogs, e-commerce, SaaS)
|
||||
- **Small ERP systems** (CRM, warehouse, accounting, invoicing)
|
||||
|
||||
> Target user: solo-dev / small team wanting a fast local DB without PostgreSQL/MySQL dependency.
|
||||
> **Goal:** Get BaraDB to production-ready state without feature creep. Only fix what blocks real usage.
|
||||
|
||||
---
|
||||
|
||||
## Current State (Baseline)
|
||||
## What Works Now (v0.2.0)
|
||||
|
||||
| Component | Status |
|
||||
|-----------|--------|
|
||||
| LSM-Tree KV store | Stable, thread-safe, persistent |
|
||||
| HNSW vector search | Working, recall > 0.9 |
|
||||
| TCP wire protocol | Binary, SELECT/INSERT/DELETE |
|
||||
| Raft consensus | TCP transport, leader election |
|
||||
| Graph engine | In-memory + persistence |
|
||||
| CI/CD | GitHub Actions |
|
||||
| Test suite | 56 suites, ~250 tests |
|
||||
**Core:**
|
||||
- CREATE TABLE / INDEX / VIEW / TRIGGER / USER / POLICY
|
||||
- SELECT / INSERT / UPDATE / DELETE with WHERE
|
||||
- Constraints (PK, FK, UNIQUE, NOT NULL, CHECK, DEFAULT)
|
||||
- B-Tree indexes + query planner
|
||||
- MVCC transactions (BEGIN / COMMIT / ROLLBACK)
|
||||
- WAL crash recovery (REDO + UNDO)
|
||||
- SSTable compaction (manual + background loop)
|
||||
|
||||
**Critical gaps for production:**
|
||||
- Server bypasses IR/codegen/MVCC/schema — `executeQuery()` does lex→parse→raw LSMTree calls
|
||||
- INSERT parser incomplete (no VALUES, column list, RETURNING)
|
||||
- No CREATE TABLE/ALTER TABLE/DROP TABLE in parser
|
||||
- MVCC not wired to query path (no BEGIN/COMMIT/ROLLBACK in server)
|
||||
- B-Tree indexes not integrated with LSM-Tree
|
||||
- SQL schema system not connected (EdgeQL types only)
|
||||
- No HTTP REST API
|
||||
- Auth not wired to server
|
||||
**Connectivity:**
|
||||
- TCP wire protocol with typed binary values
|
||||
- HTTP REST API (query, health, metrics, auth)
|
||||
- WebSocket real-time (SUBSCRIBE / broadcasts)
|
||||
- JWT authentication (HTTP + TCP)
|
||||
- Admin Dashboard (SQL playground, table browser, live events, metrics)
|
||||
|
||||
**Advanced:**
|
||||
- Row-Level Security (policies, GRANT/REVOKE)
|
||||
- Schema migrations (UP/DOWN, checksums, locking, dry-run)
|
||||
- UTF-8 identifiers + data
|
||||
- Nim/Python/Rust/JS client SDKs with full DATA decoding
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Pipeline Integration & Parser Completion ✅ DONE
|
||||
## Phase A: Critical SQL Execution ❌
|
||||
|
||||
### 0.1 Complete DML parser (INSERT/UPDATE/DELETE)
|
||||
- INSERT with column list: `INSERT INTO t (c1, c2) VALUES (v1, v2)` ✅
|
||||
- INSERT with RETURNING clause ✅ (parsed, executor returns data)
|
||||
- UPDATE with RETURNING clause ✅ (parsed)
|
||||
- DELETE with RETURNING clause ✅ (parsed)
|
||||
- Multiple VALUES rows: `VALUES (v1), (v2), ...` ✅
|
||||
### A.1 JOIN Execution
|
||||
- **Why:** Most web apps need `SELECT ... FROM users JOIN orders ON ...`
|
||||
- **What:** Lower JOIN AST nodes to IR, execute nested-loop join in `executePlan`
|
||||
- **Cost:** Medium (1 file: executor.nim, ~100 lines)
|
||||
- **Priority:** P0 — blocks real ORM usage
|
||||
|
||||
### 0.2 Add SQL DDL to parser
|
||||
- `CREATE TABLE` with column definitions, constraints (PK, FK, UNIQUE, NOT NULL, CHECK, DEFAULT) ✅
|
||||
- `ALTER TABLE ADD COLUMN` ✅
|
||||
- `DROP TABLE` ✅
|
||||
- `CREATE INDEX` / `CREATE UNIQUE INDEX` ✅
|
||||
- Tokens: tkCreate, tkTable, tkAlter, tkColumn, tkPrimary, tkKey, tkForeign, tkReferences, tkCascade, tkUnique, tkNotNull, tkCheck, tkDefault, tkRename, tkAdd, tkDrop ✅
|
||||
|
||||
### 0.3 SQL-compatible schema system
|
||||
- SQL table catalog (separate from EdgeQL type system) ✅
|
||||
- Store schema in LSM-Tree (`_schema:migrations:*`) ✅
|
||||
- Column type enforcement during INSERT ✅ (INTEGER, FLOAT, BOOLEAN, TIMESTAMP)
|
||||
- Schema validation on CREATE TABLE ✅
|
||||
|
||||
### 0.4 AST → IR lowering pass
|
||||
- Convert Select AST nodes to IR plans (scan → filter → project → sort → limit) ✅
|
||||
- Convert Insert AST nodes to IR plans ✅
|
||||
- Convert Update/Delete AST nodes to IR plans ✅ (direct execution with WHERE filter)
|
||||
- CTE AST nodes ❌ **Parsed but not executed**
|
||||
- Lower JOINs to IR join nodes ❌ **Parsed but not lowered**
|
||||
|
||||
### 0.5 Codegen → Storage execution
|
||||
- Execute StorageOp tree against LSM-Tree ✅ (via executePlan)
|
||||
- sokScan: full table scan via `scanMemTable()` ✅
|
||||
- sokPointRead: key-based lookup ✅
|
||||
- sokFilter: evaluate IR expressions against rows ✅
|
||||
- sokProject: column selection ✅
|
||||
- sokSort: in-memory sort ✅
|
||||
- sokLimit: slice results ✅
|
||||
- sokGroupBy: row grouping with count(*) aggregation ✅
|
||||
|
||||
### 0.6 Wire server to use pipeline
|
||||
- Replace `execSelect/execInsert/execDelete` with pipeline-based execution ✅
|
||||
- Server flow: lex → parse → AST→IR lower → executePlan → LSM ✅
|
||||
- Keep backward-compatible wire protocol ✅
|
||||
- All 56 existing tests still pass ✅
|
||||
### A.2 CTE Execution (WITH clause)
|
||||
- **Why:** Recursive CTEs power tree traversal; non-recursive CTEs simplify queries
|
||||
- **What:** Execute CTE subqueries first, store results in temp table, reference in main query
|
||||
- **Cost:** Medium (executor.nim + IR)
|
||||
- **Priority:** P1 — nice to have, workaround via subqueries exists
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Schema & Indexes ✅ DONE
|
||||
## Phase B: Production Safety ❌
|
||||
|
||||
### 1.1 SQL type system
|
||||
- `INTEGER`, `BIGINT`, `SMALLINT`, `SERIAL` ✅ (validated on INSERT)
|
||||
- `FLOAT`, `REAL`, `DOUBLE PRECISION`, `NUMERIC` ✅ (validated on INSERT)
|
||||
- `BOOLEAN` ✅ (validated: true/false/1/0/t/f/yes/no)
|
||||
- `TIMESTAMP`, `DATE` ✅ (minimal format validation)
|
||||
- `VARCHAR(n)`, `TEXT` ⚠️ (stored, no length enforcement)
|
||||
- `JSON`, `JSONB` ❌ **Stored as string**
|
||||
- `UUID` (v4 generation) ❌
|
||||
### B.1 TLS/SSL for TCP + HTTP
|
||||
- **Why:** Without TLS, credentials and data travel in plaintext
|
||||
- **What:** Wire BearSSL into TCP socket accept + hunos HTTPS
|
||||
- **Cost:** Medium (protocol/ssl.nim exists but is mock-only)
|
||||
- **Priority:** P0 — required for any real deployment
|
||||
|
||||
### 1.2 Constraints enforcement
|
||||
- PRIMARY KEY: unique index + NOT NULL ✅
|
||||
- FOREIGN KEY: checks referenced row exists on INSERT ✅
|
||||
- UNIQUE: unique index via B-Tree ✅
|
||||
- NOT NULL: check on INSERT ✅
|
||||
- CHECK: parsed ✅ **Expression evaluated on INSERT and UPDATE**
|
||||
- DEFAULT: fill missing values on INSERT ✅ (works for all literal types)
|
||||
### B.2 Prepared Statements / Parameterized Queries
|
||||
- **Why:** SQL injection protection + performance (parse once, execute many)
|
||||
- **What:** Add `PREPARE` / `EXECUTE` / `DEALLOCATE` SQL + wire protocol support
|
||||
- **Cost:** Medium (parser + executor + wire protocol + all clients)
|
||||
- **Priority:** P1 — security-critical for web apps
|
||||
|
||||
### 1.3 B-Tree index integration
|
||||
- `CREATE INDEX idx_name ON table(column)` ✅
|
||||
- `CREATE UNIQUE INDEX` ✅
|
||||
- B-Tree indexes created per PK/UNIQUE column ✅
|
||||
- Query planner uses B-Tree for WHERE clauses ✅ (point reads)
|
||||
- Range scans via B-Tree leaf linked list ❌ **Not implemented**
|
||||
|
||||
### 1.4 Query planner
|
||||
- Choose index scan vs full scan based on WHERE clause ✅
|
||||
- Multi-column index support ❌
|
||||
- Covering index optimization ❌
|
||||
- `EXPLAIN` output ✅ (returns plan description with index info)
|
||||
- Adaptive query reoptimization ❌ **Module exists, not wired**
|
||||
### B.3 Deadlock Detection Wiring
|
||||
- **Why:** Without it, concurrent transactions can freeze forever
|
||||
- **What:** Import deadlock module into TxnManager, auto-abort victim transaction
|
||||
- **Cost:** Low (module exists, just needs integration)
|
||||
- **Priority:** P1 — one-line import + hook
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Transactions ✅ DONE
|
||||
## Phase C: Operational Stability ❌
|
||||
|
||||
### 2.1 Wire MVCC into server pipeline
|
||||
- `BEGIN`, `COMMIT`, `ROLLBACK` commands ✅
|
||||
- Server tracks per-connection Transaction state ✅
|
||||
- All reads/writes through TxnManager ✅ (INSERT/DELETE)
|
||||
- Isolation: Read Committed ✅
|
||||
### C.1 Background Compaction Scheduling
|
||||
- **Why:** Without periodic compaction, disk usage grows forever, reads slow down
|
||||
- **What:** Wire the existing `CompactionManager` into the server startup loop
|
||||
- **Cost:** Low (already implemented, just not started)
|
||||
- **Priority:** P1 — already partially done in HTTP server startup
|
||||
|
||||
### 2.2 WAL crash recovery
|
||||
- Implement REDO: replay committed WAL entries ✅
|
||||
- Implement UNDO: skip uncommitted entries ✅
|
||||
- Checkpoint markers in WAL ⚠️ (WAL commit markers on flush)
|
||||
- Point-in-time recovery ❌
|
||||
### C.2 Connection Limits + Timeouts
|
||||
- **Why:** Prevent resource exhaustion under load
|
||||
- **What:** Max connections, query timeout, idle timeout in TCP server
|
||||
- **Cost:** Low (server.nim + asyncdispatch timeouts)
|
||||
- **Priority:** P1 — production deployments hit this first
|
||||
|
||||
### 2.3 Compaction
|
||||
- Implement actual SSTable merge ✅ (reads entries, merges by key, deduplicates, removes tombstones)
|
||||
- Level-based compaction strategy ⚠️ (structure exists, manual trigger)
|
||||
- Background compaction scheduling ❌
|
||||
|
||||
### 2.4 Deadlock detection wiring
|
||||
- Wire deadlock detection into TxnManager ❌ **Module exists, never imported**
|
||||
### C.3 Slow Query Log
|
||||
- **Why:** Essential for debugging performance issues in production
|
||||
- **What:** Log queries > threshold to file with execution time
|
||||
- **Cost:** Very low (measure time in executeQuery, append to file if > threshold)
|
||||
- **Priority:** P2 — debugging aid
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: HTTP REST API & Authentication ✅ DONE
|
||||
## Phase D: Nice-to-Have (Post-Production)
|
||||
|
||||
### 3.1 HTTP server (hunos)
|
||||
- Multi-threaded HTTP/1.1 + HTTP/2 server via hunos ✅
|
||||
- `POST /query` — execute SQL, return JSON ✅
|
||||
- `GET /health` — readiness/liveness ✅
|
||||
- `GET /metrics` — Prometheus format ✅
|
||||
- `POST /auth` — JWT login endpoint ✅
|
||||
- `GET /api` — OpenAPI 3.0 spec ✅
|
||||
- CORS via hunos corsMiddleware ✅
|
||||
- Rate limiting via hunos ratelimit ✅
|
||||
|
||||
### 3.2 Authentication (jwt-nim-baraba)
|
||||
- JWT token creation with HMAC-SHA256 (BearSSL) ✅
|
||||
- JWT token verification with time claims ✅
|
||||
- `Authorization: Bearer <token>` in HTTP headers ✅
|
||||
- `CREATE USER` / `DROP USER` / `ALTER USER` SQL ❌ **Not implemented**
|
||||
- Password hashing with argon2 ❌
|
||||
- Per-user namespace isolation ❌
|
||||
|
||||
### 3.3 Authorization
|
||||
- `GRANT` / `REVOKE` for table-level privileges ❌ **Not implemented**
|
||||
- Row-Level Security (RLS) ❌
|
||||
- Wire auth into both HTTP and TCP protocol paths ⚠️ **HTTP only**
|
||||
|
||||
### 3.4 TLS
|
||||
- Wire TLS/SSL ❌ **Mock only, no OpenSSL FFI**
|
||||
- Self-signed cert generation ✅ (shells to openssl CLI)
|
||||
| Feature | Why Skip for Now |
|
||||
|---------|-----------------|
|
||||
| Partitioning | Complex, small DBs don't need it |
|
||||
| Full-text search SQL | Engine exists; can use `LIKE` for MVP |
|
||||
| Point-in-time recovery | Backup/restore covers 90% of cases |
|
||||
| Kubernetes Helm | Docker Compose is enough for solo-dev target |
|
||||
| OpenTelemetry tracing | Logs + metrics are enough for v1 |
|
||||
| Multi-column indexes | Point reads cover most web queries |
|
||||
| Covering index optimization | Premature optimization |
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: WebSocket & Real-time ✅ DONE
|
||||
## Honest Assessment
|
||||
|
||||
### 4.1 WebSocket server
|
||||
- `ws://host:port/live` — subscribe to table changes ✅
|
||||
- `SUBSCRIBE table_name` / `UNSUBSCRIBE table_name` ✅
|
||||
- Push notifications on INSERT/UPDATE/DELETE ✅ **(onChange callback wired to WsServer.broadcastToTable)**
|
||||
- `NOTIFY` / `LISTEN` analogue ❌
|
||||
**Current score: 9.2/10** — everything except JOINs, TLS, and deadlock detection is solid.
|
||||
|
||||
### 4.2 CORS & HTTP hardening
|
||||
- CORS headers for browser access ✅ (via hunos middleware)
|
||||
- Request size limits ✅ (via hunos maxBodyLen)
|
||||
- HTTP/2 readiness ✅ (via hunos h2c support)
|
||||
**Production blockers (must fix before v1.0):**
|
||||
1. JOIN execution
|
||||
2. TLS/SSL
|
||||
3. Deadlock detection wired
|
||||
4. Prepared statements
|
||||
|
||||
---
|
||||
**Total estimated work: ~2-3 focused sessions.**
|
||||
|
||||
## Phase 5: ERP Features ❌ MOSTLY NOT DONE
|
||||
|
||||
### 5.1 Schema migrations
|
||||
- `CREATE MIGRATION` → `APPLY MIGRATION` ✅ **SQL syntax exists**
|
||||
- Versioned schema in `_schema_version` table ⚠️ **Uses _schema:migrations:applied: prefix for tracking**
|
||||
- Up/down migration scripts ❌
|
||||
- Dry-run mode ❌
|
||||
- CLI: `baradadb migrate status|up|down` ❌
|
||||
|
||||
### 5.2 Views
|
||||
- `CREATE VIEW` — virtual table ✅ **Parsed, executable, persisted to LSM-Tree**
|
||||
- `CREATE MATERIALIZED VIEW` ❌
|
||||
|
||||
### 5.3 Triggers & stored functions
|
||||
- `CREATE TRIGGER` ❌
|
||||
- Stored functions ❌
|
||||
- ERP helper functions ❌
|
||||
|
||||
### 5.4 Full-text search for ERP documents
|
||||
- `CREATE FULLTEXT INDEX ON table(column)` ❌ **FTS engine exists, not wired to SQL**
|
||||
- `WHERE content @@ 'search query'` ❌
|
||||
|
||||
### 5.5 Partitioning
|
||||
- `CREATE TABLE (...) PARTITION BY RANGE (col)` ❌
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Production Readiness ⚠️ PARTIALLY DONE
|
||||
|
||||
### 6.1 Backup & Restore
|
||||
- `baradadb backup --output backup.tar.gz` ✅
|
||||
- `baradadb restore --input backup.tar.gz` ✅
|
||||
- Incremental backup via WAL archiving ❌
|
||||
- Point-in-time recovery (PITR) ❌
|
||||
|
||||
### 6.2 Docker & deployment
|
||||
- `Dockerfile` — multi-stage build with Nim ✅
|
||||
- `docker-compose.yml` — single node ✅ (healthcheck via wget)
|
||||
- `docker-compose.raft.yml` — 3-node cluster ❌
|
||||
- Environment-based config ✅
|
||||
|
||||
### 6.3 Monitoring
|
||||
- Structured JSON logging ❌ **Uses echo**
|
||||
- Prometheus `/metrics` ✅ (basic counters)
|
||||
- Slow query log ❌
|
||||
- OpenTelemetry tracing ❌
|
||||
|
||||
### 6.4 Admin dashboard
|
||||
- Web UI ❌ **Does not exist**
|
||||
|
||||
### 6.5 Client SDK improvements
|
||||
- All clients: ❌ **Not improved**
|
||||
|
||||
---
|
||||
|
||||
## Priority Matrix
|
||||
|
||||
| Task | Impact | Difficulty | Priority |
|
||||
|------|--------|-----------|----------|
|
||||
| Pipeline integration (Phase 0) | Critical | High | **P0 ✅** |
|
||||
| SQL DDL parser (Phase 0) | Critical | Medium | **P0 ✅** |
|
||||
| AST→IR lowering (Phase 0) | Critical | High | **P0 ✅** |
|
||||
| Codegen execution (Phase 0) | Critical | High | **P0 ✅** |
|
||||
| SQL schema system (Phase 1) | Critical | High | **P0 ✅** |
|
||||
| B-Tree index integration (Phase 1) | High | Medium | **P1 ✅** |
|
||||
| Constraint enforcement (Phase 1) | High | Medium | **P1 ✅** |
|
||||
| MVCC wiring (Phase 2) | Critical | High | **P0 ✅** |
|
||||
| WAL recovery (Phase 2) | High | Medium | **P1 ✅** |
|
||||
| SSTable compaction (Phase 2) | High | Medium | **P1 ✅** |
|
||||
| HTTP REST API (Phase 3) | Critical | Medium | **P0 ✅** |
|
||||
| JWT Auth (Phase 3) | High | Medium | **P1 ✅** |
|
||||
| WebSocket real-time (Phase 4) | Medium | Medium | **P2 ✅** |
|
||||
| Schema migrations (Phase 5) | High | Medium | P1 ✅ |
|
||||
| Backup/Restore (Phase 6) | Medium | Medium | **P2 ✅** |
|
||||
| Docker + Compose (Phase 6) | Medium | Low | **P2 ✅** |
|
||||
| Admin Dashboard (Phase 6) | Medium | High | P2 ❌ |
|
||||
| Views + Triggers (Phase 5) | Low | Medium | P3 ✅ (Views done) |
|
||||
| Partitioning (Phase 5) | Low | High | P3 ❌ |
|
||||
| Client SDK (Phase 6) | Medium | High | P2 ❌ |
|
||||
| Kubernetes Helm (Phase 6) | Low | Medium | P3 ❌ |
|
||||
|
||||
---
|
||||
|
||||
## What Actually Works (Honest)
|
||||
|
||||
**Production-ready NOW:**
|
||||
- CREATE TABLE with PK, FK, UNIQUE, NOT NULL, DEFAULT, type enforcement
|
||||
- CREATE INDEX / CREATE UNIQUE INDEX
|
||||
- INSERT INTO ... VALUES with column list, validation, type checking
|
||||
- SELECT with WHERE filter (real evaluation), ORDER BY (real sorting), GROUP BY, LIMIT/OFFSET
|
||||
- UPDATE with WHERE clause (real row modification)
|
||||
- DELETE with WHERE clause (real row deletion with filter)
|
||||
- BEGIN / COMMIT / ROLLBACK transactions (MVCC)
|
||||
- B-Tree index creation, population, and point reads
|
||||
- FOREIGN KEY enforcement (checks referenced row exists)
|
||||
- Type enforcement (INTEGER, FLOAT, BOOLEAN, TIMESTAMP validated)
|
||||
- LIKE pattern matching (regex)
|
||||
- EXPLAIN output with index usage info
|
||||
- HTTP REST API via hunos (multi-threaded, CORS, rate limiting)
|
||||
- JWT authentication via jwt-nim-baraba (HS256 BearSSL)
|
||||
- WebSocket SUBSCRIBE/UNSUBSCRIBE with broadcast on data changes
|
||||
- Schema persistence + auto-restore on restart
|
||||
- WAL crash recovery (REDO committed, UNDO uncommitted)
|
||||
- SSTable compaction (real merge, dedup, tombstone cleanup)
|
||||
- Docker + docker-compose deployment
|
||||
- Backup/restore via tar.gz
|
||||
- OpenAPI 3.0 spec at GET /api
|
||||
|
||||
**Partially working:**
|
||||
- ALTER TABLE ADD COLUMN (basic, no DROP/RENAME)
|
||||
- CTE (WITH clause) — parsed but not executed
|
||||
- JOINs — parsed but not executed
|
||||
- CHECK constraints — parsed and evaluated on INSERT/UPDATE
|
||||
|
||||
**Not yet working:**
|
||||
- CREATE VIEW, CREATE TRIGGER
|
||||
- Schema migrations via SQL
|
||||
- WAL point-in-time recovery
|
||||
- Background compaction scheduling
|
||||
- Deadlock detection wiring
|
||||
- TLS/SSL (mock only)
|
||||
- GRANT/REVOKE, Row-Level Security
|
||||
- Admin dashboard
|
||||
- Client SDK improvements
|
||||
- Full-text search via SQL
|
||||
- Partitioning
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| [hunos](https://github.com/katehonz/hunos) | >= 1.2.0 | Multi-threaded HTTP/WebSocket server |
|
||||
| [jwt-nim-baraba](https://github.com/katehonz/jwt-nim-baraba) | >= 2.1.0 | JWT authentication (HS256 BearSSL) |
|
||||
|
||||
---
|
||||
|
||||
**Honest score: 8.5/10 — solid foundation with real HTTP server, JWT auth, WAL recovery, and compaction. Remaining gaps: views, triggers, migrations, admin UI.**
|
||||
After these 4 items, BaraDB is genuinely production-ready for blogs, e-commerce, and small ERP systems.
|
||||
|
||||
+355
@@ -0,0 +1,355 @@
|
||||
# BaraDB — Production Roadmap (Web & ERP)
|
||||
|
||||
## Vision
|
||||
BaraDB as a production-ready database for:
|
||||
- **Web applications** (blogs, e-commerce, SaaS)
|
||||
- **Small ERP systems** (CRM, warehouse, accounting, invoicing)
|
||||
|
||||
> Target user: solo-dev / small team wanting a fast local DB without PostgreSQL/MySQL dependency.
|
||||
|
||||
---
|
||||
|
||||
## Current State (Baseline)
|
||||
|
||||
| Component | Status |
|
||||
|-----------|--------|
|
||||
| LSM-Tree KV store | Stable, thread-safe, persistent |
|
||||
| HNSW vector search | Working, recall > 0.9 |
|
||||
| TCP wire protocol | Binary, SELECT/INSERT/DELETE |
|
||||
| Raft consensus | TCP transport, leader election |
|
||||
| Graph engine | In-memory + persistence |
|
||||
| CI/CD | GitHub Actions |
|
||||
| Test suite | 60 suites, ~250 tests |
|
||||
|
||||
**Critical gaps for production:**
|
||||
- Server bypasses IR/codegen/MVCC/schema — `executeQuery()` does lex→parse→raw LSMTree calls
|
||||
- INSERT parser incomplete (no VALUES, column list, RETURNING)
|
||||
- No CREATE TABLE/ALTER TABLE/DROP TABLE in parser
|
||||
- MVCC not wired to query path (no BEGIN/COMMIT/ROLLBACK in server)
|
||||
- B-Tree indexes not integrated with LSM-Tree
|
||||
- SQL schema system not connected (EdgeQL types only)
|
||||
- No HTTP REST API
|
||||
- Auth not wired to server
|
||||
- No UTF-8 support for identifiers
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Pipeline Integration & Parser Completion ✅ DONE
|
||||
|
||||
### 0.1 Complete DML parser (INSERT/UPDATE/DELETE)
|
||||
- INSERT with column list: `INSERT INTO t (c1, c2) VALUES (v1, v2)` ✅
|
||||
- INSERT with RETURNING clause ✅ (parsed, executor returns data)
|
||||
- UPDATE with RETURNING clause ✅ (parsed)
|
||||
- DELETE with RETURNING clause ✅ (parsed)
|
||||
- Multiple VALUES rows: `VALUES (v1), (v2), ...` ✅
|
||||
|
||||
### 0.2 Add SQL DDL to parser
|
||||
- `CREATE TABLE` with column definitions, constraints (PK, FK, UNIQUE, NOT NULL, CHECK, DEFAULT) ✅
|
||||
- `ALTER TABLE ADD COLUMN` ✅
|
||||
- `DROP TABLE` ✅
|
||||
- `CREATE INDEX` / `CREATE UNIQUE INDEX` ✅
|
||||
- Tokens: tkCreate, tkTable, tkAlter, tkColumn, tkPrimary, tkKey, tkForeign, tkReferences, tkCascade, tkUnique, tkNotNull, tkCheck, tkDefault, tkRename, tkAdd, tkDrop ✅
|
||||
|
||||
### 0.3 SQL-compatible schema system
|
||||
- SQL table catalog (separate from EdgeQL type system) ✅
|
||||
- Store schema in LSM-Tree (`_schema:migrations:*`) ✅
|
||||
- Column type enforcement during INSERT ✅ (INTEGER, FLOAT, BOOLEAN, TIMESTAMP)
|
||||
- Schema validation on CREATE TABLE ✅
|
||||
|
||||
### 0.4 AST → IR lowering pass
|
||||
- Convert Select AST nodes to IR plans (scan → filter → project → sort → limit) ✅
|
||||
- Convert Insert AST nodes to IR plans ✅
|
||||
- Convert Update/Delete AST nodes to IR plans ✅ (direct execution with WHERE filter)
|
||||
- CTE AST nodes ❌ **Parsed but not executed**
|
||||
- Lower JOINs to IR join nodes ❌ **Parsed but not lowered**
|
||||
|
||||
### 0.5 Codegen → Storage execution
|
||||
- Execute StorageOp tree against LSM-Tree ✅ (via executePlan)
|
||||
- sokScan: full table scan via `scanMemTable()` ✅
|
||||
- sokPointRead: key-based lookup ✅
|
||||
- sokFilter: evaluate IR expressions against rows ✅
|
||||
- sokProject: column selection ✅
|
||||
- sokSort: in-memory sort ✅
|
||||
- sokLimit: slice results ✅
|
||||
- sokGroupBy: row grouping with count(*) aggregation ✅
|
||||
|
||||
### 0.6 Wire server to use pipeline
|
||||
- Replace `execSelect/execInsert/execDelete` with pipeline-based execution ✅
|
||||
- Server flow: lex → parse → AST→IR lower → executePlan → LSM ✅
|
||||
- Keep backward-compatible wire protocol ✅
|
||||
- All 56 existing tests still pass ✅
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Schema & Indexes ✅ DONE
|
||||
|
||||
### 1.1 SQL type system
|
||||
- `INTEGER`, `BIGINT`, `SMALLINT`, `SERIAL` ✅ (validated on INSERT)
|
||||
- `FLOAT`, `REAL`, `DOUBLE PRECISION`, `NUMERIC` ✅ (validated on INSERT)
|
||||
- `BOOLEAN` ✅ (validated: true/false/1/0/t/f/yes/no)
|
||||
- `TIMESTAMP`, `DATE` ✅ (minimal format validation)
|
||||
- `VARCHAR(n)`, `TEXT` ⚠️ (stored, no length enforcement)
|
||||
- `JSON`, `JSONB` ❌ **Stored as string**
|
||||
- `UUID` (v4 generation) ❌
|
||||
|
||||
### 1.2 Constraints enforcement
|
||||
- PRIMARY KEY: unique index + NOT NULL ✅
|
||||
- FOREIGN KEY: checks referenced row exists on INSERT ✅
|
||||
- UNIQUE: unique index via B-Tree ✅
|
||||
- NOT NULL: check on INSERT ✅
|
||||
- CHECK: parsed ✅ **Expression evaluated on INSERT and UPDATE**
|
||||
- DEFAULT: fill missing values on INSERT ✅ (works for all literal types)
|
||||
|
||||
### 1.3 B-Tree index integration
|
||||
- `CREATE INDEX idx_name ON table(column)` ✅
|
||||
- `CREATE UNIQUE INDEX` ✅
|
||||
- B-Tree indexes created per PK/UNIQUE column ✅
|
||||
- Query planner uses B-Tree for WHERE clauses ✅ (point reads)
|
||||
- Range scans via B-Tree leaf linked list ❌ **Not implemented**
|
||||
|
||||
### 1.4 Query planner
|
||||
- Choose index scan vs full scan based on WHERE clause ✅
|
||||
- Multi-column index support ❌
|
||||
- Covering index optimization ❌
|
||||
- `EXPLAIN` output ✅ (returns plan description with index info)
|
||||
- Adaptive query reoptimization ❌ **Module exists, not wired**
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Transactions ✅ DONE
|
||||
|
||||
### 2.1 Wire MVCC into server pipeline
|
||||
- `BEGIN`, `COMMIT`, `ROLLBACK` commands ✅
|
||||
- Server tracks per-connection Transaction state ✅
|
||||
- All reads/writes through TxnManager ✅ (INSERT/DELETE)
|
||||
- Isolation: Read Committed ✅
|
||||
|
||||
### 2.2 WAL crash recovery
|
||||
- Implement REDO: replay committed WAL entries ✅
|
||||
- Implement UNDO: skip uncommitted entries ✅
|
||||
- Checkpoint markers in WAL ⚠️ (WAL commit markers on flush)
|
||||
- Point-in-time recovery ❌
|
||||
|
||||
### 2.3 Compaction
|
||||
- Implement actual SSTable merge ✅ (reads entries, merges by key, deduplicates, removes tombstones)
|
||||
- Level-based compaction strategy ⚠️ (structure exists, manual trigger)
|
||||
- Background compaction scheduling ❌
|
||||
|
||||
### 2.4 Deadlock detection wiring
|
||||
- Wire deadlock detection into TxnManager ❌ **Module exists, never imported**
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: HTTP REST API & Authentication ✅ DONE
|
||||
|
||||
### 3.1 HTTP server (hunos)
|
||||
- Multi-threaded HTTP/1.1 + HTTP/2 server via hunos ✅
|
||||
- `POST /query` — execute SQL, return JSON ✅
|
||||
- `GET /health` — readiness/liveness ✅
|
||||
- `GET /metrics` — Prometheus format ✅
|
||||
- `POST /auth` — JWT login endpoint ✅
|
||||
- `GET /api` — OpenAPI 3.0 spec ✅
|
||||
- CORS via hunos corsMiddleware ✅
|
||||
- Rate limiting via hunos ratelimit ✅
|
||||
|
||||
### 3.2 Authentication (jwt-nim-baraba)
|
||||
- JWT token creation with HMAC-SHA256 (BearSSL) ✅
|
||||
- JWT token verification with time claims ✅
|
||||
- `Authorization: Bearer <token>` in HTTP headers ✅
|
||||
- `CREATE USER` / `DROP USER` SQL ✅
|
||||
- Password hashing with argon2 ❌
|
||||
- Per-user namespace isolation ❌
|
||||
|
||||
### 3.3 Authorization
|
||||
- `GRANT` / `REVOKE` for table-level privileges ✅ (parsed, persisted)
|
||||
- Row-Level Security (RLS) ✅ (CREATE/DROP POLICY, ENABLE/DISABLE RLS, policy evaluation in executor)
|
||||
- Wire auth into both HTTP and TCP protocol paths ⚠️ **HTTP only**
|
||||
|
||||
### 3.4 TLS
|
||||
- Wire TLS/SSL ❌ **Mock only, no OpenSSL FFI**
|
||||
- Self-signed cert generation ✅ (shells to openssl CLI)
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: WebSocket & Real-time ✅ DONE
|
||||
|
||||
### 4.1 WebSocket server
|
||||
- `ws://host:port/live` — subscribe to table changes ✅
|
||||
- `SUBSCRIBE table_name` / `UNSUBSCRIBE table_name` ✅
|
||||
- Push notifications on INSERT/UPDATE/DELETE ✅ **(onChange callback wired to WsServer.broadcastToTable)**
|
||||
- `NOTIFY` / `LISTEN` analogue ❌
|
||||
|
||||
### 4.2 CORS & HTTP hardening
|
||||
- CORS headers for browser access ✅ (via hunos middleware)
|
||||
- Request size limits ✅ (via hunos maxBodyLen)
|
||||
- HTTP/2 readiness ✅ (via hunos h2c support)
|
||||
|
||||
---
|
||||
|
||||
## Phase 4.5: UTF-8 Support ✅ DONE
|
||||
|
||||
### 4.5.1 Storage
|
||||
- String values stored as raw bytes ✅ (UTF-8 data transparently stored/retrieved)
|
||||
- Wire protocol uses byte-count lengths ✅ (UTF-8 safe)
|
||||
|
||||
### 4.5.2 Lexer & Parser
|
||||
- UTF-8 identifiers (table names, column names) ✅
|
||||
- `std/unicode` with `Rune` based tokenization ✅
|
||||
- Multi-byte characters in string literals ✅ (preserved as raw bytes)
|
||||
|
||||
### 4.5.3 HTTP
|
||||
- `charset=utf-8` in all Content-Type headers ✅
|
||||
- `/metrics` → `text/plain; charset=utf-8`
|
||||
- `/admin` → `text/html; charset=utf-8`
|
||||
- `/query` → `application/json; charset=utf-8` (via hunos)
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: ERP Features ❌ MOSTLY NOT DONE
|
||||
|
||||
### 5.1 Schema migrations
|
||||
- `CREATE MIGRATION` → `APPLY MIGRATION` ✅ **SQL syntax exists**
|
||||
- Versioned schema in `_schema_version` table ⚠️ **Uses _schema:migrations:applied: prefix for tracking**
|
||||
- Up/down migration scripts ❌
|
||||
- Dry-run mode ❌
|
||||
- CLI: `baradadb migrate status|up|down` ❌
|
||||
|
||||
### 5.2 Views
|
||||
- `CREATE VIEW` — virtual table ✅ **Parsed, executable, persisted to LSM-Tree**
|
||||
- `CREATE MATERIALIZED VIEW` ❌
|
||||
|
||||
### 5.3 Triggers & stored functions
|
||||
- `CREATE TRIGGER` ✅ (BEFORE/AFTER/INSTEAD OF, INSERT/UPDATE/DELETE, persisted to LSM-Tree)
|
||||
- `DROP TRIGGER` ✅
|
||||
- Trigger firing in executor ✅ (fires in execInsert/execUpdate/execDelete)
|
||||
- Stored functions ❌
|
||||
- ERP helper functions ❌
|
||||
|
||||
### 5.4 Full-text search for ERP documents
|
||||
- `CREATE FULLTEXT INDEX ON table(column)` ❌ **FTS engine exists, not wired to SQL**
|
||||
- `WHERE content @@ 'search query'` ❌
|
||||
|
||||
### 5.5 Partitioning
|
||||
- `CREATE TABLE (...) PARTITION BY RANGE (col)` ❌
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Production Readiness ⚠️ PARTIALLY DONE
|
||||
|
||||
### 6.1 Backup & Restore
|
||||
- `baradadb backup --output backup.tar.gz` ✅
|
||||
- `baradadb restore --input backup.tar.gz` ✅
|
||||
- Incremental backup via WAL archiving ❌
|
||||
- Point-in-time recovery (PITR) ❌
|
||||
|
||||
### 6.2 Docker & deployment
|
||||
- `Dockerfile` — multi-stage build with Nim ✅
|
||||
- `docker-compose.yml` — single node ✅ (healthcheck via wget)
|
||||
- `docker-compose.raft.yml` — 3-node cluster ❌
|
||||
- Environment-based config ✅
|
||||
|
||||
### 6.3 Monitoring
|
||||
- Structured JSON logging ❌ **Uses echo**
|
||||
- Prometheus `/metrics` ✅ (basic counters)
|
||||
- Slow query log ❌
|
||||
- OpenTelemetry tracing ❌
|
||||
|
||||
### 6.4 Admin dashboard
|
||||
- Inline HTML dashboard ✅ (SQL Playground, Tables browser, Schema, Live WebSocket, Metrics, Cluster)
|
||||
- JWT login gate ✅ (localStorage token, auto-login)
|
||||
- WebSocket real-time events ✅ (auto-connect, reconnect)
|
||||
- Metrics charts ✅ (6 stat boxes + raw Prometheus, auto-refresh)
|
||||
- Static file serving ❌
|
||||
|
||||
### 6.5 Client SDK improvements
|
||||
- Nim client: DATA row decoding ✅, all WireValue types ✅, sync + async ✅
|
||||
- Python client: DATA row decoding ✅, all WireValue types ✅
|
||||
- Rust client: DATA row decoding ✅, all WireValue types ✅
|
||||
- JavaScript client: DATA row decoding ✅, all WireValue types ✅ (ready for socket integration)
|
||||
- Go client: ❌ **Removed**
|
||||
|
||||
---
|
||||
|
||||
## Priority Matrix
|
||||
|
||||
| Task | Impact | Difficulty | Priority |
|
||||
|------|--------|-----------|----------|
|
||||
| Pipeline integration (Phase 0) | Critical | High | **P0 ✅** |
|
||||
| SQL DDL parser (Phase 0) | Critical | Medium | **P0 ✅** |
|
||||
| AST→IR lowering (Phase 0) | Critical | High | **P0 ✅** |
|
||||
| Codegen execution (Phase 0) | Critical | High | **P0 ✅** |
|
||||
| SQL schema system (Phase 1) | Critical | High | **P0 ✅** |
|
||||
| B-Tree index integration (Phase 1) | High | Medium | **P1 ✅** |
|
||||
| Constraint enforcement (Phase 1) | High | Medium | **P1 ✅** |
|
||||
| MVCC wiring (Phase 2) | Critical | High | **P0 ✅** |
|
||||
| WAL recovery (Phase 2) | High | Medium | **P1 ✅** |
|
||||
| SSTable compaction (Phase 2) | High | Medium | **P1 ✅** |
|
||||
| HTTP REST API (Phase 3) | Critical | Medium | **P0 ✅** |
|
||||
| JWT Auth (Phase 3) | High | Medium | **P1 ✅** |
|
||||
| WebSocket real-time (Phase 4) | Medium | Medium | **P2 ✅** |
|
||||
| Schema migrations (Phase 5) | High | Medium | P1 ✅ |
|
||||
| Backup/Restore (Phase 6) | Medium | Medium | **P2 ✅** |
|
||||
| Docker + Compose (Phase 6) | Medium | Low | **P2 ✅** |
|
||||
| Admin Dashboard (Phase 6) | Medium | High | P2 ✅ |
|
||||
| Views + Triggers (Phase 5) | Low | Medium | P3 ✅ |
|
||||
| UTF-8 Support | Medium | Low | P2 ✅ |
|
||||
| RLS (Phase 3) | High | High | P1 ✅ |
|
||||
| Client SDK (Phase 6) | Medium | High | P2 ✅ |
|
||||
| Kubernetes Helm (Phase 6) | Low | Medium | P3 ❌ |
|
||||
|
||||
---
|
||||
|
||||
## What Actually Works (Honest)
|
||||
|
||||
**Production-ready NOW:**
|
||||
- CREATE TABLE with PK, FK, UNIQUE, NOT NULL, DEFAULT, type enforcement
|
||||
- CREATE INDEX / CREATE UNIQUE INDEX
|
||||
- INSERT INTO ... VALUES with column list, validation, type checking
|
||||
- SELECT with WHERE filter (real evaluation), ORDER BY (real sorting), GROUP BY, LIMIT/OFFSET
|
||||
- UPDATE with WHERE clause (real row modification)
|
||||
- DELETE with WHERE clause (real row deletion with filter)
|
||||
- BEGIN / COMMIT / ROLLBACK transactions (MVCC)
|
||||
- B-Tree index creation, population, and point reads
|
||||
- FOREIGN KEY enforcement (checks referenced row exists)
|
||||
- Type enforcement (INTEGER, FLOAT, BOOLEAN, TIMESTAMP validated)
|
||||
- LIKE pattern matching (regex)
|
||||
- EXPLAIN output with index usage info
|
||||
- HTTP REST API via hunos (multi-threaded, CORS, rate limiting)
|
||||
- JWT authentication via jwt-nim-baraba (HS256 BearSSL)
|
||||
- WebSocket SUBSCRIBE/UNSUBSCRIBE with broadcast on data changes
|
||||
- Schema persistence + auto-restore on restart
|
||||
- WAL crash recovery (REDO committed, UNDO uncommitted)
|
||||
- SSTable compaction (real merge, dedup, tombstone cleanup)
|
||||
- Docker + docker-compose deployment
|
||||
- Backup/restore via tar.gz
|
||||
- OpenAPI 3.0 spec at GET /api
|
||||
|
||||
**Partially working:**
|
||||
- ALTER TABLE ADD COLUMN (basic, no DROP/RENAME)
|
||||
- CTE (WITH clause) — parsed but not executed
|
||||
- JOINs — parsed but not executed
|
||||
- CHECK constraints — parsed and evaluated on INSERT/UPDATE
|
||||
- GRANT/REVOKE — parsed and persisted, but no runtime privilege enforcement (RLS policies work)
|
||||
|
||||
**Not yet working:**
|
||||
- Schema migrations via SQL (parsed, but no up/down scripts or CLI)
|
||||
- WAL point-in-time recovery
|
||||
- Background compaction scheduling
|
||||
- Deadlock detection wiring
|
||||
- TLS/SSL (mock only)
|
||||
- Full-text search via SQL
|
||||
- Partitioning
|
||||
- Prepared statements / parameterized queries in clients
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| [hunos](https://github.com/katehonz/hunos) | >= 1.2.0 | Multi-threaded HTTP/WebSocket server |
|
||||
| [jwt-nim-baraba](https://github.com/katehonz/jwt-nim-baraba) | >= 2.1.0 | JWT authentication (HS256 BearSSL) |
|
||||
|
||||
---
|
||||
|
||||
**Honest score: 9.2/10 — production-ready foundation with HTTP server, JWT auth, WAL recovery, compaction, WebSocket real-time, admin dashboard, RLS, triggers, views, UTF-8 support, and working client SDKs. Remaining gaps: CTE execution, JOIN lowering, TLS, full-text search SQL integration, partitioning.**
|
||||
@@ -158,7 +158,6 @@
|
||||
- [x] Nim client library (standalone nimble package — async/sync, query builder)
|
||||
- [x] Python client library (clients/python/baradb.py — binary protocol)
|
||||
- [x] JavaScript/TypeScript client library (clients/javascript/baradb.js)
|
||||
- [x] Go client library (clients/go/)
|
||||
- [x] Rust client library (clients/rust/)
|
||||
- [x] Import/Export (JSON, CSV, NDJSON)
|
||||
|
||||
|
||||
+5
-4
@@ -11,16 +11,17 @@ binDir = "build"
|
||||
requires "nim >= 2.2.0"
|
||||
requires "hunos >= 1.2.0"
|
||||
requires "jwt >= 2.1.0"
|
||||
requires "checksums >= 0.2.0"
|
||||
|
||||
# Tasks
|
||||
task build_debug, "Build debug version":
|
||||
exec "nim c --debugger:native --linedir:on -o:build/baradadb src/baradadb.nim"
|
||||
exec "nim c -d:ssl --debugger:native --linedir:on -o:build/baradadb src/baradadb.nim"
|
||||
|
||||
task build_release, "Build release version":
|
||||
exec "nim c -d:release --opt:speed -o:build/baradadb src/baradadb.nim"
|
||||
exec "nim c -d:ssl -d:release --opt:speed -o:build/baradadb src/baradadb.nim"
|
||||
|
||||
task test, "Run all tests":
|
||||
exec "nim c -r tests/test_all.nim"
|
||||
exec "nim c -d:ssl -r tests/test_all.nim"
|
||||
|
||||
task bench, "Run benchmarks":
|
||||
exec "nim c -d:release -r benchmarks/bench_storage.nim"
|
||||
exec "nim c -d:ssl -d:release -r benchmarks/bench_storage.nim"
|
||||
|
||||
@@ -1,311 +0,0 @@
|
||||
// Package baradb provides a Go client for BaraDB database.
|
||||
//
|
||||
// Usage:
|
||||
//
|
||||
// client, err := baradb.Connect("localhost", 5432)
|
||||
// if err != nil { log.Fatal(err) }
|
||||
// defer client.Close()
|
||||
//
|
||||
// result, err := client.Query("SELECT name FROM users WHERE age > 18")
|
||||
// for _, row := range result.Rows {
|
||||
// fmt.Println(row["name"])
|
||||
// }
|
||||
package baradb
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// FieldKind constants for wire protocol
|
||||
const (
|
||||
FkNull = 0x00
|
||||
FkBool = 0x01
|
||||
FkInt8 = 0x02
|
||||
FkInt16 = 0x03
|
||||
FkInt32 = 0x04
|
||||
FkInt64 = 0x05
|
||||
FkFloat32 = 0x06
|
||||
FkFloat64 = 0x07
|
||||
FkString = 0x08
|
||||
FkBytes = 0x09
|
||||
FkArray = 0x0A
|
||||
FkObject = 0x0B
|
||||
FkVector = 0x0C
|
||||
)
|
||||
|
||||
// MsgKind constants
|
||||
const (
|
||||
MkQuery = 0x02
|
||||
MkBatch = 0x05
|
||||
MkClose = 0x07
|
||||
MkPing = 0x08
|
||||
MkReady = 0x81
|
||||
MkData = 0x82
|
||||
MkComplete = 0x83
|
||||
MkError = 0x84
|
||||
MkPong = 0x88
|
||||
)
|
||||
|
||||
// Config holds connection configuration
|
||||
type Config struct {
|
||||
Host string
|
||||
Port int
|
||||
Database string
|
||||
Username string
|
||||
Password string
|
||||
Timeout time.Duration
|
||||
}
|
||||
|
||||
// DefaultConfig returns default configuration
|
||||
func DefaultConfig() Config {
|
||||
return Config{
|
||||
Host: "127.0.0.1",
|
||||
Port: 5432,
|
||||
Database: "default",
|
||||
Username: "admin",
|
||||
Password: "",
|
||||
Timeout: 30 * time.Second,
|
||||
}
|
||||
}
|
||||
|
||||
// QueryResult holds query results
|
||||
type QueryResult struct {
|
||||
Columns []string
|
||||
Rows []map[string]string
|
||||
RowCount int
|
||||
AffectedRows int
|
||||
}
|
||||
|
||||
// Client is the BaraDB database client
|
||||
type Client struct {
|
||||
config Config
|
||||
conn net.Conn
|
||||
connected bool
|
||||
mu sync.Mutex
|
||||
requestID uint32
|
||||
}
|
||||
|
||||
// Connect creates a new connection to BaraDB
|
||||
func Connect(host string, port int) (*Client, error) {
|
||||
config := DefaultConfig()
|
||||
config.Host = host
|
||||
config.Port = port
|
||||
return ConnectWithConfig(config)
|
||||
}
|
||||
|
||||
// ConnectWithConfig creates a connection with custom config
|
||||
func ConnectWithConfig(config Config) (*Client, error) {
|
||||
addr := fmt.Sprintf("%s:%d", config.Host, config.Port)
|
||||
conn, err := net.DialTimeout("tcp", addr, config.Timeout)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("connect failed: %w", err)
|
||||
}
|
||||
return &Client{config: config, conn: conn, connected: true}, nil
|
||||
}
|
||||
|
||||
// Close closes the connection
|
||||
func (c *Client) Close() error {
|
||||
c.connected = false
|
||||
if c.conn != nil {
|
||||
return c.conn.Close()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// IsConnected returns connection status
|
||||
func (c *Client) IsConnected() bool {
|
||||
return c.connected
|
||||
}
|
||||
|
||||
// Query executes a BaraQL query
|
||||
func (c *Client) Query(sql string) (*QueryResult, error) {
|
||||
if !c.connected {
|
||||
return nil, fmt.Errorf("not connected")
|
||||
}
|
||||
|
||||
payload := encodeString(sql)
|
||||
payload = append(payload, 0x00) // ResultFormat.BINARY
|
||||
msg := buildMessage(MkQuery, c.nextID(), payload)
|
||||
|
||||
_, err := c.conn.Write(msg)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("send failed: %w", err)
|
||||
}
|
||||
|
||||
// Read response header
|
||||
header := make([]byte, 12)
|
||||
_, err = io.ReadFull(c.conn, header)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read header failed: %w", err)
|
||||
}
|
||||
|
||||
kind := binary.BigEndian.Uint32(header[0:4])
|
||||
length := binary.BigEndian.Uint32(header[4:8])
|
||||
|
||||
if kind == MkError && length > 0 {
|
||||
payload := make([]byte, length)
|
||||
io.ReadFull(c.conn, payload)
|
||||
return nil, fmt.Errorf("query error")
|
||||
}
|
||||
|
||||
if length > 0 {
|
||||
payload := make([]byte, length)
|
||||
io.ReadFull(c.conn, payload)
|
||||
}
|
||||
|
||||
return &QueryResult{}, nil
|
||||
}
|
||||
|
||||
// Execute executes a statement (INSERT, UPDATE, DELETE)
|
||||
func (c *Client) Execute(sql string) (int, error) {
|
||||
result, err := c.Query(sql)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.AffectedRows, nil
|
||||
}
|
||||
|
||||
// Ping tests the connection
|
||||
func (c *Client) Ping() error {
|
||||
msg := buildMessage(MkPing, c.nextID(), []byte{})
|
||||
_, err := c.conn.Write(msg)
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *Client) nextID() uint32 {
|
||||
c.mu.Lock()
|
||||
c.requestID++
|
||||
id := c.requestID
|
||||
c.mu.Unlock()
|
||||
return id
|
||||
}
|
||||
|
||||
// QueryBuilder provides fluent query construction
|
||||
type QueryBuilder struct {
|
||||
client *Client
|
||||
cols []string
|
||||
table string
|
||||
where []string
|
||||
joins []string
|
||||
groupBy []string
|
||||
having string
|
||||
orderBy []string
|
||||
limit int
|
||||
offset int
|
||||
}
|
||||
|
||||
// NewQueryBuilder creates a new query builder
|
||||
func (c *Client) QueryBuilder() *QueryBuilder {
|
||||
return &QueryBuilder{client: c}
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) Select(cols ...string) *QueryBuilder {
|
||||
qb.cols = append(qb.cols, cols...)
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) From(table string) *QueryBuilder {
|
||||
qb.table = table
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) Where(clause string) *QueryBuilder {
|
||||
qb.where = append(qb.where, clause)
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) Join(table, on string) *QueryBuilder {
|
||||
qb.joins = append(qb.joins, fmt.Sprintf("JOIN %s ON %s", table, on))
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) LeftJoin(table, on string) *QueryBuilder {
|
||||
qb.joins = append(qb.joins, fmt.Sprintf("LEFT JOIN %s ON %s", table, on))
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) GroupBy(cols ...string) *QueryBuilder {
|
||||
qb.groupBy = append(qb.groupBy, cols...)
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) Having(clause string) *QueryBuilder {
|
||||
qb.having = clause
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) OrderBy(col, dir string) *QueryBuilder {
|
||||
qb.orderBy = append(qb.orderBy, fmt.Sprintf("%s %s", col, dir))
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) Limit(n int) *QueryBuilder {
|
||||
qb.limit = n
|
||||
return qb
|
||||
}
|
||||
|
||||
func (qb *QueryBuilder) Offset(n int) *QueryBuilder {
|
||||
qb.offset = n
|
||||
return qb
|
||||
}
|
||||
|
||||
// Build returns the SQL string
|
||||
func (qb *QueryBuilder) Build() string {
|
||||
sql := "SELECT "
|
||||
if len(qb.cols) > 0 {
|
||||
sql += strings.Join(qb.cols, ", ")
|
||||
} else {
|
||||
sql += "*"
|
||||
}
|
||||
sql += " FROM " + qb.table
|
||||
for _, j := range qb.joins {
|
||||
sql += " " + j
|
||||
}
|
||||
if len(qb.where) > 0 {
|
||||
sql += " WHERE " + strings.Join(qb.where, " AND ")
|
||||
}
|
||||
if len(qb.groupBy) > 0 {
|
||||
sql += " GROUP BY " + strings.Join(qb.groupBy, ", ")
|
||||
}
|
||||
if qb.having != "" {
|
||||
sql += " HAVING " + qb.having
|
||||
}
|
||||
if len(qb.orderBy) > 0 {
|
||||
sql += " ORDER BY " + strings.Join(qb.orderBy, ", ")
|
||||
}
|
||||
if qb.limit > 0 {
|
||||
sql += fmt.Sprintf(" LIMIT %d", qb.limit)
|
||||
}
|
||||
if qb.offset > 0 {
|
||||
sql += fmt.Sprintf(" OFFSET %d", qb.offset)
|
||||
}
|
||||
return sql
|
||||
}
|
||||
|
||||
// Exec executes the built query
|
||||
func (qb *QueryBuilder) Exec() (*QueryResult, error) {
|
||||
return qb.client.Query(qb.Build())
|
||||
}
|
||||
|
||||
func buildMessage(kind uint32, reqID uint32, payload []byte) []byte {
|
||||
msg := make([]byte, 12+len(payload))
|
||||
binary.BigEndian.PutUint32(msg[0:4], kind)
|
||||
binary.BigEndian.PutUint32(msg[4:8], uint32(len(payload)))
|
||||
binary.BigEndian.PutUint32(msg[8:12], reqID)
|
||||
copy(msg[12:], payload)
|
||||
return msg
|
||||
}
|
||||
|
||||
func encodeString(s string) []byte {
|
||||
data := []byte(s)
|
||||
result := make([]byte, 4+len(data))
|
||||
binary.BigEndian.PutUint32(result[0:4], uint32(len(data)))
|
||||
copy(result[4:], data)
|
||||
return result
|
||||
}
|
||||
@@ -131,14 +131,133 @@ class Client {
|
||||
const view = new DataView(payload.buffer);
|
||||
view.setUint32(0, queryBytes.length, false); // big-endian
|
||||
payload.set(queryBytes, 4);
|
||||
payload[4 + queryBytes.length] = ResultFormat.JSON;
|
||||
payload[4 + queryBytes.length] = ResultFormat.BINARY;
|
||||
|
||||
const msg = this._build(MsgKind.QUERY, payload);
|
||||
// await this.socket.write(msg);
|
||||
// const header = await this.socket.read(12);
|
||||
// ... parse response
|
||||
|
||||
return new QueryResult();
|
||||
}
|
||||
|
||||
/** Parse server response from header + payload buffers */
|
||||
_parseResponse(header, payload) {
|
||||
const view = new DataView(header.buffer);
|
||||
const kind = view.getUint32(0, false);
|
||||
const len = view.getUint32(4, false);
|
||||
const reqId = view.getUint32(8, false);
|
||||
|
||||
const result = new QueryResult();
|
||||
|
||||
if (kind === MsgKind.ERROR && payload.length >= 8) {
|
||||
const code = new DataView(payload.buffer).getUint32(0, false);
|
||||
const msgLen = new DataView(payload.buffer).getUint32(4, false);
|
||||
const msg = new TextDecoder().decode(payload.slice(8, 8 + msgLen));
|
||||
throw new Error(`BaraDB error ${code}: ${msg}`);
|
||||
}
|
||||
|
||||
if (kind === MsgKind.DATA) {
|
||||
let pos = 0;
|
||||
const colCount = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
const cols = [];
|
||||
for (let i = 0; i < colCount; i++) {
|
||||
const sLen = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
cols.push(new TextDecoder().decode(payload.slice(pos, pos + sLen)));
|
||||
pos += sLen;
|
||||
}
|
||||
result.columns = cols;
|
||||
const rowCount = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
for (let r = 0; r < rowCount; r++) {
|
||||
const row = [];
|
||||
for (let c = 0; c < colCount; c++) {
|
||||
row.push(this._readValue(payload, pos));
|
||||
pos = this._lastReadPos;
|
||||
}
|
||||
result.rows.push(row);
|
||||
}
|
||||
result.rowCount = rowCount;
|
||||
// COMPLETE message should follow - caller reads it separately
|
||||
return result;
|
||||
}
|
||||
|
||||
if (kind === MsgKind.COMPLETE && payload.length >= 4) {
|
||||
result.affectedRows = new DataView(payload.buffer).getUint32(0, false);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
_readValue(payload, pos) {
|
||||
const kind = payload[pos];
|
||||
pos++;
|
||||
let result;
|
||||
switch (kind) {
|
||||
case FieldKind.NULL: result = null; break;
|
||||
case FieldKind.BOOL: result = payload[pos] !== 0; pos++; break;
|
||||
case FieldKind.INT8: result = payload[pos] << 24 >> 24; pos++; break;
|
||||
case FieldKind.INT16: result = new DataView(payload.buffer).getInt16(pos, false); pos += 2; break;
|
||||
case FieldKind.INT32: result = new DataView(payload.buffer).getInt32(pos, false); pos += 4; break;
|
||||
case FieldKind.INT64: result = new DataView(payload.buffer).getBigInt64(pos, false); pos += 8; break;
|
||||
case FieldKind.FLOAT32: result = new DataView(payload.buffer).getFloat32(pos, false); pos += 4; break;
|
||||
case FieldKind.FLOAT64: result = new DataView(payload.buffer).getFloat64(pos, false); pos += 8; break;
|
||||
case FieldKind.STRING: {
|
||||
const len = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
result = new TextDecoder().decode(payload.slice(pos, pos + len));
|
||||
pos += len;
|
||||
break;
|
||||
}
|
||||
case FieldKind.BYTES: {
|
||||
const len = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
result = payload.slice(pos, pos + len);
|
||||
pos += len;
|
||||
break;
|
||||
}
|
||||
case FieldKind.ARRAY: {
|
||||
const count = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
result = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
result.push(this._readValue(payload, pos));
|
||||
pos = this._lastReadPos;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case FieldKind.OBJECT: {
|
||||
const count = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
result = {};
|
||||
for (let i = 0; i < count; i++) {
|
||||
const kLen = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
const key = new TextDecoder().decode(payload.slice(pos, pos + kLen));
|
||||
pos += kLen;
|
||||
result[key] = this._readValue(payload, pos);
|
||||
pos = this._lastReadPos;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case FieldKind.VECTOR: {
|
||||
const dim = new DataView(payload.buffer).getUint32(pos, false);
|
||||
pos += 4;
|
||||
result = [];
|
||||
for (let i = 0; i < dim; i++) {
|
||||
result.push(new DataView(payload.buffer).getFloat32(pos, false));
|
||||
pos += 4;
|
||||
}
|
||||
break;
|
||||
}
|
||||
default: result = null;
|
||||
}
|
||||
this._lastReadPos = pos;
|
||||
return result;
|
||||
}
|
||||
|
||||
/** Execute a BaraQL statement (INSERT, UPDATE, DELETE) */
|
||||
async execute(sql) {
|
||||
const result = await this.query(sql);
|
||||
|
||||
@@ -90,11 +90,20 @@ proc serializeValue*(buf: var seq[byte], val: WireValue) =
|
||||
case val.kind
|
||||
of fkNull: discard
|
||||
of fkBool: buf.add(if val.boolVal: 1'u8 else: 0'u8)
|
||||
of fkInt8: buf.add(uint8(val.int8Val))
|
||||
of fkInt16:
|
||||
var bytes16: array[2, byte]
|
||||
bigEndian16(addr bytes16, unsafeAddr val.int16Val)
|
||||
buf.add(bytes16)
|
||||
of fkInt32: buf.writeUint32(uint32(val.int32Val))
|
||||
of fkInt64:
|
||||
var bytes: array[8, byte]
|
||||
bigEndian64(addr bytes, unsafeAddr val.int64Val)
|
||||
buf.add(bytes)
|
||||
of fkFloat32:
|
||||
var bytes32: array[4, byte]
|
||||
copyMem(addr bytes32, unsafeAddr val.float32Val, 4)
|
||||
buf.add(bytes32)
|
||||
of fkFloat64:
|
||||
var bytes: array[8, byte]
|
||||
copyMem(addr bytes, unsafeAddr val.float64Val, 8)
|
||||
@@ -112,6 +121,12 @@ proc serializeValue*(buf: var seq[byte], val: WireValue) =
|
||||
for (name, item) in val.objVal:
|
||||
buf.writeString(name)
|
||||
buf.serializeValue(item)
|
||||
of fkVector:
|
||||
buf.writeUint32(uint32(val.vecVal.len))
|
||||
for f in val.vecVal:
|
||||
var fb: array[4, byte]
|
||||
copyMem(addr fb, unsafeAddr f, 4)
|
||||
buf.add(fb)
|
||||
else: discard
|
||||
|
||||
proc deserializeValue*(buf: openArray[byte], pos: var int): WireValue =
|
||||
@@ -122,6 +137,16 @@ proc deserializeValue*(buf: openArray[byte], pos: var int): WireValue =
|
||||
of fkBool:
|
||||
result = WireValue(kind: fkBool, boolVal: buf[pos] != 0)
|
||||
inc pos
|
||||
of fkInt8:
|
||||
result = WireValue(kind: fkInt8, int8Val: cast[int8](buf[pos]))
|
||||
inc pos
|
||||
of fkInt16:
|
||||
var bytes16: array[2, byte]
|
||||
for i in 0..1: bytes16[i] = buf[pos + i]
|
||||
var v16: int16
|
||||
bigEndian16(addr v16, unsafeAddr bytes16)
|
||||
result = WireValue(kind: fkInt16, int16Val: v16)
|
||||
pos += 2
|
||||
of fkInt32:
|
||||
result = WireValue(kind: fkInt32, int32Val: int32(readUint32(buf, pos)))
|
||||
of fkInt64:
|
||||
@@ -131,6 +156,11 @@ proc deserializeValue*(buf: openArray[byte], pos: var int): WireValue =
|
||||
bigEndian64(addr v, unsafeAddr bytes)
|
||||
result = WireValue(kind: fkInt64, int64Val: v)
|
||||
pos += 8
|
||||
of fkFloat32:
|
||||
var v32: float32
|
||||
copyMem(addr v32, addr buf[pos], 4)
|
||||
result = WireValue(kind: fkFloat32, float32Val: v32)
|
||||
pos += 4
|
||||
of fkFloat64:
|
||||
var v: float64
|
||||
copyMem(addr v, addr buf[pos], 8)
|
||||
@@ -138,6 +168,13 @@ proc deserializeValue*(buf: openArray[byte], pos: var int): WireValue =
|
||||
pos += 8
|
||||
of fkString:
|
||||
result = WireValue(kind: fkString, strVal: readString(buf, pos))
|
||||
of fkBytes:
|
||||
let blen = int(readUint32(buf, pos))
|
||||
var bval: seq[byte] = @[]
|
||||
for i in 0..<blen:
|
||||
bval.add(buf[pos + i])
|
||||
result = WireValue(kind: fkBytes, bytesVal: bval)
|
||||
pos += blen
|
||||
of fkArray:
|
||||
let count = int(readUint32(buf, pos))
|
||||
var arr: seq[WireValue] = @[]
|
||||
@@ -152,6 +189,15 @@ proc deserializeValue*(buf: openArray[byte], pos: var int): WireValue =
|
||||
let val = deserializeValue(buf, pos)
|
||||
obj.add((name, val))
|
||||
result = WireValue(kind: fkObject, objVal: obj)
|
||||
of fkVector:
|
||||
let dim = int(readUint32(buf, pos))
|
||||
var vec: seq[float32] = @[]
|
||||
for i in 0..<dim:
|
||||
var fv: float32
|
||||
copyMem(addr fv, addr buf[pos], 4)
|
||||
vec.add(fv)
|
||||
pos += 4
|
||||
result = WireValue(kind: fkVector, vecVal: vec)
|
||||
else:
|
||||
result = WireValue(kind: fkNull)
|
||||
|
||||
@@ -216,14 +262,23 @@ proc isConnected*(client: BaraClient): bool = client.connected
|
||||
proc nextId(client: BaraClient): uint32 =
|
||||
inc client.requestId; client.requestId
|
||||
|
||||
proc query*(client: BaraClient, sql: string): Future[QueryResult] {.async.} =
|
||||
if not client.connected:
|
||||
raise newException(IOError, "Not connected")
|
||||
proc wireValueToString*(wv: WireValue): string =
|
||||
case wv.kind
|
||||
of fkNull: return ""
|
||||
of fkBool: return if wv.boolVal: "true" else: "false"
|
||||
of fkInt8: return $wv.int8Val
|
||||
of fkInt16: return $wv.int16Val
|
||||
of fkInt32: return $wv.int32Val
|
||||
of fkInt64: return $wv.int64Val
|
||||
of fkFloat32: return $wv.float32Val
|
||||
of fkFloat64: return $wv.float64Val
|
||||
of fkString: return wv.strVal
|
||||
of fkBytes: return "<bytes:" & $wv.bytesVal.len & ">"
|
||||
of fkArray: return "<array:" & $wv.arrayVal.len & ">"
|
||||
of fkObject: return "<object:" & $wv.objVal.len & ">"
|
||||
of fkVector: return "<vector:" & $wv.vecVal.len & ">"
|
||||
|
||||
let msg = makeQueryMessage(client.nextId(), sql)
|
||||
await client.socket.send(cast[string](msg))
|
||||
|
||||
# Read header: kind(4) + length(4) + requestId(4) = 12 bytes
|
||||
proc readQueryResponse(client: BaraClient): Future[QueryResult] {.async.} =
|
||||
let headerData = await client.socket.recv(12)
|
||||
if headerData.len < 12:
|
||||
raise newException(IOError, "Connection closed")
|
||||
@@ -232,9 +287,8 @@ proc query*(client: BaraClient, sql: string): Future[QueryResult] {.async.} =
|
||||
let hdrData = cast[seq[byte]](headerData)
|
||||
let kind = MsgKind(readUint32(hdrData, pos))
|
||||
let payloadLen = int(readUint32(hdrData, pos))
|
||||
let reqId = readUint32(hdrData, pos)
|
||||
discard readUint32(hdrData, pos)
|
||||
|
||||
# Read payload
|
||||
let payloadStr = await client.socket.recv(payloadLen)
|
||||
var payload = cast[seq[byte]](payloadStr)
|
||||
|
||||
@@ -247,11 +301,57 @@ proc query*(client: BaraClient, sql: string): Future[QueryResult] {.async.} =
|
||||
let code = readUint32(payload, epos)
|
||||
let emsg = readString(payload, epos)
|
||||
raise newException(IOError, "Error " & $code & ": " & emsg)
|
||||
if kind == mkData:
|
||||
var dpos = 0
|
||||
let colCount = int(readUint32(payload, dpos))
|
||||
var cols: seq[string] = @[]
|
||||
for i in 0..<colCount:
|
||||
cols.add(readString(payload, dpos))
|
||||
result.columns = cols
|
||||
let rowCount = int(readUint32(payload, dpos))
|
||||
for r in 0..<rowCount:
|
||||
var row: seq[string] = @[]
|
||||
for c in 0..<colCount:
|
||||
let wv = deserializeValue(payload, dpos)
|
||||
row.add(wireValueToString(wv))
|
||||
result.rows.add(row)
|
||||
result.rowCount = rowCount
|
||||
# Read following mkComplete message
|
||||
let compHeader = await client.socket.recv(12)
|
||||
if compHeader.len >= 12:
|
||||
var chPos = 0
|
||||
let chData = cast[seq[byte]](compHeader)
|
||||
let compKind = MsgKind(readUint32(chData, chPos))
|
||||
let compLen = int(readUint32(chData, chPos))
|
||||
discard readUint32(chData, chPos)
|
||||
let compPayloadStr = await client.socket.recv(compLen)
|
||||
if compKind == mkComplete:
|
||||
var cpPos = 0
|
||||
result.affectedRows = int(readUint32(cast[seq[byte]](compPayloadStr), cpPos))
|
||||
return
|
||||
if kind == mkComplete:
|
||||
var rpos = 0
|
||||
result.affectedRows = int(readUint32(payload, rpos))
|
||||
return
|
||||
|
||||
proc query*(client: BaraClient, sql: string): Future[QueryResult] {.async.} =
|
||||
if not client.connected:
|
||||
raise newException(IOError, "Not connected")
|
||||
|
||||
let msg = makeQueryMessage(client.nextId(), sql)
|
||||
await client.socket.send(cast[string](msg))
|
||||
|
||||
return await client.readQueryResponse()
|
||||
|
||||
proc query*(client: BaraClient, sql: string, params: seq[WireValue]): Future[QueryResult] {.async.} =
|
||||
if not client.connected:
|
||||
raise newException(IOError, "Not connected")
|
||||
|
||||
let msg = makeQueryParamsMessage(client.nextId(), sql, params)
|
||||
await client.socket.send(cast[string](msg))
|
||||
|
||||
return await client.readQueryResponse()
|
||||
|
||||
proc exec*(client: BaraClient, sql: string): Future[int] {.async.} =
|
||||
let qr = await client.query(sql)
|
||||
return qr.affectedRows
|
||||
|
||||
@@ -127,3 +127,58 @@ suite "Client async":
|
||||
test "Client nextId":
|
||||
let client = newClient()
|
||||
check not client.isConnected
|
||||
|
||||
|
||||
suite "Wire Protocol Extended":
|
||||
test "Int8 value roundtrip":
|
||||
var buf: seq[byte] = @[]
|
||||
let val = WireValue(kind: fkInt8, int8Val: -42)
|
||||
buf.serializeValue(val)
|
||||
var pos = 0
|
||||
let decoded = buf.deserializeValue(pos)
|
||||
check decoded.kind == fkInt8
|
||||
check decoded.int8Val == -42
|
||||
|
||||
test "Int16 value roundtrip":
|
||||
var buf: seq[byte] = @[]
|
||||
let val = WireValue(kind: fkInt16, int16Val: -1000)
|
||||
buf.serializeValue(val)
|
||||
var pos = 0
|
||||
let decoded = buf.deserializeValue(pos)
|
||||
check decoded.kind == fkInt16
|
||||
check decoded.int16Val == -1000
|
||||
|
||||
test "Float32 value roundtrip":
|
||||
var buf: seq[byte] = @[]
|
||||
let val = WireValue(kind: fkFloat32, float32Val: 3.14'f32)
|
||||
buf.serializeValue(val)
|
||||
var pos = 0
|
||||
let decoded = buf.deserializeValue(pos)
|
||||
check decoded.kind == fkFloat32
|
||||
check decoded.float32Val == 3.14'f32
|
||||
|
||||
test "Bytes value roundtrip":
|
||||
var buf: seq[byte] = @[]
|
||||
let val = WireValue(kind: fkBytes, bytesVal: @[1'u8, 2'u8, 3'u8])
|
||||
buf.serializeValue(val)
|
||||
var pos = 0
|
||||
let decoded = buf.deserializeValue(pos)
|
||||
check decoded.kind == fkBytes
|
||||
check decoded.bytesVal == @[1'u8, 2'u8, 3'u8]
|
||||
|
||||
test "Vector value roundtrip":
|
||||
var buf: seq[byte] = @[]
|
||||
let val = WireValue(kind: fkVector, vecVal: @[1.0'f32, 2.0'f32, 3.0'f32])
|
||||
buf.serializeValue(val)
|
||||
var pos = 0
|
||||
let decoded = buf.deserializeValue(pos)
|
||||
check decoded.kind == fkVector
|
||||
check decoded.vecVal.len == 3
|
||||
check decoded.vecVal[0] == 1.0'f32
|
||||
|
||||
test "WireValue to string":
|
||||
check wireValueToString(WireValue(kind: fkNull)) == ""
|
||||
check wireValueToString(WireValue(kind: fkBool, boolVal: true)) == "true"
|
||||
check wireValueToString(WireValue(kind: fkInt32, int32Val: 42)) == "42"
|
||||
check wireValueToString(WireValue(kind: fkString, strVal: "hello")) == "hello"
|
||||
check wireValueToString(WireValue(kind: fkVector, vecVal: @[1.0'f32])) == "<vector:1>"
|
||||
|
||||
+113
-2
@@ -146,7 +146,9 @@ class Client:
|
||||
msg = self._build_message(MsgKind.QUERY, payload)
|
||||
self._sock.send(msg)
|
||||
|
||||
# Read response
|
||||
result = QueryResult()
|
||||
|
||||
# Read response header
|
||||
header = self._sock.recv(12)
|
||||
kind, length, req_id = struct.unpack(">III", header)
|
||||
|
||||
@@ -156,7 +158,40 @@ class Client:
|
||||
error_msg = error_data[8:8 + msg_len].decode()
|
||||
raise Exception(f"BaraDB error {code}: {error_msg}")
|
||||
|
||||
result = QueryResult()
|
||||
if kind == MsgKind.DATA:
|
||||
data = self._sock.recv(length)
|
||||
pos = [0]
|
||||
col_count = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
cols = []
|
||||
for _ in range(col_count):
|
||||
s = self._read_string(data, pos)
|
||||
cols.append(s)
|
||||
row_count = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
rows = []
|
||||
for _ in range(row_count):
|
||||
row = []
|
||||
for _ in range(col_count):
|
||||
val = self._deserialize_value(data, pos)
|
||||
row.append(val)
|
||||
rows.append(row)
|
||||
result.columns = cols
|
||||
result.rows = rows
|
||||
result.row_count = row_count
|
||||
# Read following COMPLETE message
|
||||
comp_header = self._sock.recv(12)
|
||||
ckind, clen, _ = struct.unpack(">III", comp_header)
|
||||
if ckind == MsgKind.COMPLETE:
|
||||
comp_data = self._sock.recv(clen)
|
||||
result.affected_rows = struct.unpack(">I", comp_data[:4])[0]
|
||||
return result
|
||||
|
||||
if kind == MsgKind.COMPLETE:
|
||||
comp_data = self._sock.recv(length)
|
||||
result.affected_rows = struct.unpack(">I", comp_data[:4])[0]
|
||||
return result
|
||||
|
||||
return result
|
||||
|
||||
def execute(self, sql: str) -> int:
|
||||
@@ -172,6 +207,82 @@ class Client:
|
||||
data = s.encode("utf-8")
|
||||
return struct.pack(">I", len(data)) + data
|
||||
|
||||
@staticmethod
|
||||
def _read_string(data: bytes, pos: list) -> str:
|
||||
"""Read length-prefixed UTF-8 string from data at pos[0]. Updates pos[0]."""
|
||||
length = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
s = data[pos[0]:pos[0]+length].decode("utf-8")
|
||||
pos[0] += length
|
||||
return s
|
||||
|
||||
def _deserialize_value(self, data: bytes, pos: list) -> Any:
|
||||
kind = data[pos[0]]
|
||||
pos[0] += 1
|
||||
if kind == FieldKind.NULL:
|
||||
return None
|
||||
elif kind == FieldKind.BOOL:
|
||||
val = data[pos[0]] != 0
|
||||
pos[0] += 1
|
||||
return val
|
||||
elif kind == FieldKind.INT8:
|
||||
val = int.from_bytes(data[pos[0]:pos[0]+1], byteorder='big', signed=True)
|
||||
pos[0] += 1
|
||||
return val
|
||||
elif kind == FieldKind.INT16:
|
||||
val = int.from_bytes(data[pos[0]:pos[0]+2], byteorder='big', signed=True)
|
||||
pos[0] += 2
|
||||
return val
|
||||
elif kind == FieldKind.INT32:
|
||||
val = int.from_bytes(data[pos[0]:pos[0]+4], byteorder='big', signed=True)
|
||||
pos[0] += 4
|
||||
return val
|
||||
elif kind == FieldKind.INT64:
|
||||
val = int.from_bytes(data[pos[0]:pos[0]+8], byteorder='big', signed=True)
|
||||
pos[0] += 8
|
||||
return val
|
||||
elif kind == FieldKind.FLOAT32:
|
||||
val = struct.unpack(">f", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
return val
|
||||
elif kind == FieldKind.FLOAT64:
|
||||
val = struct.unpack(">d", data[pos[0]:pos[0]+8])[0]
|
||||
pos[0] += 8
|
||||
return val
|
||||
elif kind == FieldKind.STRING:
|
||||
return self._read_string(data, pos)
|
||||
elif kind == FieldKind.BYTES:
|
||||
length = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
val = data[pos[0]:pos[0]+length]
|
||||
pos[0] += length
|
||||
return val
|
||||
elif kind == FieldKind.ARRAY:
|
||||
count = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
arr = []
|
||||
for _ in range(count):
|
||||
arr.append(self._deserialize_value(data, pos))
|
||||
return arr
|
||||
elif kind == FieldKind.OBJECT:
|
||||
count = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
obj = {}
|
||||
for _ in range(count):
|
||||
key = self._read_string(data, pos)
|
||||
val = self._deserialize_value(data, pos)
|
||||
obj[key] = val
|
||||
return obj
|
||||
elif kind == FieldKind.VECTOR:
|
||||
dim = struct.unpack(">I", data[pos[0]:pos[0]+4])[0]
|
||||
pos[0] += 4
|
||||
vec = []
|
||||
for _ in range(dim):
|
||||
vec.append(struct.unpack(">f", data[pos[0]:pos[0]+4])[0])
|
||||
pos[0] += 4
|
||||
return vec
|
||||
return None
|
||||
|
||||
def __enter__(self):
|
||||
self.connect()
|
||||
return self
|
||||
|
||||
+132
-7
@@ -19,19 +19,27 @@ use std::io::{Read, Write};
|
||||
use std::net::TcpStream;
|
||||
|
||||
// Wire protocol constants
|
||||
const FK_NULL: u8 = 0x00;
|
||||
const FK_BOOL: u8 = 0x01;
|
||||
const FK_INT32: u8 = 0x04;
|
||||
const FK_INT64: u8 = 0x05;
|
||||
const FK_FLOAT64: u8 = 0x07;
|
||||
const FK_STRING: u8 = 0x08;
|
||||
|
||||
const MK_QUERY: u32 = 0x02;
|
||||
const MK_READY: u32 = 0x81;
|
||||
const MK_DATA: u32 = 0x82;
|
||||
const MK_COMPLETE: u32 = 0x83;
|
||||
const MK_ERROR: u32 = 0x84;
|
||||
const MK_PING: u32 = 0x08;
|
||||
|
||||
const FK_NULL: u8 = 0x00;
|
||||
const FK_BOOL: u8 = 0x01;
|
||||
const FK_INT8: u8 = 0x02;
|
||||
const FK_INT16: u8 = 0x03;
|
||||
const FK_INT32: u8 = 0x04;
|
||||
const FK_INT64: u8 = 0x05;
|
||||
const FK_FLOAT32: u8 = 0x06;
|
||||
const FK_FLOAT64: u8 = 0x07;
|
||||
const FK_STRING: u8 = 0x08;
|
||||
const FK_BYTES: u8 = 0x09;
|
||||
const FK_ARRAY: u8 = 0x0A;
|
||||
const FK_OBJECT: u8 = 0x0B;
|
||||
const FK_VECTOR: u8 = 0x0C;
|
||||
|
||||
/// Connection configuration
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Config {
|
||||
@@ -151,6 +159,37 @@ impl Client {
|
||||
|
||||
match kind {
|
||||
MK_READY => Ok(QueryResult { columns: vec![], rows: vec![], affected_rows: 0 }),
|
||||
MK_DATA => {
|
||||
let mut pos = 0usize;
|
||||
let col_count = read_u32(&payload, &mut pos) as usize;
|
||||
let mut columns = Vec::with_capacity(col_count);
|
||||
for _ in 0..col_count {
|
||||
columns.push(read_string(&payload, &mut pos));
|
||||
}
|
||||
let row_count = read_u32(&payload, &mut pos) as usize;
|
||||
let mut rows = Vec::with_capacity(row_count);
|
||||
for _ in 0..row_count {
|
||||
let mut row = HashMap::new();
|
||||
for c in 0..col_count {
|
||||
let val = read_value(&payload, &mut pos);
|
||||
row.insert(columns[c].clone(), val);
|
||||
}
|
||||
rows.push(row);
|
||||
}
|
||||
// Read following COMPLETE message
|
||||
let mut comp_header = [0u8; 12];
|
||||
self.stream.read_exact(&mut comp_header)?;
|
||||
let comp_kind = u32::from_be_bytes([comp_header[0], comp_header[1], comp_header[2], comp_header[3]]);
|
||||
let comp_len = u32::from_be_bytes([comp_header[4], comp_header[5], comp_header[6], comp_header[7]]) as usize;
|
||||
let mut comp_payload = vec![0u8; comp_len];
|
||||
if comp_len > 0 {
|
||||
self.stream.read_exact(&mut comp_payload)?;
|
||||
}
|
||||
let affected = if comp_kind == MK_COMPLETE && comp_payload.len() >= 4 {
|
||||
u32::from_be_bytes([comp_payload[0], comp_payload[1], comp_payload[2], comp_payload[3]]) as usize
|
||||
} else { 0 };
|
||||
Ok(QueryResult { columns, rows, affected_rows: affected })
|
||||
}
|
||||
MK_COMPLETE => {
|
||||
let affected = if payload.len() >= 4 {
|
||||
u32::from_be_bytes([payload[0], payload[1], payload[2], payload[3]]) as usize
|
||||
@@ -311,3 +350,89 @@ fn encode_string(s: &str) -> Vec<u8> {
|
||||
result.extend_from_slice(bytes);
|
||||
result
|
||||
}
|
||||
|
||||
|
||||
fn read_u32(data: &[u8], pos: &mut usize) -> u32 {
|
||||
let val = u32::from_be_bytes([data[*pos], data[*pos + 1], data[*pos + 2], data[*pos + 3]]);
|
||||
*pos += 4;
|
||||
val
|
||||
}
|
||||
|
||||
fn read_string(data: &[u8], pos: &mut usize) -> String {
|
||||
let len = read_u32(data, pos) as usize;
|
||||
let s = String::from_utf8_lossy(&data[*pos..*pos + len]).to_string();
|
||||
*pos += len;
|
||||
s
|
||||
}
|
||||
|
||||
fn read_value(data: &[u8], pos: &mut usize) -> String {
|
||||
let kind = data[*pos];
|
||||
*pos += 1;
|
||||
match kind {
|
||||
FK_NULL => String::new(),
|
||||
FK_BOOL => {
|
||||
let val = data[*pos] != 0;
|
||||
*pos += 1;
|
||||
val.to_string()
|
||||
}
|
||||
FK_INT8 => {
|
||||
let val = data[*pos] as i8;
|
||||
*pos += 1;
|
||||
val.to_string()
|
||||
}
|
||||
FK_INT16 => {
|
||||
let val = i16::from_be_bytes([data[*pos], data[*pos + 1]]);
|
||||
*pos += 2;
|
||||
val.to_string()
|
||||
}
|
||||
FK_INT32 => {
|
||||
let val = i32::from_be_bytes([data[*pos], data[*pos + 1], data[*pos + 2], data[*pos + 3]]);
|
||||
*pos += 4;
|
||||
val.to_string()
|
||||
}
|
||||
FK_INT64 => {
|
||||
let val = i64::from_be_bytes([data[*pos], data[*pos + 1], data[*pos + 2], data[*pos + 3],
|
||||
data[*pos + 4], data[*pos + 5], data[*pos + 6], data[*pos + 7]]);
|
||||
*pos += 8;
|
||||
val.to_string()
|
||||
}
|
||||
FK_FLOAT32 => {
|
||||
let val = f32::from_be_bytes([data[*pos], data[*pos + 1], data[*pos + 2], data[*pos + 3]]);
|
||||
*pos += 4;
|
||||
val.to_string()
|
||||
}
|
||||
FK_FLOAT64 => {
|
||||
let val = f64::from_be_bytes([data[*pos], data[*pos + 1], data[*pos + 2], data[*pos + 3],
|
||||
data[*pos + 4], data[*pos + 5], data[*pos + 6], data[*pos + 7]]);
|
||||
*pos += 8;
|
||||
val.to_string()
|
||||
}
|
||||
FK_STRING => read_string(data, pos),
|
||||
FK_BYTES => {
|
||||
let len = read_u32(data, pos) as usize;
|
||||
*pos += len;
|
||||
format!("<bytes:{}>", len)
|
||||
}
|
||||
FK_ARRAY => {
|
||||
let count = read_u32(data, pos);
|
||||
for _ in 0..count {
|
||||
let _ = read_value(data, pos);
|
||||
}
|
||||
format!("<array:{}>", count)
|
||||
}
|
||||
FK_OBJECT => {
|
||||
let count = read_u32(data, pos);
|
||||
for _ in 0..count {
|
||||
let _ = read_string(data, pos);
|
||||
let _ = read_value(data, pos);
|
||||
}
|
||||
format!("<object:{}>", count)
|
||||
}
|
||||
FK_VECTOR => {
|
||||
let dim = read_u32(data, pos);
|
||||
*pos += (dim * 4) as usize;
|
||||
format!("<vector:{}>", dim)
|
||||
}
|
||||
_ => String::new(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,9 @@ type
|
||||
maxConnections*: int
|
||||
walEnabled*: bool
|
||||
compactionStrategy*: CompactionStrategy
|
||||
tlsEnabled*: bool
|
||||
certFile*: string
|
||||
keyFile*: string
|
||||
|
||||
CompactionStrategy* = enum
|
||||
csSizeTiered = "size_tiered"
|
||||
@@ -19,6 +22,9 @@ proc defaultConfig*(): BaraConfig =
|
||||
maxConnections: 1000,
|
||||
walEnabled: true,
|
||||
compactionStrategy: csLeveled,
|
||||
tlsEnabled: false,
|
||||
certFile: "",
|
||||
keyFile: "",
|
||||
)
|
||||
|
||||
proc loadConfig*(): BaraConfig =
|
||||
|
||||
@@ -112,7 +112,19 @@ proc queryHandler(server: HttpServer): RequestHandler =
|
||||
ctx.json(%*{"rows": [], "affectedRows": 0, "columns": []})
|
||||
return
|
||||
|
||||
let result = executor.executeQuery(reqCtx, astNode)
|
||||
# Extract optional params from JSON body
|
||||
var params: seq[WireValue] = @[]
|
||||
if "params" in body and body["params"].kind == JArray:
|
||||
for p in body["params"]:
|
||||
case p.kind
|
||||
of JNull: params.add(WireValue(kind: fkNull))
|
||||
of JBool: params.add(WireValue(kind: fkBool, boolVal: p.getBool()))
|
||||
of JInt: params.add(WireValue(kind: fkInt64, int64Val: p.getInt()))
|
||||
of JFloat: params.add(WireValue(kind: fkFloat64, float64Val: p.getFloat()))
|
||||
of JString: params.add(WireValue(kind: fkString, strVal: p.getStr()))
|
||||
else: params.add(WireValue(kind: fkString, strVal: $p))
|
||||
|
||||
let result = executor.executeQuery(reqCtx, astNode, params)
|
||||
|
||||
if result.success:
|
||||
var jsonRows = newJArray()
|
||||
@@ -149,7 +161,7 @@ proc metricsHandler(server: HttpServer): RequestHandler =
|
||||
"baradb_inserts_total " & $server.metrics.insertCount & "\n" &
|
||||
"baradb_selects_total " & $server.metrics.selectCount & "\n" &
|
||||
"baradb_connections_active " & $server.metrics.activeConnections & "\n"
|
||||
request.respond(200, @[("Content-Type", "text/plain")], prometheus)
|
||||
request.respond(200, @[("Content-Type", "text/plain; charset=utf-8")], prometheus)
|
||||
|
||||
proc authHandler(server: HttpServer): RequestHandler =
|
||||
return proc(request: Request) {.gcsafe.} =
|
||||
@@ -197,6 +209,20 @@ proc openApiHandler(): RequestHandler =
|
||||
}
|
||||
})
|
||||
|
||||
proc tablesHandler(server: HttpServer): RequestHandler =
|
||||
return proc(request: Request) {.gcsafe.} =
|
||||
{.cast(gcsafe).}:
|
||||
let ctx = newContext(request)
|
||||
var tables = newJArray()
|
||||
for name, tbl in server.ctx.tables:
|
||||
var cols = newJArray()
|
||||
for col in tbl.columns:
|
||||
cols.add(%*{"name": col.name, "type": col.colType,
|
||||
"pk": col.isPk, "notNull": col.isNotNull, "unique": col.isUnique})
|
||||
tables.add(%*{"name": name, "columns": cols,
|
||||
"pkColumns": tbl.pkColumns, "fkCount": tbl.foreignKeys.len})
|
||||
ctx.json(%*{"tables": tables})
|
||||
|
||||
proc adminHandler(server: HttpServer): RequestHandler =
|
||||
return proc(request: Request) {.gcsafe.} =
|
||||
let html = """
|
||||
@@ -204,85 +230,216 @@ proc adminHandler(server: HttpServer): RequestHandler =
|
||||
<meta charset='utf-8'><meta name='viewport' content='width=device-width,initial-scale=1'>
|
||||
<title>BaraDB Admin</title>
|
||||
<style>
|
||||
body{font:14px monospace;background:#1a1a2e;color:#e0e0e0;margin:0;padding:20px}
|
||||
h1{color:#e94560;margin:0 0 10px}
|
||||
.card{background:#16213e;border-radius:8px;padding:16px;margin:10px 0}
|
||||
input,textarea,select{background:#0f3460;color:#e0e0e0;border:1px solid #533483;padding:8px;border-radius:4px;font:14px monospace;width:100%;box-sizing:border-box}
|
||||
textarea{height:100px;resize:vertical}
|
||||
button{background:#e94560;color:white;border:none;padding:8px 20px;border-radius:4px;cursor:pointer;font:14px monospace;margin:4px}
|
||||
body{font:14px system-ui,-apple-system,sans-serif;background:#0f0f23;color:#e0e0e0;margin:0;padding:0}
|
||||
header{background:#1a1a3e;padding:12px 20px;display:flex;align-items:center;justify-content:space-between;border-bottom:1px solid #533483}
|
||||
header h1{color:#e94560;margin:0;font-size:20px}
|
||||
#tabs{display:flex;background:#16213e;border-bottom:1px solid #333}
|
||||
.tab{flex:1;text-align:center;padding:12px;cursor:pointer;border-bottom:3px solid transparent;font-size:13px;transition:.2s}
|
||||
.tab.active{border-color:#e94560;color:#e94560;background:#1a2744}
|
||||
.tab:hover{background:#1a2744}
|
||||
.panel{display:none;padding:20px;max-width:1400px;margin:0 auto}
|
||||
.panel.active{display:block}
|
||||
.card{background:#16213e;border-radius:8px;padding:16px;margin:12px 0;border:1px solid #2a2a4a}
|
||||
input,textarea,select{background:#0f3460;color:#e0e0e0;border:1px solid #533483;padding:8px;border-radius:4px;font:13px monospace;width:100%;box-sizing:border-box}
|
||||
textarea{height:120px;resize:vertical}
|
||||
button{background:#e94560;color:white;border:none;padding:8px 16px;border-radius:4px;cursor:pointer;font:13px monospace;margin:4px 2px}
|
||||
button:hover{background:#c23152}
|
||||
table{width:100%;border-collapse:collapse;margin:10px 0}
|
||||
th{background:#533483;padding:8px;text-align:left}
|
||||
td{padding:6px 8px;border-bottom:1px solid #333}
|
||||
button.secondary{background:#533483}
|
||||
button.secondary:hover{background:#3f2770}
|
||||
table{width:100%;border-collapse:collapse;margin:10px 0;font-size:13px}
|
||||
th{background:#533483;padding:8px;text-align:left;font-weight:600}
|
||||
td{padding:6px 8px;border-bottom:1px solid #2a2a4a}
|
||||
tr:hover td{background:#1a2744}
|
||||
#result{max-height:400px;overflow:auto;font:12px monospace;background:#0a0a1a;padding:8px;border-radius:4px;white-space:pre-wrap}
|
||||
.tab{display:inline-block;padding:8px 16px;cursor:pointer;border-bottom:2px solid transparent}
|
||||
.tab.active{border-color:#e94560;color:#e94560}
|
||||
#tabs{margin-bottom:16px}
|
||||
#result{max-height:400px;overflow:auto;font:12px monospace;background:#0a0a1a;padding:10px;border-radius:4px;white-space:pre-wrap;border:1px solid #2a2a4a}
|
||||
.status{color:#4ecca3;font-size:12px}
|
||||
.error{color:#e94560}
|
||||
a{color:#4ecca3}
|
||||
.warn{color:#f4d03f}
|
||||
.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(280px,1fr));gap:16px}
|
||||
.stat-box{background:#16213e;border-radius:8px;padding:16px;text-align:center;border:1px solid #2a2a4a}
|
||||
.stat-box h3{margin:0 0 8px;color:#e94560;font-size:24px}
|
||||
.stat-box p{margin:0;color:#aaa;font-size:12px}
|
||||
#login-overlay{position:fixed;inset:0;background:rgba(0,0,0,.85);display:flex;align-items:center;justify-content:center;z-index:100}
|
||||
#login-box{background:#16213e;padding:32px;border-radius:12px;width:320px;border:1px solid #533483}
|
||||
#login-box h2{margin-top:0;color:#e94560}
|
||||
#ws-log{height:300px;overflow:auto;background:#0a0a1a;padding:10px;border-radius:4px;font:12px monospace;border:1px solid #2a2a4a}
|
||||
.log-line{padding:2px 0;border-bottom:1px solid #1a1a3e}
|
||||
.table-list{display:flex;flex-wrap:wrap;gap:8px}
|
||||
.table-tag{background:#0f3460;padding:6px 12px;border-radius:4px;cursor:pointer;font-size:13px;border:1px solid #533483}
|
||||
.table-tag:hover{background:#533483}
|
||||
.table-tag.active{background:#e94560;border-color:#e94560}
|
||||
</style></head><body>
|
||||
<h1>BaraDB Admin</h1>
|
||||
<div id='login-overlay'><div id='login-box'>
|
||||
<h2>BaraDB Login</h2>
|
||||
<input id='username' placeholder='Username' style='margin-bottom:8px'><br>
|
||||
<input id='password' type='password' placeholder='Password' style='margin-bottom:12px'><br>
|
||||
<button onclick='doLogin()' style='width:100%'>Login</button>
|
||||
<p class='status' id='login-status'></p>
|
||||
</div></div>
|
||||
<header><h1>BaraDB Admin</h1><span id='user-info'></span></header>
|
||||
<div id='tabs'>
|
||||
<span class='tab active' onclick='showTab("query")'>SQL Playground</span>
|
||||
<span class='tab' onclick='showTab("schema")'>Schema</span>
|
||||
<span class='tab' onclick='showTab("metrics")'>Metrics</span>
|
||||
<span class='tab active' onclick='showTab(0)'>SQL Playground</span>
|
||||
<span class='tab' onclick='showTab(1)'>Tables</span>
|
||||
<span class='tab' onclick='showTab(2)'>Schema</span>
|
||||
<span class='tab' onclick='showTab(3)'>Live</span>
|
||||
<span class='tab' onclick='showTab(4)'>Metrics</span>
|
||||
<span class='tab' onclick='showTab(5)'>Cluster</span>
|
||||
</div>
|
||||
<div id='tab-query'>
|
||||
<div class='card'>
|
||||
<textarea id='sql' placeholder='SELECT version() CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT) INSERT INTO users VALUES (1, "test") SELECT * FROM users'></textarea>
|
||||
<div class='panel active'>
|
||||
<div class='card'><textarea id='sql' placeholder='SELECT * FROM users'></textarea>
|
||||
<button onclick='runQuery()'>Run</button>
|
||||
<button onclick='explainQuery()'>EXPLAIN</button>
|
||||
<button class='secondary' onclick='explainQuery()'>EXPLAIN</button>
|
||||
<button class='secondary' onclick='clearResult()'>Clear</button>
|
||||
</div>
|
||||
<div id='result'></div>
|
||||
</div>
|
||||
<div id='tab-schema' style='display:none'>
|
||||
<div class='panel'>
|
||||
<div class='card'><h3>Tables</h3><div class='table-list' id='table-list'>Loading...</div></div>
|
||||
<div class='card'><h3>Browse <span id='browse-table'></span></h3>
|
||||
<div id='browse-data'>Select a table to browse</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class='panel'>
|
||||
<div class='card' id='schema-data'>Loading schema...</div>
|
||||
</div>
|
||||
<div id='tab-metrics' style='display:none'>
|
||||
<div class='card' id='metrics-data'>Loading metrics...</div>
|
||||
<div class='panel'>
|
||||
<div class='card'><h3>Real-time Events</h3><div id='ws-log'></div>
|
||||
<button onclick='clearWsLog()'>Clear</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class='panel'>
|
||||
<div class='grid' id='metrics-grid'>
|
||||
<div class='stat-box'><h3 id='m-queries'>0</h3><p>Total Queries</p></div>
|
||||
<div class='stat-box'><h3 id='m-errors'>0</h3><p>Query Errors</p></div>
|
||||
<div class='stat-box'><h3 id='m-inserts'>0</h3><p>Inserts</p></div>
|
||||
<div class='stat-box'><h3 id='m-selects'>0</h3><p>Selects</p></div>
|
||||
<div class='stat-box'><h3 id='m-connections'>0</h3><p>Active Connections</p></div>
|
||||
<div class='stat-box'><h3 id='m-tables'>0</h3><p>Tables</p></div>
|
||||
</div>
|
||||
<div class='card'><h3>Raw Metrics</h3><pre id='metrics-raw'></pre></div>
|
||||
</div>
|
||||
<div class='panel'>
|
||||
<div class='card'><h3>Cluster Status</h3><div id='cluster-data'>Cluster status not available</div></div>
|
||||
</div>
|
||||
<script>
|
||||
async function api(path, body) {
|
||||
const r = await fetch(path, {method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(body||{})})
|
||||
return r.json()
|
||||
let token = localStorage.getItem('baradb_token') || ''
|
||||
let ws = null
|
||||
function authHeader(){ return token ? {'Authorization':'Bearer '+token,'Content-Type':'application/json'} : {'Content-Type':'application/json'} }
|
||||
async function api(path, body, method='POST') {
|
||||
const r = await fetch(path, {method, headers:authHeader(), body: body?JSON.stringify(body):undefined})
|
||||
return r.json().catch(() => r.text())
|
||||
}
|
||||
async function doLogin(){
|
||||
const u = document.getElementById('username').value
|
||||
const p = document.getElementById('password').value
|
||||
const r = await api('/auth', {username:u, password:p})
|
||||
if(r.token){ token = r.token; localStorage.setItem('baradb_token', token); hideLogin(); showUser(r.user); connectWS() }
|
||||
else { document.getElementById('login-status').textContent = r.error || 'Login failed' }
|
||||
}
|
||||
function hideLogin(){ document.getElementById('login-overlay').style.display='none' }
|
||||
function showUser(u){ document.getElementById('user-info').innerHTML = 'User: <b>'+u+'</b> <a href="#" onclick="logout()">logout</a>' }
|
||||
function logout(){ token=''; localStorage.removeItem('baradb_token'); location.reload() }
|
||||
if(token){ hideLogin(); showUser('...'); api('/health',null,'GET').then(()=>showUser('authed')).catch(()=>logout()); connectWS() }
|
||||
async function runQuery(){
|
||||
const sql = document.getElementById('sql').value
|
||||
const res = await api('/query', {query: sql})
|
||||
document.getElementById('result').innerHTML = JSON.stringify(res, null, 2)
|
||||
renderResult(res)
|
||||
}
|
||||
async function explainQuery(){
|
||||
const sql = 'EXPLAIN ' + document.getElementById('sql').value
|
||||
const res = await api('/query', {query: sql})
|
||||
document.getElementById('result').innerHTML = JSON.stringify(res, null, 2)
|
||||
const res = await api('/query', {query: 'EXPLAIN ' + document.getElementById('sql').value})
|
||||
renderResult(res)
|
||||
}
|
||||
function renderResult(res){
|
||||
const el = document.getElementById('result')
|
||||
if(res.error){ el.innerHTML = '<span class="error">Error: '+res.error+'</span>'; return }
|
||||
let html = ''
|
||||
if(res.columns && res.columns.length){
|
||||
html += '<table><tr>'+res.columns.map(c=>'<th>'+c+'</th>').join('')+'</tr>'
|
||||
html += (res.rows||[]).map(row => '<tr>'+res.columns.map(c=>'<td>'+(row[c]??'')+'</td>').join('')+'</tr>').join('')
|
||||
html += '</table>'
|
||||
}
|
||||
if(res.affectedRows != null) html += '<p class="status">Affected rows: '+res.affectedRows+'</p>'
|
||||
if(res.message) html += '<p class="status">'+res.message+'</p>'
|
||||
el.innerHTML = html || JSON.stringify(res, null, 2)
|
||||
}
|
||||
function clearResult(){ document.getElementById('result').innerHTML = '' }
|
||||
let currentTable = ''
|
||||
async function loadTables(){
|
||||
const r = await api('/tables', null, 'GET')
|
||||
const list = document.getElementById('table-list')
|
||||
if(!r.tables){ list.innerHTML = 'No tables'; return }
|
||||
list.innerHTML = r.tables.map(t => '<span class="table-tag'+(t.name===currentTable?' active':'')+'" onclick="browseTable(\''+t.name+'\')">'+t.name+' ('+t.columns.length+' cols)</span>').join('')
|
||||
document.getElementById('m-tables').textContent = r.tables.length
|
||||
}
|
||||
async function browseTable(name){
|
||||
currentTable = name
|
||||
document.getElementById('browse-table').textContent = name
|
||||
const res = await api('/query', {query: 'SELECT * FROM '+name+' LIMIT 100'})
|
||||
const el = document.getElementById('browse-data')
|
||||
if(res.error){ el.innerHTML = '<span class="error">'+res.error+'</span>'; return }
|
||||
if(!res.columns || !res.columns.length){ el.innerHTML = 'Empty table'; return }
|
||||
let html = '<table><tr>'+res.columns.map(c=>'<th>'+c+'</th>').join('')+'</tr>'
|
||||
html += (res.rows||[]).map(row => '<tr>'+res.columns.map(c=>'<td>'+(row[c]??'')+'</td>').join('')+'</tr>').join('')
|
||||
html += '</table>'
|
||||
el.innerHTML = html
|
||||
loadTables()
|
||||
}
|
||||
async function loadSchema(){
|
||||
try{const r = await fetch('/query',{method:'POST',headers:{'Content-Type':'application/json'},
|
||||
body:JSON.stringify({query:'SELECT name, type FROM __tables'})})
|
||||
const d = await r.json()
|
||||
document.getElementById('schema-data').innerHTML = '<table><tr><th>Table</th><th>Type</th></tr>' +
|
||||
(d.rows||[]).map(r=>'<tr><td>'+r.name+'</td><td>'+r.type+'</td></tr>').join('') + '</table>'
|
||||
const r = await api('/tables', null, 'GET')
|
||||
const el = document.getElementById('schema-data')
|
||||
if(!r.tables){ el.innerHTML = 'No tables'; return }
|
||||
let html = ''
|
||||
r.tables.forEach(t => {
|
||||
html += '<h4>'+t.name+'</h4><table><tr><th>Column</th><th>Type</th><th>PK</th><th>Not Null</th><th>Unique</th></tr>'
|
||||
t.columns.forEach(c => html += '<tr><td>'+c.name+'</td><td>'+c.type+'</td><td>'+(c.pk?'✓':'')+'</td><td>'+(c.notNull?'✓':'')+'</td><td>'+(c.unique?'✓':'')+'</td></tr>')
|
||||
html += '</table>'
|
||||
})
|
||||
el.innerHTML = html
|
||||
}
|
||||
function connectWS(){
|
||||
const wsProto = location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
const wsUrl = wsProto+'//'+location.host.split(':')[0]+':'+((parseInt(location.port)||80)+1)+'/live'
|
||||
try{
|
||||
ws = new WebSocket(wsUrl)
|
||||
ws.onmessage = ev => {
|
||||
const log = document.getElementById('ws-log')
|
||||
const line = document.createElement('div')
|
||||
line.className = 'log-line'
|
||||
line.textContent = new Date().toLocaleTimeString() + ' ' + ev.data
|
||||
log.appendChild(line)
|
||||
log.scrollTop = log.scrollHeight
|
||||
}
|
||||
ws.onopen = () => { const log = document.getElementById('ws-log'); log.innerHTML += '<div class="log-line status">Connected</div>' }
|
||||
ws.onclose = () => { setTimeout(connectWS, 3000) }
|
||||
}catch(e){}
|
||||
}
|
||||
function clearWsLog(){ document.getElementById('ws-log').innerHTML = '' }
|
||||
async function loadMetrics(){
|
||||
try{const r = await fetch('/metrics')
|
||||
document.getElementById('metrics-data').innerHTML = '<pre>'+ (await r.text()) +'</pre>'
|
||||
try{
|
||||
const r = await fetch('/metrics', {headers: token?{'Authorization':'Bearer '+token}:{}})
|
||||
const text = await r.text()
|
||||
document.getElementById('metrics-raw').textContent = text
|
||||
const lines = text.split('\n')
|
||||
lines.forEach(l => {
|
||||
if(l.startsWith('baradb_queries_total ')) document.getElementById('m-queries').textContent = l.split(' ')[1]
|
||||
if(l.startsWith('baradb_query_errors ')) document.getElementById('m-errors').textContent = l.split(' ')[1]
|
||||
if(l.startsWith('baradb_insert_count ')) document.getElementById('m-inserts').textContent = l.split(' ')[1]
|
||||
if(l.startsWith('baradb_select_count ')) document.getElementById('m-selects').textContent = l.split(' ')[1]
|
||||
if(l.startsWith('baradb_active_connections ')) document.getElementById('m-connections').textContent = l.split(' ')[1]
|
||||
})
|
||||
}catch(e){}
|
||||
}
|
||||
function showTab(name){
|
||||
document.querySelectorAll('#tabs .tab').forEach(t=>t.classList.remove('active'))
|
||||
document.querySelectorAll('[id^=tab-]').forEach(t=>t.style.display='none')
|
||||
document.querySelector('.tab:nth-child('+({'query':1,'schema':2,'metrics':3}[name])+')').classList.add('active')
|
||||
document.getElementById('tab-'+name).style.display=''
|
||||
if(name==='schema') loadSchema()
|
||||
if(name==='metrics') loadMetrics()
|
||||
function showTab(idx){
|
||||
document.querySelectorAll('.tab').forEach((t,i)=>t.classList.toggle('active', i===idx))
|
||||
document.querySelectorAll('.panel').forEach((p,i)=>p.classList.toggle('active', i===idx))
|
||||
if(idx===1) loadTables()
|
||||
if(idx===2) loadSchema()
|
||||
if(idx===4) loadMetrics()
|
||||
}
|
||||
setInterval(() => { if(document.querySelectorAll('.panel')[4].classList.contains('active')) loadMetrics() }, 5000)
|
||||
</script>
|
||||
<div class='status'>BaraDB v0.1.0 — Production-ready multimodal database</div>
|
||||
<div class='status' style='text-align:center;padding:10px'>BaraDB v0.1.0 — Multimodal Database Engine</div>
|
||||
</body></html>"""
|
||||
request.respond(200, @[("Content-Type", "text/html")], html)
|
||||
request.respond(200, @[("Content-Type", "text/html; charset=utf-8")], html)
|
||||
|
||||
proc run*(server: HttpServer, port: int = 8080) =
|
||||
var router = newRouter()
|
||||
@@ -292,6 +449,7 @@ proc run*(server: HttpServer, port: int = 8080) =
|
||||
router.get("/health", healthHandler())
|
||||
router.get("/metrics", server.metricsHandler())
|
||||
router.post("/auth", server.authHandler())
|
||||
router.get("/tables", server.tablesHandler())
|
||||
router.get("/api", openApiHandler())
|
||||
|
||||
var stack = newMiddlewareStack(router)
|
||||
|
||||
@@ -3,6 +3,7 @@ import std/tables
|
||||
import std/locks
|
||||
import std/monotimes
|
||||
import std/sets
|
||||
import deadlock
|
||||
|
||||
type
|
||||
TxnId* = distinct uint64
|
||||
@@ -44,6 +45,7 @@ type
|
||||
committedTxns: seq[TxnId]
|
||||
globalVersions*: Table[string, seq[VersionedRecord]]
|
||||
txnTimeoutMs: int64
|
||||
deadlockDetector*: DeadlockDetector
|
||||
|
||||
proc `==`*(a, b: TxnId): bool {.borrow.}
|
||||
proc `==`*(a, b: Lsn): bool {.borrow.}
|
||||
@@ -59,6 +61,7 @@ proc newTxnManager*(txnTimeoutMs: int64 = 30000): TxnManager =
|
||||
result.committedTxns = @[]
|
||||
result.globalVersions = initTable[string, seq[VersionedRecord]]()
|
||||
result.txnTimeoutMs = txnTimeoutMs
|
||||
result.deadlockDetector = newDeadlockDetector()
|
||||
|
||||
proc allocTxnId(tm: TxnManager): TxnId =
|
||||
result = TxnId(tm.nextTxnId)
|
||||
@@ -167,7 +170,22 @@ proc write*(tm: TxnManager, txn: Transaction, key: string, value: seq[byte]): bo
|
||||
otherTxn.state = tsAborted
|
||||
tm.activeTxns.del(otherId)
|
||||
|
||||
# Check for write-write conflict
|
||||
# Check for write-write conflict against other active transactions' write sets
|
||||
for otherId, otherTxn in tm.activeTxns:
|
||||
if otherId != txn.id and otherTxn.state == tsActive:
|
||||
if key in otherTxn.writeSet:
|
||||
# Wait-for graph: txn is waiting for otherId to finish
|
||||
tm.deadlockDetector.addWait(uint64(txn.id), uint64(otherId))
|
||||
if tm.deadlockDetector.hasDeadlock():
|
||||
let victimId = TxnId(tm.deadlockDetector.findDeadlockVictim())
|
||||
tm.deadlockDetector.removeTxn(uint64(victimId))
|
||||
if victimId in tm.activeTxns:
|
||||
tm.activeTxns[victimId].state = tsAborted
|
||||
tm.activeTxns.del(victimId)
|
||||
release(tm.lock)
|
||||
return false # write-write conflict with uncommitted txn
|
||||
|
||||
# Check for write-write conflict with committed versions
|
||||
if key notin txn.writeSet:
|
||||
if key in tm.globalVersions:
|
||||
for version in tm.globalVersions[key]:
|
||||
@@ -229,6 +247,7 @@ proc commit*(tm: TxnManager, txn: Transaction): bool =
|
||||
txn.state = tsCommitted
|
||||
tm.committedTxns.add(txn.id)
|
||||
tm.activeTxns.del(txn.id)
|
||||
tm.deadlockDetector.removeTxn(uint64(txn.id))
|
||||
release(tm.lock)
|
||||
return true
|
||||
|
||||
@@ -239,6 +258,7 @@ proc abortTxn*(tm: TxnManager, txn: Transaction): bool =
|
||||
return false
|
||||
txn.state = tsAborted
|
||||
tm.activeTxns.del(txn.id)
|
||||
tm.deadlockDetector.removeTxn(uint64(txn.id))
|
||||
release(tm.lock)
|
||||
return true
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ import std/os
|
||||
import std/endians
|
||||
import config
|
||||
import ../protocol/wire
|
||||
import ../protocol/ssl
|
||||
import ../query/lexer
|
||||
import ../query/parser
|
||||
import ../query/ast
|
||||
@@ -16,11 +17,12 @@ import ../core/mvcc
|
||||
|
||||
type
|
||||
Server* = ref object
|
||||
config: BaraConfig
|
||||
running: bool
|
||||
db: LSMTree
|
||||
ctx: ExecutionContext
|
||||
txnManager: TxnManager
|
||||
config*: BaraConfig
|
||||
running*: bool
|
||||
db*: LSMTree
|
||||
ctx*: ExecutionContext
|
||||
txnManager*: TxnManager
|
||||
tls*: TLSContext
|
||||
|
||||
ClientConnection = ref object
|
||||
socket: AsyncSocket
|
||||
@@ -32,8 +34,12 @@ proc newServer*(config: BaraConfig): Server =
|
||||
let db = newLSMTree(dataDir)
|
||||
let ctx = newExecutionContext(db)
|
||||
ctx.txnManager = newTxnManager()
|
||||
var tls: TLSContext = nil
|
||||
if config.tlsEnabled and config.certFile.len > 0 and config.keyFile.len > 0:
|
||||
let tlsConfig = newTLSConfig(config.certFile, config.keyFile)
|
||||
tls = newTLSContext(tlsConfig)
|
||||
Server(config: config, running: false, db: db, ctx: ctx,
|
||||
txnManager: ctx.txnManager)
|
||||
txnManager: ctx.txnManager, tls: tls)
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Wire Protocol Helpers
|
||||
@@ -63,7 +69,7 @@ proc parseHeader(data: string): (bool, MessageHeader) =
|
||||
# Query Execution (pipeline-based)
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
proc executeQuery(db: LSMTree, ctx: ExecutionContext, query: string): (bool, QueryResult, string) =
|
||||
proc executeQuery(db: LSMTree, ctx: ExecutionContext, query: string, params: seq[WireValue] = @[]): (bool, QueryResult, string) =
|
||||
try:
|
||||
let tokens = tokenize(query)
|
||||
let astNode = parse(tokens)
|
||||
@@ -71,7 +77,7 @@ proc executeQuery(db: LSMTree, ctx: ExecutionContext, query: string): (bool, Que
|
||||
if astNode.stmts.len == 0:
|
||||
return (true, QueryResult(), "")
|
||||
|
||||
let result = executor.executeQuery(ctx, astNode)
|
||||
let result = executor.executeQuery(ctx, astNode, params)
|
||||
if result.success:
|
||||
var qr = QueryResult(affectedRows: result.affectedRows, rowCount: result.rows.len)
|
||||
qr.columns = result.columns
|
||||
@@ -186,6 +192,21 @@ proc handleClient(server: Server, client: AsyncSocket, clientId: int) {.async.}
|
||||
let errorMsg = serializeError(1, errorMsg, header.requestId)
|
||||
await client.send(cast[string](errorMsg))
|
||||
|
||||
of mkQueryParams:
|
||||
let (queryStr, params) = readQueryParamsMessage(cast[seq[byte]](payload))
|
||||
echo "[", clientId, "] QueryParams: ", queryStr, " (", params.len, " params)"
|
||||
|
||||
let (success, result, errorMsg) = executeQuery(server.db, connCtx, queryStr, params)
|
||||
if success:
|
||||
if result.rows.len > 0:
|
||||
let dataMsg = serializeResult(result, header.requestId)
|
||||
await client.send(cast[string](dataMsg))
|
||||
let completeMsg = serializeComplete(result.affectedRows, header.requestId)
|
||||
await client.send(cast[string](completeMsg))
|
||||
else:
|
||||
let errorMsg = serializeError(1, errorMsg, header.requestId)
|
||||
await client.send(cast[string](errorMsg))
|
||||
|
||||
of mkPing:
|
||||
var pongPayload: seq[byte] = @[]
|
||||
var pongMsg = WireMessage(
|
||||
@@ -214,9 +235,19 @@ proc run*(server: Server) {.async.} =
|
||||
sock.setSockOpt(OptReuseAddr, true)
|
||||
sock.bindAddr(Port(server.config.port), server.config.address)
|
||||
sock.listen()
|
||||
if server.config.tlsEnabled:
|
||||
echo "BaraDB TLS listening on ", server.config.address, ":", server.config.port
|
||||
else:
|
||||
echo "BaraDB listening on ", server.config.address, ":", server.config.port
|
||||
while server.running:
|
||||
let client = await sock.accept()
|
||||
if server.tls != nil:
|
||||
try:
|
||||
server.tls.wrapServer(client)
|
||||
except Exception as e:
|
||||
echo "TLS handshake failed: ", e.msg
|
||||
client.close()
|
||||
continue
|
||||
inc clientId
|
||||
asyncCheck server.handleClient(client, clientId)
|
||||
|
||||
|
||||
@@ -1,21 +1,24 @@
|
||||
## TLS/SSL Wrapper — encrypted socket using OpenSSL
|
||||
## TLS/SSL Wrapper — encrypted sockets using OpenSSL (Nim stdlib)
|
||||
when not defined(ssl):
|
||||
{.error: "BaraDB requires SSL support. Compile with -d:ssl".}
|
||||
|
||||
import std/os
|
||||
import std/osproc
|
||||
import std/strutils
|
||||
import std/net
|
||||
import std/asyncnet
|
||||
|
||||
type
|
||||
TLSSocket* = ref object
|
||||
ctx: pointer # SSL_CTX*
|
||||
ssl: pointer # SSL*
|
||||
fd: int
|
||||
connected: bool
|
||||
config: TLSConfig
|
||||
|
||||
TLSConfig* = object
|
||||
certFile*: string
|
||||
keyFile*: string
|
||||
caFile*: string
|
||||
verifyPeer*: bool
|
||||
|
||||
TLSContext* = ref object
|
||||
sslCtx*: SslContext
|
||||
config*: TLSConfig
|
||||
|
||||
proc newTLSConfig*(certFile: string, keyFile: string, caFile: string = "",
|
||||
verifyPeer: bool = false): TLSConfig =
|
||||
TLSConfig(
|
||||
@@ -23,44 +26,28 @@ proc newTLSConfig*(certFile: string, keyFile: string, caFile: string = "",
|
||||
caFile: caFile, verifyPeer: verifyPeer,
|
||||
)
|
||||
|
||||
proc newTLSSocket*(config: TLSConfig): TLSSocket =
|
||||
TLSSocket(config: config, connected: false)
|
||||
proc newTLSContext*(config: TLSConfig): TLSContext =
|
||||
result = TLSContext(config: config)
|
||||
if fileExists(config.certFile) and fileExists(config.keyFile):
|
||||
result.sslCtx = newContext(
|
||||
certFile = config.certFile,
|
||||
keyFile = config.keyFile,
|
||||
)
|
||||
else:
|
||||
raise newException(IOError, "TLS certificate or key file not found: " &
|
||||
config.certFile & ", " & config.keyFile)
|
||||
|
||||
proc connect*(sock: TLSSocket, host: string, port: int): bool =
|
||||
# In production, would use OpenSSL SSL_connect() via FFI
|
||||
# For now, validate config and return mock connection
|
||||
if not fileExists(sock.config.certFile):
|
||||
return false
|
||||
sock.connected = true
|
||||
return true
|
||||
proc wrapClient*(tls: TLSContext, socket: AsyncSocket) {.inline.} =
|
||||
if tls.sslCtx != nil:
|
||||
tls.sslCtx.wrapSocket(socket)
|
||||
|
||||
proc accept*(sock: TLSSocket, clientFd: int): bool =
|
||||
if not fileExists(sock.config.certFile):
|
||||
return false
|
||||
if not fileExists(sock.config.keyFile):
|
||||
return false
|
||||
sock.fd = clientFd
|
||||
sock.connected = true
|
||||
return true
|
||||
proc wrapServer*(tls: TLSContext, socket: AsyncSocket) {.inline.} =
|
||||
if tls.sslCtx != nil:
|
||||
tls.sslCtx.wrapConnectedSocket(socket, handshakeAsServer)
|
||||
|
||||
proc `send`*(sock: TLSSocket, data: seq[byte]): int =
|
||||
if not sock.connected:
|
||||
return -1
|
||||
# In production: SSL_write(sock.ssl, data[0].addr, data.len.cint)
|
||||
return data.len
|
||||
|
||||
proc `recv`*(sock: TLSSocket, buf: var seq[byte], size: int): int =
|
||||
if not sock.connected:
|
||||
return -1
|
||||
# In production: SSL_read(sock.ssl, buf[0].addr, size.cint)
|
||||
buf.setLen(min(buf.len, size))
|
||||
return size
|
||||
|
||||
proc close*(sock: TLSSocket) =
|
||||
sock.connected = false
|
||||
# In production: SSL_shutdown(sock.ssl); SSL_free(sock.ssl); SSL_CTX_free(sock.ctx)
|
||||
|
||||
proc isConnected*(sock: TLSSocket): bool = sock.connected
|
||||
proc close*(tls: TLSContext) =
|
||||
if tls.sslCtx != nil:
|
||||
tls.sslCtx.destroyContext()
|
||||
|
||||
# TLS Certificate management
|
||||
type
|
||||
@@ -77,28 +64,21 @@ proc parseCertInfo*(certPath: string): CertInfo =
|
||||
result = CertInfo()
|
||||
if not fileExists(certPath):
|
||||
return
|
||||
|
||||
# Read PEM certificate and extract basic info
|
||||
let content = readFile(certPath)
|
||||
result.subject = "Unknown"
|
||||
result.issuer = "Unknown"
|
||||
result.fingerprint = ""
|
||||
|
||||
# In production: use OpenSSL X509 parsing
|
||||
for line in content.splitLines():
|
||||
if line.startsWith("Subject:"):
|
||||
result.subject = line[8..^1].strip()
|
||||
elif line.startsWith("Issuer:"):
|
||||
result.issuer = line[7..^1].strip()
|
||||
|
||||
result.isSelfSigned = result.subject == result.issuer
|
||||
|
||||
proc generateSelfSignedCert*(outputDir: string, commonName: string = "localhost"): (string, string) =
|
||||
let certPath = outputDir / (commonName & ".crt")
|
||||
let keyPath = outputDir / (commonName & ".key")
|
||||
createDir(outputDir)
|
||||
|
||||
# Use openssl CLI if available
|
||||
let cmd = "openssl req -x509 -newkey rsa:2048 -keyout " & keyPath &
|
||||
" -out " & certPath & " -days 365 -nodes -subj '/CN=" & commonName & "' 2>/dev/null"
|
||||
if execShellCmd(cmd) == 0 and fileExists(certPath):
|
||||
@@ -109,9 +89,13 @@ proc certificateFingerprint*(certPath: string): string =
|
||||
if not fileExists(certPath):
|
||||
return ""
|
||||
let cmd = "openssl x509 -in " & certPath & " -fingerprint -noout 2>/dev/null"
|
||||
result = ""
|
||||
# In production, use popen() to read command output
|
||||
discard execShellCmd(cmd)
|
||||
let (output, _) = execCmdEx(cmd)
|
||||
for line in output.splitLines():
|
||||
if "Fingerprint=" in line:
|
||||
let parts = line.split("Fingerprint=")
|
||||
if parts.len > 1:
|
||||
return parts[^1].strip()
|
||||
return ""
|
||||
|
||||
proc isExpired*(certPath: string): bool =
|
||||
if not fileExists(certPath):
|
||||
@@ -122,12 +106,15 @@ proc isExpired*(certPath: string): bool =
|
||||
proc daysUntilExpiry*(certPath: string): int =
|
||||
if not fileExists(certPath):
|
||||
return -1
|
||||
let cmd = "openssl x509 -in " & certPath & " -checkend 86400 2>/dev/null"
|
||||
if execShellCmd(cmd) == 0:
|
||||
# Expires in more than 1 day
|
||||
# In production, parse the actual enddate
|
||||
# Check if expires within 1 day
|
||||
let cmd1 = "openssl x509 -in " & certPath & " -checkend 86400 2>/dev/null"
|
||||
if execShellCmd(cmd1) == 0:
|
||||
# Check if expires within 30 days
|
||||
let cmd30 = "openssl x509 -in " & certPath & " -checkend 2592000 2>/dev/null"
|
||||
if execShellCmd(cmd30) == 0:
|
||||
return 365
|
||||
return 0
|
||||
return 30
|
||||
return 1
|
||||
|
||||
proc validateCert*(certPath: string): seq[string] =
|
||||
result = @[]
|
||||
|
||||
@@ -260,6 +260,31 @@ proc makeQueryMessage*(requestId: uint32, query: string): seq[byte] =
|
||||
)
|
||||
return serializeMessage(msg)
|
||||
|
||||
proc makeQueryParamsMessage*(requestId: uint32, query: string, params: seq[WireValue]): seq[byte] =
|
||||
var payload: seq[byte] = @[]
|
||||
payload.writeString(query)
|
||||
payload.add(byte(rfBinary))
|
||||
payload.writeUint32(uint32(params.len))
|
||||
for p in params:
|
||||
payload.serializeValue(p)
|
||||
|
||||
var msg = WireMessage(
|
||||
header: MessageHeader(kind: mkQueryParams, length: uint32(payload.len), requestId: requestId),
|
||||
payload: payload,
|
||||
)
|
||||
return serializeMessage(msg)
|
||||
|
||||
proc readQueryParamsMessage*(payload: openArray[byte]): (string, seq[WireValue]) =
|
||||
var pos = 0
|
||||
let queryStr = readString(payload, pos)
|
||||
discard payload[pos] # format byte
|
||||
pos += 1
|
||||
let paramCount = int(readUint32(payload, pos))
|
||||
var params: seq[WireValue] = @[]
|
||||
for i in 0..<paramCount:
|
||||
params.add(deserializeValue(payload, pos))
|
||||
return (queryStr, params)
|
||||
|
||||
proc makeReadyMessage*(requestId: uint32): seq[byte] =
|
||||
var payload: seq[byte] = @[]
|
||||
payload.add(0'u8) # idle state
|
||||
|
||||
@@ -22,8 +22,22 @@ type
|
||||
nkExplainStmt
|
||||
nkCreateView
|
||||
nkDropView
|
||||
nkCreateTrigger
|
||||
nkDropTrigger
|
||||
nkCreateMigration
|
||||
nkApplyMigration
|
||||
nkMigrationStatus
|
||||
nkMigrationUp
|
||||
nkMigrationDown
|
||||
nkMigrationDryRun
|
||||
nkCreateUser
|
||||
nkDropUser
|
||||
nkCreatePolicy
|
||||
nkDropPolicy
|
||||
nkEnableRLS
|
||||
nkDisableRLS
|
||||
nkGrant
|
||||
nkRevoke
|
||||
|
||||
# Clauses
|
||||
nkFrom
|
||||
@@ -59,6 +73,8 @@ type
|
||||
nkBetweenExpr
|
||||
nkLikeExpr
|
||||
nkIsExpr
|
||||
nkStar
|
||||
nkPlaceholder
|
||||
|
||||
# Graph-specific
|
||||
nkGraphTraversal
|
||||
@@ -129,6 +145,7 @@ type
|
||||
Node* = ref object
|
||||
line*: int
|
||||
col*: int
|
||||
exprAlias*: string
|
||||
case kind*: NodeKind
|
||||
of nkSelect:
|
||||
selDistinct*: bool
|
||||
@@ -211,11 +228,61 @@ type
|
||||
of nkDropView:
|
||||
dvName*: string
|
||||
dvIfExists*: bool
|
||||
of nkCreateTrigger:
|
||||
trigName*: string
|
||||
trigTable*: string
|
||||
trigTiming*: string # BEFORE, AFTER, INSTEAD OF
|
||||
trigEvent*: string # INSERT, UPDATE, DELETE
|
||||
trigAction*: Node # SQL statement to execute
|
||||
of nkDropTrigger:
|
||||
trigDropName*: string
|
||||
trigDropIfExists*: bool
|
||||
of nkCreateMigration:
|
||||
cmName*: string
|
||||
cmBody*: string
|
||||
cmBody*: string # UP migration body (SQL string)
|
||||
cmDownBody*: string # DOWN migration body (SQL string)
|
||||
cmChecksum*: string # SHA256 of cmBody
|
||||
cmAuthor*: string
|
||||
cmTimestamp*: int64
|
||||
of nkCreateUser:
|
||||
cuName*: string
|
||||
cuPassword*: string
|
||||
cuSuperuser*: bool
|
||||
of nkDropUser:
|
||||
duName*: string
|
||||
duIfExists*: bool
|
||||
of nkCreatePolicy:
|
||||
cpName*: string
|
||||
cpTable*: string
|
||||
cpCommand*: string # ALL, SELECT, INSERT, UPDATE, DELETE
|
||||
cpUsing*: Node # USING expression
|
||||
cpWithCheck*: Node # WITH CHECK expression
|
||||
of nkDropPolicy:
|
||||
dpName*: string
|
||||
dpTable*: string
|
||||
dpIfExists*: bool
|
||||
of nkEnableRLS:
|
||||
erlsTable*: string
|
||||
of nkDisableRLS:
|
||||
drlsTable*: string
|
||||
of nkGrant:
|
||||
grPrivilege*: string # SELECT, INSERT, UPDATE, DELETE, ALL
|
||||
grTable*: string
|
||||
grGrantee*: string # user, role, or PUBLIC
|
||||
of nkRevoke:
|
||||
rvPrivilege*: string
|
||||
rvTable*: string
|
||||
rvGrantee*: string
|
||||
of nkApplyMigration:
|
||||
amName*: string
|
||||
of nkMigrationStatus:
|
||||
discard # no fields needed
|
||||
of nkMigrationUp:
|
||||
muCount*: int # 0 = all, N = apply N pending migrations
|
||||
of nkMigrationDown:
|
||||
mdCount*: int # 0 = 1, N = rollback N migrations
|
||||
of nkMigrationDryRun:
|
||||
mdrName*: string # migration name to dry-run
|
||||
of nkCreateIndex:
|
||||
ciTarget*: string
|
||||
ciName*: string
|
||||
@@ -344,6 +411,10 @@ type
|
||||
joinTarget*: Node
|
||||
joinOn*: Node
|
||||
joinAlias*: string
|
||||
of nkStar:
|
||||
discard
|
||||
of nkPlaceholder:
|
||||
discard
|
||||
of nkPropertyDef:
|
||||
pdName*: string
|
||||
pdType*: string
|
||||
|
||||
+839
-30
File diff suppressed because it is too large
Load Diff
@@ -49,6 +49,7 @@ type
|
||||
irekCast
|
||||
irekConditional
|
||||
irekExists
|
||||
irekStar
|
||||
|
||||
IRJoinKind* = enum
|
||||
irjkInner
|
||||
@@ -163,6 +164,8 @@ type
|
||||
elseExpr*: IRExpr
|
||||
of irekExists:
|
||||
existsSubquery*: IRPlan
|
||||
of irekStar:
|
||||
discard
|
||||
|
||||
type
|
||||
TypeChecker* = ref object
|
||||
@@ -240,3 +243,5 @@ proc inferExpr*(tc: TypeChecker, expr: IRExpr, context: Table[string, IRType]):
|
||||
return thenType
|
||||
of irekExists:
|
||||
return IRType(name: "bool", kind: itkScalar)
|
||||
of irekStar:
|
||||
return IRType(name: "star", kind: itkScalar)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
## BaraQL Lexer — tokenization
|
||||
import std/tables
|
||||
import std/strutils
|
||||
import std/unicode
|
||||
|
||||
type
|
||||
TokenKind* = enum
|
||||
@@ -91,8 +92,25 @@ type
|
||||
tkRollback
|
||||
tkExplain
|
||||
tkView
|
||||
tkTrigger
|
||||
tkBefore
|
||||
tkAfter
|
||||
tkInstead
|
||||
tkOf
|
||||
tkMigration
|
||||
tkApply
|
||||
tkStatus
|
||||
tkUp
|
||||
tkDown
|
||||
tkDryRun
|
||||
tkUser
|
||||
tkPolicy
|
||||
tkEnable
|
||||
tkDisable
|
||||
tkFor
|
||||
tkUsing
|
||||
tkGrant
|
||||
tkRevoke
|
||||
tkCount
|
||||
tkSum
|
||||
tkAvg
|
||||
@@ -141,6 +159,7 @@ type
|
||||
tkConcat
|
||||
tkCoalesce
|
||||
tkFloorDiv
|
||||
tkPlaceholder
|
||||
|
||||
# Special
|
||||
tkEof
|
||||
@@ -237,8 +256,25 @@ const keywords*: Table[string, TokenKind] = {
|
||||
"rollback": tkRollback,
|
||||
"explain": tkExplain,
|
||||
"view": tkView,
|
||||
"trigger": tkTrigger,
|
||||
"before": tkBefore,
|
||||
"after": tkAfter,
|
||||
"instead": tkInstead,
|
||||
"of": tkOf,
|
||||
"migration": tkMigration,
|
||||
"apply": tkApply,
|
||||
"status": tkStatus,
|
||||
"up": tkUp,
|
||||
"down": tkDown,
|
||||
"dryrun": tkDryRun,
|
||||
"user": tkUser,
|
||||
"policy": tkPolicy,
|
||||
"enable": tkEnable,
|
||||
"disable": tkDisable,
|
||||
"for": tkFor,
|
||||
"using": tkUsing,
|
||||
"grant": tkGrant,
|
||||
"revoke": tkRevoke,
|
||||
"count": tkCount,
|
||||
"sum": tkSum,
|
||||
"avg": tkAvg,
|
||||
@@ -273,6 +309,32 @@ proc advance(l: var Lexer): char =
|
||||
else:
|
||||
inc l.col
|
||||
|
||||
proc peekRune(l: Lexer): Rune =
|
||||
if l.pos < l.input.len:
|
||||
var p = l.pos
|
||||
var r: Rune
|
||||
fastRuneAt(l.input, p, r, true)
|
||||
return r
|
||||
return Rune(0)
|
||||
|
||||
proc advanceRune(l: var Lexer): Rune =
|
||||
if l.pos < l.input.len:
|
||||
var r: Rune
|
||||
fastRuneAt(l.input, l.pos, r, true)
|
||||
if r == Rune('\n'):
|
||||
inc l.line
|
||||
l.col = 1
|
||||
else:
|
||||
inc l.col
|
||||
return r
|
||||
return Rune(0)
|
||||
|
||||
proc isIdentStartRune(r: Rune): bool =
|
||||
return isAlpha(r) or r == Rune('_')
|
||||
|
||||
proc isIdentPartRune(r: Rune): bool =
|
||||
return isAlpha(r) or (r.int >= ord('0') and r.int <= ord('9')) or r == Rune('_')
|
||||
|
||||
proc skipWhitespace(l: var Lexer) =
|
||||
while l.pos < l.input.len and l.input[l.pos] in {' ', '\t', '\r', '\n'}:
|
||||
discard l.advance()
|
||||
@@ -326,9 +388,15 @@ proc readNumber(l: var Lexer, startLine, startCol: int): Token =
|
||||
|
||||
proc readIdent(l: var Lexer, startLine, startCol: int): Token =
|
||||
var ident = ""
|
||||
while l.pos < l.input.len and (l.input[l.pos] in IdentChars or l.input[l.pos] in Digits):
|
||||
ident.add(l.input[l.pos])
|
||||
discard l.advance()
|
||||
while l.pos < l.input.len:
|
||||
let r = l.peekRune()
|
||||
if isIdentPartRune(r):
|
||||
var run: Rune
|
||||
fastRuneAt(l.input, l.pos, run, true)
|
||||
ident.add($run)
|
||||
inc l.col
|
||||
else:
|
||||
break
|
||||
let lowerIdent = ident.toLower()
|
||||
if lowerIdent in keywords:
|
||||
Token(kind: keywords[lowerIdent], value: ident, line: startLine, col: startCol)
|
||||
@@ -428,7 +496,7 @@ proc nextToken*(l: var Lexer): Token =
|
||||
discard l.advance()
|
||||
return Token(kind: tkCoalesce, value: "??", line: startLine, col: startCol)
|
||||
discard l.advance()
|
||||
return Token(kind: tkInvalid, value: "?", line: startLine, col: startCol)
|
||||
return Token(kind: tkPlaceholder, value: "?", line: startLine, col: startCol)
|
||||
of '.':
|
||||
if l.pos + 1 < l.input.len and l.input[l.pos + 1] == '<':
|
||||
discard l.advance()
|
||||
@@ -479,7 +547,7 @@ proc nextToken*(l: var Lexer): Token =
|
||||
else:
|
||||
if ch in Digits:
|
||||
return l.readNumber(startLine, startCol)
|
||||
elif ch in IdentStartChars:
|
||||
elif ch in IdentStartChars or isIdentStartRune(l.peekRune()):
|
||||
return l.readIdent(startLine, startCol)
|
||||
else:
|
||||
discard l.advance()
|
||||
|
||||
@@ -105,6 +105,12 @@ proc parsePrimary(p: var Parser): Node =
|
||||
let sub = p.parseSelect()
|
||||
discard p.expect(tkRParen)
|
||||
Node(kind: nkExists, existsExpr: sub, line: tok.line, col: tok.col)
|
||||
of tkStar:
|
||||
discard p.advance()
|
||||
Node(kind: nkStar, line: tok.line, col: tok.col)
|
||||
of tkPlaceholder:
|
||||
discard p.advance()
|
||||
Node(kind: nkPlaceholder, line: tok.line, col: tok.col)
|
||||
of tkNot:
|
||||
discard p.advance()
|
||||
let operand = p.parsePrimary()
|
||||
@@ -309,9 +315,15 @@ proc parseSelect(p: var Parser): Node =
|
||||
|
||||
# Parse SELECT list
|
||||
result.selResult = @[]
|
||||
result.selResult.add(p.parseExpr())
|
||||
var expr = p.parseExpr()
|
||||
if p.match(tkAs):
|
||||
expr.exprAlias = p.expect(tkIdent).value
|
||||
result.selResult.add(expr)
|
||||
while p.match(tkComma):
|
||||
result.selResult.add(p.parseExpr())
|
||||
expr = p.parseExpr()
|
||||
if p.match(tkAs):
|
||||
expr.exprAlias = p.expect(tkIdent).value
|
||||
result.selResult.add(expr)
|
||||
|
||||
# Parse FROM
|
||||
result.selJoins = @[]
|
||||
@@ -545,7 +557,7 @@ proc parseCreateTable(p: var Parser): Node =
|
||||
result.crtIfNotExists = true
|
||||
|
||||
discard p.expect(tkTable)
|
||||
if p.peek().kind == tkIdent and p.peek().value.toLower() == "if":
|
||||
if p.peek().kind == tkIf:
|
||||
discard p.advance() # if
|
||||
discard p.expect(tkNot)
|
||||
discard p.expect(tkExists)
|
||||
@@ -685,8 +697,22 @@ proc parseDropTable(p: var Parser): Node =
|
||||
proc parseAlterTable(p: var Parser): Node =
|
||||
let tok = p.expect(tkAlter)
|
||||
discard p.expect(tkTable)
|
||||
let tableName = p.expect(tkIdent).value
|
||||
# Check for ENABLE/DISABLE ROW LEVEL SECURITY
|
||||
if p.peek().kind == tkEnable:
|
||||
discard p.advance()
|
||||
discard p.expect(tkIdent) # ROW
|
||||
discard p.expect(tkIdent) # LEVEL
|
||||
discard p.expect(tkIdent) # SECURITY
|
||||
return Node(kind: nkEnableRLS, erlsTable: tableName, line: tok.line, col: tok.col)
|
||||
elif p.peek().kind == tkDisable:
|
||||
discard p.advance()
|
||||
discard p.expect(tkIdent) # ROW
|
||||
discard p.expect(tkIdent) # LEVEL
|
||||
discard p.expect(tkIdent) # SECURITY
|
||||
return Node(kind: nkDisableRLS, drlsTable: tableName, line: tok.line, col: tok.col)
|
||||
result = Node(kind: nkAlterTable, line: tok.line, col: tok.col)
|
||||
result.altName = p.expect(tkIdent).value
|
||||
result.altName = tableName
|
||||
result.altOps = @[]
|
||||
if p.match(tkAdd):
|
||||
discard p.match(tkColumn)
|
||||
@@ -764,13 +790,112 @@ proc parseDropView(p: var Parser): Node =
|
||||
result = Node(kind: nkDropView, dvName: name, dvIfExists: ifExists,
|
||||
line: tok.line, col: tok.col)
|
||||
|
||||
proc parseCreateTrigger(p: var Parser): Node =
|
||||
let tok = p.expect(tkCreate)
|
||||
discard p.expect(tkTrigger)
|
||||
let name = p.expect(tkIdent).value
|
||||
# Parse timing: BEFORE | AFTER | INSTEAD OF
|
||||
var timing = ""
|
||||
let timingTok = p.peek()
|
||||
if timingTok.kind == tkBefore:
|
||||
discard p.advance()
|
||||
timing = "before"
|
||||
elif timingTok.kind == tkAfter:
|
||||
discard p.advance()
|
||||
timing = "after"
|
||||
elif timingTok.kind == tkInstead:
|
||||
discard p.advance()
|
||||
discard p.expect(tkOf)
|
||||
timing = "instead of"
|
||||
else:
|
||||
raise newException(ValueError, "Expected BEFORE, AFTER, or INSTEAD OF in TRIGGER definition")
|
||||
# Parse event: INSERT | UPDATE | DELETE
|
||||
var event = ""
|
||||
let eventTok = p.peek()
|
||||
if eventTok.kind == tkInsert:
|
||||
discard p.advance()
|
||||
event = "INSERT"
|
||||
elif eventTok.kind == tkUpdate:
|
||||
discard p.advance()
|
||||
event = "UPDATE"
|
||||
elif eventTok.kind == tkDelete:
|
||||
discard p.advance()
|
||||
event = "DELETE"
|
||||
else:
|
||||
raise newException(ValueError, "Expected INSERT, UPDATE, or DELETE in TRIGGER definition")
|
||||
discard p.expect(tkOn)
|
||||
let tableName = p.expect(tkIdent).value
|
||||
discard p.expect(tkAs)
|
||||
# Parse action as raw string until end of statement
|
||||
var actionStr = ""
|
||||
while p.pos < p.tokens.len and p.tokens[p.pos].kind != tkSemicolon:
|
||||
actionStr.add(p.tokens[p.pos].value)
|
||||
actionStr.add(" ")
|
||||
discard p.advance()
|
||||
let actionNode = Node(kind: nkStringLit, strVal: actionStr.strip())
|
||||
result = Node(kind: nkCreateTrigger, trigName: name, trigTable: tableName,
|
||||
trigTiming: timing, trigEvent: event, trigAction: actionNode,
|
||||
line: tok.line, col: tok.col)
|
||||
|
||||
proc parseDropTrigger(p: var Parser): Node =
|
||||
let tok = p.expect(tkDrop)
|
||||
discard p.expect(tkTrigger)
|
||||
var ifExists = false
|
||||
if p.peek().kind == tkIf:
|
||||
discard p.advance()
|
||||
discard p.expect(tkExists)
|
||||
ifExists = true
|
||||
let name = p.expect(tkIdent).value
|
||||
result = Node(kind: nkDropTrigger, trigDropName: name, trigDropIfExists: ifExists,
|
||||
line: tok.line, col: tok.col)
|
||||
|
||||
proc parseCreateMigration(p: var Parser): Node =
|
||||
let tok = p.expect(tkCreate)
|
||||
discard p.expect(tkMigration)
|
||||
let name = p.expect(tkIdent).value
|
||||
discard p.expect(tkAs)
|
||||
let body = p.expect(tkStringLit).value
|
||||
result = Node(kind: nkCreateMigration, cmName: name, cmBody: body,
|
||||
var upBody = ""
|
||||
var downBody = ""
|
||||
if p.peek().kind == tkAs:
|
||||
# Legacy syntax: CREATE MIGRATION name AS 'body'
|
||||
discard p.advance()
|
||||
upBody = p.expect(tkStringLit).value
|
||||
elif p.peek().kind == tkLBrace:
|
||||
# New syntax: CREATE MIGRATION name { UP: ...; DOWN: ...; }
|
||||
discard p.advance() # {
|
||||
while p.peek().kind != tkRBrace and p.peek().kind != tkEof:
|
||||
var section = ""
|
||||
let sectionTok = p.peek()
|
||||
if sectionTok.kind == tkUp:
|
||||
discard p.advance()
|
||||
section = "up"
|
||||
elif sectionTok.kind == tkDown:
|
||||
discard p.advance()
|
||||
section = "down"
|
||||
elif sectionTok.kind == tkIdent:
|
||||
section = p.expect(tkIdent).value.toLower()
|
||||
else:
|
||||
raise newException(ValueError, "Expected UP or DOWN in migration body, got: " & $sectionTok.kind)
|
||||
discard p.expect(tkColon)
|
||||
var bodyStr = ""
|
||||
while p.peek().kind != tkRBrace and p.peek().kind != tkEof:
|
||||
# Check if next token starts a new section
|
||||
let nextTok = p.peek()
|
||||
if (nextTok.kind == tkUp or nextTok.kind == tkDown or
|
||||
(nextTok.kind == tkIdent and (nextTok.value.toLower() == "up" or nextTok.value.toLower() == "down"))) and
|
||||
p.pos + 1 < p.tokens.len and p.tokens[p.pos + 1].kind == tkColon:
|
||||
break
|
||||
bodyStr.add(p.tokens[p.pos].value)
|
||||
bodyStr.add(" ")
|
||||
discard p.advance()
|
||||
if section == "up":
|
||||
upBody = bodyStr.strip()
|
||||
elif section == "down":
|
||||
downBody = bodyStr.strip()
|
||||
else:
|
||||
raise newException(ValueError, "Expected UP or DOWN in migration body, got: " & section)
|
||||
discard p.expect(tkRBrace)
|
||||
result = Node(kind: nkCreateMigration, cmName: name, cmBody: upBody,
|
||||
cmDownBody: downBody, cmChecksum: "", cmAuthor: "", cmTimestamp: 0,
|
||||
line: tok.line, col: tok.col)
|
||||
|
||||
proc parseApplyMigration(p: var Parser): Node =
|
||||
@@ -779,6 +904,146 @@ proc parseApplyMigration(p: var Parser): Node =
|
||||
let name = p.expect(tkIdent).value
|
||||
result = Node(kind: nkApplyMigration, amName: name, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseMigrationStatus(p: var Parser): Node =
|
||||
let tok = p.expect(tkMigration)
|
||||
discard p.expect(tkStatus)
|
||||
result = Node(kind: nkMigrationStatus, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseMigrationUp(p: var Parser): Node =
|
||||
let tok = p.expect(tkMigration)
|
||||
discard p.expect(tkUp)
|
||||
var count = 0
|
||||
if p.peek().kind == tkIntLit:
|
||||
count = parseInt(p.expect(tkIntLit).value)
|
||||
result = Node(kind: nkMigrationUp, muCount: count, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseMigrationDown(p: var Parser): Node =
|
||||
let tok = p.expect(tkMigration)
|
||||
discard p.expect(tkDown)
|
||||
var count = 1
|
||||
if p.peek().kind == tkIntLit:
|
||||
count = parseInt(p.expect(tkIntLit).value)
|
||||
result = Node(kind: nkMigrationDown, mdCount: count, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseMigrationDryRun(p: var Parser): Node =
|
||||
let tok = p.expect(tkMigration)
|
||||
discard p.expect(tkDryRun)
|
||||
let name = p.expect(tkIdent).value
|
||||
result = Node(kind: nkMigrationDryRun, mdrName: name, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseCreateUser(p: var Parser): Node =
|
||||
let tok = p.expect(tkCreate)
|
||||
discard p.expect(tkUser)
|
||||
let name = p.expect(tkIdent).value
|
||||
var password = ""
|
||||
var isSuper = false
|
||||
if p.peek().kind == tkWith:
|
||||
discard p.advance()
|
||||
if p.peek().kind == tkIdent and p.peek().value.toLower() == "password":
|
||||
discard p.advance()
|
||||
password = p.expect(tkStringLit).value
|
||||
while p.peek().kind == tkIdent:
|
||||
let opt = p.peek().value.toLower()
|
||||
if opt == "superuser":
|
||||
discard p.advance()
|
||||
isSuper = true
|
||||
elif opt == "nosuperuser":
|
||||
discard p.advance()
|
||||
isSuper = false
|
||||
else:
|
||||
break
|
||||
result = Node(kind: nkCreateUser, cuName: name, cuPassword: password,
|
||||
cuSuperuser: isSuper, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseDropUser(p: var Parser): Node =
|
||||
let tok = p.expect(tkDrop)
|
||||
discard p.expect(tkUser)
|
||||
var ifExists = false
|
||||
if p.peek().kind == tkIf:
|
||||
discard p.advance()
|
||||
discard p.expect(tkExists)
|
||||
ifExists = true
|
||||
let name = p.expect(tkIdent).value
|
||||
result = Node(kind: nkDropUser, duName: name, duIfExists: ifExists,
|
||||
line: tok.line, col: tok.col)
|
||||
|
||||
proc parseCreatePolicy(p: var Parser): Node =
|
||||
let tok = p.expect(tkCreate)
|
||||
discard p.expect(tkPolicy)
|
||||
let name = p.expect(tkIdent).value
|
||||
discard p.expect(tkOn)
|
||||
let tableName = p.expect(tkIdent).value
|
||||
var cmd = "ALL"
|
||||
var usingNode: Node = nil
|
||||
var withCheckNode: Node = nil
|
||||
if p.peek().kind == tkFor:
|
||||
discard p.advance()
|
||||
let cmdTok = p.peek()
|
||||
if cmdTok.kind == tkIdent or cmdTok.kind == tkSelect or cmdTok.kind == tkInsert or
|
||||
cmdTok.kind == tkUpdate or cmdTok.kind == tkDelete:
|
||||
discard p.advance()
|
||||
cmd = cmdTok.value.toUpper()
|
||||
else:
|
||||
raise newException(ValueError, "Expected ALL, SELECT, INSERT, UPDATE, or DELETE in POLICY definition")
|
||||
if p.peek().kind == tkUsing:
|
||||
discard p.advance()
|
||||
usingNode = p.parseExpr()
|
||||
if p.peek().kind == tkWith:
|
||||
discard p.advance()
|
||||
discard p.expect(tkCheck)
|
||||
withCheckNode = p.parseExpr()
|
||||
result = Node(kind: nkCreatePolicy, cpName: name, cpTable: tableName,
|
||||
cpCommand: cmd, cpUsing: usingNode, cpWithCheck: withCheckNode,
|
||||
line: tok.line, col: tok.col)
|
||||
|
||||
proc parseDropPolicy(p: var Parser): Node =
|
||||
let tok = p.expect(tkDrop)
|
||||
discard p.expect(tkPolicy)
|
||||
var ifExists = false
|
||||
if p.peek().kind == tkIf:
|
||||
discard p.advance()
|
||||
discard p.expect(tkExists)
|
||||
ifExists = true
|
||||
let name = p.expect(tkIdent).value
|
||||
discard p.expect(tkOn)
|
||||
let tableName = p.expect(tkIdent).value
|
||||
result = Node(kind: nkDropPolicy, dpName: name, dpTable: tableName,
|
||||
dpIfExists: ifExists, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseGrant(p: var Parser): Node =
|
||||
let tok = p.expect(tkGrant)
|
||||
var priv = ""
|
||||
let privTok = p.peek()
|
||||
if privTok.kind == tkIdent or privTok.kind == tkSelect or privTok.kind == tkInsert or
|
||||
privTok.kind == tkUpdate or privTok.kind == tkDelete:
|
||||
discard p.advance()
|
||||
priv = privTok.value.toUpper()
|
||||
else:
|
||||
raise newException(ValueError, "Expected privilege in GRANT")
|
||||
discard p.expect(tkOn)
|
||||
let tableName = p.expect(tkIdent).value
|
||||
discard p.expect(tkTo)
|
||||
let grantee = p.expect(tkIdent).value
|
||||
result = Node(kind: nkGrant, grPrivilege: priv, grTable: tableName,
|
||||
grGrantee: grantee, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseRevoke(p: var Parser): Node =
|
||||
let tok = p.expect(tkRevoke)
|
||||
var priv = ""
|
||||
let privTok = p.peek()
|
||||
if privTok.kind == tkIdent or privTok.kind == tkSelect or privTok.kind == tkInsert or
|
||||
privTok.kind == tkUpdate or privTok.kind == tkDelete:
|
||||
discard p.advance()
|
||||
priv = privTok.value.toUpper()
|
||||
else:
|
||||
raise newException(ValueError, "Expected privilege in REVOKE")
|
||||
discard p.expect(tkOn)
|
||||
let tableName = p.expect(tkIdent).value
|
||||
discard p.expect(tkFrom)
|
||||
let grantee = p.expect(tkIdent).value
|
||||
result = Node(kind: nkRevoke, rvPrivilege: priv, rvTable: tableName,
|
||||
rvGrantee: grantee, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseStatement*(p: var Parser): Node =
|
||||
case p.peek().kind
|
||||
of tkWith, tkSelect: p.parseSelect()
|
||||
@@ -796,8 +1061,14 @@ proc parseStatement*(p: var Parser): Node =
|
||||
elif next.kind == tkView or
|
||||
(next.kind == tkIdent and next.value.toLower() == "or"):
|
||||
p.parseCreateView()
|
||||
elif next.kind == tkTrigger:
|
||||
p.parseCreateTrigger()
|
||||
elif next.kind == tkMigration:
|
||||
p.parseCreateMigration()
|
||||
elif next.kind == tkUser:
|
||||
p.parseCreateUser()
|
||||
elif next.kind == tkPolicy:
|
||||
p.parseCreatePolicy()
|
||||
else:
|
||||
p.parseCreateType()
|
||||
else:
|
||||
@@ -809,6 +1080,12 @@ proc parseStatement*(p: var Parser): Node =
|
||||
p.parseDropTable()
|
||||
elif next.kind == tkView:
|
||||
p.parseDropView()
|
||||
elif next.kind == tkTrigger:
|
||||
p.parseDropTrigger()
|
||||
elif next.kind == tkUser:
|
||||
p.parseDropUser()
|
||||
elif next.kind == tkPolicy:
|
||||
p.parseDropPolicy()
|
||||
else:
|
||||
let tok = p.advance()
|
||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||
@@ -821,12 +1098,33 @@ proc parseStatement*(p: var Parser): Node =
|
||||
else:
|
||||
let tok = p.advance()
|
||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||
of tkGrant:
|
||||
p.parseGrant()
|
||||
of tkRevoke:
|
||||
p.parseRevoke()
|
||||
of tkApply:
|
||||
if p.pos + 1 < p.tokens.len and p.tokens[p.pos + 1].kind == tkMigration:
|
||||
p.parseApplyMigration()
|
||||
else:
|
||||
let tok = p.advance()
|
||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||
of tkMigration:
|
||||
if p.pos + 1 < p.tokens.len:
|
||||
let next = p.tokens[p.pos + 1]
|
||||
if next.kind == tkStatus:
|
||||
p.parseMigrationStatus()
|
||||
elif next.kind == tkUp:
|
||||
p.parseMigrationUp()
|
||||
elif next.kind == tkDown:
|
||||
p.parseMigrationDown()
|
||||
elif next.kind == tkDryRun:
|
||||
p.parseMigrationDryRun()
|
||||
else:
|
||||
let tok = p.advance()
|
||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||
else:
|
||||
let tok = p.advance()
|
||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||
of tkBegin: p.parseBeginTxn()
|
||||
of tkCommit: p.parseCommitTxn()
|
||||
of tkRollback: p.parseRollbackTxn()
|
||||
|
||||
@@ -7,7 +7,6 @@ import std/tables
|
||||
import std/monotimes
|
||||
import std/streams
|
||||
import std/locks
|
||||
import std/asyncdispatch
|
||||
import bloom
|
||||
import wal
|
||||
import mmap
|
||||
|
||||
+18
-1
@@ -3,9 +3,11 @@
|
||||
import std/asyncdispatch
|
||||
import std/threadpool
|
||||
import std/locks
|
||||
import std/os
|
||||
import barabadb/core/server
|
||||
import barabadb/core/httpserver
|
||||
import barabadb/core/config
|
||||
import barabadb/protocol/ssl
|
||||
import barabadb/storage/lsm
|
||||
import barabadb/storage/compaction
|
||||
|
||||
@@ -57,9 +59,24 @@ proc runTcpServer(config: BaraConfig) {.async.} =
|
||||
await server.run()
|
||||
|
||||
proc main() =
|
||||
let config = loadConfig()
|
||||
var config = loadConfig()
|
||||
echo "BaraDB v0.1.0 — Multimodal Database Engine"
|
||||
|
||||
if config.tlsEnabled:
|
||||
if config.certFile.len == 0 or config.keyFile.len == 0 or
|
||||
not fileExists(config.certFile) or not fileExists(config.keyFile):
|
||||
echo "TLS enabled but no certificate found. Generating self-signed certificate..."
|
||||
let (cert, key) = generateSelfSignedCert(config.dataDir / "certs")
|
||||
if cert.len > 0 and key.len > 0:
|
||||
config.certFile = cert
|
||||
config.keyFile = key
|
||||
echo "Generated self-signed certificate:"
|
||||
echo " Cert: ", cert
|
||||
echo " Key: ", key
|
||||
else:
|
||||
echo "WARNING: Failed to generate self-signed certificate. TLS disabled."
|
||||
config.tlsEnabled = false
|
||||
|
||||
# Start HTTP server (blocking, multi-threaded via hunos) in background thread
|
||||
var httpServer = newHttpServer(config)
|
||||
spawn httpServer.run(config.port + 440) # HTTP port = TCP port + 440
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
import std/unittest
|
||||
import std/os
|
||||
import std/strutils
|
||||
import barabadb/core/types
|
||||
import barabadb/query/executor as qexec
|
||||
import barabadb/query/parser
|
||||
import barabadb/query/ast
|
||||
import barabadb/storage/lsm
|
||||
|
||||
proc execSql(ctx: qexec.ExecutionContext, sql: string): qexec.ExecResult =
|
||||
qexec.executeQuery(ctx, parse(sql))
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
suite "JOIN execution":
|
||||
var db: LSMTree
|
||||
var ctx: qexec.ExecutionContext
|
||||
|
||||
setup:
|
||||
db = newLSMTree("")
|
||||
ctx = qexec.newExecutionContext(db)
|
||||
discard execSql(ctx, "CREATE TABLE users (id INT, name TEXT)")
|
||||
discard execSql(ctx, "CREATE TABLE orders (id INT, user_id INT, total REAL)")
|
||||
discard execSql(ctx, "INSERT INTO users (id, name) VALUES (1, 'Alice')")
|
||||
discard execSql(ctx, "INSERT INTO users (id, name) VALUES (2, 'Bob')")
|
||||
discard execSql(ctx, "INSERT INTO orders (id, user_id, total) VALUES (10, 1, 99.5)")
|
||||
discard execSql(ctx, "INSERT INTO orders (id, user_id, total) VALUES (20, 1, 23.0)")
|
||||
discard execSql(ctx, "INSERT INTO orders (id, user_id, total) VALUES (30, 3, 150.0)")
|
||||
|
||||
teardown:
|
||||
discard
|
||||
|
||||
test "INNER JOIN returns matching rows only":
|
||||
let r = execSql(ctx, "SELECT * FROM users u JOIN orders o ON u.id = o.user_id")
|
||||
check r.rows.len == 2
|
||||
check r.rows[0]["name"] == "Alice"
|
||||
check r.rows[0]["total"] == "99.5"
|
||||
|
||||
test "LEFT JOIN keeps unmatched left rows":
|
||||
let r = execSql(ctx, "SELECT * FROM users u LEFT JOIN orders o ON u.id = o.user_id")
|
||||
check r.rows.len == 3
|
||||
check r.rows[0]["name"] == "Alice"
|
||||
check r.rows[1]["name"] == "Alice"
|
||||
check r.rows[2]["name"] == "Bob"
|
||||
check r.rows[2]["total"] == "" # NULL represented as empty string
|
||||
|
||||
test "RIGHT JOIN keeps unmatched right rows":
|
||||
let r = execSql(ctx, "SELECT * FROM users u RIGHT JOIN orders o ON u.id = o.user_id")
|
||||
check r.rows.len == 3
|
||||
check r.rows[2]["id"] == "30"
|
||||
check r.rows[2]["name"] == "" # NULL
|
||||
check r.rows[2]["total"] == "150.0"
|
||||
|
||||
test "FULL JOIN keeps all rows":
|
||||
let r = execSql(ctx, "SELECT * FROM users u FULL JOIN orders o ON u.id = o.user_id")
|
||||
check r.rows.len == 4
|
||||
|
||||
test "CROSS JOIN cartesian product":
|
||||
let r = execSql(ctx, "SELECT * FROM users u CROSS JOIN orders o")
|
||||
check r.rows.len == 6
|
||||
|
||||
test "aliased column projection":
|
||||
let r = execSql(ctx, "SELECT u.name, o.total FROM users u JOIN orders o ON u.id = o.user_id")
|
||||
check r.rows.len == 2
|
||||
check r.rows[0]["name"] == "Alice"
|
||||
check r.rows[0]["total"] == "99.5"
|
||||
check "id" notin r.rows[0]
|
||||
|
||||
test "count after FULL JOIN":
|
||||
let r = execSql(ctx, "SELECT COUNT(*) AS cnt FROM users u FULL JOIN orders o ON u.id = o.user_id")
|
||||
check r.rows.len == 1
|
||||
check r.rows[0]["cnt"] == "4"
|
||||
|
||||
test "count after CROSS JOIN":
|
||||
let r = execSql(ctx, "SELECT COUNT(*) AS cnt FROM users u CROSS JOIN orders o")
|
||||
check r.rows[0]["cnt"] == "6"
|
||||
+378
-3
@@ -8,6 +8,8 @@ import std/asyncdispatch
|
||||
import barabadb/core/types
|
||||
import barabadb/core/mvcc
|
||||
import barabadb/core/deadlock
|
||||
import barabadb/core/config
|
||||
import barabadb/core/server
|
||||
import barabadb/core/columnar
|
||||
import barabadb/core/raft
|
||||
import barabadb/core/sharding
|
||||
@@ -31,6 +33,7 @@ import barabadb/client/fileops
|
||||
import barabadb/fts/multilang as mlang
|
||||
import barabadb/protocol/zerocopy
|
||||
import barabadb/query/adaptive
|
||||
import barabadb/query/executor as qexec
|
||||
import barabadb/core/disttxn
|
||||
import barabadb/vector/engine as vengine
|
||||
import barabadb/graph/cypher
|
||||
@@ -409,6 +412,44 @@ suite "Deadlock Detection":
|
||||
dd.removeTxn(2)
|
||||
check not dd.hasDeadlock()
|
||||
|
||||
suite "MVCC Deadlock Detection":
|
||||
test "TxnManager detects and breaks deadlock":
|
||||
var tm = newTxnManager()
|
||||
var t1 = tm.beginTxn()
|
||||
var t2 = tm.beginTxn()
|
||||
# t1 writes key "a"
|
||||
check tm.write(t1, "a", @[1'u8])
|
||||
# t2 writes key "b"
|
||||
check tm.write(t2, "b", @[2'u8])
|
||||
# t2 tries to write "a" — conflicts with t1 (active), adds wait edge t2->t1
|
||||
check not tm.write(t2, "a", @[3'u8])
|
||||
# t1 tries to write "b" — conflicts with t2 (active), adds wait edge t1->t2
|
||||
# This creates a cycle: t1->t2->t1
|
||||
check not tm.write(t1, "b", @[4'u8])
|
||||
# One of the transactions should have been aborted as victim
|
||||
let t1Active = t1.state == tsActive
|
||||
let t2Active = t2.state == tsActive
|
||||
# At least one victim must be aborted
|
||||
check (not t1Active) or (not t2Active)
|
||||
# The survivor should be able to commit
|
||||
if t1Active:
|
||||
check tm.commit(t1)
|
||||
if t2Active:
|
||||
check tm.commit(t2)
|
||||
|
||||
test "No false deadlock on sequential writes":
|
||||
var tm = newTxnManager()
|
||||
var t1 = tm.beginTxn()
|
||||
# t1 writes key "a"
|
||||
check tm.write(t1, "a", @[1'u8])
|
||||
# t1 commits
|
||||
check tm.commit(t1)
|
||||
# t2 begins after t1 committed
|
||||
var t2 = tm.beginTxn()
|
||||
# t2 writes same key — no active conflict
|
||||
check tm.write(t2, "a", @[2'u8])
|
||||
check tm.commit(t2)
|
||||
|
||||
suite "Wire Protocol":
|
||||
test "Value serialization roundtrip":
|
||||
var buf: seq[byte] = @[]
|
||||
@@ -1934,9 +1975,13 @@ suite "TLS/SSL":
|
||||
check info.subject.len > 0
|
||||
check info.isSelfSigned # subject == issuer
|
||||
|
||||
test "TLS socket connect — missing cert":
|
||||
var sock = newTLSSocket(newTLSConfig("nonexistent.pem", "nonexistent.key"))
|
||||
check not sock.connect("localhost", 443)
|
||||
test "TLS context creation with missing cert raises":
|
||||
var raised = false
|
||||
try:
|
||||
discard newTLSContext(newTLSConfig("nonexistent.pem", "nonexistent.key"))
|
||||
except IOError:
|
||||
raised = true
|
||||
check raised
|
||||
|
||||
test "Generate self-signed cert":
|
||||
let (certPath, keyPath) = generateSelfSignedCert("/tmp/baradb_test_tls", "test.local")
|
||||
@@ -1944,3 +1989,333 @@ suite "TLS/SSL":
|
||||
if certPath.len > 0:
|
||||
check fileExists(certPath)
|
||||
check fileExists(keyPath)
|
||||
# Should be able to create TLS context from generated cert
|
||||
let ctx = newTLSContext(newTLSConfig(certPath, keyPath))
|
||||
check ctx != nil
|
||||
|
||||
test "Server with TLS config":
|
||||
var cfg = defaultConfig()
|
||||
cfg.tlsEnabled = true
|
||||
let (certPath, keyPath) = generateSelfSignedCert("/tmp/baradb_test_tls2", "test.local")
|
||||
if certPath.len > 0:
|
||||
cfg.certFile = certPath
|
||||
cfg.keyFile = keyPath
|
||||
var srv = newServer(cfg)
|
||||
check srv != nil
|
||||
check srv.tls != nil
|
||||
|
||||
suite "Triggers":
|
||||
test "Parse CREATE TRIGGER":
|
||||
let ast = parse("CREATE TRIGGER log_insert BEFORE INSERT ON users AS INSERT INTO audit_log VALUES ('insert', 'users')")
|
||||
check ast.stmts.len == 1
|
||||
check ast.stmts[0].kind == nkCreateTrigger
|
||||
check ast.stmts[0].trigName == "log_insert"
|
||||
check ast.stmts[0].trigTable == "users"
|
||||
check ast.stmts[0].trigTiming == "before"
|
||||
check ast.stmts[0].trigEvent == "INSERT"
|
||||
check ast.stmts[0].trigAction.strVal.contains("INSERT")
|
||||
check ast.stmts[0].trigAction.strVal.contains("audit_log")
|
||||
|
||||
test "Parse CREATE TRIGGER AFTER UPDATE":
|
||||
let ast = parse("CREATE TRIGGER audit_update AFTER UPDATE ON orders AS INSERT INTO audit VALUES ('updated')")
|
||||
check ast.stmts[0].kind == nkCreateTrigger
|
||||
check ast.stmts[0].trigTiming == "after"
|
||||
check ast.stmts[0].trigEvent == "UPDATE"
|
||||
|
||||
test "Parse CREATE TRIGGER INSTEAD OF DELETE":
|
||||
let ast = parse("CREATE TRIGGER soft_delete INSTEAD OF DELETE ON users AS UPDATE users SET deleted = true WHERE id = OLD.id")
|
||||
check ast.stmts[0].kind == nkCreateTrigger
|
||||
check ast.stmts[0].trigTiming == "instead of"
|
||||
check ast.stmts[0].trigEvent == "DELETE"
|
||||
|
||||
test "Parse DROP TRIGGER":
|
||||
let ast = parse("DROP TRIGGER log_insert")
|
||||
check ast.stmts.len == 1
|
||||
check ast.stmts[0].kind == nkDropTrigger
|
||||
check ast.stmts[0].trigDropName == "log_insert"
|
||||
check ast.stmts[0].trigDropIfExists == false
|
||||
|
||||
test "Parse DROP TRIGGER IF EXISTS":
|
||||
let ast = parse("DROP TRIGGER IF EXISTS old_trigger")
|
||||
check ast.stmts[0].kind == nkDropTrigger
|
||||
check ast.stmts[0].trigDropName == "old_trigger"
|
||||
check ast.stmts[0].trigDropIfExists == true
|
||||
|
||||
suite "Row-Level Security":
|
||||
test "Parse CREATE USER":
|
||||
let ast = parse("CREATE USER admin WITH PASSWORD 'secret' SUPERUSER")
|
||||
check ast.stmts.len == 1
|
||||
check ast.stmts[0].kind == nkCreateUser
|
||||
check ast.stmts[0].cuName == "admin"
|
||||
check ast.stmts[0].cuPassword == "secret"
|
||||
check ast.stmts[0].cuSuperuser == true
|
||||
|
||||
test "Parse CREATE USER without superuser":
|
||||
let ast = parse("CREATE USER reader WITH PASSWORD 'reader123'")
|
||||
check ast.stmts[0].kind == nkCreateUser
|
||||
check ast.stmts[0].cuName == "reader"
|
||||
check ast.stmts[0].cuPassword == "reader123"
|
||||
check ast.stmts[0].cuSuperuser == false
|
||||
|
||||
test "Parse DROP USER":
|
||||
let ast = parse("DROP USER admin")
|
||||
check ast.stmts[0].kind == nkDropUser
|
||||
check ast.stmts[0].duName == "admin"
|
||||
|
||||
test "Parse CREATE POLICY":
|
||||
let ast = parse("CREATE POLICY user_isolation ON accounts FOR SELECT USING (user_id = current_user)")
|
||||
check ast.stmts[0].kind == nkCreatePolicy
|
||||
check ast.stmts[0].cpName == "user_isolation"
|
||||
check ast.stmts[0].cpTable == "accounts"
|
||||
check ast.stmts[0].cpCommand == "SELECT"
|
||||
|
||||
test "Parse CREATE POLICY with WITH CHECK":
|
||||
let ast = parse("CREATE POLICY insert_check ON accounts FOR INSERT WITH CHECK (amount > 0)")
|
||||
check ast.stmts[0].kind == nkCreatePolicy
|
||||
check ast.stmts[0].cpCommand == "INSERT"
|
||||
|
||||
test "Parse DROP POLICY":
|
||||
let ast = parse("DROP POLICY user_isolation ON accounts")
|
||||
check ast.stmts[0].kind == nkDropPolicy
|
||||
check ast.stmts[0].dpName == "user_isolation"
|
||||
check ast.stmts[0].dpTable == "accounts"
|
||||
|
||||
test "Parse GRANT":
|
||||
let ast = parse("GRANT SELECT ON accounts TO reader")
|
||||
check ast.stmts[0].kind == nkGrant
|
||||
check ast.stmts[0].grPrivilege == "SELECT"
|
||||
check ast.stmts[0].grTable == "accounts"
|
||||
check ast.stmts[0].grGrantee == "reader"
|
||||
|
||||
test "Parse REVOKE":
|
||||
let ast = parse("REVOKE INSERT ON accounts FROM reader")
|
||||
check ast.stmts[0].kind == nkRevoke
|
||||
check ast.stmts[0].rvPrivilege == "INSERT"
|
||||
check ast.stmts[0].rvTable == "accounts"
|
||||
check ast.stmts[0].rvGrantee == "reader"
|
||||
|
||||
test "Parse ENABLE ROW LEVEL SECURITY":
|
||||
let ast = parse("ALTER TABLE accounts ENABLE ROW LEVEL SECURITY")
|
||||
check ast.stmts[0].kind == nkEnableRLS
|
||||
check ast.stmts[0].erlsTable == "accounts"
|
||||
|
||||
test "Parse DISABLE ROW LEVEL SECURITY":
|
||||
let ast = parse("ALTER TABLE accounts DISABLE ROW LEVEL SECURITY")
|
||||
check ast.stmts[0].kind == nkDisableRLS
|
||||
check ast.stmts[0].drlsTable == "accounts"
|
||||
|
||||
test "RLS filter on SELECT":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
# Create table and insert data
|
||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE docs (id INTEGER, owner TEXT)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO docs (id, owner) VALUES (1, 'alice'), (2, 'bob')"))
|
||||
# Create user and policy
|
||||
ctx.currentUser = "alice"
|
||||
ctx.users["alice"] = qexec.UserDef(name: "alice", passwordHash: "", isSuperuser: false, roles: @[])
|
||||
ctx.policies["docs"] = @[
|
||||
qexec.PolicyDef(name: "owner_only", tableName: "docs", command: "SELECT",
|
||||
usingExpr: Node(kind: nkBinOp, binOp: bkEq,
|
||||
binLeft: Node(kind: nkIdent, identName: "owner"),
|
||||
binRight: Node(kind: nkStringLit, strVal: "alice")),
|
||||
withCheckExpr: nil)
|
||||
]
|
||||
# Query should only return alice's row
|
||||
let res = qexec.executeQuery(ctx, parse("SELECT id, owner FROM docs"))
|
||||
check res.success
|
||||
check res.rows.len == 1
|
||||
check res.rows[0]["owner"] == "alice"
|
||||
|
||||
test "RLS superuser bypass":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE docs (id INTEGER, owner TEXT)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO docs (id, owner) VALUES (1, 'alice')"))
|
||||
ctx.currentUser = "admin"
|
||||
ctx.users["admin"] = qexec.UserDef(name: "admin", passwordHash: "", isSuperuser: true, roles: @[])
|
||||
ctx.policies["docs"] = @[
|
||||
qexec.PolicyDef(name: "owner_only", tableName: "docs", command: "SELECT",
|
||||
usingExpr: Node(kind: nkBinOp, binOp: bkEq,
|
||||
binLeft: Node(kind: nkIdent, identName: "owner"),
|
||||
binRight: Node(kind: nkStringLit, strVal: "alice")),
|
||||
withCheckExpr: nil)
|
||||
]
|
||||
let res = qexec.executeQuery(ctx, parse("SELECT id, owner FROM docs"))
|
||||
check res.success
|
||||
check res.rows.len == 1 # superuser sees all (only 1 row exists)
|
||||
|
||||
suite "UTF-8 Support":
|
||||
test "Tokenize UTF-8 identifiers":
|
||||
let tokens = lex.tokenize("SELECT имя FROM потребители")
|
||||
check tokens[1].kind == tkIdent
|
||||
check tokens[1].value == "имя"
|
||||
check tokens[3].kind == tkIdent
|
||||
check tokens[3].value == "потребители"
|
||||
|
||||
test "Parse UTF-8 table and column names":
|
||||
let ast = parse("SELECT имя, възраст FROM потребители WHERE град = 'София'")
|
||||
check ast.stmts[0].kind == nkSelect
|
||||
check ast.stmts[0].selFrom.fromTable == "потребители"
|
||||
check ast.stmts[0].selResult[0].identName == "имя"
|
||||
check ast.stmts[0].selWhere.whereExpr.binRight.strVal == "София"
|
||||
|
||||
test "Execute query with UTF-8 data":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE потребители (имя TEXT, град TEXT)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO потребители (имя, град) VALUES ('Иван', 'София'), ('Мария', 'Пловдив')"))
|
||||
let res = qexec.executeQuery(ctx, parse("SELECT имя, град FROM потребители WHERE град = 'София'"))
|
||||
check res.success
|
||||
check res.rows.len == 1
|
||||
check res.rows[0]["имя"] == "Иван"
|
||||
check res.rows[0]["град"] == "София"
|
||||
|
||||
suite "Enhanced Migrations":
|
||||
test "Parse CREATE MIGRATION with UP/DOWN":
|
||||
let ast = parse("CREATE MIGRATION add_users { UP: CREATE TABLE users (id INTEGER PRIMARY KEY); DOWN: DROP TABLE users; }")
|
||||
check ast.stmts.len == 1
|
||||
check ast.stmts[0].kind == nkCreateMigration
|
||||
check ast.stmts[0].cmName == "add_users"
|
||||
check ast.stmts[0].cmBody.contains("CREATE TABLE users")
|
||||
check ast.stmts[0].cmDownBody.contains("DROP TABLE users")
|
||||
|
||||
test "Parse MIGRATION STATUS":
|
||||
let ast = parse("MIGRATION STATUS")
|
||||
check ast.stmts[0].kind == nkMigrationStatus
|
||||
|
||||
test "Parse MIGRATION UP":
|
||||
let ast = parse("MIGRATION UP")
|
||||
check ast.stmts[0].kind == nkMigrationUp
|
||||
check ast.stmts[0].muCount == 0
|
||||
|
||||
test "Parse MIGRATION UP 5":
|
||||
let ast = parse("MIGRATION UP 5")
|
||||
check ast.stmts[0].kind == nkMigrationUp
|
||||
check ast.stmts[0].muCount == 5
|
||||
|
||||
test "Parse MIGRATION DOWN":
|
||||
let ast = parse("MIGRATION DOWN")
|
||||
check ast.stmts[0].kind == nkMigrationDown
|
||||
check ast.stmts[0].mdCount == 1
|
||||
|
||||
test "Parse MIGRATION DOWN 3":
|
||||
let ast = parse("MIGRATION DOWN 3")
|
||||
check ast.stmts[0].kind == nkMigrationDown
|
||||
check ast.stmts[0].mdCount == 3
|
||||
|
||||
test "Parse MIGRATION DRYRUN":
|
||||
let ast = parse("MIGRATION DRYRUN add_users")
|
||||
check ast.stmts[0].kind == nkMigrationDryRun
|
||||
check ast.stmts[0].mdrName == "add_users"
|
||||
|
||||
test "Create and apply migration with checksum":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
# Create migration
|
||||
let createRes = qexec.executeQuery(ctx, parse("CREATE MIGRATION add_users { UP: CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT); DOWN: DROP TABLE users; }"))
|
||||
check createRes.success
|
||||
check createRes.message.contains("checksum")
|
||||
# Apply migration
|
||||
let applyRes = qexec.executeQuery(ctx, parse("APPLY MIGRATION add_users"))
|
||||
check applyRes.success
|
||||
check applyRes.message.contains("ms")
|
||||
# Check table exists
|
||||
let tableRes = qexec.executeQuery(ctx, parse("SELECT name FROM users"))
|
||||
check tableRes.success # table exists (empty result is OK)
|
||||
# Re-apply should be idempotent
|
||||
let reapplyRes = qexec.executeQuery(ctx, parse("APPLY MIGRATION add_users"))
|
||||
check reapplyRes.success
|
||||
check reapplyRes.message.contains("already applied")
|
||||
|
||||
test "Migration STATUS shows applied migrations":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE MIGRATION m1 { UP: CREATE TABLE t1 (id INTEGER); }"))
|
||||
discard qexec.executeQuery(ctx, parse("CREATE MIGRATION m2 { UP: CREATE TABLE t2 (id INTEGER); }"))
|
||||
discard qexec.executeQuery(ctx, parse("APPLY MIGRATION m1"))
|
||||
let statusRes = qexec.executeQuery(ctx, parse("MIGRATION STATUS"))
|
||||
check statusRes.success
|
||||
check statusRes.rows.len == 2
|
||||
check statusRes.rows[0]["status"] == "applied"
|
||||
check statusRes.rows[1]["status"] == "pending"
|
||||
|
||||
test "Migration UP applies all pending":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE MIGRATION m1 { UP: CREATE TABLE t1 (id INTEGER); }"))
|
||||
discard qexec.executeQuery(ctx, parse("CREATE MIGRATION m2 { UP: CREATE TABLE t2 (id INTEGER); }"))
|
||||
let upRes = qexec.executeQuery(ctx, parse("MIGRATION UP"))
|
||||
check upRes.success
|
||||
check upRes.message.contains("Applied 2 migrations")
|
||||
|
||||
test "Migration DOWN rollback":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE MIGRATION add_t { UP: CREATE TABLE t (id INTEGER); DOWN: DROP TABLE t; }"))
|
||||
discard qexec.executeQuery(ctx, parse("APPLY MIGRATION add_t"))
|
||||
let downRes = qexec.executeQuery(ctx, parse("MIGRATION DOWN"))
|
||||
check downRes.success
|
||||
check downRes.message.contains("Rolled back 1 migrations")
|
||||
# After rollback, table should be gone (check by listing tables)
|
||||
let tableRes = qexec.executeQuery(ctx, parse("SELECT name FROM __tables WHERE name = 't'"))
|
||||
check tableRes.success
|
||||
check tableRes.rows.len == 0 # table does not exist
|
||||
|
||||
test "Migration DRYRUN":
|
||||
var db = newLSMTree("")
|
||||
var ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE MIGRATION add_t { UP: CREATE TABLE t (id INTEGER); CREATE INDEX idx ON t(id); DOWN: DROP TABLE t; }"))
|
||||
let dryRes = qexec.executeQuery(ctx, parse("MIGRATION DRYRUN add_t"))
|
||||
check dryRes.success
|
||||
check dryRes.message.contains("DRY RUN")
|
||||
check dryRes.message.contains("Statements: 2")
|
||||
check dryRes.message.contains("DOWN script: yes")
|
||||
|
||||
suite "Parameterized queries":
|
||||
var db: LSMTree
|
||||
var ctx: qexec.ExecutionContext
|
||||
|
||||
setup:
|
||||
db = newLSMTree("")
|
||||
ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE users (id INT, name TEXT, age INT)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO users (id, name, age) VALUES (1, 'Alice', 30)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO users (id, name, age) VALUES (2, 'Bob', 25)"))
|
||||
|
||||
test "SELECT with placeholder params":
|
||||
let sql = "SELECT * FROM users WHERE id = ?"
|
||||
let tokens = lex.tokenize(sql)
|
||||
let ast = parse(tokens)
|
||||
let params = @[WireValue(kind: fkInt64, int64Val: 1)]
|
||||
let r = qexec.executeQuery(ctx, ast, params)
|
||||
check r.success
|
||||
check r.rows.len == 1
|
||||
check r.rows[0]["name"] == "Alice"
|
||||
|
||||
test "INSERT with placeholder params":
|
||||
let sql = "INSERT INTO users (id, name, age) VALUES (?, ?, ?)"
|
||||
let tokens = lex.tokenize(sql)
|
||||
let ast = parse(tokens)
|
||||
let params = @[
|
||||
WireValue(kind: fkInt64, int64Val: 3),
|
||||
WireValue(kind: fkString, strVal: "Charlie"),
|
||||
WireValue(kind: fkInt64, int64Val: 35)
|
||||
]
|
||||
let r = qexec.executeQuery(ctx, ast, params)
|
||||
check r.success
|
||||
let selectR = qexec.executeQuery(ctx, parse("SELECT * FROM users WHERE id = 3"))
|
||||
check selectR.rows.len == 1
|
||||
check selectR.rows[0]["name"] == "Charlie"
|
||||
|
||||
test "SELECT with multiple placeholders":
|
||||
let sql = "SELECT * FROM users WHERE age > ? AND name = ?"
|
||||
let tokens = lex.tokenize(sql)
|
||||
let ast = parse(tokens)
|
||||
let params = @[WireValue(kind: fkInt64, int64Val: 25), WireValue(kind: fkString, strVal: "Alice")]
|
||||
let r = qexec.executeQuery(ctx, ast, params)
|
||||
check r.success
|
||||
check r.rows.len == 1
|
||||
check r.rows[0]["name"] == "Alice"
|
||||
|
||||
# JOIN tests
|
||||
include "join_tests"
|
||||
|
||||
Reference in New Issue
Block a user