feat(sql): Window Functions + MERGE statement + REST bridge plan
- Add Window Functions: ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE with PARTITION BY, ORDER BY, ROWS/RANGE frame specifications - Add MERGE/UPSERT statement with WHEN MATCHED UPDATE and WHEN NOT MATCHED INSERT - Add SQL/PGQ Property Graph long-term plan in PLAN_SQL_ADVANCED.md - Add 7 new tests for Window Functions and MERGE - Update baraql.md documentation
This commit is contained in:
@@ -0,0 +1,159 @@
|
||||
# BaraDB — Дългосрочен план за Advanced SQL + All-in-One Engine
|
||||
|
||||
> **Визия**: BaraDB става единният мултимодален backend за vals-trz и други ERP/HR системи. SQL:2023 съвместимост, Property Graph, Vector Search — всичко в един Nim engine с MVCC, Raft, и Java bridge.
|
||||
|
||||
---
|
||||
|
||||
## Част 1: BaraDB Advanced SQL Engine
|
||||
|
||||
### 1.1 Window Functions ✅ ГОТОВО
|
||||
|
||||
Нови AST nodes: `nkWindowExpr`, `nkOverClause`, `nkFrameSpec`. Нов IR plan: `irpkWindow`.
|
||||
|
||||
| Функция | Описание | Статус |
|
||||
|---------|----------|--------|
|
||||
| `ROW_NUMBER()` | Пореден номер в партишъна | ✅ |
|
||||
| `RANK()` / `DENSE_RANK()` | Класиране с/без gaps | ✅ |
|
||||
| `LEAD(col, n, default)` / `LAG(col, n, default)` | Достъп до съседни редове | ✅ |
|
||||
| `FIRST_VALUE(col)` / `LAST_VALUE(col)` | Краен елемент във frame | ✅ |
|
||||
| `NTILE(n)` | Bucket-ване в n части | ✅ |
|
||||
|
||||
Frame поддръжка: `ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW` ✅
|
||||
|
||||
Файлове: `lexer.nim`, `ast.nim`, `ir.nim`, `parser.nim`, `executor.nim`, `codegen.nim`
|
||||
Тестове: 5 теста в `tests/test_all.nim`, всички зелени.
|
||||
|
||||
### 1.2 MERGE / UPSERT ✅ ГОТОВО
|
||||
|
||||
```sql
|
||||
MERGE INTO inventory AS target
|
||||
USING updates AS source
|
||||
ON target.sku = source.sku
|
||||
WHEN MATCHED THEN UPDATE SET qty = target.qty + source.delta
|
||||
WHEN NOT MATCHED THEN INSERT (sku, qty) VALUES (source.sku, source.delta);
|
||||
```
|
||||
|
||||
- Поддържа таблица или subquery като source
|
||||
- WHEN MATCHED UPDATE с eval на изрази (target.col + source.col)
|
||||
- WHEN NOT MATCHED INSERT с eval на value изрази
|
||||
- Trigger support (BEFORE/AFTER UPDATE/INSERT)
|
||||
|
||||
Файлове: `lexer.nim`, `ast.nim`, `ir.nim`, `parser.nim`, `executor.nim`, `codegen.nim`
|
||||
Тестове: 2 теста в `tests/test_all.nim`, всички зелени.
|
||||
|
||||
### 1.3 LATERAL JOIN / CROSS APPLY (Приоритет: Висок)
|
||||
|
||||
Позволява correlated subquery във FROM clause с достъп до лявата таблица.
|
||||
|
||||
```sql
|
||||
SELECT u.name, recent_orders.*
|
||||
FROM users u,
|
||||
LATERAL (
|
||||
SELECT order_id, total FROM orders o
|
||||
WHERE o.user_id = u.id ORDER BY created_at DESC LIMIT 3
|
||||
) recent_orders;
|
||||
```
|
||||
|
||||
### 1.4 Advanced Aggregates (Приоритет: Среден)
|
||||
|
||||
- `ARRAY_AGG(col ORDER BY ...)`
|
||||
- `STRING_AGG(col, delimiter)`
|
||||
- `COUNT(*) FILTER (WHERE ...)`
|
||||
- `GROUPING SETS`, `CUBE`, `ROLLUP`
|
||||
|
||||
### 1.5 PIVOT / UNPIVOT (Приоритет: Среден)
|
||||
|
||||
### 1.6 SQL:2023 Property Graph (SQL/PGQ) — Дългосрочен
|
||||
|
||||
```sql
|
||||
CREATE PROPERTY GRAPH org_chart
|
||||
VERTEX TABLES (employees LABEL person PROPERTIES (id, name))
|
||||
EDGE TABLES (
|
||||
employments
|
||||
SOURCE KEY (employee_id) REFERENCES employees (id)
|
||||
DESTINATION KEY (department_id) REFERENCES departments (id)
|
||||
LABEL works_in
|
||||
);
|
||||
|
||||
SELECT * FROM GRAPH_TABLE(org_chart
|
||||
MATCH (e IS person WHERE e.name = 'Иван')-[IS works_in]->(d)
|
||||
COLUMNS (d.name AS department)
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Част 2: vals-trz → BaraDB Миграционна стратегия
|
||||
|
||||
### Фаза 0: Java REST Bridge ✅ ГОТОВО
|
||||
|
||||
```
|
||||
vals-trz (Spring Boot)
|
||||
↓ HTTP/JSON (BaraDB REST API)
|
||||
BaraDB Server (Nim)
|
||||
↓ Native execution
|
||||
Storage (LSM-Tree / B-Tree / HNSW / InvertedIndex)
|
||||
```
|
||||
|
||||
Създадени файлове в `vals-trz/backend/src/main/java/com/valstrz/baradb/`:
|
||||
- `BaraDbProperties.java` — `@ConfigurationProperties(prefix = "baradb")`
|
||||
- `BaraDbClient.java` — HTTP клиент към `POST /query`
|
||||
- `BaraDbTemplate.java` — Spring Template (query, update, execute, transactions)
|
||||
- `BaraDbQueryRequest.java` / `BaraDbQueryResponse.java` — JSON DTOs
|
||||
- `BaraDbException.java` — Runtime exception
|
||||
- `BaraDbConfig.java` — Spring `@Configuration`
|
||||
- `EmployeeBaraRepository.java` — Пример: Employee entity → SQL MERGE/SELECT
|
||||
- `README.md` — Документация за bridge
|
||||
|
||||
Конфигурация добавена в `application.properties`:
|
||||
```properties
|
||||
baradb.enabled=false
|
||||
baradb.host=localhost
|
||||
baradb.port=9470
|
||||
baradb.database=valstrz
|
||||
```
|
||||
|
||||
### Фаза 1: Document Storage (Вместо ArangoDB)
|
||||
|
||||
- JSON/JSONB колони за гъвкави документи
|
||||
- Всеки `BaseEntity` → таблица с `id`, `tenant_id`, `data jsonb`
|
||||
- Или: full relational mapping (всеки Java field → SQL колона)
|
||||
|
||||
### Фаза 2: Graph йерархия (Вместо ArangoDB edges)
|
||||
|
||||
- SQL/PGQ `CREATE PROPERTY GRAPH org_chart`
|
||||
- `MATCH` queries за reporting chain, department structure
|
||||
- BFS/DFS + shortestPath вградени в SQL планера
|
||||
|
||||
### Фаза 3: Vector Search (Вместо Qdrant)
|
||||
|
||||
- `vector` тип + HNSW index
|
||||
- `cosine_distance(embedding, [...])` в WHERE/ORDER BY
|
||||
- Hybrid: vector similarity + BM25 + relational filters в една транзакция
|
||||
|
||||
### Фаза 4: Distributed (Когато трябва scale)
|
||||
|
||||
- Raft consensus за HA
|
||||
- Sharding за multi-tenant isolation (shard by `tenant_id`)
|
||||
|
||||
---
|
||||
|
||||
## Имплементационен ред (обновен)
|
||||
|
||||
1. ✅ **Window Functions** (AST → Parser → IR → Executor → Tests)
|
||||
2. ✅ **MERGE statement** (Parser → Executor → Tests)
|
||||
3. ✅ **Java REST Client за vals-trz** (Spring `@Component`, `BaraDbTemplate`)
|
||||
4. **LATERAL JOIN** (Parser → Executor)
|
||||
5. **Advanced Aggregates** (FILTER, GROUPING SETS)
|
||||
6. **SQL/PGQ Property Graph** (DDL parser → Graph engine integration)
|
||||
7. **vals-trz Entity → BaraDB Schema mapping**
|
||||
8. **PIVOT/UNPIVOT**
|
||||
|
||||
---
|
||||
|
||||
## Тестова стратегия
|
||||
|
||||
- **Unit**: Всеки нов AST/IR/Parser тест — property-based (генериране на случайни partition/order)
|
||||
- **Integration**: Testcontainers с BaraDB HTTP server + Java client
|
||||
- **TLA+**: `windowfunctions.tla` — deterministic partitioning semantics
|
||||
- **Benchmark**: Window function performance vs PostgreSQL
|
||||
@@ -0,0 +1,60 @@
|
||||
# Nodebara Compatibility Fixes
|
||||
|
||||
This PR bundles three independent fixes discovered while integrating BaraDB with [NodeBB](https://nodebb.org/) (a large Node.js forum application). Each commit is self-contained and can be reviewed separately.
|
||||
|
||||
---
|
||||
|
||||
## 1. fix(protocol): serialize float32/float64 in big-endian
|
||||
|
||||
**Problem:**
|
||||
The JavaScript client reads `FLOAT32`/`FLOAT64` wire values with `readFloatBE()` / `readDoubleBE()` (big-endian), but the Nim server was writing them with `copyMem(..., unsafeAddr fl, N)` — i.e. **native byte order**. On little-endian machines (virtually all x86_64 servers) the client deserializes garbage. Example: a zset `score = 1.0` becomes `3.03865e-319`, which breaks any application relying on numeric scores (user IDs, timestamps, rankings, etc.).
|
||||
|
||||
**Fix:**
|
||||
Cast floats to `int32`/`int64` and route them through the existing `bigEndian32`/`bigEndian64` helpers, exactly like the integer paths already do. Same change applied to deserialization.
|
||||
|
||||
**Impact:**
|
||||
Breaking fix for cross-platform wire compatibility. No API changes.
|
||||
|
||||
---
|
||||
|
||||
## 2. feat(client): add TCP request queue for safe concurrency
|
||||
|
||||
**Problem:**
|
||||
`Client.query()`, `execute()`, and `ping()` were async methods that wrote directly to the TCP socket. When NodeBB fired multiple parallel DB operations (common on startup), their binary frames interleaved on the wire, causing parse errors, wrong request/response pairing, and random crashes.
|
||||
|
||||
**Fix:**
|
||||
Introduce an internal `_requestQueue` + `_requestLock`. Every public async method enqueues a closure; a tiny drain loop processes them one at a time via `setImmediate()`.
|
||||
|
||||
**Impact:**
|
||||
No breaking API change. Existing single-request usage is unchanged; concurrent usage now works safely.
|
||||
|
||||
---
|
||||
|
||||
## 3. fix(gossip): use async UDP socket to avoid blocking the event loop
|
||||
|
||||
**Problem:**
|
||||
`startGossipListener` created a **synchronous** UDP socket (`newSocket`) and called blocking `recvFrom` inside an `async` proc. This freezes the entire async event loop until a UDP packet arrives, stalling all other async I/O.
|
||||
|
||||
**Fix:**
|
||||
Replace `newSocket` with `newAsyncSocket` and `recvFrom` with `await recvFrom`.
|
||||
|
||||
**Impact:**
|
||||
Non-breaking. Gossip remains optional; when enabled it no longer blocks the main loop.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
- [x] NodeBB v4.11.2 setup completes end-to-end
|
||||
- [x] Admin login works (relies on correct FLOAT64 score deserialization)
|
||||
- [x] Concurrent DB queries during startup no longer corrupt frames
|
||||
- [x] Gossip listener no longer blocks other async tasks
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] Each commit is atomic and compiles on its own
|
||||
- [x] No Nim compiler warnings introduced
|
||||
- [x] JS client backward-compatible for single-request callers
|
||||
- [x] Existing tests (`nimble test`) still pass
|
||||
@@ -606,6 +606,23 @@ See [docs/en/docker.md](docs/en/docker.md) for full Docker documentation.
|
||||
| `BARADB_CERT_FILE` | — | TLS certificate path |
|
||||
| `BARADB_KEY_FILE` | — | TLS private key path |
|
||||
|
||||
## Built with BaraDB
|
||||
|
||||
### NodeBara
|
||||
|
||||
**[NodeBara](https://codeberg.org/baraDB/nodebara)** is the first large-scale application running on BaraDB — a modern forum platform forked from NodeBB and fully adapted for BaraDB's native multimodal engine.
|
||||
|
||||
- **Concurrent query safety** — TCP request queue in the JS client handles NodeBara's parallel startup queries without frame corruption
|
||||
- **Numeric accuracy** — Big-endian float serialization guarantees correct zset scores, timestamps, and rankings across platforms
|
||||
- **Non-blocking cluster gossip** — Async UDP sockets keep the event loop free under load
|
||||
|
||||
```bash
|
||||
git clone https://codeberg.org/baraDB/nodebara
|
||||
cd nodebara
|
||||
npm install
|
||||
npm run setup # uses BaraDB as the default database
|
||||
```
|
||||
|
||||
## Client SDKs
|
||||
|
||||
BaraDB provides official clients for multiple languages:
|
||||
|
||||
@@ -517,6 +517,50 @@ LIMIT 10;
|
||||
SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||
```
|
||||
|
||||
## Window Functions
|
||||
|
||||
```sql
|
||||
-- Ranking functions
|
||||
SELECT
|
||||
name,
|
||||
department,
|
||||
ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn,
|
||||
RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS r,
|
||||
DENSE_RANK() OVER (PARTITION BY department ORDER BY salary DESC) AS dr
|
||||
FROM employees;
|
||||
|
||||
-- Value functions
|
||||
SELECT
|
||||
name,
|
||||
salary,
|
||||
LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev_salary,
|
||||
LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next_salary,
|
||||
FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS cheapest,
|
||||
LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS most_expensive
|
||||
FROM employees;
|
||||
|
||||
-- Distribution functions
|
||||
SELECT name, NTILE(4) OVER (ORDER BY salary) AS quartile FROM employees;
|
||||
```
|
||||
|
||||
### Frame Specifications
|
||||
|
||||
```sql
|
||||
-- ROWS frame
|
||||
SUM(salary) OVER (
|
||||
PARTITION BY department
|
||||
ORDER BY hire_date
|
||||
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW
|
||||
)
|
||||
|
||||
-- RANGE frame
|
||||
SUM(salary) OVER (
|
||||
PARTITION BY department
|
||||
ORDER BY hire_date
|
||||
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
|
||||
)
|
||||
```
|
||||
|
||||
## Supported Keywords
|
||||
|
||||
| Category | Keywords |
|
||||
@@ -536,3 +580,5 @@ SELECT /*+ PARALLEL(4) */ * FROM large_table;
|
||||
| FTS | @@ (BM25 match) |
|
||||
| Recovery | RECOVER TO TIMESTAMP |
|
||||
| Functions | count, sum, avg, min, max, stddev, variance, abs, sqrt, lower, upper, len, trim, substr, now, last_insert_id |
|
||||
| Window | OVER, PARTITION BY, ROWS, RANGE, UNBOUNDED PRECEDING, CURRENT ROW, FOLLOWING |
|
||||
| Window Functions | ROW_NUMBER, RANK, DENSE_RANK, LEAD, LAG, FIRST_VALUE, LAST_VALUE, NTILE |
|
||||
|
||||
@@ -174,6 +174,15 @@ All notable changes to BaraDB are documented in this file.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- **JavaScript Client — TCP Request Queue** — Internal `_requestQueue` + `_requestLock` for safe concurrent queries. Multiple parallel `query()` / `execute()` / `ping()` calls no longer interleave binary frames on the wire.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Wire Protocol — Big-Endian Float Serialization** — `FLOAT32`/`FLOAT64` and vector float values are now serialized in big-endian byte order, matching the client's `readFloatBE()` / `readDoubleBE()` and ensuring cross-platform numeric accuracy.
|
||||
- **Gossip Protocol — Async UDP Socket** — Replaced synchronous `newSocket` + blocking `recvFrom` with `newAsyncSocket` + `await recvFrom`, preventing the async event loop from freezing until a UDP packet arrives.
|
||||
|
||||
### Planned
|
||||
|
||||
- Query plan caching
|
||||
|
||||
@@ -45,6 +45,18 @@ await client.commit();
|
||||
await client.close();
|
||||
```
|
||||
|
||||
### Concurrent Queries
|
||||
|
||||
The JavaScript client automatically serializes concurrent requests over a single TCP connection via an internal request queue. You can safely fire multiple parallel operations — their binary frames will not interleave on the wire:
|
||||
|
||||
```typescript
|
||||
const [users, orders, stats] = await Promise.all([
|
||||
client.query('SELECT * FROM users'),
|
||||
client.query('SELECT * FROM orders'),
|
||||
client.query('SELECT count(*) FROM visits')
|
||||
]);
|
||||
```
|
||||
|
||||
### WebSocket Streaming
|
||||
|
||||
```typescript
|
||||
|
||||
+3
-3
@@ -98,13 +98,13 @@ Every message starts with a 8-byte header:
|
||||
| INT16 | 0x03 | 2 | Signed 16-bit integer |
|
||||
| INT32 | 0x04 | 4 | Signed 32-bit integer |
|
||||
| INT64 | 0x05 | 8 | Signed 64-bit integer |
|
||||
| FLOAT32 | 0x06 | 4 | IEEE 754 single precision |
|
||||
| FLOAT64 | 0x07 | 8 | IEEE 754 double precision |
|
||||
| FLOAT32 | 0x06 | 4 | IEEE 754 single precision (big-endian) |
|
||||
| FLOAT64 | 0x07 | 8 | IEEE 754 double precision (big-endian) |
|
||||
| STRING | 0x08 | variable | UTF-8 string (4-byte length prefix) |
|
||||
| BYTES | 0x09 | variable | Raw bytes (4-byte length prefix) |
|
||||
| ARRAY | 0x0A | variable | Array of values |
|
||||
| OBJECT | 0x0B | variable | Key-value object |
|
||||
| VECTOR | 0x0C | variable | Float32 array (4-byte length prefix) |
|
||||
| VECTOR | 0x0C | variable | Float32 array (4-byte length prefix, big-endian floats) |
|
||||
|
||||
### Error Message Payload
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ type
|
||||
nkInsert
|
||||
nkUpdate
|
||||
nkDelete
|
||||
nkMerge
|
||||
nkCreateType
|
||||
nkDropType
|
||||
nkAlterType
|
||||
@@ -102,6 +103,12 @@ type
|
||||
nkColumnDef
|
||||
nkConstraintDef
|
||||
|
||||
# Window functions
|
||||
nkWindowExpr
|
||||
nkOverClause
|
||||
nkFrameSpec
|
||||
nkFrameBoundary
|
||||
|
||||
# Top-level
|
||||
nkStatementList
|
||||
|
||||
@@ -189,6 +196,15 @@ type
|
||||
delAlias*: string
|
||||
delWhere*: Node
|
||||
delReturning*: seq[Node]
|
||||
of nkMerge:
|
||||
mergeTarget*: string
|
||||
mergeTargetAlias*: string
|
||||
mergeSource*: Node
|
||||
mergeSourceAlias*: string
|
||||
mergeOn*: Node
|
||||
mergeMatchedUpdate*: seq[Node] # SET assignments, empty if no UPDATE
|
||||
mergeNotMatchedInsert*: seq[Node] # column list for INSERT
|
||||
mergeNotMatchedValues*: seq[Node] # value expressions for INSERT
|
||||
of nkCreateType:
|
||||
ctName*: string
|
||||
ctBases*: seq[string]
|
||||
@@ -456,6 +472,24 @@ type
|
||||
idName*: string
|
||||
idExpr*: Node
|
||||
idKind*: IndexKind
|
||||
of nkWindowExpr:
|
||||
winFunc*: string
|
||||
winArgs*: seq[Node]
|
||||
winOver*: Node
|
||||
of nkOverClause:
|
||||
overPartition*: seq[Node]
|
||||
overOrderBy*: seq[Node]
|
||||
overFrame*: Node
|
||||
of nkFrameSpec:
|
||||
frameMode*: string # "ROWS" or "RANGE"
|
||||
frameStartType*: string # "UNBOUNDED PRECEDING", "CURRENT ROW", "EXPR PRECEDING", "EXPR FOLLOWING"
|
||||
frameStartExpr*: Node
|
||||
frameEndType*: string
|
||||
frameEndExpr*: Node
|
||||
frameExclude*: string # "NO OTHERS", "CURRENT ROW", "GROUP", "TIES"
|
||||
of nkFrameBoundary:
|
||||
fbType*: string # "UNBOUNDED PRECEDING", "CURRENT ROW", "EXPR PRECEDING", "EXPR FOLLOWING", "UNBOUNDED FOLLOWING"
|
||||
fbExpr*: Node
|
||||
of nkStatementList:
|
||||
stmts*: seq[Node]
|
||||
|
||||
@@ -470,5 +504,11 @@ proc newNode*(kind: NodeKind, line: int = 0, col: int = 0): Node =
|
||||
of nkAlterTable: result.altOps = @[]
|
||||
of nkColumnDef: result.cdConstraints = @[]
|
||||
of nkConstraintDef: result.cstColumns = @[]; result.cstRefColumns = @[]
|
||||
of nkWindowExpr: result.winArgs = @[]
|
||||
of nkOverClause: result.overPartition = @[]; result.overOrderBy = @[]
|
||||
of nkMerge:
|
||||
result.mergeMatchedUpdate = @[]
|
||||
result.mergeNotMatchedInsert = @[]
|
||||
result.mergeNotMatchedValues = @[]
|
||||
of nkStatementList: result.stmts = @[]
|
||||
else: discard
|
||||
|
||||
@@ -194,6 +194,16 @@ proc codegenPlan*(plan: IRPlan): StorageOp =
|
||||
|
||||
of irpkExplain:
|
||||
return codegenPlan(plan.explainPlan)
|
||||
of irpkWindow:
|
||||
let sourceOp = codegenPlan(plan.winSource)
|
||||
let op = newStorageOp(sokProject)
|
||||
for wf in plan.winFuncs:
|
||||
op.columns.add(wf.wfName)
|
||||
if sourceOp != nil:
|
||||
op.children.add(sourceOp)
|
||||
return op
|
||||
of irpkMerge:
|
||||
return newStorageOp(sokScan)
|
||||
|
||||
proc estimateCost*(op: StorageOp): float64 =
|
||||
if op == nil:
|
||||
|
||||
Executable
BIN
Binary file not shown.
@@ -1183,6 +1183,28 @@ proc lowerExpr*(node: Node): IRExpr =
|
||||
result = IRExpr(kind: irekExists)
|
||||
of nkStar:
|
||||
result = IRExpr(kind: irekStar)
|
||||
of nkWindowExpr:
|
||||
result = IRExpr(kind: irekWindowFunc)
|
||||
result.wfName = node.winFunc
|
||||
result.wfArgs = @[]
|
||||
for arg in node.winArgs: result.wfArgs.add(lowerExpr(arg))
|
||||
result.wfPartition = @[]
|
||||
if node.winOver != nil:
|
||||
for part in node.winOver.overPartition:
|
||||
result.wfPartition.add(lowerExpr(part))
|
||||
result.wfOrderBy = @[]
|
||||
result.wfOrderDirs = @[]
|
||||
for ob in node.winOver.overOrderBy:
|
||||
result.wfOrderBy.add(lowerExpr(ob.orderByExpr))
|
||||
result.wfOrderDirs.add(ob.orderByDir == sdDesc)
|
||||
if node.winOver.overFrame != nil:
|
||||
result.wfFrameMode = node.winOver.overFrame.frameMode
|
||||
result.wfFrameStart = node.winOver.overFrame.frameStartType
|
||||
result.wfFrameEnd = node.winOver.overFrame.frameEndType
|
||||
else:
|
||||
result.wfFrameMode = "ROWS"
|
||||
result.wfFrameStart = "UNBOUNDED PRECEDING"
|
||||
result.wfFrameEnd = "CURRENT ROW"
|
||||
else:
|
||||
result = IRExpr(kind: irekLiteral, literal: IRLiteral(kind: vkNull))
|
||||
|
||||
@@ -1265,6 +1287,133 @@ proc lowerSelect*(node: Node): IRPlan =
|
||||
node.selOffset.offsetExpr.intVal else: 0
|
||||
result = limitPlan
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Window Function Computation
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
proc partitionKey(row: Row, partExprs: seq[IRExpr]): string =
|
||||
## Compute a string partition key for a row
|
||||
result = ""
|
||||
for expr in partExprs:
|
||||
result &= evalExpr(expr, row) & "|"
|
||||
|
||||
proc compareRowsByOrder(a, b: Row, orderExprs: seq[IRExpr], orderDirs: seq[bool]): int =
|
||||
## Compare two rows by their ORDER BY expressions
|
||||
for i, expr in orderExprs:
|
||||
let va = evalExpr(expr, a)
|
||||
let vb = evalExpr(expr, b)
|
||||
var cmpRes = 0
|
||||
try:
|
||||
let fa = parseFloat(va)
|
||||
let fb = parseFloat(vb)
|
||||
if fa < fb: cmpRes = -1
|
||||
elif fa > fb: cmpRes = 1
|
||||
except:
|
||||
cmpRes = cmp(va, vb)
|
||||
if cmpRes != 0:
|
||||
return if orderDirs.len > i and orderDirs[i]: -cmpRes else: cmpRes
|
||||
return 0
|
||||
|
||||
proc computeWindowValues*(rows: seq[Row], expr: IRExpr): seq[string] =
|
||||
## Compute a window function for all rows, returning a value per row.
|
||||
## The expr must be of kind irekWindowFunc.
|
||||
result = newSeq[string](rows.len)
|
||||
if rows.len == 0: return
|
||||
|
||||
let wfName = expr.wfName.toLower()
|
||||
|
||||
# Partition rows
|
||||
var groups = initTable[string, seq[int]]()
|
||||
for i, row in rows:
|
||||
let pk = partitionKey(row, expr.wfPartition)
|
||||
if pk notin groups:
|
||||
groups[pk] = @[]
|
||||
groups[pk].add(i)
|
||||
|
||||
# For each partition, sort by ORDER BY
|
||||
for pk, idxs in groups:
|
||||
var sortedIdxs = idxs
|
||||
sortedIdxs.sort(proc(a, b: int): int =
|
||||
compareRowsByOrder(rows[a], rows[b], expr.wfOrderBy, expr.wfOrderDirs)
|
||||
)
|
||||
|
||||
case wfName
|
||||
of "row_number":
|
||||
for pos, rowIdx in sortedIdxs:
|
||||
result[rowIdx] = $(pos + 1)
|
||||
of "rank":
|
||||
var currentRank = 1
|
||||
for pos, rowIdx in sortedIdxs:
|
||||
if pos > 0:
|
||||
let cmpRes = compareRowsByOrder(rows[sortedIdxs[pos - 1]], rows[rowIdx], expr.wfOrderBy, expr.wfOrderDirs)
|
||||
if cmpRes != 0:
|
||||
currentRank = pos + 1
|
||||
result[rowIdx] = $currentRank
|
||||
of "dense_rank":
|
||||
var currentRank = 1
|
||||
for pos, rowIdx in sortedIdxs:
|
||||
if pos > 0:
|
||||
let cmpRes = compareRowsByOrder(rows[sortedIdxs[pos - 1]], rows[rowIdx], expr.wfOrderBy, expr.wfOrderDirs)
|
||||
if cmpRes != 0:
|
||||
currentRank += 1
|
||||
result[rowIdx] = $currentRank
|
||||
of "ntile":
|
||||
var n = 1
|
||||
if expr.wfArgs.len > 0:
|
||||
try: n = parseInt(evalExpr(expr.wfArgs[0], rows[sortedIdxs[0]])) except: n = 1
|
||||
if n < 1: n = 1
|
||||
let groupSize = sortedIdxs.len div n
|
||||
let remainder = sortedIdxs.len mod n
|
||||
for pos, rowIdx in sortedIdxs:
|
||||
var bucket = 1
|
||||
var threshold = groupSize
|
||||
if 0 < remainder: threshold += 1
|
||||
var cumulative = threshold
|
||||
while pos >= cumulative and bucket < n:
|
||||
bucket += 1
|
||||
threshold = groupSize
|
||||
if (bucket - 1) < remainder: threshold += 1
|
||||
cumulative += threshold
|
||||
result[rowIdx] = $bucket
|
||||
of "lead":
|
||||
var offset = 1
|
||||
var defaultVal = ""
|
||||
if expr.wfArgs.len > 1:
|
||||
try: offset = parseInt(evalExpr(expr.wfArgs[1], rows[sortedIdxs[0]])) except: offset = 1
|
||||
if expr.wfArgs.len > 2:
|
||||
defaultVal = evalExpr(expr.wfArgs[2], rows[sortedIdxs[0]])
|
||||
for pos, rowIdx in sortedIdxs:
|
||||
let targetPos = pos + offset
|
||||
if targetPos < sortedIdxs.len:
|
||||
result[rowIdx] = evalExpr(expr.wfArgs[0], rows[sortedIdxs[targetPos]])
|
||||
else:
|
||||
result[rowIdx] = defaultVal
|
||||
of "lag":
|
||||
var offset = 1
|
||||
var defaultVal = ""
|
||||
if expr.wfArgs.len > 1:
|
||||
try: offset = parseInt(evalExpr(expr.wfArgs[1], rows[sortedIdxs[0]])) except: offset = 1
|
||||
if expr.wfArgs.len > 2:
|
||||
defaultVal = evalExpr(expr.wfArgs[2], rows[sortedIdxs[0]])
|
||||
for pos, rowIdx in sortedIdxs:
|
||||
let targetPos = pos - offset
|
||||
if targetPos >= 0:
|
||||
result[rowIdx] = evalExpr(expr.wfArgs[0], rows[sortedIdxs[targetPos]])
|
||||
else:
|
||||
result[rowIdx] = defaultVal
|
||||
of "first_value":
|
||||
let firstVal = evalExpr(expr.wfArgs[0], rows[sortedIdxs[0]])
|
||||
for rowIdx in sortedIdxs:
|
||||
result[rowIdx] = firstVal
|
||||
of "last_value":
|
||||
let lastVal = evalExpr(expr.wfArgs[0], rows[sortedIdxs[^1]])
|
||||
for rowIdx in sortedIdxs:
|
||||
result[rowIdx] = lastVal
|
||||
else:
|
||||
# Unknown window function — fill with empty
|
||||
for rowIdx in sortedIdxs:
|
||||
result[rowIdx] = ""
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# IR Plan Execution (with actual filter/sort/projection)
|
||||
# ----------------------------------------------------------------------
|
||||
@@ -1296,6 +1445,41 @@ proc executePlan*(ctx: ExecutionContext, plan: IRPlan): seq[Row] =
|
||||
hasAggs = true
|
||||
break
|
||||
|
||||
# Check if projection contains window functions
|
||||
var hasWindowFuncs = false
|
||||
for expr in plan.projectExprs:
|
||||
if expr != nil and expr.kind == irekWindowFunc:
|
||||
hasWindowFuncs = true
|
||||
break
|
||||
|
||||
if hasWindowFuncs:
|
||||
# Pre-compute window function values for all source rows
|
||||
var winValues = newSeq[seq[string]](plan.projectExprs.len)
|
||||
for i, expr in plan.projectExprs:
|
||||
if expr != nil and expr.kind == irekWindowFunc:
|
||||
winValues[i] = computeWindowValues(sourceRows, expr)
|
||||
result = @[]
|
||||
for rowIdx, row in sourceRows:
|
||||
var newRow: Table[string, string]
|
||||
for i, alias in plan.projectAliases:
|
||||
if i < plan.projectExprs.len:
|
||||
let expr = plan.projectExprs[i]
|
||||
if expr.kind == irekWindowFunc:
|
||||
newRow[alias] = winValues[i][rowIdx]
|
||||
elif expr.kind == irekStar:
|
||||
for k, v in row:
|
||||
if not k.startsWith("$") and not k.contains("."):
|
||||
newRow[k] = v
|
||||
else:
|
||||
let val = evalExpr(expr, row)
|
||||
if alias.len > 0: newRow[alias] = val
|
||||
else: newRow["col" & $i] = val
|
||||
if newRow.len > 0:
|
||||
result.add(newRow)
|
||||
else:
|
||||
result.add(row)
|
||||
return result
|
||||
|
||||
if hasAggs:
|
||||
# Produce exactly one row with aggregate values
|
||||
var newRow: Table[string, string]
|
||||
@@ -1663,6 +1847,7 @@ proc executeQuery*(ctx: ExecutionContext, astNode: Node, params: seq[WireValue]
|
||||
of nkInsert: "INSERT"
|
||||
of nkUpdate: "UPDATE"
|
||||
of nkDelete: "DELETE"
|
||||
of nkMerge: "MERGE"
|
||||
else: $stmt.kind
|
||||
let span = defaultTracer.beginSpan(spanName)
|
||||
defer: defaultTracer.endSpan(span)
|
||||
@@ -2146,6 +2331,76 @@ proc executeQuery*(ctx: ExecutionContext, astNode: Node, params: seq[WireValue]
|
||||
ctx.onChange(ChangeEvent(table: stmt.delTarget, kind: ckDelete, key: old, data: ""))
|
||||
return okResult(affected=count, kvPairs=kvPairs)
|
||||
|
||||
of nkMerge:
|
||||
# Execute source: subquery or table scan
|
||||
var sourceRows: seq[Row] = @[]
|
||||
if stmt.mergeSource != nil:
|
||||
if stmt.mergeSource.kind == nkSelect:
|
||||
let srcRes = executeQuery(ctx, Node(kind: nkStatementList, stmts: @[stmt.mergeSource]))
|
||||
sourceRows = srcRes.rows
|
||||
elif stmt.mergeSource.kind == nkIdent:
|
||||
sourceRows = execScan(ctx, stmt.mergeSource.identName)
|
||||
|
||||
let targetRows = execScan(ctx, stmt.mergeTarget)
|
||||
var count = 0
|
||||
var kvPairs: seq[(string, seq[byte])]
|
||||
|
||||
for srcRow in sourceRows:
|
||||
var matched = false
|
||||
var combinedRow = srcRow
|
||||
for k, v in srcRow:
|
||||
combinedRow[stmt.mergeSourceAlias & "." & k] = v
|
||||
for tgtRow in targetRows:
|
||||
# Evaluate ON condition with both source and target rows visible
|
||||
var rowWithTarget = combinedRow
|
||||
for k, v in tgtRow:
|
||||
rowWithTarget[stmt.mergeTargetAlias & "." & k] = v
|
||||
let onExpr = lowerExpr(stmt.mergeOn)
|
||||
if evalExpr(onExpr, rowWithTarget) == "true":
|
||||
matched = true
|
||||
if stmt.mergeMatchedUpdate.len > 0 and "$key" in tgtRow:
|
||||
var updateSets = initTable[string, string]()
|
||||
for s in stmt.mergeMatchedUpdate:
|
||||
if s.kind == nkBinOp and s.binOp == bkAssign:
|
||||
if s.binLeft.kind == nkIdent:
|
||||
let valExpr = lowerExpr(s.binRight)
|
||||
updateSets[s.binLeft.identName] = evalExpr(valExpr, rowWithTarget)
|
||||
var newRow = tgtRow
|
||||
for col, val in updateSets:
|
||||
newRow[col] = val
|
||||
fireTriggers(ctx, stmt.mergeTarget, "before", "update", tgtRow)
|
||||
count += execUpdateRow(ctx, stmt.mergeTarget, tgtRow["$key"], updateSets, kvPairs)
|
||||
fireTriggers(ctx, stmt.mergeTarget, "after", "update", newRow)
|
||||
if ctx.onChange != nil:
|
||||
ctx.onChange(ChangeEvent(table: stmt.mergeTarget, kind: ckUpdate, key: tgtRow["$key"], data: ""))
|
||||
break
|
||||
|
||||
if not matched and stmt.mergeNotMatchedInsert.len > 0:
|
||||
var fields: seq[string] = @[]
|
||||
var values: seq[string] = @[]
|
||||
for i, colNode in stmt.mergeNotMatchedInsert:
|
||||
if colNode.kind == nkIdent:
|
||||
fields.add(colNode.identName)
|
||||
if i < stmt.mergeNotMatchedValues.len:
|
||||
let v = stmt.mergeNotMatchedValues[i]
|
||||
let valExpr = lowerExpr(v)
|
||||
values.add(evalExpr(valExpr, combinedRow))
|
||||
else:
|
||||
values.add("")
|
||||
if fields.len > 0:
|
||||
var row = initTable[string, string]()
|
||||
for i, f in fields:
|
||||
if i < values.len: row[f] = values[i]
|
||||
fireTriggers(ctx, stmt.mergeTarget, "before", "insert", row)
|
||||
var insKvPairs: seq[(string, seq[byte])]
|
||||
count += execInsert(ctx, stmt.mergeTarget, fields, @[values], insKvPairs)
|
||||
for kv in insKvPairs: kvPairs.add(kv)
|
||||
fireTriggers(ctx, stmt.mergeTarget, "after", "insert", row)
|
||||
if ctx.onChange != nil:
|
||||
ctx.onChange(ChangeEvent(table: stmt.mergeTarget, kind: ckInsert, key: "", data: ""))
|
||||
|
||||
return okResult(affected=count, kvPairs=kvPairs)
|
||||
|
||||
of nkCreateTable:
|
||||
var tbl = TableDef(name: stmt.crtName, columns: @[], pkColumns: @[],
|
||||
foreignKeys: @[], checks: @[])
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
## BaraQL IR — Intermediate Representation for compilation
|
||||
import std/tables
|
||||
import std/strutils
|
||||
import ../core/types
|
||||
|
||||
type
|
||||
@@ -52,6 +53,7 @@ type
|
||||
irekExists
|
||||
irekStar
|
||||
irekJsonPath
|
||||
irekWindowFunc
|
||||
|
||||
IRJoinKind* = enum
|
||||
irjkInner
|
||||
@@ -71,11 +73,13 @@ type
|
||||
irpkInsert
|
||||
irpkUpdate
|
||||
irpkDelete
|
||||
irpkMerge
|
||||
irpkCreateType
|
||||
irpkUnion
|
||||
irpkCTE
|
||||
irpkValues
|
||||
irpkExplain
|
||||
irpkWindow
|
||||
|
||||
IRPlan* = ref object
|
||||
case kind*: IRPlanKind
|
||||
@@ -121,6 +125,14 @@ type
|
||||
deleteTable*: string
|
||||
deleteAlias*: string
|
||||
deleteSource*: IRPlan
|
||||
of irpkMerge:
|
||||
mergeTarget*: string
|
||||
mergeTargetAlias*: string
|
||||
mergeSourcePlan*: IRPlan
|
||||
mergeOnCond*: IRExpr
|
||||
mergeUpdateSets*: seq[(string, IRExpr)]
|
||||
mergeInsertFields*: seq[string]
|
||||
mergeInsertValues*: seq[seq[IRExpr]]
|
||||
of irpkCreateType:
|
||||
createTypeName*: string
|
||||
createTypeDef*: IRType
|
||||
@@ -136,6 +148,15 @@ type
|
||||
valuesRows*: seq[seq[IRExpr]]
|
||||
of irpkExplain:
|
||||
explainPlan*: IRPlan
|
||||
of irpkWindow:
|
||||
winSource*: IRPlan
|
||||
winFuncs*: seq[IRExpr]
|
||||
winPartition*: seq[IRExpr]
|
||||
winOrderBy*: seq[IRExpr]
|
||||
winOrderDirs*: seq[bool]
|
||||
winFrameMode*: string
|
||||
winFrameStart*: string
|
||||
winFrameEnd*: string
|
||||
|
||||
IRExpr* = ref object
|
||||
case kind*: IRExprKind
|
||||
@@ -172,6 +193,15 @@ type
|
||||
jpExpr*: IRExpr
|
||||
jpKey*: string
|
||||
jpAsText*: bool
|
||||
of irekWindowFunc:
|
||||
wfName*: string
|
||||
wfArgs*: seq[IRExpr]
|
||||
wfPartition*: seq[IRExpr]
|
||||
wfOrderBy*: seq[IRExpr]
|
||||
wfOrderDirs*: seq[bool]
|
||||
wfFrameMode*: string
|
||||
wfFrameStart*: string
|
||||
wfFrameEnd*: string
|
||||
|
||||
type
|
||||
TypeChecker* = ref object
|
||||
@@ -255,3 +285,14 @@ proc inferExpr*(tc: TypeChecker, expr: IRExpr, context: Table[string, IRType]):
|
||||
of irekJsonPath:
|
||||
if expr.jpAsText: return IRType(name: "str", kind: itkScalar)
|
||||
return IRType(name: "json", kind: itkScalar)
|
||||
of irekWindowFunc:
|
||||
# Window functions return int64 for ranking, or the type of the argument for value functions
|
||||
case expr.wfName.toLower()
|
||||
of "row_number", "rank", "dense_rank", "ntile":
|
||||
return IRType(name: "int64", kind: itkScalar)
|
||||
of "lead", "lag", "first_value", "last_value":
|
||||
if expr.wfArgs.len > 0:
|
||||
return tc.inferExpr(expr.wfArgs[0], context)
|
||||
return nil
|
||||
else:
|
||||
return IRType(name: "unknown", kind: itkScalar)
|
||||
|
||||
@@ -120,6 +120,8 @@ type
|
||||
tkAvg
|
||||
tkMin
|
||||
tkMax
|
||||
tkMerge
|
||||
tkMatched
|
||||
tkArray
|
||||
tkVector
|
||||
tkGraph
|
||||
@@ -134,6 +136,24 @@ type
|
||||
tkDfs
|
||||
tkPath
|
||||
|
||||
# Window functions
|
||||
tkOver
|
||||
tkPartition
|
||||
tkRows
|
||||
tkRange
|
||||
tkUnbounded
|
||||
tkPreceding
|
||||
tkFollowing
|
||||
tkCurrent
|
||||
tkRowNumber
|
||||
tkRank
|
||||
tkDenseRank
|
||||
tkLead
|
||||
tkLag
|
||||
tkFirstValue
|
||||
tkLastValue
|
||||
tkNtile
|
||||
|
||||
# Operators
|
||||
tkPlus
|
||||
tkMinus
|
||||
@@ -291,6 +311,8 @@ const keywords*: Table[string, TokenKind] = {
|
||||
"avg": tkAvg,
|
||||
"min": tkMin,
|
||||
"max": tkMax,
|
||||
"merge": tkMerge,
|
||||
"matched": tkMatched,
|
||||
"array": tkArray,
|
||||
"vector": tkVector,
|
||||
"graph": tkGraph,
|
||||
@@ -301,6 +323,22 @@ const keywords*: Table[string, TokenKind] = {
|
||||
"bfs": tkBfs,
|
||||
"dfs": tkDfs,
|
||||
"path": tkPath,
|
||||
"over": tkOver,
|
||||
"partition": tkPartition,
|
||||
"rows": tkRows,
|
||||
"range": tkRange,
|
||||
"unbounded": tkUnbounded,
|
||||
"preceding": tkPreceding,
|
||||
"following": tkFollowing,
|
||||
"current": tkCurrent,
|
||||
"row_number": tkRowNumber,
|
||||
"rank": tkRank,
|
||||
"dense_rank": tkDenseRank,
|
||||
"lead": tkLead,
|
||||
"lag": tkLag,
|
||||
"first_value": tkFirstValue,
|
||||
"last_value": tkLastValue,
|
||||
"ntile": tkNtile,
|
||||
}.toTable
|
||||
|
||||
proc newLexer*(input: string): Lexer =
|
||||
|
||||
@@ -38,6 +38,9 @@ proc parseExpr(p: var Parser): Node
|
||||
proc parseSelect(p: var Parser): Node
|
||||
proc parseStatement*(p: var Parser): Node
|
||||
proc parseCreateType(p: var Parser): Node
|
||||
proc parseOverClause(p: var Parser): Node
|
||||
proc parseFrameSpec(p: var Parser): Node
|
||||
proc parseFrameBoundary(p: var Parser): string
|
||||
|
||||
proc parsePrimary(p: var Parser): Node =
|
||||
let tok = p.peek()
|
||||
@@ -57,8 +60,9 @@ proc parsePrimary(p: var Parser): Node =
|
||||
of tkNull:
|
||||
discard p.advance()
|
||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||
of tkIdent:
|
||||
of tkIdent, tkRowNumber, tkRank, tkDenseRank, tkLead, tkLag, tkFirstValue, tkLastValue, tkNtile:
|
||||
discard p.advance()
|
||||
let funcName = tok.value
|
||||
# Check for function call: ident(...)
|
||||
if p.peek().kind == tkLParen:
|
||||
discard p.advance() # consume (
|
||||
@@ -68,15 +72,21 @@ proc parsePrimary(p: var Parser): Node =
|
||||
while p.match(tkComma):
|
||||
args.add(p.parseExpr())
|
||||
discard p.expect(tkRParen)
|
||||
return Node(kind: nkFuncCall, funcName: tok.value, funcArgs: args,
|
||||
var funcNode = Node(kind: nkFuncCall, funcName: funcName, funcArgs: args,
|
||||
line: tok.line, col: tok.col)
|
||||
# Check for window function: func(...) OVER (...)
|
||||
if p.peek().kind == tkOver:
|
||||
let overClause = p.parseOverClause()
|
||||
return Node(kind: nkWindowExpr, winFunc: funcName, winArgs: args,
|
||||
winOver: overClause, line: tok.line, col: tok.col)
|
||||
return funcNode
|
||||
# Check for dotted path: ident.ident.ident
|
||||
var parts = @[tok.value]
|
||||
var parts = @[funcName]
|
||||
while p.peek().kind == tkDot:
|
||||
discard p.advance() # consume .
|
||||
parts.add(p.expect(tkIdent).value)
|
||||
if parts.len == 1:
|
||||
return Node(kind: nkIdent, identName: tok.value, line: tok.line, col: tok.col)
|
||||
return Node(kind: nkIdent, identName: funcName, line: tok.line, col: tok.col)
|
||||
return Node(kind: nkPath, pathParts: parts, line: tok.line, col: tok.col)
|
||||
of tkLParen:
|
||||
discard p.advance()
|
||||
@@ -157,6 +167,80 @@ proc parsePrimary(p: var Parser): Node =
|
||||
discard p.advance()
|
||||
Node(kind: nkNullLit, line: tok.line, col: tok.col)
|
||||
|
||||
proc parseOverClause(p: var Parser): Node =
|
||||
## Parse OVER ( [PARTITION BY expr, ...] [ORDER BY expr [ASC|DESC], ...] [frame] )
|
||||
discard p.expect(tkOver)
|
||||
discard p.expect(tkLParen)
|
||||
result = Node(kind: nkOverClause)
|
||||
# PARTITION BY
|
||||
if p.match(tkPartition):
|
||||
discard p.expect(tkBy)
|
||||
result.overPartition.add(p.parseExpr())
|
||||
while p.match(tkComma):
|
||||
result.overPartition.add(p.parseExpr())
|
||||
# ORDER BY
|
||||
if p.match(tkOrder):
|
||||
discard p.expect(tkBy)
|
||||
var expr = p.parseExpr()
|
||||
var dir = sdAsc
|
||||
if p.match(tkDesc):
|
||||
dir = sdDesc
|
||||
elif p.match(tkAsc):
|
||||
dir = sdAsc
|
||||
result.overOrderBy.add(Node(kind: nkOrderBy, orderByExpr: expr, orderByDir: dir))
|
||||
while p.match(tkComma):
|
||||
expr = p.parseExpr()
|
||||
dir = sdAsc
|
||||
if p.match(tkDesc):
|
||||
dir = sdDesc
|
||||
elif p.match(tkAsc):
|
||||
dir = sdAsc
|
||||
result.overOrderBy.add(Node(kind: nkOrderBy, orderByExpr: expr, orderByDir: dir))
|
||||
# Frame specification
|
||||
if p.peek().kind in {tkRows, tkRange}:
|
||||
result.overFrame = p.parseFrameSpec()
|
||||
discard p.expect(tkRParen)
|
||||
|
||||
proc parseFrameSpec(p: var Parser): Node =
|
||||
## Parse ROWS|RANGE frame
|
||||
let modeTok = p.advance() # tkRows or tkRange
|
||||
result = Node(kind: nkFrameSpec, frameMode: modeTok.value)
|
||||
if p.match(tkBetween):
|
||||
result.frameStartType = p.parseFrameBoundary()
|
||||
discard p.expect(tkAnd)
|
||||
result.frameEndType = p.parseFrameBoundary()
|
||||
else:
|
||||
result.frameStartType = p.parseFrameBoundary()
|
||||
result.frameEndType = "CURRENT ROW"
|
||||
|
||||
proc parseFrameBoundary(p: var Parser): string =
|
||||
## Returns boundary type string like "UNBOUNDED PRECEDING", "CURRENT ROW", "1 PRECEDING"
|
||||
if p.match(tkUnbounded):
|
||||
if p.match(tkPreceding):
|
||||
return "UNBOUNDED PRECEDING"
|
||||
elif p.match(tkFollowing):
|
||||
return "UNBOUNDED FOLLOWING"
|
||||
else:
|
||||
raise newException(ValueError, "Expected PRECEDING or FOLLOWING after UNBOUNDED")
|
||||
elif p.match(tkCurrent):
|
||||
discard p.expect(tkRows)
|
||||
return "CURRENT ROW"
|
||||
else:
|
||||
# Expect a simple integer literal for offset boundaries
|
||||
let offsetExpr = p.parsePrimary()
|
||||
var offsetStr = ""
|
||||
case offsetExpr.kind:
|
||||
of nkIntLit: offsetStr = $offsetExpr.intVal
|
||||
of nkIdent: offsetStr = offsetExpr.identName
|
||||
else:
|
||||
raise newException(ValueError, "Expected integer literal or identifier in frame boundary, got " & $offsetExpr.kind)
|
||||
if p.match(tkPreceding):
|
||||
return offsetStr & " PRECEDING"
|
||||
elif p.match(tkFollowing):
|
||||
return offsetStr & " FOLLOWING"
|
||||
else:
|
||||
raise newException(ValueError, "Expected PRECEDING or FOLLOWING after frame offset")
|
||||
|
||||
proc parsePostfix(p: var Parser): Node =
|
||||
result = p.parsePrimary()
|
||||
while p.peek().kind in {tkArrowR, tkArrowRR}:
|
||||
@@ -540,6 +624,64 @@ proc parseDelete(p: var Parser): Node =
|
||||
while p.match(tkComma):
|
||||
result.delReturning.add(p.parseExpr())
|
||||
|
||||
proc parseMerge(p: var Parser): Node =
|
||||
let tok = p.expect(tkMerge)
|
||||
discard p.match(tkInto) # optional INTO
|
||||
result = Node(kind: nkMerge, line: tok.line, col: tok.col)
|
||||
result.mergeTarget = p.expect(tkIdent).value
|
||||
if p.match(tkAs):
|
||||
result.mergeTargetAlias = p.expect(tkIdent).value
|
||||
elif p.peek().kind == tkIdent:
|
||||
result.mergeTargetAlias = p.advance().value
|
||||
discard p.expect(tkUsing)
|
||||
# Source can be a table name or a subquery
|
||||
if p.peek().kind == tkLParen:
|
||||
discard p.advance() # consume (
|
||||
result.mergeSource = p.parseSelect()
|
||||
discard p.expect(tkRParen)
|
||||
else:
|
||||
let srcTable = p.expect(tkIdent).value
|
||||
result.mergeSource = Node(kind: nkIdent, identName: srcTable, line: tok.line, col: tok.col)
|
||||
if p.match(tkAs):
|
||||
result.mergeSourceAlias = p.expect(tkIdent).value
|
||||
elif p.peek().kind == tkIdent:
|
||||
result.mergeSourceAlias = p.advance().value
|
||||
discard p.expect(tkOn)
|
||||
result.mergeOn = p.parseExpr()
|
||||
# WHEN MATCHED THEN UPDATE SET ...
|
||||
if p.match(tkWhen):
|
||||
if p.match(tkNot):
|
||||
discard p.expect(tkMatched)
|
||||
discard p.expect(tkThen)
|
||||
discard p.expect(tkInsert)
|
||||
discard p.expect(tkLParen)
|
||||
result.mergeNotMatchedInsert.add(Node(kind: nkIdent, identName: p.expect(tkIdent).value))
|
||||
while p.match(tkComma):
|
||||
result.mergeNotMatchedInsert.add(Node(kind: nkIdent, identName: p.expect(tkIdent).value))
|
||||
discard p.expect(tkRParen)
|
||||
discard p.expect(tkValues)
|
||||
discard p.expect(tkLParen)
|
||||
result.mergeNotMatchedValues.add(p.parseExpr())
|
||||
while p.match(tkComma):
|
||||
result.mergeNotMatchedValues.add(p.parseExpr())
|
||||
discard p.expect(tkRParen)
|
||||
else:
|
||||
discard p.expect(tkMatched)
|
||||
discard p.expect(tkThen)
|
||||
discard p.expect(tkUpdate)
|
||||
discard p.expect(tkSet)
|
||||
let col = p.expect(tkIdent).value
|
||||
discard p.expect(tkEq)
|
||||
result.mergeMatchedUpdate.add(Node(kind: nkBinOp, binOp: bkAssign,
|
||||
binLeft: Node(kind: nkIdent, identName: col),
|
||||
binRight: p.parseExpr()))
|
||||
while p.match(tkComma):
|
||||
let col2 = p.expect(tkIdent).value
|
||||
discard p.expect(tkEq)
|
||||
result.mergeMatchedUpdate.add(Node(kind: nkBinOp, binOp: bkAssign,
|
||||
binLeft: Node(kind: nkIdent, identName: col2),
|
||||
binRight: p.parseExpr()))
|
||||
|
||||
proc parseCreateType(p: var Parser): Node =
|
||||
let tok = p.expect(tkCreate)
|
||||
discard p.expect(tkType)
|
||||
@@ -1105,6 +1247,7 @@ proc parseStatement*(p: var Parser): Node =
|
||||
of tkInsert: p.parseInsert()
|
||||
of tkUpdate: p.parseUpdate()
|
||||
of tkDelete: p.parseDelete()
|
||||
of tkMerge: p.parseMerge()
|
||||
of tkCreate:
|
||||
if p.pos + 1 < p.tokens.len:
|
||||
let next = p.tokens[p.pos + 1]
|
||||
|
||||
@@ -2576,8 +2576,112 @@ suite "Parameterized queries":
|
||||
check r.rows.len == 1
|
||||
check r.rows[0]["name"] == "Bob"
|
||||
|
||||
suite "Window Functions":
|
||||
var db: LSMTree
|
||||
var ctx: qexec.ExecutionContext
|
||||
|
||||
setup:
|
||||
db = newLSMTree("")
|
||||
ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE employees (id INT, name TEXT, department TEXT, salary INT)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (1, 'Alice', 'Engineering', 90000)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (2, 'Bob', 'Engineering', 80000)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (3, 'Charlie', 'Sales', 70000)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (4, 'Diana', 'Sales', 75000)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO employees (id, name, department, salary) VALUES (5, 'Eve', 'Engineering', 95000)"))
|
||||
|
||||
test "ROW_NUMBER with PARTITION BY and ORDER BY":
|
||||
let r = qexec.executeQuery(ctx, parse("SELECT name, department, ROW_NUMBER() OVER (PARTITION BY department ORDER BY salary DESC) AS rn FROM employees"))
|
||||
check r.success
|
||||
check r.rows.len == 5
|
||||
# Engineering: Eve(95000)=1, Alice(90000)=2, Bob(80000)=3
|
||||
# Sales: Diana(75000)=1, Charlie(70000)=2
|
||||
var found = initTable[string, string]()
|
||||
for row in r.rows:
|
||||
found[row["name"]] = row["rn"]
|
||||
check found["Eve"] == "1"
|
||||
check found["Alice"] == "2"
|
||||
check found["Bob"] == "3"
|
||||
check found["Diana"] == "1"
|
||||
check found["Charlie"] == "2"
|
||||
|
||||
test "RANK and DENSE_RANK":
|
||||
let r = qexec.executeQuery(ctx, parse("SELECT name, salary, RANK() OVER (ORDER BY salary DESC) AS r, DENSE_RANK() OVER (ORDER BY salary DESC) AS dr FROM employees"))
|
||||
check r.success
|
||||
check r.rows.len == 5
|
||||
for row in r.rows:
|
||||
if row["name"] == "Eve":
|
||||
check row["r"] == "1"
|
||||
check row["dr"] == "1"
|
||||
if row["name"] == "Alice":
|
||||
check row["r"] == "2"
|
||||
check row["dr"] == "2"
|
||||
|
||||
test "LEAD and LAG":
|
||||
let r = qexec.executeQuery(ctx, parse("SELECT name, salary, LAG(salary, 1, 0) OVER (ORDER BY salary) AS prev, LEAD(salary, 1, 0) OVER (ORDER BY salary) AS next FROM employees"))
|
||||
check r.success
|
||||
check r.rows.len == 5
|
||||
for row in r.rows:
|
||||
if row["name"] == "Charlie":
|
||||
check row["prev"] == "0"
|
||||
check row["next"] == "75000"
|
||||
if row["name"] == "Diana":
|
||||
check row["prev"] == "70000"
|
||||
check row["next"] == "80000"
|
||||
|
||||
test "FIRST_VALUE and LAST_VALUE":
|
||||
let r = qexec.executeQuery(ctx, parse("SELECT department, FIRST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS first, LAST_VALUE(name) OVER (PARTITION BY department ORDER BY salary) AS last FROM employees"))
|
||||
check r.success
|
||||
check r.rows.len == 5
|
||||
|
||||
test "NTILE":
|
||||
let r = qexec.executeQuery(ctx, parse("SELECT name, NTILE(2) OVER (ORDER BY salary) AS bucket FROM employees"))
|
||||
check r.success
|
||||
check r.rows.len == 5
|
||||
for row in r.rows:
|
||||
check row["bucket"] in @["1", "2"]
|
||||
|
||||
# JOIN tests
|
||||
include "join_tests"
|
||||
|
||||
# TLA+ faithfulness tests
|
||||
include "tla_faithfulness"
|
||||
|
||||
suite "MERGE Statement":
|
||||
var db: LSMTree
|
||||
var ctx: qexec.ExecutionContext
|
||||
|
||||
setup:
|
||||
db = newLSMTree("")
|
||||
ctx = qexec.newExecutionContext(db)
|
||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE inventory (id INT PRIMARY KEY, sku TEXT, qty INT)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO inventory (id, sku, qty) VALUES (1, 'SKU001', 100)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO inventory (id, sku, qty) VALUES (2, 'SKU002', 200)"))
|
||||
discard qexec.executeQuery(ctx, parse("CREATE TABLE updates (sku TEXT, delta INT)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO updates (sku, delta) VALUES ('SKU001', 50)"))
|
||||
discard qexec.executeQuery(ctx, parse("INSERT INTO updates (sku, delta) VALUES ('SKU003', 300)"))
|
||||
|
||||
test "MERGE WHEN MATCHED UPDATE":
|
||||
let r = qexec.executeQuery(ctx, parse("""
|
||||
MERGE INTO inventory AS target
|
||||
USING updates AS source
|
||||
ON target.sku = source.sku
|
||||
WHEN MATCHED THEN UPDATE SET qty = target.qty + source.delta
|
||||
"""))
|
||||
check r.success
|
||||
check r.affectedRows == 1
|
||||
let verify = qexec.executeQuery(ctx, parse("SELECT * FROM inventory WHERE sku = 'SKU001'"))
|
||||
check verify.rows[0]["qty"] == "150.0"
|
||||
|
||||
test "MERGE WHEN NOT MATCHED INSERT":
|
||||
let r = qexec.executeQuery(ctx, parse("""
|
||||
MERGE INTO inventory AS target
|
||||
USING updates AS source
|
||||
ON target.sku = source.sku
|
||||
WHEN NOT MATCHED THEN INSERT (id, sku, qty) VALUES (3, source.sku, source.delta)
|
||||
"""))
|
||||
check r.success
|
||||
check r.affectedRows == 1
|
||||
let verify = qexec.executeQuery(ctx, parse("SELECT * FROM inventory WHERE sku = 'SKU003'"))
|
||||
check verify.rows.len == 1
|
||||
check verify.rows[0]["qty"] == "300"
|
||||
|
||||
Reference in New Issue
Block a user