docs: add German (DE) documentation + update all docs for Sessions 10-12
New German documentation (docs/de/): - index.md, quickstart.md, installation.md - baraql.md, graph.md, vector.md, mcp.md Updated English documentation: - changelog.md: all Sessions 10-12 features - graph.md: SQL GRAPH_TABLE, CREATE GRAPH, all 8 algorithms, Cypher, similarity_nodes, node2vec - vector.md: hybrid RAG, chunk(), embed_text(), auto-embed, nl_to_sql(), schema_prompt() - baraql.md: new AI & Cross-Modal Functions section, updated keyword tables - mcp.md: MCP Server documentation (new file) - index.md: added German (DE) language link
This commit is contained in:
@@ -0,0 +1,114 @@
|
||||
# BaraQL — Abfragesprache-Referenz
|
||||
|
||||
BaraQL ist eine SQL-kompatible Abfragesprache mit Erweiterungen für Graph-, Vektor- und Dokumentoperationen.
|
||||
|
||||
## Datentypen
|
||||
|
||||
| Typ | Beschreibung | Beispiel |
|
||||
|------|-------------|---------|
|
||||
| `null` | Nullwert | `null` |
|
||||
| `bool` | Boolean | `true`, `false` |
|
||||
| `int64` | 64-bit Ganzzahl | `42` |
|
||||
| `float64` | 64-bit Fließkomma | `3.14` |
|
||||
| `str` | UTF-8 String | `'hello'` |
|
||||
| `vector(n)` | Float32 Vektor | `VECTOR(768)` |
|
||||
| `json` | JSON-Dokument | `{"key": "value"}` |
|
||||
|
||||
## Grundlegende Abfragen
|
||||
|
||||
```sql
|
||||
SELECT * FROM users;
|
||||
SELECT name, age FROM users WHERE age > 18;
|
||||
SELECT * FROM users ORDER BY age DESC LIMIT 10;
|
||||
```
|
||||
|
||||
## Vektor-Operationen
|
||||
|
||||
```sql
|
||||
-- Distanzberechnungen
|
||||
SELECT cosine_distance(embedding, '[0.1, 0.2, 0.3]') AS dist FROM items;
|
||||
SELECT embedding <-> '[0.1, 0.2, 0.3]' AS dist FROM items;
|
||||
|
||||
-- Hybride Suche
|
||||
SELECT hybrid_search('query', embedding, content, 10) AS result;
|
||||
```
|
||||
|
||||
## Graph-Operationen
|
||||
|
||||
```sql
|
||||
CREATE GRAPH social;
|
||||
DROP GRAPH social;
|
||||
|
||||
-- Traversierung
|
||||
SELECT * FROM GRAPH_TABLE(social MATCH (n)-[r]->(m)
|
||||
ALGORITHM bfs START 1
|
||||
COLUMNS (id, node_label));
|
||||
|
||||
-- PageRank
|
||||
SELECT * FROM GRAPH_TABLE(social ALGORITHM pagerank
|
||||
COLUMNS (id, node_label, rank));
|
||||
|
||||
-- Community Detection
|
||||
SELECT * FROM GRAPH_TABLE(social ALGORITHM community
|
||||
COLUMNS (id, node_label, community));
|
||||
```
|
||||
|
||||
## AI-Funktionen
|
||||
|
||||
```sql
|
||||
-- Text in Chunks zerlegen
|
||||
SELECT chunk('Langer Text...', 1024, 128) AS result;
|
||||
|
||||
-- Embedding generieren
|
||||
SELECT embed_text('Suchtext') AS result;
|
||||
|
||||
-- Natural Language → SQL
|
||||
SELECT nl_to_sql('Zeige alle Benutzer über 25', 'users') AS result;
|
||||
|
||||
-- Schema-Prompt für LLM
|
||||
SELECT schema_prompt('users') AS result;
|
||||
|
||||
-- Cypher-Übersetzung
|
||||
SELECT cypher('MATCH (a)-[r]->(b) RETURN a.node_label') AS result;
|
||||
|
||||
-- Knotenähnlichkeit
|
||||
SELECT similarity_nodes('social', 'jaccard') AS result;
|
||||
|
||||
-- Graph-Embeddings
|
||||
SELECT node2vec_embed('social', 64) AS result;
|
||||
```
|
||||
|
||||
## Joins
|
||||
|
||||
```sql
|
||||
SELECT u.name, o.amount
|
||||
FROM users u
|
||||
INNER JOIN orders o ON u.id = o.user_id;
|
||||
```
|
||||
|
||||
## Aggregation
|
||||
|
||||
```sql
|
||||
SELECT department, COUNT(*), AVG(salary)
|
||||
FROM employees
|
||||
GROUP BY department
|
||||
HAVING COUNT(*) > 5;
|
||||
```
|
||||
|
||||
## Index-Erstellung
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_name ON users(name) USING btree;
|
||||
CREATE INDEX idx_vec ON docs(embedding) USING hnsw;
|
||||
CREATE INDEX idx_fts ON docs(content) USING fts;
|
||||
```
|
||||
|
||||
## Multi-Tenant / RLS
|
||||
|
||||
```sql
|
||||
CREATE POLICY tenant_policy ON orders
|
||||
FOR ALL USING (tenant_id = current_setting('app.tenant_id'));
|
||||
|
||||
SET app.tenant_id = 'company-a';
|
||||
SELECT * FROM orders; -- Automatisch gefiltert
|
||||
```
|
||||
@@ -0,0 +1,145 @@
|
||||
# Graph Engine
|
||||
|
||||
Adjazenzlisten-Speicher mit eingebauten Algorithmen für Graph-Traversierung und -Analyse.
|
||||
Vollständig integriert in den SQL-Executor via `GRAPH_TABLE()`.
|
||||
|
||||
## SQL — Graph DDL
|
||||
|
||||
### Graph erstellen
|
||||
|
||||
```sql
|
||||
CREATE GRAPH org_chart;
|
||||
```
|
||||
|
||||
Erstellt automatisch zwei Tabellen:
|
||||
- `org_chart_nodes (id INTEGER PRIMARY KEY, node_label TEXT, properties TEXT)`
|
||||
- `org_chart_edges (source_id INTEGER, dest_id INTEGER, edge_label TEXT, weight REAL)`
|
||||
|
||||
### Graph löschen
|
||||
|
||||
```sql
|
||||
DROP GRAPH org_chart;
|
||||
```
|
||||
|
||||
## SQL — Daten einfügen
|
||||
|
||||
```sql
|
||||
-- Knoten
|
||||
INSERT INTO org_chart_nodes (id, node_label) VALUES (1, 'CEO');
|
||||
INSERT INTO org_chart_nodes (id, node_label) VALUES (2, 'VP');
|
||||
|
||||
-- Kanten
|
||||
INSERT INTO org_chart_edges (source_id, dest_id, edge_label) VALUES (1, 2, 'manages');
|
||||
```
|
||||
|
||||
Alle INSERTs werden automatisch mit dem nativen Graph-Objekt synchronisiert.
|
||||
|
||||
## SQL — GRAPH_TABLE Abfragen
|
||||
|
||||
### BFS (Breitensuche)
|
||||
|
||||
```sql
|
||||
SELECT * FROM GRAPH_TABLE(org_chart MATCH (n)-[r]->(m)
|
||||
ALGORITHM bfs
|
||||
START 1 MAXDEPTH 2
|
||||
COLUMNS (id, node_label));
|
||||
```
|
||||
|
||||
### DFS (Tiefensuche)
|
||||
|
||||
```sql
|
||||
SELECT * FROM GRAPH_TABLE(org_chart MATCH (n)-[r]->(m)
|
||||
ALGORITHM dfs START 1
|
||||
COLUMNS (id, node_label));
|
||||
```
|
||||
|
||||
### PageRank
|
||||
|
||||
```sql
|
||||
SELECT id, node_label, rank FROM GRAPH_TABLE(org_chart
|
||||
ALGORITHM pagerank
|
||||
COLUMNS (id, node_label, rank))
|
||||
ORDER BY rank DESC;
|
||||
```
|
||||
|
||||
### Community Detection (Louvain)
|
||||
|
||||
```sql
|
||||
SELECT id, node_label, community FROM GRAPH_TABLE(org_chart
|
||||
ALGORITHM community
|
||||
COLUMNS (id, node_label, community));
|
||||
```
|
||||
|
||||
### Kürzester Pfad (Shortest Path)
|
||||
|
||||
```sql
|
||||
SELECT * FROM GRAPH_TABLE(org_chart
|
||||
ALGORITHM shortest_path
|
||||
START 1 END 3
|
||||
COLUMNS (id, node_label));
|
||||
```
|
||||
|
||||
### Dijkstra (gewichtete kürzeste Pfade)
|
||||
|
||||
```sql
|
||||
SELECT * FROM GRAPH_TABLE(org_chart
|
||||
ALGORITHM dijkstra START 1
|
||||
COLUMNS (id, node_label, distance));
|
||||
```
|
||||
|
||||
## SQL-Funktionen
|
||||
|
||||
### Knotenähnlichkeit
|
||||
|
||||
```sql
|
||||
-- Jaccard-Ähnlichkeit zwischen allen Knotenpaaren
|
||||
SELECT similarity_nodes('social', 'jaccard') AS result;
|
||||
|
||||
-- Adamic-Adar-Ähnlichkeit
|
||||
SELECT similarity_nodes('social', 'adamic_adar') AS result;
|
||||
```
|
||||
|
||||
### Node2Vec Embeddings
|
||||
|
||||
```sql
|
||||
-- Graphstruktur-Embeddings generieren (64 Dimensionen)
|
||||
SELECT node2vec_embed('social', 64) AS result;
|
||||
```
|
||||
|
||||
## Cypher-Kompatibilität
|
||||
|
||||
```sql
|
||||
-- Cypher-Syntax automatisch nach GRAPH_TABLE übersetzen
|
||||
SELECT cypher('MATCH (a)-[r]->(b) WHERE a.node_label = ''CEO'' RETURN b.node_label') AS result;
|
||||
```
|
||||
|
||||
## Algorithmen
|
||||
|
||||
| Algorithmus | Beschreibung | SQL |
|
||||
|-------------|--------------|-----|
|
||||
| `bfs` | Breitensuche | `ALGORITHM bfs` |
|
||||
| `dfs` | Tiefensuche | `ALGORITHM dfs` |
|
||||
| `dijkstra` | Gewichtete kürzeste Pfade | `ALGORITHM dijkstra` |
|
||||
| `pageRank` | Knoten-Wichtigkeit | `ALGORITHM pagerank` |
|
||||
| `louvain` | Community Detection | `ALGORITHM community` |
|
||||
| `shortestPath` | Kürzester Pfad | `ALGORITHM shortest_path START X END Y` |
|
||||
| `similarityNodes` | Knotenähnlichkeit | `similarity_nodes()` |
|
||||
| `node2vec` | Graph Embeddings | `node2vec_embed()` |
|
||||
|
||||
## Native Nim API
|
||||
|
||||
```nim
|
||||
import barabadb/graph/engine
|
||||
|
||||
var g = newGraph()
|
||||
let alice = g.addNode("Person", {"name": "Alice"}.toTable)
|
||||
let bob = g.addNode("Person", {"name": "Bob"}.toTable)
|
||||
discard g.addEdge(alice, bob, "knows")
|
||||
|
||||
let bfs = g.bfs(alice)
|
||||
let path = g.shortestPath(alice, bob)
|
||||
let ranks = g.pageRank()
|
||||
let communities = louvain(g)
|
||||
let similarities = g.similarityNodes(smJaccard)
|
||||
let embeddings = g.node2vec(64, 10, 5)
|
||||
```
|
||||
@@ -0,0 +1,64 @@
|
||||
# BaraDB Dokumentation
|
||||
|
||||
**Eine multimodale Datenbank-Engine — 100% Nim, null Abhängigkeiten.**
|
||||
|
||||
## Sprachen
|
||||
|
||||
- [English](../en/)
|
||||
- [Български (Bulgarisch)](../bg/)
|
||||
- [Deutsch (German)](../de/)
|
||||
- [Русский (Russisch)](../ru/)
|
||||
- [فارسی (Farsi)](../fa/)
|
||||
- [中文 (Chinesisch)](../zh/)
|
||||
- [Türkçe (Türkisch)](../tr/)
|
||||
- [العربية (Arabisch)](../ar/)
|
||||
|
||||
---
|
||||
|
||||
## Schnellstart
|
||||
|
||||
- [Installation](installation.md)
|
||||
- [Schnellstart](quickstart.md)
|
||||
- [Architektur](architecture.md)
|
||||
- [Konfiguration](configuration.md)
|
||||
|
||||
## Kernkonzepte
|
||||
|
||||
- [BaraQL Abfragesprache](baraql.md)
|
||||
- [Speicher-Engines](storage.md)
|
||||
- [Schema System](schema.md)
|
||||
|
||||
## Engines
|
||||
|
||||
- [LSM-Tree Speicher](lsm.md)
|
||||
- [B-Tree Index](btree.md)
|
||||
- [Vektor-Suche](vector.md)
|
||||
- [Graph Engine](graph.md)
|
||||
- [Volltext-Suche](fts.md)
|
||||
- [Spaltenbasierte Speicherung](columnar.md)
|
||||
|
||||
## API & Clients
|
||||
|
||||
- [Client SDKs](clients.md)
|
||||
- [Binärprotokoll](api-binary.md)
|
||||
- [HTTP/REST API](api-http.md)
|
||||
- [MCP Server](mcp.md)
|
||||
|
||||
## Betrieb
|
||||
|
||||
- [Performance](performance.md)
|
||||
- [Sicherheit](security.md)
|
||||
- [Monitoring](monitoring.md)
|
||||
- [Backup & Recovery](backup.md)
|
||||
- [Fehlerbehebung](troubleshooting.md)
|
||||
|
||||
## Erweitert
|
||||
|
||||
- [Transaktionen & MVCC](transactions.md)
|
||||
- [Verteilte Systeme](distributed.md)
|
||||
- [Docker Deployment](docker.md)
|
||||
- [Änderungsprotokoll](changelog.md)
|
||||
|
||||
---
|
||||
|
||||
*Um eine neue Sprache hinzuzufügen, erstellen Sie einen neuen Ordner in `docs/` mit dem Sprachcode (z.B. `docs/de/`).*
|
||||
@@ -0,0 +1,57 @@
|
||||
# BaraDB — Installation
|
||||
|
||||
## Voraussetzungen
|
||||
|
||||
- **Nim >= 2.2.0** (`curl https://nim-lang.org/choosenim/init.sh -sSf | sh`)
|
||||
- **Git**
|
||||
- **OpenSSL** (für TLS)
|
||||
|
||||
## Aus dem Quellcode bauen
|
||||
|
||||
```bash
|
||||
git clone https://github.com/katehonz/barabaDB.git
|
||||
cd barabadb
|
||||
nimble build_release
|
||||
```
|
||||
|
||||
Die Binärdateien werden im `build/` Verzeichnis erstellt:
|
||||
- `build/baradadb` — Datenbank-Server (TCP + HTTP)
|
||||
- `build/baramcp` — MCP Server für AI-Agenten
|
||||
|
||||
## Debug-Build
|
||||
|
||||
```bash
|
||||
nimble build_debug
|
||||
```
|
||||
|
||||
## Tests ausführen
|
||||
|
||||
```bash
|
||||
nimble test
|
||||
```
|
||||
|
||||
## Docker
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## Verifizierung
|
||||
|
||||
```bash
|
||||
./build/baradadb --version
|
||||
# BaraDB v1.1.2 — Multimodal Database Engine
|
||||
|
||||
./build/baramcp --data-dir ./data &
|
||||
echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' | ./build/baramcp
|
||||
```
|
||||
|
||||
## Manuelle Kompilierung
|
||||
|
||||
```bash
|
||||
# Server
|
||||
nim c -d:release --opt:speed -o:build/baradadb src/baradadb.nim
|
||||
|
||||
# MCP Server
|
||||
nim c -d:release --opt:speed -o:build/baramcp src/baramcp.nim
|
||||
```
|
||||
+108
@@ -0,0 +1,108 @@
|
||||
# MCP Server (Model Context Protocol)
|
||||
|
||||
BaraDB enthält einen eingebauten MCP-Server, der es AI-Agenten (Claude, Cursor, etc.) ermöglicht, direkt mit der Datenbank zu interagieren.
|
||||
|
||||
## Schnellstart
|
||||
|
||||
```bash
|
||||
./build/baramcp --data-dir ./data
|
||||
```
|
||||
|
||||
Der Server startet im STDIO-Modus und akzeptiert JSON-RPC 2.0 Nachrichten.
|
||||
|
||||
## Verfügbare Tools
|
||||
|
||||
### 1. `query` — SQL ausführen
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "query",
|
||||
"arguments": {
|
||||
"sql": "SELECT * FROM users WHERE age > ?",
|
||||
"params": [25],
|
||||
"tenant_id": "company-a",
|
||||
"user_id": "alice"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Parameterisierte Abfragen mit `?`-Platzhaltern. Multi-Tenant-Support über `tenant_id` und `user_id`.
|
||||
|
||||
### 2. `vector_search` — Semantische Suche
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "vector_search",
|
||||
"arguments": {
|
||||
"table": "docs",
|
||||
"column": "embedding",
|
||||
"query_vector": [0.1, 0.2, 0.3],
|
||||
"k": 5,
|
||||
"metric": "cosine",
|
||||
"tenant_id": "company-a"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Unterstützte Metriken: `cosine`, `euclidean`, `dot_product`, `manhattan`.
|
||||
|
||||
### 3. `schema_inspect` — Schema erkunden
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "schema_inspect",
|
||||
"arguments": {
|
||||
"table": "users",
|
||||
"tenant_id": "company-a"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Gibt Tabellen, Spalten, Typen, Primärschlüssel, Fremdschlüssel, Indizes und RLS-Policies zurück.
|
||||
|
||||
## Konfiguration in Claude Desktop
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"baradb": {
|
||||
"command": "/pfad/zu/build/baramcp",
|
||||
"args": ["--data-dir", "/pfad/zu/daten"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Konfiguration in Cursor
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"baradb": {
|
||||
"command": "/pfad/zu/build/baramcp",
|
||||
"args": ["--data-dir", "~/.baradb/data"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Multi-Tenant Isolation
|
||||
|
||||
Jede MCP-Anfrage kann `tenant_id` und `user_id` enthalten. Diese werden als Session-Variablen gesetzt:
|
||||
|
||||
- `app.tenant_id` — für RLS-Filterung
|
||||
- `app.user_id` — für `current_user`-Referenzen
|
||||
|
||||
RLS-Policies filtern die Daten automatisch basierend auf diesen Variablen.
|
||||
|
||||
## JSON-RPC 2.0 Protokoll
|
||||
|
||||
Der Server verwendet JSON-RPC 2.0 über STDIO:
|
||||
|
||||
```json
|
||||
// Anfrage
|
||||
{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {...}}
|
||||
|
||||
// Antwort
|
||||
{"jsonrpc": "2.0", "id": 1, "result": {"content": [{"type": "text", "text": "..."}]}}
|
||||
```
|
||||
@@ -0,0 +1,134 @@
|
||||
# BaraDB — Schnellstart
|
||||
|
||||
## Server starten
|
||||
|
||||
```bash
|
||||
./build/baradadb
|
||||
```
|
||||
|
||||
Der Server startet standardmäßig auf `localhost:9470`.
|
||||
|
||||
## Verbindung via CLI
|
||||
|
||||
```bash
|
||||
./build/baradadb --shell
|
||||
```
|
||||
|
||||
## MCP Server (AI Agenten)
|
||||
|
||||
```bash
|
||||
./build/baramcp --data-dir ./data
|
||||
```
|
||||
|
||||
Der MCP Server startet im STDIO-Modus und stellt 3 Tools für AI-Agenten bereit: `query`, `vector_search`, `schema_inspect`.
|
||||
|
||||
## Grundlegende Operationen
|
||||
|
||||
### Tabelle erstellen
|
||||
|
||||
```sql
|
||||
CREATE TABLE users (
|
||||
id INTEGER PRIMARY KEY,
|
||||
name TEXT,
|
||||
email TEXT,
|
||||
age INTEGER
|
||||
);
|
||||
```
|
||||
|
||||
### Daten einfügen
|
||||
|
||||
```sql
|
||||
INSERT INTO users (id, name, email, age) VALUES (1, 'Alice', 'alice@test.com', 30);
|
||||
INSERT INTO users (id, name, email, age) VALUES (2, 'Bob', 'bob@test.com', 25);
|
||||
```
|
||||
|
||||
### Daten abfragen
|
||||
|
||||
```sql
|
||||
SELECT name, age FROM users WHERE age > 18;
|
||||
```
|
||||
|
||||
### Indizes erstellen
|
||||
|
||||
```sql
|
||||
-- BTree Index
|
||||
CREATE INDEX idx_name ON users(name) USING btree;
|
||||
|
||||
-- Volltext-Index
|
||||
CREATE INDEX idx_email_fts ON users(email) USING fts;
|
||||
|
||||
-- Vektor-Index
|
||||
CREATE INDEX idx_vec ON items(embedding) USING hnsw;
|
||||
```
|
||||
|
||||
## Vector Search
|
||||
|
||||
```sql
|
||||
CREATE TABLE docs (id INTEGER PRIMARY KEY, content TEXT, embedding VECTOR(768));
|
||||
CREATE INDEX docs_vec ON docs(embedding) USING hnsw;
|
||||
|
||||
-- Ähnlichkeitssuche
|
||||
SELECT id, cosine_distance(embedding, '[0.1, 0.2, ...]') AS dist
|
||||
FROM docs ORDER BY dist ASC LIMIT 10;
|
||||
```
|
||||
|
||||
## Graph Engine
|
||||
|
||||
```sql
|
||||
CREATE GRAPH social;
|
||||
INSERT INTO social_nodes (id, node_label) VALUES (1, 'Alice'), (2, 'Bob');
|
||||
INSERT INTO social_edges (source_id, dest_id) VALUES (1, 2);
|
||||
|
||||
-- BFS Traversal
|
||||
SELECT * FROM GRAPH_TABLE(social MATCH (n)-[r]->(m) ALGORITHM bfs COLUMNS (id, node_label));
|
||||
|
||||
-- PageRank
|
||||
SELECT * FROM GRAPH_TABLE(social ALGORITHM pagerank COLUMNS (id, node_label, rank));
|
||||
|
||||
-- Community Detection (Louvain)
|
||||
SELECT * FROM GRAPH_TABLE(social ALGORITHM community COLUMNS (id, node_label, community));
|
||||
|
||||
-- Kürzester Pfad
|
||||
SELECT * FROM GRAPH_TABLE(social ALGORITHM shortest_path START 1 END 2 COLUMNS (id, node_label));
|
||||
|
||||
-- Knoten-Ähnlichkeit (Jaccard)
|
||||
SELECT similarity_nodes('social', 'jaccard') AS result;
|
||||
```
|
||||
|
||||
## AI Pipeline
|
||||
|
||||
```sql
|
||||
-- Text in Chunks zerlegen
|
||||
SELECT chunk('Langer Text hier...', 1024, 128) AS result;
|
||||
|
||||
-- Embedding generieren (mit konfiguriertem externen Service)
|
||||
SELECT embed_text('Suchanfrage') AS result;
|
||||
|
||||
-- Schema-Prompt für LLM generieren
|
||||
SELECT schema_prompt('users') AS result;
|
||||
|
||||
-- Natural Language → SQL (mit konfiguriertem LLM)
|
||||
SELECT nl_to_sql('Zeige alle Benutzer über 25', 'users') AS result;
|
||||
|
||||
-- Cypher zu BaraQL übersetzen
|
||||
SELECT cypher('MATCH (a)-[r]->(b) RETURN a.node_label, b.node_label') AS result;
|
||||
```
|
||||
|
||||
## HTTP/REST API
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:9470/query \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"query": "SELECT * FROM users"}'
|
||||
```
|
||||
|
||||
## Konfiguration
|
||||
|
||||
```bash
|
||||
# Umgebungsvariablen
|
||||
export BARADB_DATA_DIR=./data
|
||||
export BARADB_EMBED_ENDPOINT=http://localhost:11434/api/embeddings
|
||||
export BARADB_EMBED_MODEL=nomic-embed-text
|
||||
export BARADB_LLM_ENDPOINT=http://localhost:11434/api/generate
|
||||
export BARADB_LLM_MODEL=llama3
|
||||
```
|
||||
@@ -0,0 +1,129 @@
|
||||
# Vektor-Suche
|
||||
|
||||
Native HNSW und IVF-PQ Indizes für Ähnlichkeitssuche mit vollständiger SQL-Integration.
|
||||
|
||||
## SQL — Vektor-Spalten
|
||||
|
||||
```sql
|
||||
CREATE TABLE items (
|
||||
id INT PRIMARY KEY,
|
||||
embedding VECTOR(768)
|
||||
);
|
||||
```
|
||||
|
||||
Der `VECTOR(n)`-Typ speichert float32-Arrays mit fester Dimension `n`.
|
||||
|
||||
## Vektoren einfügen
|
||||
|
||||
```sql
|
||||
INSERT INTO items (id, embedding) VALUES (1, '[0.1, 0.2, 0.3, ...]');
|
||||
```
|
||||
|
||||
## Vektor-Distanzfunktionen
|
||||
|
||||
```sql
|
||||
-- Kosinus-Distanz (0 = identisch, 1 = orthogonal)
|
||||
SELECT id, cosine_distance(embedding, '[0.1, 0.2, 0.3]') AS dist FROM items;
|
||||
|
||||
-- Euklidische / L2 Distanz
|
||||
SELECT id, euclidean_distance(embedding, '[0.1, 0.2, 0.3]') AS dist FROM items;
|
||||
SELECT id, embedding <-> '[0.1, 0.2, 0.3]' AS dist FROM items;
|
||||
|
||||
-- Inneres Produkt (negativ für Minimierung)
|
||||
SELECT id, inner_product(embedding, '[0.1, 0.2, 0.3]') AS dist FROM items;
|
||||
|
||||
-- Manhattan / L1 Distanz
|
||||
SELECT id, l1_distance(embedding, '[0.1, 0.2, 0.3]') AS dist FROM items;
|
||||
```
|
||||
|
||||
## Vektor-Indizes
|
||||
|
||||
```sql
|
||||
-- HNSW-Index für approximative Nächste-Nachbarn-Suche
|
||||
CREATE INDEX idx_items_vec ON items(embedding) USING hnsw;
|
||||
|
||||
-- Der Index wird bei INSERT und UPDATE automatisch aktualisiert
|
||||
```
|
||||
|
||||
## Hybrid RAG Search
|
||||
|
||||
```sql
|
||||
-- Kombinierte Vektor- + Volltext-Suche mit RRF-Reranking
|
||||
SELECT hybrid_search('AI query', embedding, content, 10) AS result;
|
||||
|
||||
-- Gefilterte hybride Suche
|
||||
SELECT hybrid_search_filtered('AI query', embedding, content, 10, 'category', 'news') AS result;
|
||||
```
|
||||
|
||||
## AI Pipeline
|
||||
|
||||
### Text-Chunking
|
||||
|
||||
```sql
|
||||
-- Text in überlappende Chunks zerlegen
|
||||
SELECT chunk('Langer Text hier...', 1024, 128) AS result;
|
||||
|
||||
-- Ergebnis: [{"index":0, "text":"...", "size":124}, ...]
|
||||
```
|
||||
|
||||
### Embedding-Generierung
|
||||
|
||||
```sql
|
||||
-- Externen Embedding-Service aufrufen (konfiguriert via Umgebungsvariablen)
|
||||
SELECT embed_text('Suchtext hier') AS result;
|
||||
```
|
||||
|
||||
Umgebungsvariablen für den Embedder:
|
||||
```bash
|
||||
export BARADB_EMBED_ENDPOINT=http://localhost:11434/api/embeddings
|
||||
export BARADB_EMBED_MODEL=nomic-embed-text
|
||||
```
|
||||
|
||||
### Auto-Embedding bei INSERT
|
||||
|
||||
Wenn eine VECTOR-Spalte NULL ist, aber eine TEXT-Spalte einen Wert hat, wird das Embedding automatisch generiert (falls ein Embedder konfiguriert ist).
|
||||
|
||||
```sql
|
||||
CREATE TABLE docs (id INTEGER PRIMARY KEY, content TEXT, embedding VECTOR(768));
|
||||
CREATE INDEX docs_vec ON docs(embedding) USING hnsw;
|
||||
|
||||
-- embedding wird automatisch gefüllt
|
||||
INSERT INTO docs (id, content) VALUES (1, 'Dieser Text wird automatisch embedded');
|
||||
```
|
||||
|
||||
## Distanzmetriken
|
||||
|
||||
| Metrik | SQL-Funktion | Beschreibung |
|
||||
|--------|-------------|-------------|
|
||||
| `cosine` | `cosine_distance(a, b)` | Kosinus-Distanz |
|
||||
| `euclidean` | `euclidean_distance(a, b)` / `<->` | L2-Distanz |
|
||||
| `dotproduct` | `inner_product(a, b)` | Negatives Skalarprodukt |
|
||||
| `manhattan` | `l1_distance(a, b)` | L1-Distanz |
|
||||
|
||||
## Index-Typen
|
||||
|
||||
### HNSW (Standard)
|
||||
|
||||
```nim
|
||||
import barabadb/vector/engine
|
||||
var hnsw = newHNSWIndex(dimensions = 128, m = 16, efConstruction = 200)
|
||||
```
|
||||
|
||||
### IVF-PQ
|
||||
|
||||
```nim
|
||||
var ivfpq = newIVFPQIndex(dimensions = 128, numCentroids = 256, subQuantizers = 8)
|
||||
```
|
||||
|
||||
## Native Nim API
|
||||
|
||||
```nim
|
||||
import barabadb/vector/engine
|
||||
|
||||
var idx = newHNSWIndex(dimensions = 128)
|
||||
idx.insert(1, @[1.0'f32, 0.0'f32, ...], {"category": "A"}.toTable)
|
||||
|
||||
let results = idx.search(queryVector, k = 10)
|
||||
let filtered = idx.searchWithFilter(queryVector, k = 10,
|
||||
filter = proc(meta: Table[string, string]): bool = return "category" in meta)
|
||||
```
|
||||
Reference in New Issue
Block a user