feat: rate limiting, shared DB instance, TCP_NODELAY fix

- Add rate limiter to TCP and HTTP servers
- Share single LSMTree between TCP and HTTP servers
- Replace unsafe cast[string]/cast[seq[byte]] with safe helpers
- Stabilize gossip UDP socket with exponential backoff
- Fix TCP_NODELAY via low-level setSockOptInt(IPPROTO_TCP)
  (asyncnet.setSockOpt(OptNoDelay) uses SOL_SOCKET and fails)
- Apply TCP_NODELAY to server and websocket sockets
This commit is contained in:
2026-05-18 19:33:40 +03:00
parent 7e6a45e6b7
commit 8afa998516
13 changed files with 240 additions and 47 deletions
+23 -2
View File
@@ -34,10 +34,31 @@ Multiple communication protocols:
- **WebSocket** (`core/websocket.nim`): Full-duplex streaming
- **Embedded** (`storage/lsm.nim`): Direct in-process access
### Server Architecture
The TCP and HTTP servers share a single LSMTree instance to ensure data consistency:
```
┌─────────────────────────────────────────────────────────┐
│ SHARED STORAGE │
│ LSMTree Instance │
├────────────────────────┬────────────────────────────────┤
│ TCP Server │ HTTP Server │
│ (Binary Protocol) │ (REST API) │
│ Port: 9472 │ Port: 9912 │
│ TCP_NODELAY: ON │ Multi-threaded │
└────────────────────────┴────────────────────────────────┘
```
**Key optimizations:**
- **Shared LSMTree** — Both servers operate on the same database instance, eliminating data inconsistency
- **TCP_NODELAY** — Enabled on both listening and client sockets for lower latency on small messages
- **Safe byte conversion** — Proper `bytesToString`/`stringToBytes` functions instead of unsafe `cast` operations
### Connection Management
- **Connection Pool** (`protocol/pool.nim`): Min/max connection limits with idle timeout
- **Rate Limiting** (`protocol/ratelimit.nim`): Token-bucket global and per-client limits
- **Rate Limiting** (`protocol/ratelimit.nim`): Token-bucket global and per-client limits, integrated in both TCP and HTTP handlers
- **Authentication** (`protocol/auth.nim`): JWT with HMAC-SHA256 and role-based access
- **TLS/SSL** (`protocol/ssl.nim`): TLS 1.3 with auto-generated certificates
@@ -121,7 +142,7 @@ Client → Protocol → Auth → Parser → AST → IR → Codegen
- **Raft Consensus** (`core/raft.nim`): Leader election, log replication
- **Sharding** (`core/sharding.nim`): Hash, range, and consistent hashing
- **Replication** (`core/replication.nim`): Sync, async, semi-sync modes
- **Gossip Protocol** (`core/gossip.nim`): SWIM-like membership management
- **Gossip Protocol** (`core/gossip.nim`): SWIM-like membership management with exponential backoff error recovery
- **Distributed Transactions** (`core/disttxn.nim`): Two-phase commit
## Key Design Decisions
+15
View File
@@ -2,6 +2,21 @@
All notable changes to BaraDB are documented in this file.
## [Unreleased] — Server Architecture Improvements
### Fixed
- **Shared Storage Instance** — TCP and HTTP servers now share a single LSMTree instance instead of creating separate ones, eliminating data inconsistency between protocols
- **Unsafe Cast Operations** — Replaced all `cast[string]` and `cast[seq[byte]]` with safe `bytesToString`/`stringToBytes` helper functions that properly copy data for ARC/ORC memory management compatibility
### Changed
- **TCP_NODELAY Enabled** — Both listening and client sockets now have TCP_NODELAY set, reducing latency for small messages (queries, acks) by disabling Nagle's algorithm
- **Rate Limiter Integrated** — Token-bucket rate limiter is now active in both TCP and HTTP server handlers:
- TCP: Per-client rate limiting on query messages (error code 429)
- HTTP: Rate limiting on `/query` and `/batch` endpoints with `X-Forwarded-For` support
- **Gossip Error Handling** — Improved UDP socket error recovery with exponential backoff (100ms-5s) and socket recreation only after 10 consecutive failures
## [Unreleased] — AI-Native Platform
### Added
+4 -2
View File
@@ -45,8 +45,10 @@ BaraDB can be configured via **environment variables**, a **config file**, or **
|----------|---------|-------------|
| `BARADB_AUTH_ENABLED` | `false` | Enable authentication |
| `BARADB_JWT_SECRET` | — | JWT signing secret |
| `BARADB_RATE_LIMIT_GLOBAL` | `10000` | Global requests per second |
| `BARADB_RATE_LIMIT_PER_CLIENT` | `1000` | Per-client requests per second |
| `BARADB_RATE_LIMIT_GLOBAL` | `10000` | Global requests per minute (token-bucket) |
| `BARADB_RATE_LIMIT_PER_CLIENT` | `1000` | Per-client requests per minute (token-bucket) |
> **Note:** Rate limiting is enforced at the protocol level in both TCP and HTTP servers. TCP uses client connection ID for per-client limits. HTTP uses `X-Forwarded-For` header (for proxy setups) or a global key.
### Logging
+17
View File
@@ -11,6 +11,11 @@ BaraDB supports multiple protocols for client communication:
The binary protocol uses big-endian encoding for all multi-byte values.
### Connection Optimizations
- **TCP_NODELAY** — Enabled on both server and client sockets to minimize latency for small messages (queries, acks). This disables Nagle's algorithm, ensuring packets are sent immediately without buffering.
- **Rate Limiting** — Token-bucket rate limiter is integrated at the protocol level. Queries exceeding the rate limit receive error code 429.
### Connection Lifecycle
```
@@ -142,6 +147,18 @@ printf '\x00\x00\x00\x12\x01\x02\x00\x00\x00\x00\x08SELECT 1' > /dev/tcp/localho
Base URL: `http://localhost:9470/api/v1`
### Rate Limiting
All query endpoints (`/query`, `/batch`) are protected by a token-bucket rate limiter:
- **Global rate** — Configured via `BARADB_RATE_LIMIT_GLOBAL` (default: 10,000 requests/minute)
- **Per-client rate** — Configured via `BARADB_RATE_LIMIT_PER_CLIENT` (default: 1,000 requests/minute)
- **Client identification** — Uses `X-Forwarded-For` header when behind a proxy, otherwise uses a global key
When rate-limited, the server returns HTTP 429 with:
```json
{"error": "Rate limit exceeded"}
```
### Endpoints
#### Health