baraba/Baradb
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(backup): add dry-run, force, history log, auto-verify, rollback, disk check

Browse Source
This commit is contained in:
dimgigov
2026-05-06 23:20:08 +03:00
parent 7f2f5664fa
commit be370fb76b
2 changed files with 277 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+132 -22
View File
@@ -664,19 +664,21 @@ BARADB_LOG_LEVEL=debug ./build/baradadb
## Backup & Recovery
BaraDB includes a built-in backup manager that creates compressed tar.gz
snapshots of your data directory. The manager supports online backups
(server does not need to stop), integrity verification, retention policies,
and safe restore with automatic rollback protection.
snapshots of your data directory. The manager supports **online backups**
(server does not need to stop), **integrity verification**, **retention policies**,
**dry-run restore previews**, **automatic rollback protection**, and a full
**restore history log**.
### Quick Reference
| Command | Purpose |
|---------|---------|
| `backup backup` | Create a new snapshot |
| `backup restore` | Restore data from a snapshot |
| `backup restore` | Restore data from a snapshot (auto-verifies first) |
| `backup list` | Show all snapshots |
| `backup verify` | Check archive integrity |
| `backup cleanup` | Delete old snapshots |
| `backup verify` | Check archive integrity without extracting |
| `backup cleanup` | Delete old snapshots, keep N most recent |
| `backup history` | Show log of all restore operations |
| `backup help` | Show full help text |
### Build the Backup Tool
@@ -685,6 +687,12 @@ and safe restore with automatic rollback protection.
nim c -o:build/backup src/barabadb/core/backup.nim
```
For production use, compile with release optimizations:
```bash
nim c -d:release -o:build/backup src/barabadb/core/backup.nim
```
### Creating Backups
**Basic backup** — creates `backup_<timestamp>.tar.gz` in the current directory:
@@ -742,7 +750,8 @@ Found 3 backup(s):
### Verifying Backups
Always verify a snapshot before restoring, especially after transferring it
over the network:
over the network. The restore command does this automatically, but you can
also check manually:
```bash
./build/backup verify --input=backup_1715011200.tar.gz
@@ -758,30 +767,87 @@ A corrupted archive prints an error and exits with code 1.
### Restoring from Backup
The restore command follows a **safe restore workflow**:
1. **Verify** archive integrity automatically
2. **Prompt** for confirmation (unless `--force` is used)
3. **Move** existing data to `data/server.old_<timestamp>`
4. **Extract** the archive
5. **Rollback** automatically if extraction fails
6. **Log** the operation to `backup_history.log`
> ⚠️ **WARNING:** Restore replaces the existing data directory. The old data
> is automatically moved to `data/server.old_<timestamp>` before extraction.
> If extraction fails, the tool attempts an automatic rollback to the old data.
**Interactive restore** (asks for confirmation):
```bash
./build/backup restore --input=backup_1715011200.tar.gz
```
You will be prompted for confirmation:
You will be prompted:
```
Verifying archive before restore...
Archive is valid: backup_1715011200.tar.gz (12.45 MB)
WARNING: This will REPLACE the data in: data/server
Continue? [y/N]
```
**Force restore** — skip confirmation (for scripts and automation):
```bash
./build/backup restore --input=backup.tar.gz --force
```
**Dry-run restore** — preview what would happen without making changes:
```bash
./build/backup restore --input=backup.tar.gz --dry-run
```
Output:
```
DRY-RUN: The following actions would be performed:
1. Verify archive integrity: backup.tar.gz
2. Move existing data to: data/server.old_1778099200
3. Extract archive to: data/server
Archive size: 12.45 MB
Free space: 45.20 GB
```
**Restore to a different data directory**:
```bash
./build/backup restore --input=backup.tar.gz --data-dir=data/recovered
```
**Skip confirmation (scripting)** — pipe `y`:
**Verbose restore** (shows all steps and disk space check):
```bash
echo "y" | ./build/backup restore --input=backup.tar.gz
./build/backup restore --input=backup.tar.gz --verbose
```
### Restore History
Every restore operation is logged to `backup_history.log` in the current
directory. View the history:
```bash
./build/backup history
```
Example output:
```
Restore history:
--------------------------------------------------------------------------------
[2026-05-06 23:15:00] SUCCESS restore from /backups/backup_1715011200.tar.gz to /opt/baradb/data/server
[2026-05-06 22:30:15] FAILED restore from /backups/backup_1715007000.tar.gz to /opt/baradb/data/server
[2026-05-05 08:00:00] DRY-RUN restore from /backups/backup_1714900000.tar.gz to /opt/baradb/data/server
--------------------------------------------------------------------------------
```
### Cleanup & Retention
@@ -799,18 +865,33 @@ Delete old snapshots automatically, keeping only the N most recent:
./build/backup cleanup --keep=3 --verbose
```
### Full Option Reference
### Automated Backups with Cron
| Option | Short | Default | Description |
|--------|-------|---------|-------------|
| `--data-dir` | `-d` | `data/server` | Path to the data directory |
| `--output` | `-o` | auto-generated | Destination path for new backup |
| `--input` | `-i` | — | Source archive for restore/verify |
| `--keep` | `-k` | `5` | Number of snapshots to retain |
| `--exclude` | `-e` | — | Exclude pattern (repeatable) |
| `--level` | `-l` | `6` | Gzip compression 0-9 |
| `--verbose` | `-v` | off | Detailed progress output |
| `--help` | `-h` | — | Show help text |
Add to your crontab for daily backups at 2 AM:
```bash
# Edit crontab
crontab -e
# Add this line for daily backups
0 2 * * * cd /opt/baradb && ./build/backup backup --output=/backups/baradb_$(date +\%F).tar.gz --level=6 >> /var/log/baradb-backup.log 2>&1
# Weekly cleanup — keep last 7 snapshots
0 3 * * 0 cd /opt/baradb && ./build/backup cleanup --keep=7 >> /var/log/baradb-backup.log 2>&1
```
### Disaster Recovery Best Practices
1. **3-2-1 Rule:** Keep 3 copies, on 2 different media, with 1 offsite.
2. **Verify regularly:** Run `backup verify` on archived snapshots monthly.
3. **Test restores:** Perform a dry-run restore (`--dry-run`) weekly and a
full test restore to a staging environment monthly.
4. **Monitor disk space:** The restore command warns if free space is less
than 2× the archive size.
5. **Keep old data:** After restore, the previous data is preserved as
`data/server.old_<timestamp>`. Only delete it after confirming the new
data works.
6. **Log audit trail:** Use `backup history` to track all restore operations.
### Nim API
@@ -832,13 +913,42 @@ for b in backups:
# Verify without extracting
let valid = verifyArchive("snapshot.tar.gz")
# Restore (destructive!)
# Restore with rollback protection
let restored = restoreDataDir("snapshot.tar.gz", "data/server")
# Dry-run restore — preview without changes
let preview = restoreDataDir("snapshot.tar.gz", "data/server", dryRun = true)
# Cleanup retention
cleanupOldBackups("data/server", keepLast = 5)
# Read restore history
for entry in readHistory():
echo entry
```
### Full Option Reference
| Option | Short | Default | Description |
|--------|-------|---------|-------------|
| `--data-dir` | `-d` | `data/server` | Path to the data directory |
| `--output` | `-o` | auto-generated | Destination path for new backup |
| `--input` | `-i` | — | Source archive for restore/verify |
| `--keep` | `-k` | `5` | Number of snapshots to retain |
| `--exclude` | `-e` | — | Exclude pattern (repeatable) |
| `--level` | `-l` | `6` | Gzip compression 0-9 |
| `--dry-run` | — | off | Preview restore without changes |
| `--force` | `-f` | off | Skip confirmation prompts |
| `--verbose` | `-v` | off | Detailed progress output |
| `--help` | `-h` | — | Show help text |
### Exit Codes
| Code | Meaning |
|------|---------|
| `0` | Success |
| `1` | Error (invalid args, missing files, verification or extraction failure) |
### Point-in-Time Recovery (WAL)
For fine-grained recovery, replay the WAL from a checkpoint:
src/barabadb/core/backup.nim
+145 -13
View File
@@ -6,6 +6,7 @@
## list List available snapshots with size & timestamp
## verify Check archive integrity without extracting
## cleanup Remove old snapshots keeping N most recent
## history Show restore operation log
## help Show detailed usage information
##
## Options:
@@ -15,6 +16,8 @@
## --keep, -k <N> Number of snapshots to retain (default: 5)
## --exclude, -e <PAT> Exclude pattern (can be used multiple times)
## --level, -l <0-9> Compression level for gzip (default: 6)
## --dry-run Show what would be done without doing it
## --force, -f Skip confirmation prompts
## --verbose, -v Enable verbose output
import std/os
@@ -22,7 +25,6 @@ import std/osproc
import std/strutils
import std/times
import std/algorithm
import std/sequtils
import std/parseopt
type
@@ -36,6 +38,7 @@ const
DEFAULT_DATA_DIR = "data/server"
DEFAULT_KEEP_COUNT = 5
DEFAULT_COMPRESSION = 6
HISTORY_FILE = "backup_history.log"
HELP_TEXT = """
BaraDB Backup Manager — Archive and restore your database safely
================================================================
@@ -49,6 +52,7 @@ COMMANDS:
restore Replace the current data directory with contents from a snapshot.
WARNING: This DESTROYS existing data. Use with care.
Automatically verifies archive integrity before extracting.
list Show all snapshots found next to the data directory.
Displays size, timestamp and compression ratio if known.
@@ -59,6 +63,8 @@ COMMANDS:
cleanup Delete old snapshots, keeping only the N most recent ones.
Default retention is 5 snapshots.
history Show log of all restore operations performed on this system.
help Show this help message.
OPTIONS:
@@ -68,6 +74,8 @@ OPTIONS:
-k, --keep <N> Retention count for cleanup (default: 5)
-e, --exclude <PAT> Exclude files matching pattern (repeatable)
-l, --level <0-9> Gzip compression level (default: 6, max: 9)
--dry-run Show what restore would do without changing anything
-f, --force Skip confirmation prompts (use with caution!)
-v, --verbose Print detailed progress information
EXAMPLES:
@@ -80,6 +88,12 @@ EXAMPLES:
# Restore from a specific snapshot
backup restore --input=backup_1715011200.tar.gz
# Dry-run restore — preview what will happen
backup restore --input=backup.tar.gz --dry-run
# Force restore without confirmation (automation/scripts)
backup restore --input=backup.tar.gz --force
# List all snapshots
backup list
@@ -91,6 +105,10 @@ EXAMPLES:
# Exclude WAL files from backup
backup backup --exclude="*.log" --exclude="wal/*"
EXIT CODES:
0 Success
1 General error (invalid arguments, missing files, tar failure, etc.)
"""
proc formatBytes*(bytes: int64): string =
@@ -124,6 +142,58 @@ proc parseBackupFilename*(filename: string): int64 =
except:
result = 0
proc getArchiveSize*(input: string): int64 =
## Return uncompressed size estimate from tar archive
let cmd = "tar -tzf " & quoteShell(input) & " | wc -l"
let (outStr, exitCode) = execCmdEx(cmd)
if exitCode == 0:
try:
result = parseBiggestInt(strip(outStr))
except:
result = 0
else:
result = 0
proc getFreeSpace*(path: string): int64 =
## Return free disk space in bytes for the filesystem containing path
let cmd = "df -B1 " & quoteShell(path) & " | tail -1 | awk '{print $4}'"
let (outStr, exitCode) = execCmdEx(cmd)
if exitCode == 0:
try:
result = parseBiggestInt(strip(outStr))
except:
result = -1
else:
result = -1
proc logRestore*(archive: string, dataDir: string, success: bool, dryRun: bool = false) =
## Append restore operation to history log
let logPath = getCurrentDir() / HISTORY_FILE
let status = if dryRun: "DRY-RUN" elif success: "SUCCESS" else: "FAILED"
let entry = "[" & format(getTime(), "yyyy-MM-dd HH:mm:ss") & "] " &
status & " restore from " & absolutePath(archive) &
" to " & absolutePath(dataDir) & "\n"
try:
let f = open(logPath, fmAppend)
f.write(entry)
f.close()
except:
discard # Silently fail if log cannot be written
proc readHistory*(): seq[string] =
## Read restore history log
result = @[]
let logPath = getCurrentDir() / HISTORY_FILE
if not fileExists(logPath):
return
try:
let content = readFile(logPath)
for line in splitLines(content):
if line.len > 0:
result.add(line)
except:
discard
proc backupDataDir*(dataDir: string, output: string, excludes: seq[string] = @[], compression: int = DEFAULT_COMPRESSION, verbose: bool = false): bool =
## Create a tar.gz backup of the data directory
if not dirExists(dataDir):
@@ -162,17 +232,38 @@ proc backupDataDir*(dataDir: string, output: string, excludes: seq[string] = @[]
echo " Source: ", dataDir
return true
proc restoreDataDir*(input: string, dataDir: string, verbose: bool = false): bool =
## Restore from a tar.gz backup
proc restoreDataDir*(input: string, dataDir: string, verbose: bool = false, dryRun: bool = false): bool =
## Restore from a tar.gz backup.
## When dryRun is true, only prints what would be done.
if not fileExists(input):
echo "ERROR: Backup file not found: ", input
return false
let archiveSize = getFileSize(input)
let freeSpace = getFreeSpace(parentDir(dataDir))
let oldBackupPath = dataDir & ".old_" & $getTime().toUnix()
if dryRun:
echo "DRY-RUN: The following actions would be performed:"
echo " 1. Verify archive integrity: ", input
echo " 2. Move existing data to: ", oldBackupPath
echo " 3. Extract archive to: ", dataDir
echo " Archive size: ", formatBytes(archiveSize)
if freeSpace >= 0:
echo " Free space: ", formatBytes(freeSpace)
else:
echo " Free space: unable to determine"
return true
# Check free space
if freeSpace >= 0 and freeSpace < archiveSize * 2:
echo "WARNING: Free space (", formatBytes(freeSpace), ") may be insufficient."
echo " Archive is ", formatBytes(archiveSize), " — at least 2x is recommended."
if dirExists(dataDir):
let backupOld = dataDir & ".old_" & $getTime().toUnix()
if verbose:
echo "Moving existing data to: ", backupOld
moveDir(dataDir, backupOld)
echo "Moving existing data to: ", oldBackupPath
moveDir(dataDir, oldBackupPath)
createDir(dataDir)
@@ -185,10 +276,18 @@ proc restoreDataDir*(input: string, dataDir: string, verbose: bool = false): boo
echo "ERROR: tar extraction failed with exit code ", exitCode
if outputStr.len > 0:
echo outputStr
# Attempt rollback
if dirExists(oldBackupPath):
echo "Attempting rollback..."
removeDir(dataDir)
moveDir(oldBackupPath, dataDir)
echo "Rollback complete. Data restored to previous state."
return false
echo "Restored successfully from: ", input
echo " Target: ", dataDir
if dirExists(oldBackupPath):
echo " Old data preserved at: ", oldBackupPath
return true
proc verifyArchive*(input: string, verbose: bool = false): bool =
@@ -276,6 +375,18 @@ proc cleanupOldBackups*(dataDir: string, keepLast: int = DEFAULT_KEEP_COUNT, ver
echo "Cleanup complete."
proc printHistory*() =
## Display restore history
let entries = readHistory()
if entries.len == 0:
echo "No restore history found."
return
echo "Restore history:"
echo repeat("-", 80)
for entry in entries:
echo entry
echo repeat("-", 80)
# =============================================================================
# CLI Entry Point
# =============================================================================
@@ -288,6 +399,8 @@ when isMainModule:
excludes: seq[string] = @[]
compression = DEFAULT_COMPRESSION
verbose = false
dryRun = false
force = false
for kind, key, val in getopt():
case kind
@@ -311,6 +424,8 @@ when isMainModule:
if compression < 0 or compression > 9:
quit("ERROR: --level must be between 0 and 9", 1)
except: quit("ERROR: --level must be a number", 1)
of "dry-run": dryRun = true
of "force", "f": force = true
of "verbose", "v": verbose = true
of "help", "h":
echo HELP_TEXT
@@ -333,13 +448,27 @@ when isMainModule:
of "restore":
if target.len == 0:
quit("ERROR: restore requires --input=<file.tar.gz>\nUse 'backup help' for usage.", 1)
echo "WARNING: This will REPLACE the data in: ", dataDir
echo "Continue? [y/N] "
let answer = readLine(stdin)
if answer.toLowerAscii() notin ["y", "yes"]:
quit("Restore cancelled", 0)
let ok = restoreDataDir(target, dataDir, verbose)
if not ok:
# Always verify first unless dry-run
if not dryRun:
echo "Verifying archive before restore..."
let vok = verifyArchive(target, verbose)
if not vok:
logRestore(target, dataDir, false)
quit("Restore aborted: archive verification failed.", 1)
if not dryRun and not force:
echo "WARNING: This will REPLACE the data in: ", dataDir
echo "Continue? [y/N] "
let answer = readLine(stdin)
if answer.toLowerAscii() notin ["y", "yes"]:
echo "Restore cancelled."
logRestore(target, dataDir, false, dryRun = false)
quit(0)
let ok = restoreDataDir(target, dataDir, verbose, dryRun)
logRestore(target, dataDir, ok, dryRun)
if not ok and not dryRun:
quit("Restore failed", 1)
of "list":
@@ -356,6 +485,9 @@ when isMainModule:
of "cleanup":
cleanupOldBackups(dataDir, keepCount, verbose)
of "history":
printHistory()
of "help":
echo HELP_TEXT