Wiki

Backup & Recovery

How QuoteNode handles database and file backups — age-encrypted archives, manifest-driven restore, recovery kits, and runtime profiles.

Backup & Recovery

QuoteNode includes a built-in backup system that protects your commercial data against hardware failures, accidental deletions, and operational incidents.

What Is Backed Up

Each backup creates an archive containing:

Database dump — a full PostgreSQL dump in custom format (pg_dump --format=custom), including all tables, sequences, and constraints.
File storage — all uploaded files (product images, company logos) and generated PDFs, archived as files.tar.gz with relative paths.
Backup manifest — a backup-manifest.json file containing source version, encryption mode, database payload mode, checksums, and restore compatibility metadata.
Integrity checksums — a checksums.sha256 file containing SHA-256 hashes of all archive components.

The three encryption layers

A backup involves three independent secrets/settings. “Encrypted backup” is ambiguous — always name the layer:

AGE private identity — unlocks only the outer .tar.age archive. It comes from the Recovery Kit and is used at restore/verify time (RESTORE_AGE_IDENTITY_FILE). It does not contain or replace DB_ENCRYPTION_KEY.
DB_ENCRYPTION_KEY — decrypts the APP_ENCRYPTED database fields and the operational secrets (MFA, SMTP, offer-email payloads). Back it up separately from the archive, and never pass it as a command-line argument.
ENCRYPT_PII — the target runtime storage policy. It does not decrypt an archive; changing it never substitutes for the correct key or the restore postprocess.

Keep your .env / deployment secrets and the Recovery Kit in separate protected locations.

Archive Encryption

New encrypted backups use age recipient encryption (AGE_RECIPIENT). This is the only supported encryption model for new archives.

The backup process uses only a public age recipient (age1...) — no private key is needed on the server to create backups.
Encrypted archives have a .tar.age extension.
The encrypted archive SHA-256 checksum is stored alongside the archive for operator-side verification.
Decryption requires the matching private age identity, which is held exclusively in the operator’s recovery kit.
The raw private age identity is never stored in the database, logs, DTOs, or process arguments after setup confirmation.

Unencrypted local backups

Local unencrypted backups (.tar) are allowed only when all of the following are true:

BACKUP_ARCHIVE_ENCRYPTION_MODE=NONE
BACKUP_ALLOW_UNENCRYPTED_LOCAL_BACKUP=true
Destination is local storage
BACKUP_DATABASE_PAYLOAD_MODE=APP_ENCRYPTED
The administrator has explicitly acknowledged the risk in the UI

Unencrypted archives are blocked for remote storage and for PII-decrypted payloads.

Configuration (.env). The age recipient is generated by the Recovery Kit (Admin → Security → Backup) and stored in the instance — you do not put a key in .env. Set only the two mode variables:

BACKUP_ENABLED=true
# Second, dedicated switch — default off. Set true only on the backup-worker to run scheduled backups.
BACKUP_SCHEDULER_ENABLED=true
BACKUP_ARCHIVE_ENCRYPTION_MODE=AGE_RECIPIENT
BACKUP_DATABASE_PAYLOAD_MODE=APP_ENCRYPTED
# Alternative — encrypted archive + decrypted PII (cross-instance portability):
# BACKUP_DATABASE_PAYLOAD_MODE=PII_DECRYPTED
BACKUP_ALLOW_UNENCRYPTED_LOCAL_BACKUP=false

To disable archive encryption (local only, not recommended — PII still stays encrypted in the payload):

BACKUP_ARCHIVE_ENCRYPTION_MODE=NONE
BACKUP_ALLOW_UNENCRYPTED_LOCAL_BACKUP=true

With AGE_RECIPIENT, if no recipient has been generated yet the backup fails loudly rather than writing an unprotected archive. NONE + PII_DECRYPTED is rejected.

Database Payload Modes

Mode	Description
`APP_ENCRYPTED`	Default. PII fields remain encrypted with the application `DB_ENCRYPTION_KEY`. Restoring onto the same key (matching fingerprint) needs no extra step; restoring onto a different key requires the original source key (supplied via `PII_REKEY_OLD_KEY`) for an offline rekey, otherwise the restore is blocked before any change.
`PII_DECRYPTED`	PII fields are decrypted into a temporary database before archiving. User password hashes remain as-is. Requires `AGE_RECIPIENT` archive encryption. Enables cross-instance portability without the old DB key — at the cost of resetting the operational secrets (see below).

When restoring a PII_DECRYPTED backup into a target with ENCRYPT_PII=true, the restore re-encrypts PII with the target’s DB_ENCRYPTION_KEY before the application serves traffic — this requires a target key to be configured, otherwise the restore is blocked. Into a target with ENCRYPT_PII=false the PII stays in plaintext and the restore requires an explicit RESTORE_ACKNOWLEDGE_PLAINTEXT_PII acknowledgement.

Recovery Kit

During setup, the administrator generates (or imports) an age recipient for encrypted backups. The recovery kit is a ZIP file containing:

quotenode-age-identity.txt — the private age identity (returned only once)
quotenode-age-recipient.txt — the public recipient and fingerprint
quotenode-backup-verify.sh — local verification script (Linux/macOS)
quotenode-backup-verify.ps1 — local verification script (Windows)
README.md — verification and restore commands

Critical: The private age identity is returned only once during kit generation. If lost, encrypted archives cannot be decrypted. Store the recovery kit in a secure offline location.

The kit is not everything. The age identity unlocks only the outer archive. It does not contain or replace DB_ENCRYPTION_KEY, which protects the PII and operational secrets inside the archive. Preserve DB_ENCRYPTION_KEY separately, and never store it in the kit.

The administrator must confirm that the kit was saved and tested before the setup wizard marks backup as production-ready.

Local Backup Verification

Downloaded backups can be verified outside the running application using the recovery kit scripts:

scripts/quotenode-backup-verify.sh \
  --archive /path/backup.tar.age \
  --identity /path/quotenode-age-identity.txt \
  --expected-sha256 <sha256-from-ui-or-sidecar>

The verifier:

Checks the encrypted archive SHA-256 against the expected value or .sha256 sidecar.
Decrypts with age -d into a private temp directory.
Validates tar entries (rejects absolute paths, .., symlinks, device files).
Validates backup-manifest.json exists and is a supported version.
Runs sha256sum -c checksums.sha256.
Runs pg_restore --list db.dump when pg_restore is available.
Removes decrypted temp data by default (use --keep-decrypted to retain).

Online Backup (Default)

Online backups run while the application is serving requests. There is no downtime.

pg_dump in custom format creates a transactionally consistent snapshot without locking tables or blocking queries.
File storage is archived in parallel — uploaded files and PDFs are immutable after creation, so the archive is always consistent.
PostgreSQL does not need to stop for a consistent database dump.

How Backups Run: Choosing a Topology

There are two ways to run scheduled backups continuously. Both produce identical archives — they differ only in where the backup job executes.

Topology	When to use	Containers
Dedicated worker (default)	Production. Backups run in a separate `backup-worker` container, so they never compete with web traffic for memory or CPU.	web backend + `backup-worker`
In-process	Custom single-backend setups with their own Compose file (the release packages do not use it). The web backend also runs backups itself — one less container to operate.	web backend only

The backup job itself is the same shell pipeline (pg_dump → archive → encrypt) in both cases, run outside the JVM heap, so its memory cost is modest.

How to select a topology

Two environment variables work together:

JOBS_MODE decides which container actually runs the scheduled backup job.
BACKUP_RUNTIME_PROFILE is a label recorded in backup logs and manifests so you can see, after the fact, how each backup was produced.

Topology	Web backend	Backup worker
Dedicated worker (default)	`JOBS_MODE=web` + `BACKUP_RUNTIME_PROFILE=WORKER`	`JOBS_MODE=backup-only` container
In-process	`JOBS_MODE=all` + `BACKUP_RUNTIME_PROFILE=WEB_MAINTENANCE`	(none)

The release packages use the dedicated worker topology for every installation (server, LAN, laptop, NAS and Coolify): both Compose files include a backup-worker service and set JOBS_MODE per service. In the standalone package this Compose setting takes precedence over .env, so in-process mode (JOBS_MODE=all + BACKUP_RUNTIME_PROFILE=WEB_MAINTENANCE) requires a Compose file you maintain yourself.

Runtime profile reference

BACKUP_RUNTIME_PROFILE accepts four values. The first two are for the continuous topologies above; the last two are for one-off, operator-driven backups.

Profile	Description
`WORKER`	Default. A permanent `backup-worker` container runs scheduled backups via cron.
`WEB_MAINTENANCE`	The web backend performs scheduled backups directly. No extra worker container needed.
`ONE_SHOT`	An external cron or operator runs `operator/backup-one-shot.sh` and exits. No permanent worker.
`OFFLINE_MAINTENANCE`	Automated script announces downtime, stops frontend/backend/backup-worker, keeps PostgreSQL running, runs backup, restarts services, and verifies health.

One-shot backup

COMPOSE_PROJECT_NAME=quotenode COMPOSE_FILE=$PWD/docker-compose.yml COMPOSE_ENV_FILE=$PWD/.env \
  BACKUP_ARCHIVE_ENCRYPTION_MODE=AGE_RECIPIENT \
  BACKUP_AGE_RECIPIENT=age1... \
  ./operator/backup-one-shot.sh

Offline maintenance

COMPOSE_PROJECT_NAME=quotenode COMPOSE_FILE=$PWD/docker-compose.yml COMPOSE_ENV_FILE=$PWD/.env \
  BACKUP_OFFLINE_NOTICE_SECONDS=300 \
  ./operator/backup-offline-maintenance.sh

The offline maintenance script:

Announces scheduled maintenance (configurable grace period).
Optionally enables a static maintenance page while services are down.
Stops frontend, backend, and backup-worker (keeps PostgreSQL running).
Runs a one-shot backup.
Restarts all services.
Verifies backend health.
Writes a host-level JSONL maintenance journal.

If the script fails at any point, a safety trap restarts services automatically.

Use --dry-run to preview the plan without executing:

./operator/backup-offline-maintenance.sh --dry-run

Scheduling

Default schedule: 2:00 AM daily (0 0 2 * * *, configurable via BACKUP_CRON)
Activation (two switches): scheduled backups run only when both BACKUP_ENABLED=true and BACKUP_SCHEDULER_ENABLED=true. BACKUP_SCHEDULER_ENABLED is a dedicated kill-switch that defaults to off, so enabling backups never triggers a surprise first scheduled run. Set it only on the backup-worker tier (runtime profile WORKER/WEB_MAINTENANCE), never the web tier.
Manual trigger: Administrators can trigger an immediate backup from the admin panel or via API (POST /api/v1/admin/backup/trigger). The trigger returns immediately (HTTP 202) while the backup runs asynchronously.
Concurrency protection: A database-backed lock ensures only one backup runs at a time across all workers and manual triggers.

For ONE_SHOT and OFFLINE_MAINTENANCE profiles, the internal scheduler is disabled — backups are managed externally.

Storage Options

Local storage (default)

Backups are stored in BACKUP_LOCAL_DIR (default: /app/data/backups), mapped to a Docker volume.

For production, local backups should be supplemented with remote copies.

Remote storage via rclone

If BACKUP_RCLONE_REMOTE is configured, backups are automatically uploaded to a remote destination after local creation. rclone supports 70+ cloud storage providers including S3-compatible, Google Cloud Storage, Azure Blob, SFTP, and WebDAV.

Downloading Backups

Backup archive downloads require step-up authorization:

Navigate to Settings > Backup.
Click Download on the backup entry.
Enter your current password (and TOTP code if MFA is enabled).
The system issues a short-lived, one-time download grant (default: 120 seconds).
The archive downloads automatically.

The grant token is stored only as a SHA-256 hash. Grant creation and download are audited and other administrators are notified.

Note: Direct download without a grant returns 403. Backups on remote storage cannot be downloaded through the admin panel.

Metadata & Audit Trail

Every backup is tracked in the backup_logs table with:

Status (RUNNING, SUCCESS, FAILED)
Initiated by (SCHEDULER or ADMIN)
Start/completion time
Size (bytes) and SHA-256 checksum
Destination (local path or remote URL)
Manifest version and source app version
Archive encryption mode and fingerprint
Database payload mode
Runtime profile
Verification status (NOT_RUN, PASSED, FAILED, SKIPPED_TIMEOUT)
Restore test status and restore compatibility status

Restoration

Using `operator/restore.sh`

The canonical restore script handles the full lifecycle:

COMPOSE_PROJECT_NAME=quotenode COMPOSE_FILE_PATH=$PWD/docker-compose.yml COMPOSE_ENV_FILE=$PWD/.env \
  DB_USERNAME="$(grep '^DB_USERNAME=' .env | cut -d= -f2-)" \
  DB_NAME="$(grep '^DB_NAME=' .env | cut -d= -f2-)" \
  DB_PASSWORD="$(grep '^DB_PASSWORD=' .env | cut -d= -f2-)" \
  RESTORE_JOURNAL_DIR=$PWD/restore-journal \
  RESTORE_AGE_IDENTITY_FILE=/path/to/quotenode-age-identity.txt \
  [email protected] \
  RESTORE_ADMIN_PASSWORD_FILE=/path/to/password \
  ./operator/restore.sh /path/to/backup.tar.age

The script:

Extracts and validates the archive (manifest, checksums, tar entry safety).
Runs preflight checks (disk capacity, PostgreSQL version, DB connectivity).
Creates a target safety backup before any destructive operation.
Writes a host-level restore journal (JSONL, survives DB replacement).
Restores the database.
Stages files into a temp directory and swaps them into place.
Preserves target identity (instance ID, backup encryption metadata, license state).
Clears sessions, public offer links, and notification tokens.
Resets or force-changes admin access.
Re-encrypts PII if the backup is PII_DECRYPTED and the target has ENCRYPT_PII=true.

Cross-instance restore

When restoring a backup from a different installation, the default mode is PRESERVE_TARGET_IDENTITY:

Target instance ID, backup age recipient, recovery-kit metadata, and license state are preserved.
Sessions, public offer links, and notification tokens are cleared.
Admin password is reset/force-changed.
Business data is fully replaced (no merge).

Restore compatibility

The restore preflight validates:

Older backup into newer app: allowed (compatible).
Newer backup into older app: blocked (NEWER_SCHEMA).
Missing manifest: blocked (NO_MANIFEST).
APP_ENCRYPTED onto a different key with no verified source key: blocked before destructive work (SOURCE_DB_ENCRYPTION_KEY_REQUIRED) — supply the original key via PII_REKEY_OLD_KEY to rekey instead; a mismatched key reports SOURCE_DB_ENCRYPTION_KEY_MISMATCH.
Plaintext-PII target without acknowledgement: blocked (PLAINTEXT_PII_TARGET_NOT_ACKNOWLEDGED) — set RESTORE_ACKNOWLEDGE_PLAINTEXT_PII=true if intended.
PII_DECRYPTED without archive encryption: blocked (PII_DECRYPTED_NO_ARCHIVE_ENCRYPTION).

A blocked compatibility result mutates nothing.

Operational secrets when the key changes

MFA TOTP secrets, SMTP passwords and offer-email payloads are encrypted with the source DB_ENCRYPTION_KEY. On restore:

same key — everything stays readable, nothing to do;
different key + verified source key (PII_REKEY_OLD_KEY) — they are rekeyed to the target key automatically;
no verified source key (only reachable for PII_DECRYPTED) — they cannot be recovered and are reset after an explicit RESTORE_ACKNOWLEDGE_OPERATIONAL_SECRET_RESET=true: MFA registrations are removed (users re-enrol), SMTP credentials are cleared and outbound email stays disabled until re-entered, and offer-email bearer payloads are scrubbed.

Rollback guarantees

The destructive restore is a single rollback unit:

the source archive is never modified;
any failure before the database is written leaves the target unchanged;
a failure after the write rolls the database and files back together from the target safety backup;
if that rollback itself fails, the application is left stopped with all recovery artifacts preserved — do not start it; recover from the safety archive.

Admin access guarantee

Restore guarantees admin access through one of:

RESTORE_ADMIN_PASSWORD_FILE — operator-supplied password (set with forcePasswordChange=true)
RESTORE_ADMIN_PASSWORD — for local/test use only
Auto-generated one-time reset token — printed to console and host-level journal

Dry run

RESTORE_DRY_RUN=true ./operator/restore.sh /path/to/backup.tar.age

Validates the archive, checks checksums and manifest, and reports the restore plan without modifying anything.

Disaster Recovery — Rebuilding from Scratch

Your server is gone. You have a backup archive (.tar.age or .tar) and a new server with Docker. Here is how to restore.

Step 1 — Prepare the new server

mkdir quotenode && cd quotenode

Set up docker-compose.yml and .env as described in the Installation Guide.

Critical for APP_ENCRYPTED backups: the simplest recovery reuses the same DB_ENCRYPTION_KEY. Restoring onto a different key still works if you supply the original key via PII_REKEY_OLD_KEY for an offline rekey. Without the original key, APP_ENCRYPTED PII cannot be recovered (only a PII_DECRYPTED archive can, and its operational secrets must then be reset) — never generate a random replacement key.

Critical for encrypted archives: You need the recovery kit private identity file (quotenode-age-identity.txt) to decrypt the archive. It is a separate secret from DB_ENCRYPTION_KEY — you may need both.

Step 2 — Copy the backup and restore script

scp backup.tar.age user@new-server:~/quotenode/
# operator/restore.sh comes with the release package (the release that created the backup, or a later one)
curl -fsSLO https://quotenode.dev/releases/v0.9.8-alpha/quotenode-deploy-v0.9.8-alpha.tar.gz
tar -xzf quotenode-deploy-v0.9.8-alpha.tar.gz --strip-components=1

Step 3 — Run the restore

# Only a successful restore starts the remaining services (the restore itself starts PostgreSQL, the backend and
# the frontend). On failure nothing else starts and the restore's exit status is kept: read its message.
# Default installations keep PII unencrypted at rest (ENCRYPT_PII=false); a restore into such a target requires the
# explicit RESTORE_ACKNOWLEDGE_PLAINTEXT_PII acknowledgement below, for this command only. A target with
# ENCRYPT_PII=true in .env does not use it.
export COMPOSE_PROJECT_NAME=quotenode
COMPOSE_FILE_PATH=$PWD/docker-compose.yml COMPOSE_ENV_FILE=$PWD/.env \
  DB_USERNAME="$(grep '^DB_USERNAME=' .env | cut -d= -f2-)" \
  DB_NAME="$(grep '^DB_NAME=' .env | cut -d= -f2-)" \
  DB_PASSWORD="$(grep '^DB_PASSWORD=' .env | cut -d= -f2-)" \
  RESTORE_JOURNAL_DIR=$PWD/restore-journal \
  RESTORE_AGE_IDENTITY_FILE=./quotenode-age-identity.txt \
  [email protected] \
  RESTORE_ADMIN_PASSWORD_FILE=./admin-password \
  RESTORE_ACKNOWLEDGE_PLAINTEXT_PII=true \
  ./operator/restore.sh --fresh-install backup.tar.age \
  && docker compose up -d

If the restore reports a failure, do not start QuoteNode by hand: it would start an empty or incompletely restored installation. Read the message, fix the cause (for example the identity file or DB_ENCRYPTION_KEY) and run the block again. If the restore says it left the application stopped with recovery artifacts preserved, follow Rollback guarantees above.

Step 4 — Verify

Open your domain and log in. All customers, offers, products, and settings should be exactly as they were at backup time.

Tested disaster recovery

The full backup → isolated restore path is exercised end to end as an automated disaster-recovery drill (scenarios A–G): same-key restore, cross-key rekey, plaintext-PII acknowledgement, operational-secret reset, wrong-identity refusal, and post-restore rollback. It runs against a real PostgreSQL in a disposable environment and asserts the rollback and key-handling guarantees described above.

Backup Retention

Local: BACKUP_RETENTION_DAILY (default: 7) most recent successful backups are kept.
Remote: configure lifecycle policies on your storage provider.
PII-decrypted backups should have stricter retention due to their sensitive nature.

Backup Timeout

Each backup has a configurable timeout (default: 30 minutes via BACKUP_TIMEOUT_MINUTES). If exceeded, the backup is terminated and recorded as FAILED.

Recommended Strategy

Enable daily automatic backups with AGE_RECIPIENT encryption and remote storage.
Generate a recovery kit during setup and store it in a secure offline location.
Verify your recovery kit against a real backup using the included verifier scripts.
Test restore procedures periodically — a backup that has never been tested is not a backup.
Download a backup at least monthly to an offline location (requires step-up authorization).
Monitor backup status in the admin panel — warnings appear for stale, failed, or unencrypted backups.
Keep at least 7 days of backup history (the default).

Backup & Recovery

What Is Backed Up

The three encryption layers

Archive Encryption

Unencrypted local backups

Database Payload Modes

Recovery Kit

Local Backup Verification

Online Backup (Default)

How Backups Run: Choosing a Topology

How to select a topology

Runtime profile reference

One-shot backup

Offline maintenance

Scheduling

Storage Options

Local storage (default)

Remote storage via rclone

Downloading Backups

Metadata & Audit Trail

Restoration

Using operator/restore.sh

Cross-instance restore

Restore compatibility

Operational secrets when the key changes

Rollback guarantees

Admin access guarantee

Dry run

Disaster Recovery — Rebuilding from Scratch

Step 1 — Prepare the new server

Step 2 — Copy the backup and restore script

Step 3 — Run the restore

Step 4 — Verify

Tested disaster recovery

Backup Retention

Backup Timeout

Recommended Strategy

Using `operator/restore.sh`