Skip to content

Backup Runbook

This page converts the pre-reinstall checklist into the current scripted backup path. Restoration remains the priority, but future restore exercises depend on producing the same kind of complete, verifiable backup again.

Control Point

Run backup scripts on the Proxmox host as root. The Proxmox host owns:

  • pct and qm guest access.
  • Redirection into the NAS backup tree when NAS storage is active.

As of July 1, 2026, pvesm status showed only local and local-lvm; the historical network-backup-syn CIFS mount was not present. Recapture or restore the NAS backup target before running this backup path unchanged.

Do not run these scripts inside the Docker LXC.

Backup Root

Default:

/mnt/pve/network-backup-syn/proxmox-rebuild-${RUN_ID}

RUN_ID defaults to a timestamp, but the answer file may provide one. The prepare script refuses a backup root that is not on the expected CIFS mount. CIFS mode bits may not enforce 0700, so verify Synology ACLs before writing secret-bearing files.

This default is historical until network-backup-syn is restored and validated.

Scripted Flow

On Proxmox:

cp /path/to/repo/scripts/backup/backup-answer.env.example /root/backup-answer.env
chmod 600 /root/backup-answer.env

Edit /root/backup-answer.env, then run stages:

/path/to/repo/scripts/backup/00-prepare-backup-root.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/01-capture-proxmox.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/02-capture-dns.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/03-capture-docker-definitions.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/04-export-databases.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/05-archive-applications.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/06-archive-websites.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/07-verify-backup.sh --answer-file /root/backup-answer.env
/path/to/repo/scripts/backup/08-encrypt-and-copy-backup.sh --answer-file /root/backup-answer.env

run-backup.sh is available for a practiced run and requires:

CONFIRM_PRODUCTION_BACKUP=run-production-backup

Captured Material

The scripted path captures:

  • Proxmox configs, guest definitions, host networking, storage, and versions.
  • DNS reference from CT 107, plus optional manually supplied Pi-hole Teleporter archive.
  • Docker compose files, .env files, Dockerfiles, network inspect JSON, container inspect JSON, and compose project list.
  • PostgreSQL globals and per-database custom dumps.
  • MariaDB logical dumps.
  • MongoDB archive when a mongodb container exists.
  • Application archives for retained Docker projects, excluding logs, caches, and temporary paths.
  • Website working-tree archives when source repositories are not enough.
  • Optional pfSense raw XML supplied by the operator.
  • Checksums for produced artifacts.
  • Optional encrypted archive and second physical copy.

Manual Inputs

Some items must still be exported or approved manually:

  • pfSense raw encrypted XML from the pfSense UI. A redacted XML is not a recovery backup.
  • Pi-hole Teleporter archive from the Pi-hole UI.
  • Cloudflared token rotation and storage in the secret manager.
  • LXC 105 damaged raw image copy plan. Prefer ddrescue with a map file. Use dd only as a documented fallback. Never run modifying fsck against the only copy.
  • RustDesk identity retention decision if the service is still needed.
  • Second physical destination and age recipients file.

Verification

07-verify-backup.sh verifies:

  • No zero-length files.
  • tar -tzf for tar archives.
  • gzip -t for compressed SQL and MongoDB archives.
  • pg_restore --list for PostgreSQL dumps using the PostgreSQL container.
  • SHA-256 checksums for artifacts.

After backup verification, perform a restore exercise with the restore scripts. A backup is not considered proven until at least PostgreSQL, Forgejo, Vaultwarden, and Adminer are restored and validated in a test target.

Resume Behavior

  • Existing backup roots are reused when the same RUN_ID is supplied.
  • Stages write temporary files and rename them into place after success.
  • Rerun the failed stage after fixing the cause.
  • Do not remove partial .tmp files until you know whether they contain useful incident evidence.
  • Re-run 07-verify-backup.sh after any resumed capture.

Exclusions

Do not back up:

  • Docker runtime metadata under /var/lib/docker.
  • Docker images, stopped containers, build caches, package caches, and logs.
  • Pi-hole query history.
  • pfSense RRD or volatile extra data.
  • Generated static sites when source content is available and verified.
  • Database data directories as the primary backup for PostgreSQL, MariaDB, or MongoDB.