Backups & restore

The safety net for a box with no replica

A single server running your database in a container has no failover twin to fall back on. Sproobo's answer is off-box backups that actually get exercised — dumped outside the container, verified by checksum, and periodically restored for real to prove they work.

How a backup runs

Every backup runs the engine's own dump command inside the running service container via docker exec — never against a host-installed client:

  • PostgreSQL — pg_dump -Fc (custom format).
  • MySQL / MariaDB — mysqldump --single-transaction.
  • Redis — redis-cli --rdb -, streaming the RDB snapshot out.
  • MongoDB — mongodump --archive, authenticated against the admin database.

The dump is zstd-compressed and streamed straight to an off-box, S3-compatible target (Hetzner Object Storage, Backblaze B2, Cloudflare R2, or any generic S3 endpoint), with a sha256 computed in-flight. A Backup row records the object key, size, checksum, and timing. The bind-mounted data directory itself is never touched — a backup is a read-only logical dump, today. Physical, WAL-level point-in-time recovery is on the roadmap but not shipped yet.

Retention & scheduling

Each service has its own backup schedule: an enabled flag, a time-zone-aware local window, and GFS retention — daily, weekly, and monthly generations kept on their own cadence instead of one flat expiry.

Encryption today is TLS in transit plus whatever at-rest encryption your target provides. Client-side envelope encryption of the dump itself is on the roadmap, not shipped yet.

Restoring a backup

Restoring is an ADMIN-gated dashboard action, and it refuses to run against anything unverified: only a backup that finished succeeded and carries both a checksum and an object key is restorable — a pending, failed, or otherwise unverified dump is rejected outright.

Once approved, the agent downloads the object, re-verifies its sha256, decodes the zstd stream, and pipes it straight into the engine's own restore command inside the already-running container — the mirror image of the backup step, over the same docker exec path (pg_restore --clean --if-exists, mongorestore --drop, and so on — each engine's own idempotent restore flags). There's no host postgres user and no peer-auth dependency the way a bare-metal restore would need — the exact host-coupling failure this replaces from v2.

Every restore is audited. Today it's a dashboard-only action — the CLI's sproobo backups command (and its MCP equivalent) only lists backup tasks; there's no CLI or MCP command to trigger a restore yet.

Restore-verification is a scheduled, automated proof.

On a recurring daily window (toggled per service), Sproobo takes the latest successful backup and restores it into a disposable, throwaway container on sproobo-int — no published port, a scratch data directory that isn't treated as sacred. It's health-probed before and after the restore, then torn down. A pass stamps the backup restore-verified; a failure alerts you instead of leaving you to find out the hard way during an actual incident. That's the closest thing to a replica this single-box model has.

Version upgrades are backup-gated

Changing a service's engine version is never an in-place image-tag swap. It runs as its own gated task, and it cannot even be requested without a verified backup already in hand:

01

A fresh, separately versioned data directory is bound

The current version's directory is left completely untouched on disk — the upgrade never writes into it.

02

The service container is recreated on the new image

Same name, so bound apps see no config change — but this does mean the service is briefly unavailable while the new-version container starts, unlike a side-by-side app deploy.

03

The verified pre-upgrade backup loads into it

The same dump/restore recipe as any other restore, landing in the fresh, empty instance once it's health-probed as reachable.

04

A final probe confirms the upgraded service is answering

Now running the target version, with its data restored from the verified dump.

If the task fails or is canceled at any point, the control plane immediately reverts its recorded desired version back to the one you upgraded from and marks the upgrade failed. Because the prior data directory was never touched and the pre-upgrade backup is still sitting in the target, nothing is ever lost or overwritten — but re-establishing the service on the older version afterward is a follow-up action, not an automatic one.

Requesting the upgrade itself goes through upgrade_service — the only always-approval-gated destructive write on the whole platform. Every other write can be pre-approved for automation; this one can't, no matter what the caller's key allows. It needs an ADMIN role and a verified backup id, and the storage target it restores from is resolved server-side, never accepted as an argument. See Databases & services for how a version is chosen at create time in the first place.

A backup is a hard precondition, not a suggestion.

An off-box backup is mandatory before any destructive or upgrade operation touches a stateful service's data — the same rule that governs the rest of the platform's trust model. See Security model.

Failure modes

  • A dump fails — the backup is marked failed with a reason, the schedule retries later, and you're alerted; no partial object is ever kept.
  • A restore is requested against an unverified backup — refused up front; nothing unverified is ever piped into a live database.
  • A restore-verification fails — the throwaway container is torn down, the backup stays unmarked, and you're alerted.
  • Disk pressure during an upgrade — a pre-flight free-disk check blocks the whole operation before anything destructive starts, rather than filling the box mid-swap.

Next