Disaster Recovery & Business Continuity

1. The recovery surface — what Versus holds

Everything you must reconstruct lives in a small number of places. Protect these and you can rebuild the whole system.

2. RPO / RTO by form factor

RPO = how much data you can afford to lose (the age of your newest usable backup). RTO = how long you can be down. Your numbers follow the form factor you deployed (see the On-Premise Installation Guide) and the cadence you pick.

3. What to back up (and how)

PostgreSQL — the primary

Postgres is the one thing you cannot lose. At minimum, take a nightly logical dump off the box:

pg_dump --format=custom --no-owner \
  "postgres://versus:PASS@pg-primary:5432/versus" \
  > versus-$(date +%F).dump

For a low RPO, add continuous WAL archiving + base backups → point-in-time recovery(pgBackRest, Barman, or the CloudNativePG operator on Kubernetes). Encrypt backups at rest and store them off-box and off-site — a backup on the same VM or array is not a backup.

Object storage — device backups & media

Point BACKUP_S3_* at durable, versioned storage and replicate it off-site:

BACKUP_S3_BUCKET=versus-backups
BACKUP_S3_ENDPOINT=https://minio.yourco.internal   # omit for AWS S3
BACKUP_S3_REGION=us-east-1                          # 'auto' for R2
BACKUP_S3_ACCESS_KEY_ID=...
BACKUP_S3_SECRET_ACCESS_KEY=...

Enable bucket versioning (so an overwrite or ransomware event is recoverable) and cross-site replication (S3/R2 rules, or MinIO site-replication). Durable device-config backups land here too — see Backup & Config Protection.

The keys, the license, and config

• FIELD_ENCRYPTION_KEY + SECRET_KEY — escrow in your secrets manager or a sealed offline copy, separately from the database.
• .veslic license — keep the file (and any re-host file) with your config. See Activating an On-Premise License.
• Config / env / values.yaml — version it like code so a rebuild is reproducible.

Collectors

Collectors are effectively stateless and re-enrollable — they dial the gateway outbound over a bearer-authenticated TLS channel on 443. In Mode A (local vault, credentials never leave your network), back up each collector host's vault material per your collector runbook; otherwise a rebuilt collector simply re-enrolls.

4. Restore — the recovery runbook

Rebuilding from nothing, in order. Order matters — the keys must be in place before the data, or encrypted columns won't decrypt.

5. High availability vs disaster recovery

They solve different problems, and you generally want both:

• High availability (staying up) absorbs the loss of a component with no downtime — a web pod dies, a Postgres primary fails over. That's form factor B(Kubernetes multi-replica + operator Postgres failover) or C (load-balanced appliances + HA Postgres).
• Disaster recovery (coming back) rebuilds after a disaster — data corruption, ransomware, operator error, or the loss of a whole site. That's backups + the runbook above.

When you run more than one app node (Option C), pin the per-process singletons so nodes don't double-fire or diverge: run the schedulers/pollers on one node (DISABLE_INPROC_SCHEDULERS=1 on the others), enable LB sticky sessions, and share object storage and the media path. The installation guide has the full table.

6. Multi-site / metro DR

To survive a whole-site loss, run a warm standby at a second site:

• Database: asynchronous streaming replication (or shipped WAL + base backups) to a standby Postgres at the DR site. RPO = replication lag.
• Object storage: cross-site bucket replication or MinIO site-replication, so backups and media exist at both sites.
• Keys & license: the FIELD_ENCRYPTION_KEY, SECRET_KEY, and a valid .veslic must be at the DR site before you need them.
• Failover: promote the standby Postgres, bring up the DR app pointed at it, repoint DNS/LB, and re-point collectors at the DR gateway address (outbound-only, so a stable DR DNS name makes this a no-op).

RTO is then promote + boot + DNS — minutes, not hours.

7. Air-gapped DR

Nothing here requires the internet:

• Backups stay in-network — an in-cluster MinIO or a NAS. Off-site means shipping encrypted media, or replicating to a second isolated site over your own link.
• Licensing is a node-locked .veslic file re-uploaded on the recovered node; keep spare/re-host files on hand so licensing never blocks a recovery, and understand the grace period.
• Identity has no internet dependency — the built-in local backend with TOTP MFA (or an in-network LDAP/AD) authenticates offline, so a recovered node isn't waiting on an external IdP.

8. Business continuity — operating while degraded

Versus fails soft, which buys you time during an incident:

• A lapsed license doesn't destroy anything. Past the grace period an install goes read-only, not dark — data and history are intact and it returns to full function the moment a valid license is applied.
• A Versus outage doesn't take down your network. Managed devices keep forwarding and you retain out-of-band (SSH/console) access to every one. Versus is how you manage the estate, not how packets flow through it.
• Features degrade gracefully. AI-assisted features fall back quietly when a model endpoint is unavailable; core inventory, workflows and monitoring keep working.

In your own runbook, note that while Versus is being restored, device changes revert to your out-of-band process — and the restored system re-discovers current state on its next sweep.

9. Test your DR (game days)

A backup you have never restored is a hypothesis. Prove it on a schedule:

• Quarterly restore drill. Restore last night's backup into an isolatedenvironment, set the escrowed keys, and verify: the app starts, you can log in, inventory and history are present, and a vaulted secret decrypts — the single best proof your field key and DB match.
• Measure the real RTO. Time the drill end to end; that number, not the target, is your actual RTO. Tune cadence and automation until it meets your objective.
• Rotate the incident owner through the runbook so more than one person has done it.

10. Checklist