Docs / Restore and Fire Drills

Restore and Fire Drills

You can restore any backup from the command line or the console. On a schedule, SafeGrd also restores each surface's newest backup, checks what came back and signs a dated record of the result.

Restore a backup

# into a PostgreSQL target
safegrd restore --snapshot snap-1b094c0af8 --target "$DATABASE_URL"
# files or mail, onto disk
safegrd restore --snapshot snap-94dec39a5e --target-dir ./recovered

restore downloads the encrypted backup from storage, decrypts it on this machine with your private key, decompresses it and writes it to the target. It reads the key from key_path or SAFEGRD_PRIVATE_KEY. On a host other than the one that took the backup, pass --key-path or --private-key.

Restore from the console

On a paid plan, you can restore any completed snapshot from the console. Your host runs the restore, and the console shows how it went.

  1. Create a database account that can write to the target. The backup's own role is read-only, so it cannot restore.
  2. Click Restore on the backup you want. The button is on the Snapshots page, in the Snapshots tab of the backup's surface, and at the top of the backup's detail.
  3. Choose what to restore:
    • the whole backup, into an empty database
    • chosen schemas (PostgreSQL only)
    • chosen tables, from any PostgreSQL backup or from an incremental backup of MySQL, MariaDB, SQLite or MongoDB (one collection)
    • the rows of one table that a SQL condition matches (PostgreSQL only). They go back by primary key into the table as it is in the target.
    A files surface restores into an empty directory on the host, or into a bucket. You can also start from the Tables tab of a database surface. Restore table on each table opens the dialog with that table picked, and so does Restore rows on a PostgreSQL surface.
  4. Type the target's URL with the password of the account from step 1.
  5. Read the summary, type the surface's name, and press Restore.
The Restore dialog for shop-db at its confirm step: the whole backup into db.acme.example:5432/shop_restored, with the surface's name typed and the Restore button. The Restore dialog for shop-db at its confirm step: the whole backup into db.acme.example:5432/shop_restored, with the surface's name typed and the Restore button.
The last step of a restore: what is restored, into which database, and the surface's name typed to confirm.

The restore appears in the surface's Restores tab. It waits there until your host checks in, then shows how many tables and rows came back. Only owners and admins see the Restore button. On Free, the console shows the safegrd restore command to run on your host instead.

The console does not restore WordPress sites or Laravel apps. Restore a WordPress site from wp-admin and a Laravel app on its host.

Restore to a moment

For a surface with point-in-time recovery, open its Point in time tab and pick a moment inside the window. The console writes the safegrd wal restore command for that moment. Run it on a host that has the surface's config and key. Each database's page has the steps: PostgreSQL, MySQL and MariaDB, MongoDB and SQLite.

Where the restore runs

Your target and its password

A backup before you restore into a live database

When you restore chosen schemas or tables, they go into a database that has other data in it, usually the live one. So SafeGrd backs up the surface first, and you can go back to how it was. If that backup fails, the restore does not run. A backup taken in the last 15 minutes counts as that backup. You can turn this off in the dialog.

The record of a restore

The Restores tab lists who asked, what was restored into which target, and how many tables and rows came back. SafeGrd signs each completed restore with its attestation key, and Certificate shows the signed record. A restore is not a Fire Drill, so it does not count toward your plan's drills, the badge or the recovery point.

What a PostgreSQL restore turns off and takes out

Fire Drills

A Fire Drill restores a snapshot somewhere disposable, counts what came back and records the result. It runs in your environment with your key. SafeGrd's server receives the report, not the data.

# in-memory dry restore, no database needed
safegrd verify --snapshot snap-1b094c0af8
# full drill into an ephemeral database
safegrd verify --snapshot snap-1b094c0af8 --sandbox-target "$SANDBOX_URL"

By default, verify does a dry run. It decrypts the stream, replays it in memory and checks table counts, row counts, column counts and extensions, with no PostgreSQL running anywhere. It checks the data but does not load the schema. Only a sandbox drill does that. Because the dry run needs no database, you can run it in CI or on a laptop.

For the full drill, pass --sandbox-target without --dry-run. The drill loads the data into a throwaway database and checks it there. SafeGrd records sandbox drills on every paid plan. On the free plan, it records the in-memory drill and refuses a sandbox drill, with the reason.

The two kinds of Fire Drill. Both start from a snapshot decrypted on the host. An in-memory drill replays it with no database and checks the counts of tables, rows and columns. A sandbox drill loads it into a throwaway database, checks the same counts, runs the database's own check such as amcheck or CHECK TABLE, then runs your own checks. Both end in a signed drill record. snapshot decrypted on the host in memory no database needed sandbox a throwaway database Counts checked tables, rows, columns Counts checked tables, rows, columns Database's own check amcheck, CHECK TABLE Your checks queries and commands drill record signed and chained
Only a sandbox drill loads the schema, so only it shows that a dump loads into a real database. Your own checks run there too.

After loading the data, a sandbox drill has the database check it:

The result is one line of the drill and part of its certificate. Sometimes the sandbox cannot run the check: PostgreSQL has no amcheck, the role may not create it, or a MongoDB user has no dbAdmin on the sandbox. Then the drill says NOT RUN with the reason, and passes or fails on its other checks. To have PostgreSQL's check run, let the sandbox's role run CREATE EXTENSION amcheck, or create the extension in the sandbox database yourself.

A database drill record also shows how each table's row count changed since the previous backup. It lists the tables that grew or shrank, largest change first, then the tables added and removed, and how many did not change. The console shows this in the drill's detail, and it is signed with the record (how).

Every plan runs scheduled Fire Drills. The free plan drills once a week, in memory. Paid plans drill weekly or daily and record sandbox drills (pricing).

The Fire Drills tab: the drill record, then each surface's last verified drill, recovery point and restore time. The Fire Drills tab: the drill record, then each surface's last verified drill, recovery point and restore time.
The Fire Drills tab. Upcoming lists each surface's next drill, overdue first. History lists every drill with what was restored and its certificate, and Badge and public page holds the badge.

A surface's Details show when its next backup and drill are due, and who starts them. SafeGrd starts the ones it runs on time. A host's daemon runs a due one at its next check-in. For a backup from cron or a CI workflow, that schedule decides the time, and the console shows the earliest time the next drill can run. Upcoming on the Fire Drills tab lists every surface's next drill, overdue ones first, with where it runs and how often the plan drills.

Your own checks

Row counts show the data came back. Your own checks show it is the data you expect: no user without an email, orders from the last day, a migration table at the right version. A sandbox drill runs each check after the restore and records its result beside the counts.

In the console, add queries under Drill checks on a database surface, each with the value it must return. The host's daemon picks them up at its next check-in, and a drill on SafeGrd's machines runs them too. In the host's config, put the same queries under drill.checks. The config also takes commands:

surfaces:
- id: app-db
drill:
checks:
- name: every user has an email
sql: SELECT count(*) FROM users WHERE email IS NULL
expect: "0"
- name: orders in the last day
sql: SELECT count(*) FROM orders WHERE created_at > now() - interval '1 day'
expect: "> 0"
- name: newest order is as new as the backup
sql: SELECT max(created_at) >= timestamptz '${SAFEGRD_SNAPSHOT_CREATED_AT}' - interval '1 hour' FROM orders
expect: "true"
- name: app smoke test
command: ./scripts/smoke.sh

A check that does not hold fails the drill. The alert names the check and says the restore matched the backup. The certificate hash covers each check's name, result and expected value, so the signed record fixes what every check returned.

An in-memory drill has no database to run the checks against. It runs without them and prints a warning with how many it skipped. On a plan with sandbox drills, the daemon drills a SQLite surface that has no drill.sandbox_url into a new file in its state directory, so its checks run, and deletes the file afterwards. safegrd doctor reads every surface's checks and reports any that cannot run.

For a single drill by hand, put the same list in a file:

safegrd verify --snapshot snap-1b094c0af8 --sandbox-target "$SANDBOX_URL" --checks checks.yaml

Read rows out of a backup

To get back a few rows without restoring the table they came from, use --query. It restores a database snapshot into a sandbox, runs one query there in a read-only transaction, and prints the rows as CSV. Column names come first, and NULL is an empty field. --out writes the CSV to a file instead of stdout.

safegrd verify --snapshot snap-1b094c0af8 --query "SELECT * FROM orders WHERE id = 4211" --out order.csv

With --sandbox-target, the restore goes into that empty database and stays there. Without it, a PostgreSQL snapshot is restored into a cluster that the command starts on the host and removes afterwards. That needs initdb and postgres of the snapshot's version or newer. --keep leaves the cluster running for a day and prints how to connect. A query run is not a drill. Nothing is reported to SafeGrd, and the MCP server does not offer it.

A full drill of a demo database, as safegrd verify prints it:

Warning: the value of --sandbox-target is visible in `ps` and shell history. Pass env:VAR or file:/path instead.
Verifying snap-20261031-135407-16c6d3 (epoch e202610-b3036e43, shop-db): checking the packs, restoring it into postgres://postgres:xxxxx@localhost:62309/drill_sandbox?sslmode=disable.
   [PASS] DecryptionIntegrity (Expected: the run, its indexes and trees open with the identity, Actual: they do)
   [PASS] RepositoryIntegrity (Expected: every pack present and agreeing with its index, Actual: 3 packs checked)
   [PASS] DigestIntegrity (Expected: f9eef402a29aca7aac7575ff9de386b4e31978b155fc35b5affd11e1923133b1, Actual: f9eef402a29aca7aac7575ff9de386b4e31978b155fc35b5affd11e1923133b1)
   [PASS] TableCountMatch (Expected: 4 tables, Actual: 4 tables)
   [PASS] RowCountMatch (Expected: 45404 rows, Actual: 45404 rows)
   [PASS] Row Count: public.customers (Expected: 1840 rows, Actual: 1840 rows)
   [PASS] Row Count: public.order_items (Expected: 31000 rows, Actual: 31000 rows)
   [PASS] Row Count: public.orders (Expected: 12500 rows, Actual: 12500 rows)
   [PASS] Row Count: public.products (Expected: 64 rows, Actual: 64 rows)
   [PASS] Newest Row: public.customers (Expected: created_at 2026-10-31T13:53:57.447761Z, Actual: created_at 2026-10-31T13:53:57.447761Z)
   [PASS] ExtensionBootCheck (Expected: [], Actual: [])
   [PASS] Schema Captured By pg_dump (Expected: schema from pg_dump, Actual: pg_dump 18.6)
   [PASS] Data Check: amcheck (Expected: every heap and B-tree index readable, Actual: 4 tables, 3 B-tree indexes, no corruption)

Fire Drill Passed
   Verification ID: verif-cc68f9a0
   Duration:        236ms
   Tables:          4
   Rows:            45,404
   Certificate:     cert_sg_d05260a52f24625bb27f4687d67b67781d72da6b13b2003c33128037d1f88ecf

Drills on SafeGrd

If a surface has no host to drill it, for example one you back up from a GitHub Actions workflow, SafeGrd can run its drills. Turn on Drill on SafeGrd on the surface. Each drill runs on a machine SafeGrd starts for it and destroys afterwards, and the record is the same one a host sends. Hosted backups and drills covers which surfaces can use it, what each machine is given, and how minutes are counted.

Recovery point and restore time

For each surface, the console shows four figures, measured from its snapshots and drills. The first three measure the recovery point. A security questionnaire compares that with your recovery point objective (RPO), and restore time with your recovery time objective (RTO). SafeGrd sets no target and guarantees none. RPO and RTO for a database backup explains both.

FigureHow it is measured
VerifiedThe time since the newest snapshot that a drill restored, and how deep that drill went. The console leads with this one, because it counts only backups that came back.
Latest backupThe time since the newest snapshot that was written. A snapshot that Threat Shield flagged still counts, because the data is there. The console marks it.
Worst gap, 30 daysThe recovery point at the surface's worst moment in the last 30 days: the longest time between two backups in those days. It counts from the last backup before the window and includes the time since the newest one. A surface backed up for less than 30 days is measured from its first backup, and the console says so. The gap is shown against the schedule. A gap over twice the schedule is red, because that is when the overdue alert fires. Time the schedule was paused in the console is left out of each gap, and the console says how much. The recovery figures and the Evidence Pack list each pause with who paused it and why.
Restore timeThe median (p50) and 95th percentile (p95) duration of this surface’s passed drills in the last 90 days. Next to them, the console shows how many drills that is, the usual size of the snapshots they restored, and where they ran.

A drill is timed from before the download, through decryption, to the end of the restore. Restore time counts only drills that wrote the data out: a sandbox drill, which loads a database, and a restore of a repository, which writes the files.

In-memory drills are left out, because they take less time than any real restore. For a database, an in-memory drill checks the backup without loading the data or building an index. For a files or mail archive, it reads the archive without writing a file. SQLite is the exception. Its in-memory drill writes the database file out and opens it, which is all a SQLite restore does, so it counts.

If a surface has no drill that counts, the console shows why instead of a number. Either the free plan drills in memory, the host could not run a sandbox drill (the console names the reason), or no drill has passed yet.

Drills that ran on SafeGrd’s machines are counted separately from drills on your host. Those machines are not yours, so a drill can take a different time there than a restore on your own hardware. To get the time on your hardware, time a restore yourself: Rehearse it before you need it.

The same figures are in GET /api/v1/orgs/{org_id}/recovery and in the Evidence Pack, each with the date it was measured.

Verifying from another machine

You can run a drill on a machine other than the one that took the backup. Point verify at the storage directly and give it the key:

safegrd verify --snapshot snap-1b094c0af8 \
  --s3-bucket my-worm-bucket --s3-endpoint https://s3.example.com \
  --s3-access-key "$AWS_ACCESS_KEY_ID" --s3-secret-key env:AWS_SECRET_ACCESS_KEY \
  --key-path ./daemon.key

This is what disaster recovery looks like: a separate machine with read access to the bucket and your key rebuilds the data from nothing. The recovery runbooks take you through it step by step. Testing that a backup restores explains the same checks with plain pg_restore and mysql.

You can also drill another host's snapshot from any host enrolled in the same organization. Run safegrd verify --snapshot there, and the console records the result as a Fire Drill on the other host's surface. The drilling host can be in any project, and it can be one enrolled only to restore. It needs the snapshot's key. SafeGrd releases a SafeGrd-managed key to it. A customer-managed key has to be on that host already. If you keep your customer-managed key on a second host this way, that host can drill the other host's backups too.