Restore and Fire Drills
You can restore any backup from the command line or the console. On a schedule, SafeGrd also restores each surface's newest backup, checks what came back and signs a dated record of the result.
Restore a backup
restore downloads the encrypted backup from storage, decrypts it on this machine
with your private key, decompresses it and writes it to the target. It reads the key from
key_path or SAFEGRD_PRIVATE_KEY. On a host other than the one that
took the backup, pass --key-path or --private-key.
Restore from the console
On a paid plan, you can restore any completed snapshot from the console. Your host runs the restore, and the console shows how it went.
- Create a database account that can write to the target. The backup's own role is read-only, so it cannot restore.
- Click Restore on the backup you want. The button is on the Snapshots page, in the Snapshots tab of the backup's surface, and at the top of the backup's detail.
- Choose what to restore:
- the whole backup, into an empty database
- chosen schemas (PostgreSQL only)
- chosen tables, from any PostgreSQL backup or from an incremental backup of MySQL, MariaDB, SQLite or MongoDB (one collection)
- the rows of one table that a SQL condition matches (PostgreSQL only). They go back by primary key into the table as it is in the target.
- Type the target's URL with the password of the account from step 1.
- Read the summary, type the surface's name, and press Restore.
The restore appears in the surface's Restores tab. It waits there until
your host checks in, then shows how many tables and rows came back. Only owners and admins
see the Restore button. On Free, the console shows the
safegrd restore command to run on your host instead.
The console does not restore WordPress sites or Laravel apps. Restore a WordPress site from wp-admin and a Laravel app on its host.
Restore to a moment
For a surface with point-in-time recovery, open its Point in time tab and pick
a moment inside the window. The console writes the safegrd wal restore command for
that moment. Run it on a host that has the surface's config and key. Each database's page has
the steps:
PostgreSQL,
MySQL and MariaDB,
MongoDB and
SQLite.
Where the restore runs
- If your host runs the daemon, the host starts the restore at its next check-in.
- For a surface SafeGrd backs up itself, a machine SafeGrd starts for the restore runs it. The surface needs a SafeGrd-managed key, and the backup has to be in hosted storage.
- Otherwise, the console gives you
safegrd restore --request rst_.... Run it on any host in your organization that has the key. It reads what you chose in the console and reports back.
Your target and its password
- Nothing in the target is dropped or overwritten. If a table already exists and has rows, the restore stops and leaves the target as it was.
- SafeGrd keeps the URL you typed sealed and gives it once to whatever runs the restore. The key access log records that. SafeGrd deletes the URL when the restore ends, or after a day if nothing picked it up.
- To put a live database back, restore into a new database beside it and point your application at that. Or restore only the broken schemas or tables into the live database.
A backup before you restore into a live database
When you restore chosen schemas or tables, they go into a database that has other data in it, usually the live one. So SafeGrd backs up the surface first, and you can go back to how it was. If that backup fails, the restore does not run. A backup taken in the last 15 minutes counts as that backup. You can turn this off in the dialog.
The record of a restore
The Restores tab lists who asked, what was restored into which target, and how many tables and rows came back. SafeGrd signs each completed restore with its attestation key, and Certificate shows the signed record. A restore is not a Fire Drill, so it does not count toward your plan's drills, the badge or the recovery point.
What a PostgreSQL restore turns off and takes out
pg_cronjobs come back turned off. The restore prints each one with its id and the statement that turns it on, so nothing runs against restored data until you say so. To keep them as they were, pass--keep-cron-active. See pg_cron jobs.- For the tables a surface names under
erasures:, erased rows stay erased. Before it commits, the restore deletes each row that was deleted after the backup was taken, and prints how many. If it cannot read the erasure record, it restores nothing. To restore anyway, pass--without-erasures. See the erasure record.
Fire Drills
A Fire Drill restores a snapshot somewhere disposable, counts what came back and records the result. It runs in your environment with your key. SafeGrd's server receives the report, not the data.
By default, verify does a dry run. It decrypts the stream, replays it in memory
and checks table counts, row counts, column counts and extensions, with no PostgreSQL
running anywhere. It checks the data but does not load the schema. Only a sandbox drill does
that. Because the dry run needs no database, you can run it in CI or on a laptop.
For the full drill, pass --sandbox-target without --dry-run. The
drill loads the data into a throwaway database and checks it there. SafeGrd records sandbox
drills on every paid plan. On the free plan, it records the in-memory drill and refuses a
sandbox drill, with the reason.
After loading the data, a sandbox drill has the database check it:
- PostgreSQL runs
amcheckover every table (PostgreSQL 14 and later) and every B-tree index. - MySQL runs
CHECK TABLEon every table. - MongoDB runs
validateon every collection. - SQLite runs
PRAGMA integrity_check.
The result is one line of the drill and part of its certificate. Sometimes the sandbox cannot
run the check: PostgreSQL has no amcheck, the role may not create it, or a
MongoDB user has no dbAdmin on the sandbox. Then the drill says
NOT RUN with the reason, and passes or fails on its other checks. To have
PostgreSQL's check run, let the sandbox's role run CREATE EXTENSION amcheck, or
create the extension in the sandbox database yourself.
A database drill record also shows how each table's row count changed since the previous backup. It lists the tables that grew or shrank, largest change first, then the tables added and removed, and how many did not change. The console shows this in the drill's detail, and it is signed with the record (how).
Every plan runs scheduled Fire Drills. The free plan drills once a week, in memory. Paid plans drill weekly or daily and record sandbox drills (pricing).
A surface's Details show when its next backup and drill are due, and who starts them. SafeGrd starts the ones it runs on time. A host's daemon runs a due one at its next check-in. For a backup from cron or a CI workflow, that schedule decides the time, and the console shows the earliest time the next drill can run. Upcoming on the Fire Drills tab lists every surface's next drill, overdue ones first, with where it runs and how often the plan drills.
Your own checks
Row counts show the data came back. Your own checks show it is the data you expect: no user without an email, orders from the last day, a migration table at the right version. A sandbox drill runs each check after the restore and records its result beside the counts.
In the console, add queries under Drill checks on a database surface, each with the
value it must return. The host's daemon picks them up at its next
check-in, and a drill on SafeGrd's machines runs them too.
In the host's config, put the same queries under drill.checks. The config also
takes commands:
- A query runs in a read-only transaction, for at most 60 seconds. The first column of its
first row is compared with
expect, which is a value ("0","true","ok") or a comparison ("> 0",">= 1000","!= 0"). Numbers compare as numbers, so42.75matches anumericcolumn. Queries run on PostgreSQL, MySQL, MariaDB and SQLite sandboxes. - In a query,
${SAFEGRD_SNAPSHOT_CREATED_AT}is replaced with the time the backup was taken, in RFC 3339 and UTC to the second (2026-10-20T09:30:00Z). Quote it. Compare your newest row with it to fail a drill of a backup whose data had stopped changing. Row counts alone do not catch that. The console's From a template writes this check for the surface's database:- PostgreSQL:
timestamptz '${SAFEGRD_SNAPSHOT_CREATED_AT}' - MySQL and MariaDB:
CAST('${SAFEGRD_SNAPSHOT_CREATED_AT}' AS DATETIME), which is UTC, so compare it withUTC_TIMESTAMP()or a column written in UTC - SQLite:
datetime('${SAFEGRD_SNAPSHOT_CREATED_AT}'), anddatetime(created_at)on the column too, since text compares letter by letter
${SAFEGRD_...}in a query is refused when you save the check. - PostgreSQL:
- A command can be set only in the host's config, not in the console, and
it never runs on SafeGrd's machines. It runs through
sh -con the host that runs the drill, for at most 5 minutes, and passes if it exits 0. It gets these environment variables:SAFEGRD_SANDBOX_URL, the sandbox's connection stringSAFEGRD_SNAPSHOT_IDSAFEGRD_SNAPSHOT_CREATED_AT, when the backup was taken (RFC 3339, UTC). Compare your newest row with it to fail a drill of a backup whose data had stopped changing.
timeout_secondson a check changes its time limit.
A check that does not hold fails the drill. The alert names the check and says the restore matched the backup. The certificate hash covers each check's name, result and expected value, so the signed record fixes what every check returned.
An in-memory drill has no database to run the checks against. It runs without them and
prints a warning with how many it skipped. On a plan with sandbox drills, the daemon drills a
SQLite surface that has no drill.sandbox_url into a new file in its state
directory, so its checks run, and deletes the file afterwards. safegrd doctor
reads every surface's checks and reports any that cannot run.
For a single drill by hand, put the same list in a file:
Read rows out of a backup
To get back a few rows without restoring the table they came from, use
--query. It restores a database snapshot into a sandbox, runs one query there in
a read-only transaction, and prints the rows as CSV. Column names come first, and NULL is an
empty field. --out writes the CSV to a file instead of stdout.
With --sandbox-target, the restore goes into that empty database and stays
there. Without it, a PostgreSQL snapshot is restored into a cluster that the command starts
on the host and removes afterwards. That needs initdb and postgres
of the snapshot's version or newer. --keep leaves the cluster running for a day
and prints how to connect. A query run is not a drill. Nothing is reported to SafeGrd, and
the MCP server does not offer it.
A full drill of a demo database, as safegrd verify prints it:
Warning: the value of --sandbox-target is visible in `ps` and shell history. Pass env:VAR or file:/path instead. Verifying snap-20261031-135407-16c6d3 (epoch e202610-b3036e43, shop-db): checking the packs, restoring it into postgres://postgres:xxxxx@localhost:62309/drill_sandbox?sslmode=disable. [PASS] DecryptionIntegrity (Expected: the run, its indexes and trees open with the identity, Actual: they do) [PASS] RepositoryIntegrity (Expected: every pack present and agreeing with its index, Actual: 3 packs checked) [PASS] DigestIntegrity (Expected: f9eef402a29aca7aac7575ff9de386b4e31978b155fc35b5affd11e1923133b1, Actual: f9eef402a29aca7aac7575ff9de386b4e31978b155fc35b5affd11e1923133b1) [PASS] TableCountMatch (Expected: 4 tables, Actual: 4 tables) [PASS] RowCountMatch (Expected: 45404 rows, Actual: 45404 rows) [PASS] Row Count: public.customers (Expected: 1840 rows, Actual: 1840 rows) [PASS] Row Count: public.order_items (Expected: 31000 rows, Actual: 31000 rows) [PASS] Row Count: public.orders (Expected: 12500 rows, Actual: 12500 rows) [PASS] Row Count: public.products (Expected: 64 rows, Actual: 64 rows) [PASS] Newest Row: public.customers (Expected: created_at 2026-10-31T13:53:57.447761Z, Actual: created_at 2026-10-31T13:53:57.447761Z) [PASS] ExtensionBootCheck (Expected: [], Actual: []) [PASS] Schema Captured By pg_dump (Expected: schema from pg_dump, Actual: pg_dump 18.6) [PASS] Data Check: amcheck (Expected: every heap and B-tree index readable, Actual: 4 tables, 3 B-tree indexes, no corruption) Fire Drill Passed Verification ID: verif-cc68f9a0 Duration: 236ms Tables: 4 Rows: 45,404 Certificate: cert_sg_d05260a52f24625bb27f4687d67b67781d72da6b13b2003c33128037d1f88ecf
Drills on SafeGrd
If a surface has no host to drill it, for example one you back up from a GitHub Actions workflow, SafeGrd can run its drills. Turn on Drill on SafeGrd on the surface. Each drill runs on a machine SafeGrd starts for it and destroys afterwards, and the record is the same one a host sends. Hosted backups and drills covers which surfaces can use it, what each machine is given, and how minutes are counted.
Recovery point and restore time
For each surface, the console shows four figures, measured from its snapshots and drills. The first three measure the recovery point. A security questionnaire compares that with your recovery point objective (RPO), and restore time with your recovery time objective (RTO). SafeGrd sets no target and guarantees none. RPO and RTO for a database backup explains both.
| Figure | How it is measured |
|---|---|
| Verified | The time since the newest snapshot that a drill restored, and how deep that drill went. The console leads with this one, because it counts only backups that came back. |
| Latest backup | The time since the newest snapshot that was written. A snapshot that Threat Shield flagged still counts, because the data is there. The console marks it. |
| Worst gap, 30 days | The recovery point at the surface's worst moment in the last 30 days: the longest time between two backups in those days. It counts from the last backup before the window and includes the time since the newest one. A surface backed up for less than 30 days is measured from its first backup, and the console says so. The gap is shown against the schedule. A gap over twice the schedule is red, because that is when the overdue alert fires. Time the schedule was paused in the console is left out of each gap, and the console says how much. The recovery figures and the Evidence Pack list each pause with who paused it and why. |
| Restore time | The median (p50) and 95th percentile (p95) duration of this surface’s passed drills in the last 90 days. Next to them, the console shows how many drills that is, the usual size of the snapshots they restored, and where they ran. |
A drill is timed from before the download, through decryption, to the end of the restore. Restore time counts only drills that wrote the data out: a sandbox drill, which loads a database, and a restore of a repository, which writes the files.
In-memory drills are left out, because they take less time than any real restore. For a database, an in-memory drill checks the backup without loading the data or building an index. For a files or mail archive, it reads the archive without writing a file. SQLite is the exception. Its in-memory drill writes the database file out and opens it, which is all a SQLite restore does, so it counts.
If a surface has no drill that counts, the console shows why instead of a number. Either the free plan drills in memory, the host could not run a sandbox drill (the console names the reason), or no drill has passed yet.
Drills that ran on SafeGrd’s machines are counted separately from drills on your host. Those machines are not yours, so a drill can take a different time there than a restore on your own hardware. To get the time on your hardware, time a restore yourself: Rehearse it before you need it.
The same figures are in GET /api/v1/orgs/{org_id}/recovery and in the Evidence
Pack, each with the date it was measured.
Verifying from another machine
You can run a drill on a machine other than the one that took the backup. Point
verify at the storage directly and give it the key:
This is what disaster recovery looks like: a separate machine with read access to the
bucket and your key rebuilds the data from nothing.
The recovery runbooks take you through it step by step.
Testing that a backup restores explains
the same checks with plain pg_restore and mysql.
You can also drill another host's snapshot from any host enrolled in the same organization.
Run safegrd verify --snapshot there, and the console records the result as a
Fire Drill on the other host's surface. The drilling host can be in any project, and it can be
one enrolled only to restore. It needs the snapshot's key. SafeGrd releases a SafeGrd-managed
key to it. A customer-managed key has to be on that host already. If you keep your
customer-managed key on a second host this way, that host can drill the other host's backups
too.