Docs / Security and key custody

Security and key custody

Your host encrypts every backup before it uploads it. The bucket keeps it under S3 Object Lock until a date set by your plan or your retention settings. Unless you choose a customer-managed key, SafeGrd keeps the key that decrypts your backups sealed and releases it only to your enrolled hosts.

The cryptography

LayerWhat is used
EncryptionAge, which uses ChaCha20-Poly1305 for authenticated encryption and X25519 for key agreement. It is a standard scheme you can audit, with nothing bespoke.
Compressionzstandard, applied before encryption on the streaming path.
AttestationEd25519 signatures over a SHA-256 hash chain.
ImplementationWritten in Go, with no wrapped proprietary blobs. Database surfaces use the engine's own client tools: pg_dump for a PostgreSQL schema, mysqldump and mysql for MySQL and MariaDB, mongodump and mongorestore for MongoDB. SQLite, files and mail need no other tool.

Encryption is asymmetric, so a host needs only the public key to back up. You need the private key only to restore or to run a Fire Drill.

The CLI sets no server-side encryption header when it uploads, because the object is already Age ciphertext. Your bucket's own encryption at rest (SSE-S3, which AWS applies to every new bucket) is up to you. It adds nothing to a backup's confidentiality.

Data keys and key-encryption keys

In envelope-encryption terms, every backup has its own data encryption key (DEK), and your age key pair is the key-encryption key (KEK) that wraps it.

KeyWhat it doesWhere it lives
DEK Age makes a random 16-byte file key for each backup, and encrypts the backup with ChaCha20-Poly1305 under a key derived from it. In the backup's header, wrapped by the KEK. Each backup gets a new one, so there is nothing to store or rotate.
KEK Your age key pair (X25519). The public half wraps each backup's DEK. The private half unwraps it for a restore or a drill. The public half is on your hosts and in SafeGrd. Where the private half lives depends on your key custody choice: a SafeGrd-managed key or a customer-managed key.
How one backup is sealed. Your host, which needs only the public half of your KEK, encrypts the data with a new file key and wraps that file key with the KEK's public half. The backup holds the wrapped file key in its header and the encrypted data in its body. To restore or drill, the KEK's private half unwraps the file key, and the file key decrypts the body. Backing up, on your host needs only the KEK's public half one backup Header: its file key (DEK) wrapped with the KEK's public half Body: your data encrypted with the file key Restoring or drilling the KEK's private half unwraps the file key the file key decrypts the body
Every backup gets a new file key, so there is nothing per backup to store or rotate. Where the KEK's private half lives is your key custody choice, below.

With a SafeGrd-managed key, SafeGrd keeps your key sealed and releases it only to your enrolled hosts, so you can restore even after losing a host. With a customer-managed key, the private half of the KEK stays on your hosts. The KEK is the only key you make a choice about.

A SafeGrd-managed key works like a key your cloud provider manages for you. SafeGrd keeps it per organization, releases it only to your enrolled hosts and audits every release. A customer-managed key never reaches SafeGrd. In cloud terms, that is client-side encryption rather than a key in a provider's KMS.

What SafeGrd's server never receives

The host sends the encrypted backup straight to the bucket, and only a record of it (sizes, checksums, row counts, lock date, drill results) to SafeGrd. your host encrypts with the public key backup (encrypted) record only your bucket or SafeGrd-hosted storage SafeGrd sizes, checksums, row counts, lock date, drill results
The backup goes from your host to the bucket and never passes through SafeGrd's server. SafeGrd receives the record it shows in the console.

Two secrets, each chosen separately

SafeGrd needs two secrets to protect a surface. You choose for each one whether SafeGrd keeps it, and choosing SafeGrd for one does not choose it for the other.

SecretWhat it doesDefault
The encryption key The age private key that decrypts a host's backups (the private half of the KEK). Restores and drills need it. Backing up needs only its public half. A SafeGrd-managed key, kept sealed by SafeGrd. You choose per host when it enrolls. --key-custody local keeps it on the host as a customer-managed key.
A surface's credential The database URL or mailbox password the host uses to read the surface, or the S3 key of a bucket SafeGrd backs up. Only backups need it. The host keeps it. You choose per surface: the setup wizard asks, or use Credential in the surface's panel to hand it to SafeGrd.

A bucket's access key belongs to the project rather than to a host. If you set the bucket up in the console, SafeGrd keeps the access key sealed. If you set it up with the CLI, the hosts keep it. Hosted storage needs no key of yours.

The encryption key

You choose each host's key mode when the host enrolls, with --key-custody. In both modes the host encrypts every backup the same way before it leaves. The difference is where the private key lives, and so how a new host gets it.

SafeGrd-managed key --key-custody safegrd, the default

  1. This host makes the key when it enrolls
  2. SafeGrd keeps it sealed
  3. Your enrolled hosts get it to restore or drill
What it gives you
SafeGrd keeps your key sealed and releases it only to your enrolled hosts, so you can restore even after losing a host.
Every release
is written to the key access record before it happens.
A copy
safegrd keygen --held writes one on an enrolled host, if you want your own.

Customer-managed key --key-custody local

  1. This host makes the key and keeps it
  2. Your copy somewhere safe, such as a password manager
  3. A new host you copy the key file to it
What it gives you
Only you can decrypt these backups. SafeGrd has the public key and nothing else.
Keep
a copy of the key file somewhere safe, before the first backup.

You lose no backups when you lose a host, in either mode. With a SafeGrd-managed key, enroll a new host and restore (the runbook). With a customer-managed key, put your copy of the key file on the new host first.

prod-app-01
$ safegrd keygen --held --out /tmp/prod-app-01.key
Private key: /tmp/prod-app-01.key (mode 0600), a copy of the SafeGrd-managed key.
Public key:  age1lq70sjlq9u8ymuxvmxqwf20trfjn5qx44578xy3gvu2fqyzsaa9scp2uzm
Fingerprint: SG:e53b6c6a:cc58f416
SafeGrd still holds this key and releases it only to your enrolled hosts. Keep this copy somewhere safe: it opens every backup sealed to this key.
Your own copy of a SafeGrd-managed key. SafeGrd still keeps the key, and the key access record lists the copy.
prod-app-01
$ safegrd keygen --out /tmp/offline.key
Private key: /tmp/offline.key (mode 0600). Keep a copy somewhere safe.
Public key:  age1k0j89dn7xm007xyd2ct7z4jxfyw7scph5tt5my6f83rmcza6tp9s3ppdmu
Fingerprint: SG:b26ce676:74b55239
A new customer-managed key. Make it on a machine kept apart from the hosts that back up, and put the public key in encryption.public_key on those hosts. They can then encrypt but cannot decrypt.

The console shows how each surface's key is managed. The CLI never sends a key you supply yourself. When SafeGrd backs up a database or bucket for you, there is no host to make a key. Those backups are sealed to a key SafeGrd makes and keeps for your organization.

Credentials

When SafeGrd holds a credential, it keeps it sealed and releases it only to the surface's host when it backs up, or to the machine SafeGrd starts for that backup. Nobody can read it back through the console or the API. A credential you keep on the host stays in its config, which points to it with an environment variable, a command or a file.

To restore from the console you need an account that can write, and the backup's own account cannot. Type that account's URL when you ask for the restore. SafeGrd seals it and gives it once to the host or the machine SafeGrd starts that runs the restore. The release is recorded in the key access log. SafeGrd deletes the URL when the restore ends, or after a day if nothing picked it up. A machine SafeGrd starts for a restore gets the snapshot's key and that one URL, for that restore only. It never receives the surface's own credential.

Whichever you pick, the archive is sealed the same way. Install and enroll shows how to choose the key when a host enrolls.

SafeGrd's own database

SafeGrd's database holds your account, your hosts' enrollment, the record of every backup and drill, and the sealed secrets above.

If a host, a token or a key file is compromised

Work through the steps in order. The first one cuts off access, and the later ones show what was reached. Each step names the console action or the command to run.

What happenedWhat to do
A host is lost or compromised
  1. Rotate the host's token: Hosts, the host's Details, Edit, Rotate token. The old node token stops working at once. The host can no longer report, fetch a held credential or receive the key.
  2. Rotate the surface's credential at the database or mail provider, whether SafeGrd or the host kept it. Whoever had the host could read its config. If SafeGrd holds the credential, save the new one from Credential in the surface's panel.
  3. Check what the host was given: Projects, the project's Storage, Key access record. It lists every release of a held secret to that host, with the time. That is what the host could have decrypted or connected to.
  4. Enroll the replacement with safegrd enroll. With a SafeGrd-managed key it can restore at once (runbook). With a customer-managed key, copy the key file to it.
A personal access token leaked
  1. Revoke it: Tokens, the token, Revoke. SafeGrd refuses every request with it from then on.
  2. A token can enroll a host, read, and ask for a backup or a drill. Check Hosts for a host you did not enroll, and the Key access record for a release to one. Activity in the console shows what changed while the token was out.
  3. Create a new token for whatever used the old one, and pass it as env:VAR or file:/path.
An agent's sign-in token leaked
  1. Revoke it: Tokens, the row that says Signed in from and the client's name, Revoke. The client's access and refresh tokens stop working on their next call.
  2. A sign-in token works only on the MCP server, and the REST API refuses it. Its access token lasts an hour. The refresh token that renews it is single-use, and presenting one twice ends the whole sign-in on its own. While it was out, it could do what a personal access token can: read, and ask for a backup or a drill.
  3. Connect the client again. It signs in afresh.
A customer-managed key file was copied
  1. Rotate the bucket's access key at your storage provider, and save the new one under the project's Storage in Projects. Someone with the key file can read a backup only if they can also download it.
  2. On the host, run safegrd init --force. It writes a fresh config and a new key, so SafeGrd sees the host as a new one. Run safegrd enroll to send the new public key, then add its surfaces again. Backups from then on are encrypted to the new key.
  3. Keep the old key file until the earlier snapshots expire, because they still open with it. Guard it as before: anyone holding it and a copy of a snapshot can read that snapshot.

The activity log

Activity in the console is the audit log for owners and admins. Each row names who did what, and when. It records:

Every release of a held key or credential appears here too, as well as in the Key access record. A host reading its own hosted storage appears only in the key access record. Owners and admins can read both and export them as CSV or JSON (GET /api/v1/orgs/{id}/events/export?category=action|access&format=csv|json). SafeGrd keeps them for 90 days, so export the log to keep it longer.

Check these claims in the CLI's source

You can check everything this page says about what leaves your host in the CLI's source, at github.com/safegrd/cli. It is licensed under the Business Source License 1.1, so you can read, modify, build and evaluate it. Any SafeGrd account comes with production use, the free plan included. Without an account you can still list, verify, restore and export your backups. If SafeGrd stops running its service, all production use is allowed. Each version becomes GPLv3 four years after its release (license).

Hardening the host