Compare commits

...

1 Commits

Author SHA1 Message Date
a15cb5b07d all: remove concept of read-write key
We don't need a read-write key: we can just SSH directly to
jim-backups@backup.jim.sh instead and run commands thay way.
Remove read-write key and document it in the README.

Also add some tools to update the README variables on updates.
2021-10-19 14:46:34 -04:00
5 changed files with 56 additions and 82 deletions

View File

@ -38,7 +38,9 @@ rebase:
git pull git pull
git checkout - git checkout -
git rebase master git rebase master
./initial-setup.sh --update-paths
systemctl daemon-reload systemctl daemon-reload
git status
# Show status of most recent backup run # Show status of most recent backup run
.PHONY: status .PHONY: status

View File

@ -16,14 +16,14 @@ Cheat sheet
*After setup, the copy of this file on the client will have the *After setup, the copy of this file on the client will have the
variables in this section filled in automatically* variables in this section filled in automatically*
### Configuration ## Configuration
Hostname: ${HOSTNAME} Hostname: ${HOSTNAME}
Base directory: ${BORG_DIR} Base directory: ${BORG_DIR}
Destination: ${BACKUP_USER}@${BACKUP_HOST} Destination: ${BACKUP_USER}@${BACKUP_HOST}
Repository: ${BACKUP_REPO} Repository: ${BACKUP_REPO}
### Commands ## Commands
See when next backup is scheduled: See when next backup is scheduled:
@ -61,12 +61,25 @@ Mount and look at files:
sudo -s # to explore as root sudo -s # to explore as root
sudo umount mnt sudo umount mnt
Prune old backups. Only run if sure local system was never compromised,
as object deletion could have been queued during append-only operations.
Requires SSH key password from bitwarden.
sudo ${BORG_DIR}/prune.sh ## Compaction and remote access
Old backups are "pruned" automatically, but because the SSH key is
append-only, no space is actually recovered on the server, it's just
marked for deletion. If you are sure that the client system was not
compromised, then you can run compaction manually directly on the
backup host by logging in via SSH (bitwarden `ssh ${BACKUP_HOST} /
${BACKUP_USER}`) and compacting there:
ssh ${BACKUP_USER}@${BACKUP_HOST} borg/borg compact --verbose --progress ${BACKUP_REPO}
This doesn't require the repo key. It shouldn't be entered on the untrusted
backup host, so for operations that need it, use a trusted host and run borg
remotely instead, e.g.:
${BORG_DIR}/Borg.bin --remote-path borg/borg info ${BACKUP_USER}@${BACKUP_HOST}:borg/${HOSTNAME}
The repo passphrase is in bitwarden `borg ${HOSTNAME}/ repo key`.
Design Design
@ -81,20 +94,13 @@ Design
- on clients (in `/opt/borg/passphrase`, for making backups) - on clients (in `/opt/borg/passphrase`, for making backups)
- in bitwarden (under `borg <hostname>`, user `repo key`) - in bitwarden (under `borg <hostname>`, user `repo key`)
- Each client has two SSH keys for connecting to the server: - Each client has two passwordless SSH keys for connecting to the server:
- `/opt/borg/ssh/id_ecdsa_appendonly` - `/opt/borg/ssh/id_ecdsa_appendonly`
- configured on server for append-only operation - configured on server for append-only operation
- used for making backups - used for making backups
- no password - `/opt/borg/ssh/id_ecdsa_notify`
- `/opt/borg/ssh/id_ecdsa` - configured on server for running `borg/notify.sh` only
- configured on server for read-write operation - used for sending email notifications on errors
- used for manual recovery, management, pruning
- password in bitwarden (under `borg <hostname>`, user `read-write ssh key`)
- Pruning requires the password and is a manual operation, and should only
be run when the client has not been compromised.
sudo /opt/borg/prune.sh
- Systemd timers start daily backups: - Systemd timers start daily backups:
@ -103,9 +109,7 @@ Design
- Backup script `/opt/borg/backup.py` uses configuration in - Backup script `/opt/borg/backup.py` uses configuration in
`/opt/borg/config.yaml` to generate our own list of files, excluding `/opt/borg/config.yaml` to generate our own list of files, excluding
anything that's too large by default. This requires borg 1.2.0b1 anything that's too large by default. This requires borg 1.2 or newer.
or newer.
Notes Notes

11
borg.sh
View File

@ -7,15 +7,6 @@ export BORG_PASSCOMMAND="cat ${BORG_DIR}/passphrase"
export BORG_BASE_DIR=${BORG_DIR} export BORG_BASE_DIR=${BORG_DIR}
export BORG_CACHE_DIR=${BORG_DIR}/cache export BORG_CACHE_DIR=${BORG_DIR}/cache
export BORG_CONFIG_DIR=${BORG_DIR}/config export BORG_CONFIG_DIR=${BORG_DIR}/config
if [ "$1" = "--rw" ] ; then export BORG_RSH="ssh -F $SSH/config -i $SSH/id_ecdsa_appendonly"
if [ "$BORG_RW_KEY_ADDED" != "1" ] ; then
echo "=== Need SSH key passphrase. Check Bitwarden for:"
echo "=== borg $HOSTNAME / read-write SSH key"
fi
export BORG_RSH="ssh -F $SSH/config -o BatchMode=no -o PreferredAuthentications=publickey -i $SSH/id_ecdsa"
shift
else
export BORG_RSH="ssh -F $SSH/config -i $SSH/id_ecdsa_appendonly"
fi
exec "${BORG_BIN}" "$@" exec "${BORG_BIN}" "$@"

View File

@ -27,8 +27,34 @@ trap 'error_handler ${BASH_SOURCE} ${LINENO} $?' ERR
set -o errexit set -o errexit
set -o errtrace set -o errtrace
if [ -e ".setup-complete" ]; then update_paths()
echo "Error: BORG_DIR $BORG_DIR was already set up; giving up." {
sed -i \
-e "s!\${HOSTNAME}!${HOSTNAME}!g" \
-e "s!\${BORG_DIR}!${BORG_DIR}!g" \
-e "s!\${BACKUP_USER}!${BACKUP_USER}!g" \
-e "s!\${BACKUP_HOST}!${BACKUP_HOST}!g" \
-e "s!\${BACKUP_REPO}!${BACKUP_REPO}!g" \
README.md
sed -i\
-e "1c#!${BORG_DIR}/.venv/bin/python" \
backup.py
}
if [ "$1" == "--update-paths" ] ; then
if [ -e "vars.sh" ]; then
echo "Updating paths"
update_paths
exit 0
else
echo "Can't update paths, not set up yet"
exit 1
fi
fi
if [ -e "vars.sh" ]; then
echo "Error: BORG_DIR $BORG_DIR already looks set up; giving up."
echo "Use \"git clean\" to return it to original state if desired" echo "Use \"git clean\" to return it to original state if desired"
exit 1 exit 1
fi fi
@ -113,7 +139,6 @@ print_random_key()
generate_keys() generate_keys()
{ {
PASS_SSH=$(print_random_key)
PASS_REPOKEY=$(print_random_key) PASS_REPOKEY=$(print_random_key)
echo "$PASS_REPOKEY" > passphrase echo "$PASS_REPOKEY" > passphrase
chmod 600 passphrase chmod 600 passphrase
@ -136,8 +161,6 @@ configure_ssh()
-C "backup-appendonly@$HOSTID" -f "$SSH/id_ecdsa_appendonly" -C "backup-appendonly@$HOSTID" -f "$SSH/id_ecdsa_appendonly"
ssh-keygen -N "" -t ecdsa \ ssh-keygen -N "" -t ecdsa \
-C "backup-notify@$HOSTID" -f "$SSH/id_ecdsa_notify" -C "backup-notify@$HOSTID" -f "$SSH/id_ecdsa_notify"
ssh-keygen -N "$PASS_SSH" -t ecdsa \
-C "backup@$HOSTID" -f "$SSH/id_ecdsa"
# Create config snippets # Create config snippets
log "Creating SSH config and wrapper script" log "Creating SSH config and wrapper script"
@ -184,7 +207,6 @@ EOF
run_ssh_command "cat >> .ssh/authorized_keys" <<EOF run_ssh_command "cat >> .ssh/authorized_keys" <<EOF
command="$cmd --append-only",restrict $(cat "$SSH/id_ecdsa_appendonly.pub") command="$cmd --append-only",restrict $(cat "$SSH/id_ecdsa_appendonly.pub")
command="borg/notify.sh",restrict $(cat "$SSH/id_ecdsa_notify.pub") command="borg/notify.sh",restrict $(cat "$SSH/id_ecdsa_notify.pub")
command="$cmd",restrict $(cat "$SSH/id_ecdsa.pub")
EOF EOF
# Test that everything worked # Test that everything worked
@ -281,21 +303,6 @@ EOF
fi fi
} }
update_paths()
{
sed -i \
-e "s!\${HOSTNAME}!${HOSTNAME}!g" \
-e "s!\${BORG_DIR}!${BORG_DIR}!g" \
-e "s!\${BACKUP_USER}!${BACKUP_USER}!g" \
-e "s!\${BACKUP_HOST}!${BACKUP_HOST}!g" \
-e "s!\${BACKUP_REPO}!${BACKUP_REPO}!g" \
README.md
sed -i\
-e "1c#!${BORG_DIR}/.venv/bin/python" \
backup.py
}
git_setup() git_setup()
{ {
if ! git checkout -b "setup-${HOSTNAME}" ; then if ! git checkout -b "setup-${HOSTNAME}" ; then
@ -324,11 +331,7 @@ update_paths
git_setup git_setup
echo echo
notice "Add these two passwords to Bitwarden:" notice "Add this password to Bitwarden:"
notice ""
notice " Name: borg ${HOSTNAME}"
notice " Username: read-write ssh key"
notice " Password: $PASS_SSH"
notice "" notice ""
notice " Name: borg ${HOSTNAME}" notice " Name: borg ${HOSTNAME}"
notice " Username: repo key" notice " Username: repo key"

View File

@ -1,26 +0,0 @@
#!/bin/bash
set -e
. "$(dirname "$0")"/vars.sh
if [ "$BORG_RW_KEY_ADDED" != "1" ] ; then
echo "Re-executing under a new ssh agent"
exec env BORG_RW_KEY_ADDED=1 ssh-agent "$0"
fi
echo "=== Please enter SSH key passphrase. Check Bitwarden for:"
echo "=== borg $HOSTNAME / read-write SSH key"
ssh-add "$(realpath "$(dirname "$0")")/ssh/id_ecdsa"
$BORG --rw prune \
--verbose \
--progress \
--stats \
--keep-within=7d \
--keep-daily=14 \
--keep-weekly=8 \
--keep-monthly=-1
$BORG --rw compact \
--verbose \
--progress