Browse Source

all: remove concept of read-write key

We don't need a read-write key: we can just SSH directly to
jim-backups@backup.jim.sh instead and run commands thay way.
Remove read-write key and document it in the README.

Also add some tools to update the README variables on updates.
master
Jim Paris 2 years ago
parent
commit
a15cb5b07d
5 changed files with 56 additions and 82 deletions
  1. +2
    -0
      Makefile
  2. +24
    -20
      README.md
  3. +1
    -10
      borg.sh
  4. +29
    -26
      initial-setup.sh
  5. +0
    -26
      prune.sh

+ 2
- 0
Makefile View File

@@ -38,7 +38,9 @@ rebase:
git pull
git checkout -
git rebase master
./initial-setup.sh --update-paths
systemctl daemon-reload
git status

# Show status of most recent backup run
.PHONY: status


+ 24
- 20
README.md View File

@@ -16,14 +16,14 @@ Cheat sheet
*After setup, the copy of this file on the client will have the
variables in this section filled in automatically*

### Configuration
## Configuration

Hostname: ${HOSTNAME}
Base directory: ${BORG_DIR}
Destination: ${BACKUP_USER}@${BACKUP_HOST}
Repository: ${BACKUP_REPO}

### Commands
## Commands

See when next backup is scheduled:

@@ -61,12 +61,25 @@ Mount and look at files:
sudo -s # to explore as root
sudo umount mnt

Prune old backups. Only run if sure local system was never compromised,
as object deletion could have been queued during append-only operations.
Requires SSH key password from bitwarden.

sudo ${BORG_DIR}/prune.sh
## Compaction and remote access

Old backups are "pruned" automatically, but because the SSH key is
append-only, no space is actually recovered on the server, it's just
marked for deletion. If you are sure that the client system was not
compromised, then you can run compaction manually directly on the
backup host by logging in via SSH (bitwarden `ssh ${BACKUP_HOST} /
${BACKUP_USER}`) and compacting there:

ssh ${BACKUP_USER}@${BACKUP_HOST} borg/borg compact --verbose --progress ${BACKUP_REPO}

This doesn't require the repo key. It shouldn't be entered on the untrusted
backup host, so for operations that need it, use a trusted host and run borg
remotely instead, e.g.:

${BORG_DIR}/Borg.bin --remote-path borg/borg info ${BACKUP_USER}@${BACKUP_HOST}:borg/${HOSTNAME}

The repo passphrase is in bitwarden `borg ${HOSTNAME}/ repo key`.


Design
@@ -81,20 +94,13 @@ Design
- on clients (in `/opt/borg/passphrase`, for making backups)
- in bitwarden (under `borg <hostname>`, user `repo key`)

- Each client has two SSH keys for connecting to the server:
- Each client has two passwordless SSH keys for connecting to the server:
- `/opt/borg/ssh/id_ecdsa_appendonly`
- configured on server for append-only operation
- used for making backups
- no password
- `/opt/borg/ssh/id_ecdsa`
- configured on server for read-write operation
- used for manual recovery, management, pruning
- password in bitwarden (under `borg <hostname>`, user `read-write ssh key`)

- Pruning requires the password and is a manual operation, and should only
be run when the client has not been compromised.

sudo /opt/borg/prune.sh
- `/opt/borg/ssh/id_ecdsa_notify`
- configured on server for running `borg/notify.sh` only
- used for sending email notifications on errors

- Systemd timers start daily backups:

@@ -103,9 +109,7 @@ Design

- Backup script `/opt/borg/backup.py` uses configuration in
`/opt/borg/config.yaml` to generate our own list of files, excluding
anything that's too large by default. This requires borg 1.2.0b1
or newer.

anything that's too large by default. This requires borg 1.2 or newer.


Notes


+ 1
- 10
borg.sh View File

@@ -7,15 +7,6 @@ export BORG_PASSCOMMAND="cat ${BORG_DIR}/passphrase"
export BORG_BASE_DIR=${BORG_DIR}
export BORG_CACHE_DIR=${BORG_DIR}/cache
export BORG_CONFIG_DIR=${BORG_DIR}/config
if [ "$1" = "--rw" ] ; then
if [ "$BORG_RW_KEY_ADDED" != "1" ] ; then
echo "=== Need SSH key passphrase. Check Bitwarden for:"
echo "=== borg $HOSTNAME / read-write SSH key"
fi
export BORG_RSH="ssh -F $SSH/config -o BatchMode=no -o PreferredAuthentications=publickey -i $SSH/id_ecdsa"
shift
else
export BORG_RSH="ssh -F $SSH/config -i $SSH/id_ecdsa_appendonly"
fi
export BORG_RSH="ssh -F $SSH/config -i $SSH/id_ecdsa_appendonly"

exec "${BORG_BIN}" "$@"

+ 29
- 26
initial-setup.sh View File

@@ -27,8 +27,34 @@ trap 'error_handler ${BASH_SOURCE} ${LINENO} $?' ERR
set -o errexit
set -o errtrace

if [ -e ".setup-complete" ]; then
echo "Error: BORG_DIR $BORG_DIR was already set up; giving up."
update_paths()
{
sed -i \
-e "s!\${HOSTNAME}!${HOSTNAME}!g" \
-e "s!\${BORG_DIR}!${BORG_DIR}!g" \
-e "s!\${BACKUP_USER}!${BACKUP_USER}!g" \
-e "s!\${BACKUP_HOST}!${BACKUP_HOST}!g" \
-e "s!\${BACKUP_REPO}!${BACKUP_REPO}!g" \
README.md

sed -i\
-e "1c#!${BORG_DIR}/.venv/bin/python" \
backup.py
}

if [ "$1" == "--update-paths" ] ; then
if [ -e "vars.sh" ]; then
echo "Updating paths"
update_paths
exit 0
else
echo "Can't update paths, not set up yet"
exit 1
fi
fi

if [ -e "vars.sh" ]; then
echo "Error: BORG_DIR $BORG_DIR already looks set up; giving up."
echo "Use \"git clean\" to return it to original state if desired"
exit 1
fi
@@ -113,7 +139,6 @@ print_random_key()

generate_keys()
{
PASS_SSH=$(print_random_key)
PASS_REPOKEY=$(print_random_key)
echo "$PASS_REPOKEY" > passphrase
chmod 600 passphrase
@@ -136,8 +161,6 @@ configure_ssh()
-C "backup-appendonly@$HOSTID" -f "$SSH/id_ecdsa_appendonly"
ssh-keygen -N "" -t ecdsa \
-C "backup-notify@$HOSTID" -f "$SSH/id_ecdsa_notify"
ssh-keygen -N "$PASS_SSH" -t ecdsa \
-C "backup@$HOSTID" -f "$SSH/id_ecdsa"

# Create config snippets
log "Creating SSH config and wrapper script"
@@ -184,7 +207,6 @@ EOF
run_ssh_command "cat >> .ssh/authorized_keys" <<EOF
command="$cmd --append-only",restrict $(cat "$SSH/id_ecdsa_appendonly.pub")
command="borg/notify.sh",restrict $(cat "$SSH/id_ecdsa_notify.pub")
command="$cmd",restrict $(cat "$SSH/id_ecdsa.pub")
EOF

# Test that everything worked
@@ -281,21 +303,6 @@ EOF
fi
}

update_paths()
{
sed -i \
-e "s!\${HOSTNAME}!${HOSTNAME}!g" \
-e "s!\${BORG_DIR}!${BORG_DIR}!g" \
-e "s!\${BACKUP_USER}!${BACKUP_USER}!g" \
-e "s!\${BACKUP_HOST}!${BACKUP_HOST}!g" \
-e "s!\${BACKUP_REPO}!${BACKUP_REPO}!g" \
README.md

sed -i\
-e "1c#!${BORG_DIR}/.venv/bin/python" \
backup.py
}

git_setup()
{
if ! git checkout -b "setup-${HOSTNAME}" ; then
@@ -324,11 +331,7 @@ update_paths
git_setup

echo
notice "Add these two passwords to Bitwarden:"
notice ""
notice " Name: borg ${HOSTNAME}"
notice " Username: read-write ssh key"
notice " Password: $PASS_SSH"
notice "Add this password to Bitwarden:"
notice ""
notice " Name: borg ${HOSTNAME}"
notice " Username: repo key"


+ 0
- 26
prune.sh View File

@@ -1,26 +0,0 @@
#!/bin/bash

set -e
. "$(dirname "$0")"/vars.sh

if [ "$BORG_RW_KEY_ADDED" != "1" ] ; then
echo "Re-executing under a new ssh agent"
exec env BORG_RW_KEY_ADDED=1 ssh-agent "$0"
fi

echo "=== Please enter SSH key passphrase. Check Bitwarden for:"
echo "=== borg $HOSTNAME / read-write SSH key"
ssh-add "$(realpath "$(dirname "$0")")/ssh/id_ecdsa"

$BORG --rw prune \
--verbose \
--progress \
--stats \
--keep-within=7d \
--keep-daily=14 \
--keep-weekly=8 \
--keep-monthly=-1

$BORG --rw compact \
--verbose \
--progress

Loading…
Cancel
Save