My backup scripts and tools
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
Jim Paris 43ceb39120 backup: support multiple roots; remove "relative absolute path" nonsense 1 month ago
.gitignore Update git setup 1 month ago
Makefile Fix some exclude/include path issues; misc setup improvements 1 month ago
Pipfile Implement filesystem scanning with configurable filters 1 month ago
Pipfile.lock Implement filesystem scanning with configurable filters 1 month ago
README.md Update README cheat sheet 1 month ago
backup.py backup: support multiple roots; remove "relative absolute path" nonsense 1 month ago
config.yaml backup: support multiple roots; remove "relative absolute path" nonsense 1 month ago
initial-setup.sh Fix shebang line after setup 1 month ago
prune.sh Continue reworking towards local copy of borg, etc 1 month ago

README.md

Initial setup

Run on client:

sudo git clone https://git.jim.sh/jim/borg-setup.git /opt/borg
sudo /opt/borg/initial-setup.sh

Customize /opt/borg/config.yaml as desired.

Cheat sheet

After setup, the copy of this file on the client will have the variables in this section filled in automatically

Configuration

Hostname: ${HOSTNAME}
Base directory: ${BORG_DIR}
Destination: ${BACKUP_USER}@${BACKUP_HOST}
Repository: ${BACKUP_REPO}

Commands

See when next backup is scheduled:

systemctl list-timers borg-backup.timer

See status of most recent backup:

systemctl status --full --lines 999999 --no-pager --all borg-backup

Watch log:

journalctl --all --follow --unit borg-backup

Start backup now:

sudo systemctl start borg-backup

Interrupt backup in progress:

sudo systemctl stop borg-backup

Show backups and related info:

sudo ${BORG_DIR}/borg.sh info
sudo ${BORG_DIR}/borg.sh list

Run Borg using the read-write SSH key:

sudo ${BORG_DIR}/borg.sh --rw list

Mount and look at files:

mkdir mnt
sudo ${BORG_DIR}/borg.sh mount :: mnt
sudo -s # to explore as root
sudo umount mnt

Prune old backups. Only run if sure local system was never compromised, as object deletion could have been queued during append-only operations. Requires SSH key password from bitwarden.

sudo ${BORG_DIR}/prune.sh

Design

  • On server, we have a separate user account “jim-backups”. Password for this account is in bitwarden in the “Backups” folder, under ssh backup.jim.sh.

  • Repository keys are repokeys, which get stored on the server, inside the repo. Passphrases are stored:

    • on clients (in /opt/borg/passphrase, for making backups)
    • in bitwarden (under borg <hostname>, user repo key)
  • Each client has two SSH keys for connecting to the server:

    • /opt/borg/ssh/id_ecdsa_appendonly
      • configured on server for append-only operation
      • used for making backups
      • no password
    • /opt/borg/ssh/id_ecdsa
      • configured on server for read-write operation
      • used for manual recovery, management, pruning
      • password in bitwarden (under borg [hostname], user read-write ssh key)
  • Pruning requires the password and is a manual operation, and should only be run when the client has not been compromised.

    sudo /opt/borg/prune.sh
    
  • Systemd timers start daily backups:

    /etc/systemd/system/borg-backup.service -> /opt/borg/borg-backup.service
    /etc/systemd/system/borg-backup.timer -> /opt/borg/borg-backup.timer
    
  • Backup script /opt/borg/backup.py uses configuration in /opt/borg/config.yaml to generate our own list of files, excluding anything that’s too large by default. This requires borg 1.2.0b1 or newer, which is why the setup scripts download a specific version.