Initial setup
=============

Run on client:

    sudo git clone https://git.jim.sh/jim/borg-setup.git /opt/borg
    sudo /opt/borg/initial-setup.sh

Customize `/opt/borg/config.yaml` as desired.



Cheat sheet
===========

*After setup, the copy of this file on the client will have the
variables in this section filled in automatically*

## Configuration

    Hostname: ${HOSTNAME}
    Base directory: ${BORG_DIR}
    Destination: ${BACKUP_USER}@${BACKUP_HOST}
    Repository: ${BACKUP_REPO}

## Commands

See when next backup is scheduled:

    systemctl list-timers borg-backup.timer

See status of most recent backup:

    systemctl status --full --lines 999999 --no-pager --all borg-backup

Watch log:

    journalctl --all --follow --unit borg-backup

Start backup now:

    sudo systemctl start borg-backup

Interrupt backup in progress:

    sudo systemctl stop borg-backup

Show backups and related info:

    sudo ${BORG_DIR}/borg.sh info
    sudo ${BORG_DIR}/borg.sh list

Run Borg using the read-write SSH key:

    sudo ${BORG_DIR}/borg.sh --rw list

Mount and look at files:

    mkdir mnt
    sudo ${BORG_DIR}/borg.sh mount :: mnt
    sudo -s # to explore as root
    sudo umount mnt


## Compaction and remote access

Old backups are "pruned" automatically, but because the SSH key is
append-only, no space is actually recovered on the server, it's just
marked for deletion.  If you are sure that the client system was not
compromised, then you can run compaction manually directly on the
backup host by logging in via SSH (bitwarden `ssh ${BACKUP_HOST} /
${BACKUP_USER}`) and compacting there:

    ssh ${BACKUP_USER}@${BACKUP_HOST} borg/borg compact --verbose --progress ${BACKUP_REPO}

This doesn't require the repo key.  That key shouldn't be entered on
the untrusted backup host, so for operations that need it, use a
trusted host and run borg remotely instead, e.g.:

    ${BORG_BIN} --remote-path borg/borg info ${BACKUP_USER}@${BACKUP_HOST}:borg/${HOSTNAME}

The repo passphrase is in bitwarden `borg ${HOSTNAME} / repo key`.


Design
======

- On server, we have a separate user account "jim-backups".  Password
  for this account is in bitwarden in the "Backups" folder, under `ssh
  backup.jim.sh`.

- Repository keys are repokeys, which get stored on the server, inside
  the repo.  Passphrases are stored:
  - on clients (in `/opt/borg/passphrase`, for making backups)
  - in bitwarden (under `borg <hostname>`, user `repo key`)

- Each client has two passwordless SSH keys for connecting to the server:
  - `/opt/borg/ssh/id_ecdsa_appendonly`
    - configured on server for append-only operation
    - used for making backups
  - `/opt/borg/ssh/id_ecdsa_notify`
    - configured on server for running `borg/notify.sh` only
    - used for sending email notifications on errors

- Systemd timers start daily backups:

      /etc/systemd/system/borg-backup.service -> /opt/borg/borg-backup.service
      /etc/systemd/system/borg-backup.timer -> /opt/borg/borg-backup.timer

- Backup script `/opt/borg/backup.py` uses configuration in
  `/opt/borg/config.yaml` to generate our own list of files, excluding
  anything that's too large by default.  This requires borg 1.2 or newer.


Notes
=====

# Building Borg binary from git

    sudo apt install python3.9 scons libacl1-dev libfuse-dev libpython3.9-dev patchelf
    git clone https://github.com/borgbackup/borg.git
    cd borg
    virtualenv --python=python3.9 borg-env
    source borg-env/bin/activate
    pip install -r requirements.d/development.txt
    pip install pyinstaller
    pip install llfuse
    pip install -e .[llfuse]
    pyinstaller --clean --noconfirm scripts/borg.exe.spec
    pip install staticx

    # for x86
    staticx -l /lib/x86_64-linux-gnu/libm.so.6 dist/borg.exe borg.x86_64

    # for ARM; see https://github.com/JonathonReinhart/staticx/issues/209
    staticx -l /lib/arm-linux-gnueabihf/libm.so.6 dist/borg.exe borg.armv7l

Then run `borg.x86_64`.  Confirm the version with `borg.armv7l --version`.

*Note:* This uses the deprecated `llfuse` instead of the newer `pyfuse3`.
`pyfuse3` doesn't work because, at minimum, it pulls in `trio` which
requires `ssl` which is explicitly excluded by
`scripts/borg.exe.spec`.