My backup scripts and tools
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

README.md 4.1 KiB

7 months ago
7 months ago
7 months ago
9 months ago
9 months ago
9 months ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143
  1. Initial setup
  2. =============
  3. Run on client:
  4. sudo git clone https://git.jim.sh/jim/borg-setup.git /opt/borg
  5. sudo /opt/borg/initial-setup.sh
  6. Customize `/opt/borg/config.yaml` as desired.
  7. Cheat sheet
  8. ===========
  9. *After setup, the copy of this file on the client will have the
  10. variables in this section filled in automatically*
  11. ## Configuration
  12. Hostname: ${HOSTNAME}
  13. Base directory: ${BORG_DIR}
  14. Destination: ${BACKUP_USER}@${BACKUP_HOST}
  15. Repository: ${BACKUP_REPO}
  16. ## Commands
  17. See when next backup is scheduled:
  18. systemctl list-timers borg-backup.timer
  19. See status of most recent backup:
  20. systemctl status --full --lines 999999 --no-pager --all borg-backup
  21. Watch log:
  22. journalctl --all --follow --unit borg-backup
  23. Start backup now:
  24. sudo systemctl start borg-backup
  25. Interrupt backup in progress:
  26. sudo systemctl stop borg-backup
  27. Show backups and related info:
  28. sudo ${BORG_DIR}/borg.sh info
  29. sudo ${BORG_DIR}/borg.sh list
  30. Run Borg using the read-write SSH key:
  31. sudo ${BORG_DIR}/borg.sh --rw list
  32. Mount and look at files:
  33. mkdir mnt
  34. sudo ${BORG_DIR}/borg.sh mount :: mnt
  35. sudo -s # to explore as root
  36. sudo umount mnt
  37. ## Compaction and remote access
  38. Old backups are "pruned" automatically, but because the SSH key is
  39. append-only, no space is actually recovered on the server, it's just
  40. marked for deletion. If you are sure that the client system was not
  41. compromised, then you can run compaction manually directly on the
  42. backup host by logging in via SSH (bitwarden `ssh ${BACKUP_HOST} /
  43. ${BACKUP_USER}`) and compacting there:
  44. ssh ${BACKUP_USER}@${BACKUP_HOST} borg/borg compact --verbose --progress ${BACKUP_REPO}
  45. This doesn't require the repo key. That key shouldn't be entered on
  46. the untrusted backup host, so for operations that need it, use a
  47. trusted host and run borg remotely instead, e.g.:
  48. ${BORG_BIN} --remote-path borg/borg info ${BACKUP_USER}@${BACKUP_HOST}:borg/${HOSTNAME}
  49. The repo passphrase is in bitwarden `borg ${HOSTNAME} / repo key`.
  50. Design
  51. ======
  52. - On server, we have a separate user account "jim-backups". Password
  53. for this account is in bitwarden in the "Backups" folder, under `ssh
  54. backup.jim.sh`.
  55. - Repository keys are repokeys, which get stored on the server, inside
  56. the repo. Passphrases are stored:
  57. - on clients (in `/opt/borg/passphrase`, for making backups)
  58. - in bitwarden (under `borg <hostname>`, user `repo key`)
  59. - Each client has two passwordless SSH keys for connecting to the server:
  60. - `/opt/borg/ssh/id_ecdsa_appendonly`
  61. - configured on server for append-only operation
  62. - used for making backups
  63. - `/opt/borg/ssh/id_ecdsa_notify`
  64. - configured on server for running `borg/notify.sh` only
  65. - used for sending email notifications on errors
  66. - Systemd timers start daily backups:
  67. /etc/systemd/system/borg-backup.service -> /opt/borg/borg-backup.service
  68. /etc/systemd/system/borg-backup.timer -> /opt/borg/borg-backup.timer
  69. - Backup script `/opt/borg/backup.py` uses configuration in
  70. `/opt/borg/config.yaml` to generate our own list of files, excluding
  71. anything that's too large by default. This requires borg 1.2 or newer.
  72. Notes
  73. =====
  74. # Building Borg binary from git
  75. sudo apt install python3.9 scons libacl1-dev libfuse-dev libpython3.9-dev patchelf
  76. git clone https://github.com/borgbackup/borg.git
  77. cd borg
  78. virtualenv --python=python3.9 borg-env
  79. source borg-env/bin/activate
  80. pip install -r requirements.d/development.txt
  81. pip install pyinstaller
  82. pip install llfuse
  83. pip install -e .[llfuse]
  84. pyinstaller --clean --noconfirm scripts/borg.exe.spec
  85. pip install staticx
  86. # for x86
  87. staticx -l /lib/x86_64-linux-gnu/libm.so.6 dist/borg.exe borg.x86_64
  88. # for ARM; see https://github.com/JonathonReinhart/staticx/issues/209
  89. staticx -l /lib/arm-linux-gnueabihf/libm.so.6 dist/borg.exe borg.armv7l
  90. Then run `borg.x86_64`. Confirm the version with `borg.armv7l --version`.
  91. *Note:* This uses the deprecated `llfuse` instead of the newer `pyfuse3`.
  92. `pyfuse3` doesn't work because, at minimum, it pulls in `trio` which
  93. requires `ssl` which is explicitly excluded by
  94. `scripts/borg.exe.spec`.