Borgmatic

Borgmatic#

Let’s say we want to backup the root (/) partition. By using Borgmatic and these scripts you get:

  • compressed and differential backups

  • optional notifications before, after and during the backup

  • possibility browse through all versions of a backup and to have optional notifications about the mount action

See also

  • A collection of scripts I have written and/or adapted that I currently use on my systems as automated tasks [1]

  • borgmatic [2]

  • How to monitor your backups - borgmatic [3]

  • How to deal with very large backups - borgmatic [4]

  • Low power consumption home backup server project - Where is it? [5]

  • Borgmatic Systemd unit service file [6]

  • Borgmatic Systemd unit timer file [7]

  • GitHub - caronc/apprise: Apprise - Push Notifications that work with just about every platform! [11]

Setup#

  1. install the dependencies

    apt-get install borgmatic python3-yaml python3-venv

  2. create the jobs directories. See reference

    mkdir -p /home/jobs/{scripts,services}/by-user/root
chmod 700 -R /home/jobs/{scripts,services}/by-user/root

  3. create a new virtual environment

    cd /home/jobs/scripts/by-user/root
python3 -m venv .venv_borgmatic_hooks
. .venv_borgmatic_hooks/bin/activate

  4. install fpyutils. See reference

  5. create the requirements_notify_unit_status.txt file

    /home/jobs/scripts/by-user/root/requirements_borgmatic_hooks.txt#
    apprise
PyYAML

  6. install the dependencies

    pip3 install -r requirements_borgmatic_hooks.txt
deactivate

  7. create the hooks script

    /home/jobs/scripts/by-user/root/borgmatic_hooks.py#
     1#!/usr/bin/env python3
 2#
 3# borgmatic_hooks.py
 4#
 5# Copyright (C) 2021-2024 Franco Masotti
 6#
 7# This program is free software: you can redistribute it and/or modify
 8# it under the terms of the GNU General Public License as published by
 9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program.  If not, see <https://www.gnu.org/licenses/>.
19r"""Send a notification after the backup operation."""
20
21import shlex
22import sys
23
24import apprise
25import yaml
26
27if __name__ == '__main__':
28    configuration_file = shlex.quote(sys.argv[1])
29    config = yaml.load(open(configuration_file), Loader=yaml.SafeLoader)
30    action = sys.argv[2]
31    borgmatic_config_file = shlex.quote(sys.argv[3])
32    repository = sys.argv[4]
33    output = sys.argv[5]
34    error = sys.argv[6]
35
36    message = action + ' on ' + repository + ' using ' + borgmatic_config_file + ': ' + '\n' + 'output: ' + output + '\n' + 'error: ' + error
37
38    # Create an Apprise instance.
39    apobj = apprise.Apprise()
40
41    # Add all of the notification services by their server url.
42    for uri in config['apprise_notifiers']['dest']:
43        apobj.add(uri)
44
45    apobj.notify(
46        body=message,
47        title=config['apprise_notifiers']['title'],
48    )

  8. create the mount script

    /home/jobs/scripts/by-user/root/borgmatic_mount.py#
     1#!/usr/bin/env python3
 2#
 3# borgmatic_mount.py
 4#
 5# Copyright (C) 2020-2022 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
 6#
 7# This program is free software: you can redistribute it and/or modify
 8# it under the terms of the GNU General Public License as published by
 9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program.  If not, see <https://www.gnu.org/licenses/>.
19r"""Mount a borg backup."""
20
21import pathlib
22import shlex
23import sys
24
25import fpyutils
26import yaml
27
28if __name__ == '__main__':
29    configuration_file = shlex.quote(sys.argv[1])
30    config = yaml.load(open(configuration_file), Loader=yaml.SafeLoader)
31    # Action may only be "mount" or "umount".
32    action = shlex.quote(sys.argv[2])
33
34    pathlib.Path(shlex.quote(config['files']['mountpoint'])).mkdir(
35        0o700, exist_ok=True, parents=True)
36    fpyutils.shell.execute_command_live_output(
37        shlex.quote(config['files']['borgmatic_binary']) + ' --config ' +
38        shlex.quote(config['files']['borgmatic_config']) + ' ' + action +
39        ' --mount-point ' + shlex.quote(config['files']['mountpoint']))
40
41    message = 'borgmatic_mount using ' + shlex.quote(
42        config['files']['borgmatic_config']
43    ) + '\n' + 'action: ' + action + '\n' + 'on: ' + shlex.quote(
44        config['files']['mountpoint'])
45    if config['notify']['gotify']['enabled']:
46        m = config['notify']['gotify']['message'] + '\n' + message
47        fpyutils.notify.send_gotify_message(
48            config['notify']['gotify']['url'],
49            config['notify']['gotify']['token'], m,
50            config['notify']['gotify']['title'],
51            config['notify']['gotify']['priority'])
52    if config['notify']['email']['enabled']:
53        fpyutils.notify.send_email(message,
54                                   config['notify']['email']['smtp_server'],
55                                   config['notify']['email']['port'],
56                                   config['notify']['email']['sender'],
57                                   config['notify']['email']['user'],
58                                   config['notify']['email']['password'],
59                                   config['notify']['email']['receiver'],
60                                   config['notify']['email']['subject'])

  9. create the borgmatic service

    /home/jobs/services/by-user/root/borgmatic.myhostname_root.service#
     1#
 2# borgmatic.myhostname_root.service
 3#
 4# Copyright (C) 2016-2020 Dan Helfman <https://projects.torsion.org/witten/borgmatic/raw/branch/master/sample/systemd/borgmatic.service>
 5#               2020-2022 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
 6#
 7# This program is free software: you can redistribute it and/or modify
 8# it under the terms of the GNU General Public License as published by
 9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program.  If not, see <https://www.gnu.org/licenses/>.
19
20[Unit]
21Description=borgmatic myhostname_root backup
22Wants=network-online.target
23After=network-online.target
24ConditionACPower=true
25Requires=backed-up-mountpoint.mount
26Requires=mnt-backups-myhostname_root.mount
27After=backed-up-mountpoint.mount
28After=mnt-backups-myhostname_root.mount
29
30[Service]
31Type=oneshot
32
33# Lower CPU and I/O priority.
34Nice=19
35CPUSchedulingPolicy=batch
36IOSchedulingClass=best-effort
37IOSchedulingPriority=7
38IOWeight=100
39
40# Do not retry.
41Restart=no
42
43# Prevent rate limiting of borgmatic log events. If you are using an older version of systemd that
44# doesn't support this (pre-240 or so), you may have to remove this option.
45LogRateLimitIntervalSec=0
46
47# Delay start to prevent backups running during boot.
48ExecStartPre=/usr/bin/sleep 1m
49ExecStart=/usr/bin/systemd-inhibit --who="borgmatic" --why="Prevent interrupting scheduled backup" /usr/bin/borgmatic --config /home/jobs/scripts/by-user/root/borgmatic.myhostname_root.yaml --syslog-verbosity 1
50
51User=root
52Group=root
53
54[Install]
55WantedBy=multi-user.target

  10. create the borgmatic timer

    /home/jobs/services/by-user/root/borgmatic.myhostname_root.timer#
     1#
 2# borgmatic.myhostname_root.timer
 3#
 4# Copyright (C) 2020-2022 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
 5#
 6# This program is free software: you can redistribute it and/or modify
 7# it under the terms of the GNU General Public License as published by
 8# the Free Software Foundation, either version 3 of the License, or
 9# (at your option) any later version.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License
17# along with this program.  If not, see <https://www.gnu.org/licenses/>.
18
19[Unit]
20Description=Once a day borgmatic myhostname_root backup
21
22[Timer]
23OnCalendar=*-*-* 3:00:00
24Persistent=true
25
26[Install]
27WantedBy=timers.target

  11. create the borgmatic mount service

    /home/jobs/services/by-user/root/borgmatic-mount.myhostname_root.service#
     1#
 2# borgmatic-mount.myhostname_root.service
 3#
 4# Copyright (C) 2020-2022 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
 5#
 6# This program is free software: you can redistribute it and/or modify
 7# it under the terms of the GNU General Public License as published by
 8# the Free Software Foundation, either version 3 of the License, or
 9# (at your option) any later version.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License
17# along with this program.  If not, see <https://www.gnu.org/licenses/>.
18
19[Unit]
20Description=borgmatic myhostname_root mount and umount
21
22[Service]
23Type=simple
24RemainAfterExit=yes
25ExecStart=/home/jobs/scripts/by-user/root/borgmatic_mount.py /home/jobs/scripts/by-user/root/borgmatic_mount.myhostname_root.yaml 'mount'
26ExecStop=/home/jobs/scripts/by-user/root/borgmatic_mount.py /home/jobs/scripts/by-user/root/borgmatic_mount.myhostname_root.yaml 'umount'
27User=root
28Group=root

  12. create a borgmatic hooks configuration file

    /home/jobs/scripts/by-user/root/borgmatic_hooks.myhostname_root.yaml#
     1#
 2# borgmatic_hooks.myhostname_your_source_mountpoint.conf
 3#
 4# Copyright (C) 2020-2024 Franco Masotti
 5#
 6# This program is free software: you can redistribute it and/or modify
 7# it under the terms of the GNU General Public License as published by
 8# the Free Software Foundation, either version 3 of the License, or
 9# (at your option) any later version.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License
17# along with this program.  If not, see <https://www.gnu.org/licenses/>.
18
19apprise_notifiers:
20  dest:
21    - 'nctalks://<string>/'
22    - 'mailtos://<string>'
23  title: 'borgmatic backup'

  13. create a borgmatic mount configuration file

    /home/jobs/scripts/by-user/root/borgmatic_mount.myhostname_root.yaml#
     1#
 2# borgmatic_mount.myhostname_root.yaml
 3#
 4# Copyright (C) 2020-2022 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
 5#
 6# This program is free software: you can redistribute it and/or modify
 7# it under the terms of the GNU General Public License as published by
 8# the Free Software Foundation, either version 3 of the License, or
 9# (at your option) any later version.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License
17# along with this program.  If not, see <https://www.gnu.org/licenses/>.
18
19files:
20    borgmatic_config: '/home/jobs/scripts/by-user/root/borgmatic.myhostname_root.yaml'
21    mountpoint: '/tmp/borgmatic.myhostname_root'
22    borgmatic_binary: '/usr/bin/borgmatic'
23
24notify:
25    email:
26        enabled: true
27        smtp_server: 'smtp.gmail.com'
28        port: 465
29        sender: 'myusername@gmail.com'
30        user: 'myusername'
31        password: 'my awesome password'
32        receiver: 'myusername@gmail.com'
33        subject: 'borgmatic mount'
34    gotify:
35        enabled: true
36        url: '<gotify url>'
37        token: '<app token>'
38        title: 'borgmatic mount'
39        message: 'borgmatic mount'
40        priority: 5

New repository#

Before enabling the backup you need to decide exactly where to backup the directory, what to exclude, etc…

In this example we are not using Borg’s encryption because:

  • it works with older versions of borg

  • data is simpler to recover in case things go wrong

  • these are not to be considered offsite backups

By using Borgmatic you can either decide to backup on a local directory or on a remote machine via SSH.

See also

  • borg init — Borg - Deduplicating Archiver 1.2.0 documentation [8]

Types of backup#

Run as user

Instruction number

root

*

If you are backing up on a local directory you simply have to create a new repository according to the configuration.

  1. create a new configuration file for the local backup

    /home/jobs/scripts/by-user/root/borgmatic.myhostname_root.yaml#
     1#
 2# borgmatic.myhostname_root.yaml
 3#
 4# Copyright (C) 2014-2020 Dan Helfman <https://torsion.org/borgmatic/docs/reference/config.yaml>
 5#               2020-2022 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
 6#
 7# This program is free software: you can redistribute it and/or modify
 8# it under the terms of the GNU General Public License as published by
 9# the Free Software Foundation, either version 3 of the License, or
10# (at your option) any later version.
11#
12# This program is distributed in the hope that it will be useful,
13# but WITHOUT ANY WARRANTY; without even the implied warranty of
14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
15# GNU General Public License for more details.
16#
17# You should have received a copy of the GNU General Public License
18# along with this program.  If not, see <https://www.gnu.org/licenses/>.
19
20location:
21    source_directories:
22        - /backed/up/mountpoint
23    repositories:
24        - /mnt/backups/myhostname_root.borg
25    exclude_patterns:
26        - /data
27        - /proc
28        - /sys
29        - /tmp
30        - /run
31        - /mnt
32        - /media
33        - /lost+found
34storage:
35    checkpoint_interval: 900
36    lock_wait: 120
37retention:
38    keep_within: 1w
39    keep_monthly: 1
40consistency:
41    checks:
42        - disabled
43output:
44    color: false
45hooks:
46    before_backup:
47        - /bin/sh -c '. /home/jobs/scripts/by-user/root/.venv_borgmatic_hooks/bin/activate && /home/jobs/scripts/by-user/root/borgmatic_hooks.py /home/jobs/scripts/by-user/root/borgmatic_hooks.myhostname_root.yaml "start" "{configuration_filename}" "{repository}" "{output}" "{error}"; deactivate'
48    after_backup:
49        - /bin/sh -c '. /home/jobs/scripts/by-user/root/.venv_borgmatic_hooks/bin/activate && /home/jobs/scripts/by-user/root/borgmatic_hooks.py /home/jobs/scripts/by-user/root/borgmatic_hooks.myhostname_root.yaml "finish" "{configuration_filename}" "{repository}" "{output}" "{error}"; deactivate'
50    on_error:
51        - /bin/sh -c '. /home/jobs/scripts/by-user/root/.venv_borgmatic_hooks/bin/activate && /home/jobs/scripts/by-user/root/borgmatic_hooks.py /home/jobs/scripts/by-user/root/borgmatic_hooks.myhostname_root.yaml "error" "{configuration_filename}" "{repository}" "{output}" "{error}"; deactivate'

  2. create the new repository

    borg init -e none /mnt/backups/myhostname_root.borg

Deploy#

  1. run the deploy script

  2. check if the backup service works

Note

If you get an unknown repo error try adding this key: storage.unknown_unencrypted_repo_access_is_ok: true

Mount#

To mount the backup you just have to run the Systemd service

systemctl start borgmatic-mount.myhostname_root.service

To unmount, stop the service

systemctl stop borgmatic-mount.myhostname_root.service

Other#

Databases#

Borgmatic also support backing up databases. Just put [] as location.source_directories and add a database object in the hooks section.

See also

  • How to backup your databases - borgmatic [9]

Running the service before computer shutdown#

See also

  • linux - How do I run a script before everything else on shutdown with systemd? - Super User [10]

  1. change some of the original borgmatic service Systemd unit file lines with these:

    ExecStart=/bin/true
RemainAfterExit=yes
TimeoutStopSec=infinity
ExecStop=/usr/bin/borgmatic --config /home/jobs/scripts/by-user/root/borgmatic.myhostname_root.yaml --syslog-verbosity 1

Although this service remains active all the time, the syncronization action runs when the system is halted using an ExecStop directive. Since we don’t know how much time the syncronization takes, a TimeoutStopSec=infinity directive is present.

Automatic backup on a removable USB drive on plug in#

  1. create a partition on a block device

  2. create a filesystem. In this example the filsystem’s UUID is ABCD-0123 and is obtained with this command

    udevadm info --name=${partition} | grep "ID_FS_UUID="

  3. add its entry in the fstab file, for example

    /etc/fstab#
    1UUID=ABCD-0123   /mnt/backups/myhostname_root.borg            btrfs       nofail,auto,rw,relatime     0 0

  4. create a new udev rule to enable the auto mounting

    /etc/udev/rules.d/99-usb-automount.rule#
    1ACTION=="add", SUBSYSTEMS=="usb", ENV{ID_FS_UUID}=="ABCD-0123", SUBSYSTEM=="block", RUN{program}+="/usr/bin/bash -c '(/usr/bin/systemctl start root.mount && systemctl start borgmatic.myhostname_root.service && /usr/bin/systemctl start udev-umount.myhostname_root.service) &'"

  5. reboot to apply the rules

  6. plug in the USB device and check if the service starts

Mounting backups on other computers#

Let’s say computer A backups on computer B, but A is offline.

You can access A’s backups via B on a third computer, C. C just needs to have SSH access to B and a copy of A’s original borgmatic files.

Footnotes

Contents