[Bug] Exclude-snapshot-root code doesn't check for per-backup-point --relative [sf#20]
bebehei opened this issue · comments
Reported by jcpotts on 2010-02-07 10:20 UTC
First, thanks for rsnapshot.
Backing-up my complete system, and wanting a separate backup for each mount point (disk partition), i find myself having to resort to this overloaded config file (where some of the "backup" lines are commented out temporarily for a shorter dry-run report):
etc \# cat rsnapshot.conf1
config\_version 1.2
snapshot\_root /mnt/usb-a2/.snapshots/
no\_create\_root 1
cmd\_cp /bin/cp
cmd\_rm /bin/rm
cmd\_rsync /usr/bin/rsync
cmd\_du /bin/du
cmd\_rsnapshot\_diff /usr/bin/rsnapshot-diff
retain daily 7
verbose 2
loglevel 3
rsync\_short\_args -aCFHAX
rsync\_long\_args --numeric-ids --relative --delete-excluded
du\_args -csh
one\_fs 1
include\_file /usr/local/etc/rsync/backup-includes
exclude\_file /usr/local/etc/rsync/backup-excludes
link\_dest 1
backup / localhost/TOP/
\#backup /home/ localhost/HOME/ rsync\_long\_args=--numeric-ids --delete-excluded --include-from=/usr/local/etc/rsync/backup-includes --exclude-from=/usr/local/etc/rsync/backup-excludes --exclude=mnt/usb-a2/.snapshots
\#backup /usr/ localhost/USR/ rsync\_long\_args=--numeric-ids --delete-excluded --include-from=/usr/local/etc/rsync/backup-includes --exclude-from=/usr/local/etc/rsync/backup-excludes --exclude=mnt/usb-a2/.snapshots
\#backup /usr/local/ localhost/USRLOCAL/ rsync\_long\_args=--numeric-ids --delete-excluded --include-from=/usr/local/etc/rsync/backup-includes --exclude-from=/usr/local/etc/rsync/backup-excludes --exclude=mnt/usb-a2/.snapshots
\#backup /var/ localhost/VAR/ rsync\_long\_args=--numeric-ids --delete-excluded --include-from=/usr/local/etc/rsync/backup-includes --exclude-from=/usr/local/etc/rsync/backup-excludes --exclude=mnt/usb-a2/.snapshots
\#backup /opt/ localhost/OPT/ rsync\_long\_args=--numeric-ids --delete-excluded --include-from=/usr/local/etc/rsync/backup-includes --exclude-from=/usr/local/etc/rsync/backup-excludes --exclude=mnt/usb-a2/.snapshots
backup /boot/ localhost/BOOT/ rsync\_long\_args=--numeric-ids --delete-excluded --include-from=/usr/local/etc/rsync/backup-includes --exclude-from=/usr/local/etc/rsync/backup-excludes --exclude=mnt/usb-a2/.snapshots
...which yields this:
etc \# rsnapshot -c rsnapshot.conf1 -t daily
mv /mnt/usb-a2/.snapshots/daily.3/ /mnt/usb-a2/.snapshots/daily.4/
mv /mnt/usb-a2/.snapshots/daily.2/ /mnt/usb-a2/.snapshots/daily.3/
mv /mnt/usb-a2/.snapshots/daily.1/ /mnt/usb-a2/.snapshots/daily.2/
mv /mnt/usb-a2/.snapshots/daily.0/ /mnt/usb-a2/.snapshots/daily.1/
/usr/bin/rsync -aCFHAXx --numeric-ids --relative --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes \
\--exclude=mnt/usb-a2/.snapshots \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/ /. \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes \
\--exclude=mnt/usb-a2/.snapshots \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/BOOT/ /boot/ \
/mnt/usb-a2/.snapshots/daily.0/localhost/BOOT/
touch /mnt/usb-a2/.snapshots/daily.0/
BTW, according to the rsync(1) man page, --delete is implied by --delete-excluded.
Notice that, where i am using the "rsync_long_args=", i'm having to specify /all/ rsync long options except the "--link-dest" one, which means both "--{in|ex}clude-from=", and the "--exclude="$snapshot_root one, since it is not enough to assemble your "rsync_long_args=" the way you otherwise would "rsync_long_args" (the one in the config file on its own line, as contrasts with the one you add to the "backup" line)
I would've preferred this configuation (where is shown only what's different from the first one):
etc # cat rsnapshot.conf2
rsync\_long\_args --numeric-ids --delete-excluded
backup / localhost/TOP/ +rsync\_long\_args=--relative
\#backup /home/ localhost/HOME/
\#backup /usr/ localhost/USR/
\#backup /usr/local/ localhost/USRLOCAL/
\#backup /var/ localhost/VAR/
\#backup /opt/ localhost/OPT/
backup /boot/ localhost/BOOT/
...which, unfortunately, gives this:
etc \# rsnapshot -c rsnapshot.conf2 -t daily
mv /mnt/usb-a2/.snapshots/daily.3/ /mnt/usb-a2/.snapshots/daily.4/
mv /mnt/usb-a2/.snapshots/daily.2/ /mnt/usb-a2/.snapshots/daily.3/
mv /mnt/usb-a2/.snapshots/daily.1/ /mnt/usb-a2/.snapshots/daily.2/
mv /mnt/usb-a2/.snapshots/daily.0/ /mnt/usb-a2/.snapshots/daily.1/
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/media /media \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/media
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/bin /bin \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/bin
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/boot /boot \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/boot
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/dev /dev \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/dev
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/etc /etc \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/etc
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/home /home \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/home
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/lib /lib \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/lib
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
/lost+found /mnt/usb-a2/.snapshots/daily.0/localhost/TOP/lost+found
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/opt /opt \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/opt
ln /mnt/usb-a2/.snapshots/daily.1/localhost/TOP/.rsync-filter \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/.rsync-filter
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
/.rsync-filter \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/.rsync-filter
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/proc /proc \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/proc
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/root /root \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/root
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/sbin /sbin \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/sbin
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/emerge-webrsync \
/emerge-webrsync \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/emerge-webrsync
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/sys /sys \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/sys
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/tmp /tmp \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/tmp
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/usr /usr \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/usr
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/var /var \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/var
ln /mnt/usb-a2/.snapshots/daily.1/localhost/TOP/sed.gs\_prefix \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/sed.gs\_prefix
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
/sed.gs\_prefix \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/sed.gs\_prefix
ln /mnt/usb-a2/.snapshots/daily.1/localhost/TOP/sed.gs\_user\_root\_suffix \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/sed.gs\_user\_root\_suffix
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
/sed.gs\_user\_root\_suffix \
/mnt/usb-a2/.snapshots/daily.0/localhost/TOP/sed.gs\_user\_root\_suffix
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes --relative \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/TOP/GNUstep \
/GNUstep /mnt/usb-a2/.snapshots/daily.0/localhost/TOP/GNUstep
/usr/bin/rsync -aCFHAXx --numeric-ids --delete-excluded \
\--include-from=/usr/local/etc/rsync/backup-includes \
\--exclude-from=/usr/local/etc/rsync/backup-excludes \
\--link-dest=/mnt/usb-a2/.snapshots/daily.1/localhost/BOOT/ /boot/ \
/mnt/usb-a2/.snapshots/daily.0/localhost/BOOT/
touch /mnt/usb-a2/.snapshots/daily.0/
...where, as you see, there is an invocation of rsync for each file under "/". In addition i end up with /bin/bin/, /etc/etc/, /sbin/sbin/, and so on.
I have also tried this variation, which does away with the "+" prefix to "rsync_long_args=", but with the same result:
etc \# cat rsnapshot.conf3
rsync\_long\_args --numeric-ids --delete-excluded
backup / localhost/TOP/ rsync\_long\_args=--numeric-ids --relative --delete-excluded
\#backup /home/ localhost/HOME/
\#backup /usr/ localhost/USR/
\#backup /usr/local/ localhost/USRLOCAL/
\#backup /var/ localhost/VAR/
\#backup /opt/ localhost/OPT/
backup /boot/ localhost/BOOT/
Which gives the same result as the previous (the second) one. Notice also that here i don't get the --{in|ex}clude-from- options, nor the --exclude=$snapshot_root for the first backup line (that of "/").
I don't care much for the stripping of the trailing slash of the source dir; as you must know, it is not trivial to rsync. Why not use the same rule as that of rsync, and simply pass it the source dir as specified ? Where "file" means "take this file, and if it is a directory, then bring all that it contains along"; and "file/" means "we bring the contents of this directory, but not itself".
Besides, one could argue that "/" has a trailing slash, so why treat it differently than "file/" ?
Commented by hashproduct on 2010-03-07 04:45 UTC
This report demonstrates two different problems:
#1. The code that avoids recursively backing up the snapshot root does not check for a --relative option in the per-backup-point rsync_long_args (or +rsync_long_args). If that happens, the code will unnecessarily adopt the legacy strategy of splitting the backup point, which actually puts the destination files in the wrong place because rsnapshot strips trailing slashes with --relative. This could be addressed specifically, but I'm hoping to remove the affected code entirely, so let's see what happens there first:
#2. Specifying a per-backup-point rsync_long_args clobbers the global {in,ex}clude{,_file} options. This is covered by a separate ticket:
https://sourceforge.net/tracker/?func=detail&aid=2942861&group\_id=88546&atid=587076
Commented by hashproduct on 2010-03-07 08:18 UTC
Changing the summary to clearly reflect issue #1.
Updated by hashproduct on 2010-03-07 08:18 UTC
- summary: unexpected behaviour of "(+)rsync_long_args=" --> Exclude-snapshot-root code doesn't check for per-backup-point --relative