Build fails because of missing rsync
nsajko opened this issue · comments
The README.md should mention that rsync is necessary.
It seems that rsync is called by make to install Linux headers.
I'm not aware of that dependency and it's certainly not intended. Can you tell where it's coming from? I'll try to find and patch it out.
OK, I found it in the top-level makefile. I'll see what can be done to fix it.
@richfelker, this is how LFS handle this -- https://www.linuxfromscratch.org/lfs/view/systemd/chapter05/linux-headers.html:
Now extract the user-visible kernel headers from the source. The recommended make target “headers_install” cannot be used, because it requires rsync, which may not be available. The headers are first placed in ./usr, then copied to the needed location.
make headers
find usr/include -name '.*' -delete
rm usr/include/Makefile
cp -rv usr/include $LFS/usr
so, for my builds I fixed it as follows:
$ fgrep 'obj_kernel_headers/.lc_built:' -A 4 litecross/Makefile
obj_kernel_headers/.lc_built: | src_kernel_headers
mkdir -p $(CURDIR)/obj_kernel_headers/staged
cd src_kernel_headers && $(MAKE) ARCH=$(LINUX_ARCH) O=$(CURDIR)/obj_kernel_headers INSTALL_HDR_PATH=$(CURDIR)/obj_kernel_headers/staged headers
cp -r obj_kernel_headers/usr/include obj_kernel_headers/staged/
find obj_kernel_headers/staged/include -name '.*' -exec rm {} +
Here is a diff (I am not currently able to do a proper PR, unfortunately):
diff --git a/litecross/Makefile b/litecross/Makefile
index 9cd691b..f06b037 100644
--- a/litecross/Makefile
+++ b/litecross/Makefile
@@ -269,8 +269,9 @@ src_kernel_headers: | $(LINUX_SRCDIR)
ln -sf "$(LINUX_SRCDIR)" $@
obj_kernel_headers/.lc_built: | src_kernel_headers
mkdir -p $(CURDIR)/obj_kernel_headers/staged
- cd src_kernel_headers && $(MAKE) ARCH=$(LINUX_ARCH) O=$(CURDIR)/obj_kernel_headers INSTALL_HDR_PATH=$(CURDIR)/obj_kernel_headers/staged headers_install
- find obj_kernel_headers/staged/include '(' -name .install -o -name ..install.cmd ')' -exec rm {} +
+ cd src_kernel_headers && $(MAKE) ARCH=$(LINUX_ARCH) O=$(CURDIR)/obj_kernel_headers INSTALL_HDR_PATH=$(CURDIR)/obj_kernel_headers/staged headers
+ cp -r obj_kernel_headers/usr/include obj_kernel_headers/staged/
+ find obj_kernel_headers/staged/include -name '.*' -exec rm {} +
touch $@
install-kernel-headers: | obj_kernel_headers/.lc_built
mkdir -p $(DESTDIR)$(OUTPUT)$(SYSROOT)/include
In order to consider this it's going to need to be a solution that does not involve invoking rm via find. That is just way too great a risk of destroying somebody's filesystem. Can we instead do a find command that copies the wanted files (instead of deleting the unwanted ones) to the destination? Or something else safe?
Well, -exec rm {} + was already there, I just adjusted it to delete all .* files. I would prefer find ... -delete, but I did not want to deviate from the current code too much.
Also, find obj_kernel_headers/staged/include -name '.*' -delete should be pretty safe, I think, since it only operates within the specified directory and below (and we just populated that directory). I am not sure that it would be easier/safer to come up with a solution that will determine what's needed to be copied and what's not. This is why kernel folks used rsync: to exclude anything that is not *.h. Anyway, my solution is just one of the options. I stumbled upon this issue, located a ticket from an year ago, and decided to share how I fixed it on my system. Perhaps, some people who stumble upon this issue will find it useful. :)
Oh, I missed that we already have it there and you just changed it. Uhg. I don't like having that at all, but at least the old version was just fixed filenames. Is there a reason for .*?
Maybe something like
find obj_kernel_headers/usr/include -name '*.h' -exec sh -c 'for i ; do f=${i#obj_kernel_headers/usr/} && mkdir -p obj_kernel_headers/staged/${f%/*} && cp "$i" obj_kernel_headers/staged/"$f" ; done' {} +
Yeah, this could work too. It will be many times slower and more prone to errors, but should work. I just ran it in the tree to see how it compares with the "copy & delete" version (I preserved the result of "copy & delete" in staged.with_delete):
$ find obj_kernel_headers/usr/include -name '*.h' -exec sh -c 'for i ; do f=${i#obj_kernel_headers/usr/} && mkdir -p obj_kernel_headers/staged/${f%/*} && cp "$i" obj_kernel_headers/staged/"$f" ; done' {} +
[galaxy@archlinux x86_64-linux-musl]$ cd obj_kernel_headers/
[galaxy@archlinux obj_kernel_headers]$ diff -puNr staged.with_delete staged | diffstat
stat.h | 138 ------------------------------------------------------------------------------------------------------------------------------------------
1 file changed, 138 deletions(-)
[galaxy@archlinux obj_kernel_headers]$ diff -puNr staged.with_delete staged | head -3
diff -puNr staged.with_delete/include/asm/stat.h staged/include/asm/stat.h
--- staged.with_delete/include/asm/stat.h 2021-09-18 02:19:27.226056858 +1000
+++ staged/include/asm/stat.h 1970-01-01 10:00:00.000000000 +1000
[galaxy@archlinux obj_kernel_headers]$ ls -ld usr/include/asm/stat.h staged/include/asm/stat.h
ls: cannot access 'staged/include/asm/stat.h': No such file or directory
-rw-r--r-- 1 galaxy galaxy 3131 Sep 18 02:19 usr/include/asm/stat.h
So, it almost works: it missed the include/asm/stat.h for some reason. I still think that "copy & delete" is a better option in this particular case. The problem with the complicated shell executed for each discovered file is that it is slow, has approximately the same security implications, and is not simple.
It is your project, @richfelker , so you decide whether you want to address the isssue at all (all in all, this can be closed by just commenting that rsync is required if one wants to use the kernel package instead of the kernel-headers one), solve it your way, or my way. For my projects, I will stick with the "copy & delete" approach with just one enhancement: will use cp -rl instead of just cp -r.
P.S To answer your question re: .*. It is the same reason as it was in your original clean up of ..install.cmd -- for each file in the source directory there is a hidden ".<source_name>.cmd" file. When we are not engaging "headers_install" we need to do the work of ensuring these *.cmd files do not end up in the target directory.