3.12 is failing to build on indigo ubuntu 64bit
tfoote opened this issue · comments
http://build.ros.org/view/Kbin_uX64/job/Kbin_uX64__parrot_arsdk__ubuntu_xenial_amd64__binary/35/
dpkg-shlibdeps -Tdebian/ros-kinetic-parrot-arsdk.substvars -l/tmp/binarydeb/ros-kinetic-parrot-arsdk-3.12.6/debian/ros-kinetic-parrot-arsdk//opt/ros/kinetic/lib/ debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libjson-c.so.2.0.2 debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libmux.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetwork.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardatatransfer.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardiscovery.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmavlink.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarsal.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libcurl.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libpomp.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmedia.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcommands.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarutils.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream2.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetworkal.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/x86_64-linux-gnu/libstdc\+\+.so.6.0.21
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libardiscovery.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarcommands.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarstream.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarstream2.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarnetworkal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarnetwork.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libmux.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: error: couldn't find library libjson-c.so.2 needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libardiscovery.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarutils.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libardatatransfer.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libmux.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libpomp.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: error: couldn't find library libjson-c.so.2 needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardatatransfer.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libardiscovery.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardatatransfer.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarutils.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardatatransfer.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcommands.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetwork.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarnetworkal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetwork.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libpomp.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libmux.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarutils.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libardiscovery.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarutils.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libmux.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarutils.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: /usr/lib/x86_64-linux-gnu/libcurl.so has an unexpected SONAME (libcurl.so.4)
dpkg-shlibdeps: warning: can't extract name and version from library name 'libcurl.so'
dpkg-shlibdeps: warning: can't extract name and version from library name 'libcurl.so'
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmedia.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libardiscovery.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmedia.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: error: couldn't find library libjson-c.so.2 needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmedia.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream2.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libmux.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream2.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libpomp.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream2.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarmedia.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream2.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetworkal.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libmux.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetworkal.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libpomp.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetworkal.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardiscovery.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarnetworkal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardiscovery.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libmux.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardiscovery.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libpomp.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardiscovery.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: error: couldn't find library libjson-c.so.2 needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardiscovery.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmavlink.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarsal.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: couldn't find library libarnetwork.so needed by debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream.so (ELF format: 'elf64-x86-64'; RPATH: '')
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarutils.so was not linked against libcurl.so (it uses none of the library's symbols)
dpkg-shlibdeps: error: cannot continue due to the errors listed above
Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file.
To help dpkg-shlibdeps find private libraries, you might need to use -l.
dh_shlibdeps: dpkg-shlibdeps -Tdebian/ros-kinetic-parrot-arsdk.substvars -l/tmp/binarydeb/ros-kinetic-parrot-arsdk-3.12.6/debian/ros-kinetic-parrot-arsdk//opt/ros/kinetic/lib/ debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libjson-c.so.2.0.2 debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libmux.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetwork.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardatatransfer.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libardiscovery.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmavlink.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarupdater.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarsal.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libcurl.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libpomp.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarmedia.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcommands.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarcontroller.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarutils.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarstream2.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/libarnetworkal.so debian/ros-kinetic-parrot-arsdk/opt/ros/kinetic/lib/parrot_arsdk/x86_64-linux-gnu/libstdc++.so.6.0.21 returned exit code 2
debian/rules:51: recipe for target 'override_dh_shlibdeps' failed
make[1]: *** [override_dh_shlibdeps] Error 2
make[1]: Leaving directory '/tmp/binarydeb/ros-kinetic-parrot-arsdk-3.12.6'
make: *** [binary] Error 2
debian/rules:23: recipe for target 'binary' failed
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2
E: Building failed
Traceback (most recent call last):
File "/tmp/ros_buildfarm/ros_buildfarm/binarydeb_job.py", line 133, in build_binarydeb
subprocess.check_call(cmd, cwd=source_dir)
File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['apt-src', 'build', 'ros-kinetic-parrot-arsdk']' returned non-zero exit status 1
Thanks @tfoote.
I've been trying to chase this issue down for a while. The pre-release tests all pass since the problem is at debian packaging level. I tried re-producing the problem locally using the instructions here [1, 2], but could not run the script successfully. I will do another pass later today.
I would really appreciate it if you share any insight about why dh_shlibdeps may fail for only a specific target platform (Xenial/64bit)?
[1] https://github.com/ros-infrastructure/ros_buildfarm/blob/master/doc/jobs/release_jobs.rst#run-the-release-job-locally
[2] https://github.com/ros-infrastructure/ros_buildfarm/blob/master/doc/environment.rst
It looks like potentially some dependencies are missing? The error about not finding libjson-c is likely the culprit.
For setting up your environment. You can now use the debian package python3-ros-buildfarm and go straight to the test. You will also need docker without sudo setup.
I was able to reproduce using this:
2000 sudo apt-get install python3-ros-buildfarm
2001 mkdir /tmp/release_job
2002 generate_release_script.py https://raw.githubusercontent.com/ros-infrastructure/ros_buildfarm_config/production/index.yaml kinetic default parrot_arsdk ubuntu xenial amd64 > /tmp/release_job/test.sh
2003 less test.sh
2004 ls /tmp/release_job/test.sh
2005 cd /tmp/release_job/
2006 cat test.sh
2007 sh test.sh
Thanks @tfoote for your instructions.
Unfortunately, I still could not run the release_job locally. The progress would advance, but I would receive ccache permission errors (during the cmake configuration step) within the container, similar to ros-infrastructure/buildfarm_deployment#113 . (target folder: /home/buildfarm/.ccache/tmp).
Nevertheless, I think I found where the problem is coming from.
parrot_arsdk(from Parrot), comes with its own build and workspace management system which itself is fairly complex.- As part of its default build process, it builds
libjsonandlibcurlfrom source. Those are used by many of its higher level libraries. - When we wrapped this process with
catkin, I deliberatively made the install step to copy all the libraries to a folder within the install location (e.g./opt/ros/kinetic/lib/parrot_arsdk/libar*.soinstead of/opt/ros/kinetic/lib/libar*.so). This path is not added toLD_LIBRARY_PATH. Any downstream dependency (such asbebop_autonomy/bebop_driver), manually set theirlink_directoriesto a variable exported by this package during theircmakeconfiguration step. The rational is to prevent custom version oflibjsonorlibcurlto be installed into a shared library install location. - Neither the
dpkg-shlibdepsstep of the build script, norldconfigknow about this folder when a release_job is being executed. As you can see,dpkg-shlibdepsis throwing a lot of warnings because it can not find any of the custom libraries thatlibar*libraries depend on. - According to man:dpkg-shlibdeps, it (4) is ok for private library dependencies (
*.so), but not for public library dependencies (*.so.x.y.z). The former causes a warning, while the latter results in an error. - On some of the target platforms (such as Xenial x64), the SDK generated libraries are linked against
libjson-c.so.2, which because of (4) can not be found bydpkg-shlibdeps. Since it looks like a public library dependency (5),dpkg-shlibdepsfails the build.
It might be difficult to trace down why this dependency happens on a certain platform only.
@tfoote In my opinion the easiest fix for this problem is to let the install step install everything (including libjson, libcurl and liblibressl) to /opt/ros/*/lib. Is this going to cause any troubles for any other package?
Another option is to provide an additional -l argument to dpkg-shlibdeps. Is there any way that we can control these options at the package level?
Unfortunately installing those custom versions of libcurl and libjson will break any other user of those libraries which is expecting the system version.
If you want to install these things they should have all their symbols renamed as well as the libraries so as not to collide with the system libraries.
If you can find a way to pass the extra libraries to dpkg-shlibdeps that probably would be the best approach. The other thing to test would be if they really need the custom forks of those libraries or if they could be overridden to use the system versions. Often these things are integrated into the build when you want a newer version. And at some point the newer version becomes available generally but it's already in the build tree and no one wants to remove it.
Thanks so much @tfoote for your suggestions.
Unfortunately installing those custom versions of libcurl and libjson will break any other user of those libraries which is expecting the system version.If you want to install these things they should have all their symbols renamed as well as the libraries so as not to collide with the system libraries.
I think that renaming these files and/or renaming their symbols will be a tedious task. Therefore, installing them to a common folder is not an option at all.
If you can find a way to pass the extra libraries to dpkg-shlibdeps that probably would be the best approach.
I agree. I figured that bloom adds these flags to the debian/rules file. I don't think the extra effort to change the logic there to make it more flexible worth it.
The other thing to test would be if they really need the custom forks of those libraries or if they could be overridden to use the system versions. Often these things are integrated into the build when you want a newer version. And at some point the newer version becomes available generally but it's already in the build tree and no one wants to remove it.
I agree. We will look into this option first.
PR #7 adds a patch that aims to temporarily fix this issue.
Similar issues encountered in release 3.14.0. Fixed in release 3.14.1.