Make Fails on Ubuntu 18.04
reefland opened this issue · comments
Based on the success of #90 I figured I'd try to run make.
The tools I installed so far for this:
sudo apt install --yes --no-install-recommends gcc gcc-multilib libzfslinux-dev python2.7 python-setuptools cython
I had to over-ride the default phython 2.7 path such as:
sudo make PYTHON=$(which python2.7) install
This yielded the following output:
/usr/bin/python2.7 setup.py install --prefix /usr/local
running install
running bdist_egg
running egg_info
creating libzfs.egg-info
writing libzfs.egg-info/PKG-INFO
writing top-level names to libzfs.egg-info/top_level.txt
writing dependency_links to libzfs.egg-info/dependency_links.txt
writing manifest file 'libzfs.egg-info/SOURCES.txt'
reading manifest file 'libzfs.egg-info/SOURCES.txt'
writing manifest file 'libzfs.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
cythoning libzfs.pyx to libzfs.c
warning: libzfs.pyx:637:24: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:637:31: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:638:29: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:638:35: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
Error compiling Cython file:
------------------------------------------------------------
...
cdef int block_flag = 0 if blocking else 1
zevent_fd = zfs_dev_fd
with nogil:
ret = libzfs.zpool_events_next(self.handle, &nvl, &dropped, block_flag, zevent_fd)
if ret != 0 or (nvl == NULL and block_flag == 0):
raise self.get_error()
^
------------------------------------------------------------
libzfs.pyx:418:20: Raising exception not allowed without gil
Error compiling Cython file:
------------------------------------------------------------
...
cdef int block_flag = 0 if blocking else 1
zevent_fd = zfs_dev_fd
with nogil:
ret = libzfs.zpool_events_next(self.handle, &nvl, &dropped, block_flag, zevent_fd)
if ret != 0 or (nvl == NULL and block_flag == 0):
raise self.get_error()
^
------------------------------------------------------------
libzfs.pyx:418:40: Calling gil-requiring function not allowed without gil
building 'libzfs' extension
creating build
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-5Z483E/python2.7-2.7.17=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c libzfs.c -o build/temp.linux-x86_64-2.7/libzfs.o -g -O2 -I/usr/include/libzfs/ -I/usr/include/libspl/ -D_MACHINE_ENDIAN_H_ -DHAVE_ISSETUGID -Werror=implicit-function-declaration
libzfs.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^~~~~
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Makefile:11: recipe for target 'install' failed
make: *** [install] Error 1
As to your suggestion to try the DEB package instead. I assume that means I can just grab the latest one, I don't need something specific to the version of ZFS on Linux (0.7.5) I'm running?
We have only tested with python 3. We dont explicitly support python 2 (never tried)
The make file is explicitly looking for 2.7 as it contains PYTHON ?= /usr/local/bin/python2.7 that's the only reason I tried that version. I'll try again with Python 3. Glad to see it doesn't need 2.7.
I yanked python2.7 and installed 3:
sudo apt install --yes --no-install-recommends gcc gcc-multilib libzfslinux-dev python3 python3-setuptools cython3
$ python3 --version
Python 3.6.9
But I get the same error as before:
$ sudo make PYTHON=$(which python3) install
/usr/bin/python3 setup.py install --prefix /usr/local
running install
running bdist_egg
running egg_info
writing libzfs.egg-info/PKG-INFO
writing dependency_links to libzfs.egg-info/dependency_links.txt
writing top-level names to libzfs.egg-info/top_level.txt
reading manifest file 'libzfs.egg-info/SOURCES.txt'
writing manifest file 'libzfs.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
cythoning libzfs.pyx to libzfs.c
warning: libzfs.pyx:637:24: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:637:31: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:638:29: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
warning: libzfs.pyx:638:35: Non-trivial type declarators in shared declaration (e.g. mix of pointers and values). Each pointer declaration should be on its own line.
Error compiling Cython file:
------------------------------------------------------------
...
cdef int block_flag = 0 if blocking else 1
zevent_fd = zfs_dev_fd
with nogil:
ret = libzfs.zpool_events_next(self.handle, &nvl, &dropped, block_flag, zevent_fd)
if ret != 0 or (nvl == NULL and block_flag == 0):
raise self.get_error()
^
------------------------------------------------------------
libzfs.pyx:418:20: Raising exception not allowed without gil
Error compiling Cython file:
------------------------------------------------------------
...
cdef int block_flag = 0 if blocking else 1
zevent_fd = zfs_dev_fd
with nogil:
ret = libzfs.zpool_events_next(self.handle, &nvl, &dropped, block_flag, zevent_fd)
if ret != 0 or (nvl == NULL and block_flag == 0):
raise self.get_error()
^
------------------------------------------------------------
libzfs.pyx:418:40: Calling gil-requiring function not allowed without gil
building 'libzfs' extension
creating build/temp.linux-x86_64-3.6
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c libzfs.c -o build/temp.linux-x86_64-3.6/libzfs.o -g -O2 -I/usr/include/libzfs/ -I/usr/include/libspl/ -D_MACHINE_ENDIAN_H_ -DHAVE_ISSETUGID -Werror=implicit-function-declaration
libzfs.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
^~~~~
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Makefile:11: recipe for target 'install' failed
make: *** [install] Error 1
Based on messages I get from trying to install the DEB packages, I think the ZFS on Ubuntu 18.04.4 LTS is just too old to work with py-libzfs without installing custom PPA's to install newer version. Looks like ZFS 0.7.12 or newer is needed, and I have 0.7.5. I'll hold out for Ubuntu 20.04 LTS and try again.
Unpacking python3-libzfs (0.3-1~truenas+2) ...
dpkg: dependency problems prevent configuration of python3-libzfs:
python3-libzfs depends on libnvpair1linux (>= 0.7.12); however:
Version of libnvpair1linux on system is 0.7.5-1ubuntu16.8.
python3-libzfs depends on libuutil1linux (>= 0.7.12); however:
Version of libuutil1linux on system is 0.7.5-1ubuntu16.8.
python3-libzfs depends on libzfs2linux (>= 0.7.12); however:
Version of libzfs2linux on system is 0.7.5-1ubuntu16.8.
python3-libzfs depends on python3 (>= 3.7~); however:
Version of python3 on system is 3.6.7-1~18.04.
Looks like ZFS 0.7.12 or newer is needed, and I have 0.7.5
You should report a bug to Ubuntu package maintainers. The 0.7.x releases only accept fixes (it's feature-freezed), so having 0.7.5 means there's a pile of fixes missing that were in 0.7.6, 0.7.7, etc.
@Hi-Angel - Canonical is well aware. They use a kernel module, not DKMS. A given version is matched to that release's kernel level. They backport some fixes but not new features. Essentially upgrading to new Ubuntu release provides an updated kernel and updated zfs version. My situation has restrictions on which versions I can use (LTS only) and building my own from source, or using 3rd party ZFS packages is not allowed. Now that newer LTS versions are available, looking at this again.
Used the py-libzfs DEB packages provided here, and installed on Ubuntu 20.04.1, installation went fine and seems to be working. I'll close the issue.
$ python
Python 3.8.2 (default, Jul 16 2020, 14:00:26)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libzfs
>>> list(libzfs.ZFS().pools)
[<libzfs.ZFSPool name 'bpool' guid '11274381276152110139'>, <libzfs.ZFSPool name 'rpool' guid '16659136442229807815'>]
Sigh, after all this... FreeNAS 11.3 ZFS replication to Ubuntu with NETCAT as the transport still doesn't work. This method is what required py-libzfs in the first place (SSH as transport works fine)
@reefland any news? I try the same, replicating to an Ubuntu 20.04, and I hope NETCAT would be able to improve speeds, as I am using a Raspberry Pi 4 with Ubuntu 20.04 64bit
EDIT: so far so good, building it from scratch, with the prefix-fix and the hardcoded python-fix (those scripts really need some robustness improvement) now the replication got another error:
umount: only root can use "--types" option I guess this is because I am using a non root, and the scripts inside truenas don't use sudo?? --> going for root as the ssh user :-/
EDIT2: Holy $hit, it worked 😄 never seen spees, exceeding 80Mb/s 💪
@blackliner - nice, good to see you got it working. I did have success with TrueNAS 12.x, but never got FreeNAS 11.x working. Ubuntu 20.04 has the python3-libzfs package, did not need to build from scratch. I now do regular push replication from TrueNAS 12 to Ubuntu 20.04 using SSH+NETCAT transport.
Using SSH or SSH+NETCAT were both able to saturate my 1Gb Ethernet ~ 110MB/sec. I did see much lower CPU with NETCAT so still worth it. I think when I bump to 2.5 Gb or 10 Gb Ethernet I'll show a clear difference where SSH alone can't saturate the NIC anymore.
as I am on a raspi for the replication target, I got a 100% speed increase with using SSH+NETCAT mode (40Mbps --> 80Mbps) + reduced CPU util. Unfortunately the 20.04 raspi distro does not seem to have the python3-libzfs package :-(
Ubuntu 20.04 has the python3-libzfs package, did not need to build from scratch. I now do regular push replication from TrueNAS 12 to Ubuntu 20.04 using SSH+NETCAT transport.
Did you have to do anything fancy to make python3-libzfs work in Ubuntu? I've installed the package via apt in 21.04 (it's called python3-pyzfs there), but when the TrueNAS side runs the replication it throws a ModuleNotFoundError: No module named 'libzfs' and fails.
I've also manually confirmed that the py-libzfs module can't be imported on the Pi running Ubuntu:
pi@raspberry[~]$ python3
Python 3.9.5 (default, May 11 2021, 08:20:37)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libzfs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'libzfs'
>>>
pyzfs and py-libzfs are different things.