0.17.16: test suite is failing in `umockdev:fails-valgrind / umockdev.py` unit

Question

0.17.16: test suite is failing in `umockdev:fails-valgrind / umockdev.py` unit

kloczek opened this issue 2 years ago · comments

+ cd umockdev-0.17.16
+ /usr/bin/meson test -C x86_64-redhat-linux-gnu --num-processes 48 --print-errorlogs
ninja: no work to do.
ninja: Entering directory `/home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/x86_64-redhat-linux-gnu'
ninja: no work to do.
1/7 umockdev / ioctl-tree                     OK               0.03s
2/7 umockdev / static-code                    OK               0.11s
3/7 umockdev:fails-valgrind / umockdev.py     FAIL             0.11s   exit status 1
>>> TOP_SRCDIR=/home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16 G_DEBUG=fatal-warnings,fatal-criticals,gc-friendly LD_LIBRARY_PATH=/home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/x86_64-redhat-linux-gnu/:/home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/x86_64-redhat-linux-gnu MALLOC_CHECK_=3 PATH=/home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/x86_64-redhat-linux-gnu:/usr/bin:/usr/sbin:/usr/local/sbin MALLOC_PERTURB_=61 GI_TYPELIB_PATH=/home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/x86_64-redhat-linux-gnu /home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/src/umockdev-wrapper /usr/bin/python3 -Wd -Werror::PendingDeprecationWarning -Werror::DeprecationWarning /home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/tests/test-umockdev.py
――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――
stderr:
Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/umockdev-0.17.16/tests/test-umockdev.py", line 29, in <module>
    gi.require_version('GUdev', '1.0')
AttributeError: module 'gi' has no attribute 'require_version'
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

4/7 umockdev:fails-valgrind / umockdev-vala   OK               1.05s
5/7 umockdev / umockdev-run                   OK               1.79s
6/7 umockdev:fails-valgrind / umockdev-record OK               2.04s
7/7 umockdev / umockdev                       OK               3.94s

Summary of Failures:

3/7 umockdev:fails-valgrind / umockdev.py     FAIL             0.11s   exit status 1

Ok:                 6
Expected Fail:      0
Fail:               1
Unexpected Pass:    0
Skipped:            0
Timeout:            0

I'm using gobject-introspection 1.75.4.

Eli Schwartz · Answer 1 · Mon Jan 30 2023 05:52:51 GMT+0800 (China Standard Time)

You do not have pygobject (the gi module) installed as a test dependency.

Tomasz Kłoczko · Answer 2 · Mon Jan 30 2023 06:47:21 GMT+0800 (China Standard Time)

No I didn't. Just added pkgconfig pygobject-3.0 dependencoes to build env and loos like it solved the issue.
I think that it would be good to add dependency('pygobject-3.0') to main meson.build.

Tomasz Kłoczko · Answer 3 · Mon Jan 30 2023 06:59:52 GMT+0800 (China Standard Time)

I think that it would be good to add dependency('pygobject-3.0') to main meson.build.

Actually no .. dependency('pygobject-3.0') it installs pygobject header files and in this case it is missing python module so probably better would be add something like import('python').find_installation('python', modules: ['pygobject'])).

Tomasz Kłoczko · Answer 4 · Mon Jan 30 2023 07:00:48 GMT+0800 (China Standard Time)

Yep .. just tested and add to build dependencies pygobject modules fixes the issue.

Eli Schwartz · Answer 5 · Mon Jan 30 2023 07:01:10 GMT+0800 (China Standard Time)

But it doesn't want the pkg-config file, which is the development component useful for compiling against the pygobject C header.

It just wants to be able to perform a runtime import (import gi), so you need the python3-gobject module, not the python3-gobject-devel development package.

Tomasz Kłoczko · Answer 6 · Mon Jan 30 2023 10:11:36 GMT+0800 (China Standard Time)

But it doesn't want the pkg-config file, which is the development component useful for compiling against the pygobject C header.

It just wants to be able to perform a runtime import (import gi), so you need the python3-gobject module, not the python3-gobject-devel development package.

Yep and that is why I've added that better would be add checking for pygobject python module (which provides gi drecrory).

Martin Pitt · Answer 7 · Mon Feb 13 2023 15:13:56 GMT+0800 (China Standard Time)

I'm not sure what to do with this.. This sounds like a downstream packaging issue? It seems solved now? Or is there anything that needs to be done upstream? Thanks!

Tomasz Kłoczko · Answer 8 · Mon Feb 13 2023 18:15:11 GMT+0800 (China Standard Time)

I'm not sure what to do with this.. This sounds like a downstream packaging issue? It seems solved now? Or is there anything that needs to be done upstream? Thanks!

As I wrote looks like in meson it is missing cecking dependecy (depends() call) for pygobject .

Martin Pitt · Answer 9 · Mon Feb 13 2023 20:08:47 GMT+0800 (China Standard Time)

Tricky.. one really doesn't need the pygobject .pc file, e.g. the Fedora and Debian packaging don't have this either. That kind of runtime dependency is not what dependency() is good at.

Eli Schwartz · Answer 10 · Mon Feb 13 2023 20:21:13 GMT+0800 (China Standard Time)

As I pointed out above, checking for a dependency() would unreasonably add an additional distro package to Debian/Fedora build dependencies.

Tomasz Kłoczko · Answer 11 · Mon Feb 13 2023 21:01:24 GMT+0800 (China Standard Time)

OK .. mesom provides python modules dependencies checking.
Something like below should probably do what is needed here 🤔

py_deps = import('python').find_installation('python3', modules: ['pygobject'])

Eli Schwartz · Answer 12 · Mon Feb 13 2023 21:08:07 GMT+0800 (China Standard Time)

That's the distro package name, not the python module name.

And it's fairly common for gi to have overrides that cause import gi to work, but not provide anything. This is why packages need to explicitly depend on python3-gobject if they use it, in addition to whichever packages might happen to provide gi overrides.

So just checking for gi won't help anyway.

And either way it's still a packaging issue.

Benjamin Berg · Answer 13 · Mon Feb 13 2023 21:47:29 GMT+0800 (China Standard Time)

In some cases I added a

try:
    import ....
except Exception as e:
    print("Missing dependencies: %s" % str(e))
    sys.exit(77)

block. Then the test just skips, and things are "fine" in some way. Obviously, you should still pull in the runtime dependency. But it makes it clear that this is a problem for you to solve and you can ignore it if you chose to do so.

That said, one does not want to accept a skipped test like this in CI for example.

Tomasz Kłoczko · Answer 14 · Mon Feb 13 2023 22:51:53 GMT+0800 (China Standard Time)

That's the distro package name, not the python module name.

pip shows

$ pip show PyGObject
Name: PyGObject
Version: 3.43.1.dev0
Summary: Python bindings for GObject Introspection
Home-page: https://pygobject.readthedocs.io
Author: James Henstridge
Author-email: james@daa.com.au
License: GNU LGPL
Location: /usr/lib64/python3.8/site-packages
Requires:
Required-by:

so pygobject as checked module name should be OK.

Tomasz Kłoczko · Answer 15 · Mon Feb 13 2023 22:52:43 GMT+0800 (China Standard Time)

yep ..

$ pip show pygobject
Name: PyGObject
Version: 3.43.1.dev0
Summary: Python bindings for GObject Introspection
Home-page: https://pygobject.readthedocs.io
Author: James Henstridge
Author-email: james@daa.com.au
License: GNU LGPL
Location: /usr/lib64/python3.8/site-packages
Requires:
Required-by:

Eli Schwartz · Answer 16 · Mon Feb 13 2023 23:06:50 GMT+0800 (China Standard Time)

Pip shows the pip package name, not the python module name.

The pip package name is usually the same as the distro package name (or at least very similar, modulo prefixes indicating the relation to python).

The python ecosystem prefers to call the thing you're viewing a "distribution" name, though. ;)

...

The point is, that meson only provides functionality for verifying that a module is importable. It will test modules: ['pygobject'] by attempting to run import pygobject. And failing.

Please. Accept that I know what I'm talking about here. I am upstream for meson, I know how meson works.

Tomasz Kłoczko · Answer 17 · Mon Feb 13 2023 23:23:52 GMT+0800 (China Standard Time)

Pip shows the pip package name, not the python module name.

OK so .. py_deps = import('python').find_installation('python3', modules: ['gi'])

Eli Schwartz · Answer 18 · Tue Feb 14 2023 00:24:14 GMT+0800 (China Standard Time)

And it's fairly common for gi to have overrides that cause import gi to work, but not provide anything. This is why packages need to explicitly depend on python3-gobject if they use it, in addition to whichever packages might happen to provide gi overrides.

So just checking for gi won't help anyway.

Martin Pitt · Answer 19 · Sun Feb 26 2023 17:51:34 GMT+0800 (China Standard Time)

At the moment, meson.build checks for python and g-ir-compiler, and if both are found, enables the test:

if python.found()    
if g_ir_compiler.found()    
  test('umockdev.py', python,    
    args: ['-Wd', '-Werror::PendingDeprecationWarning', '-Werror::DeprecationWarning',    
           testsdir / 'test-umockdev.py'],    
    depends: [preload_lib, umockdev_typelib],    
    suite: 'fails-valgrind')    
endif    
endif

I suppose this could grow a test for the python gi module; this is enough, as Fedora's python3-gobject-base provides nothing aside the gi module, so I'm not entirely sure what @eli-schwartz really means with "So just checking for gi won't help anyway".

@kloczek perhaps you can send a PR with what you would like to see here? Thanks!

Eli Schwartz · Answer 20 · Sun Feb 26 2023 20:50:41 GMT+0800 (China Standard Time)

I already explained it, but maybe this will do a better job:

It's possible to install: gi.overrides.FooBar as a python module. If you do this, python will think gi is installed, but it might not be.

One way to work around this is to make four packages instead of two, so that FooBar is one package, gi.overrides.FooBar is a second package, gi is a third package, and python3-gobject is a fourth package.

This works as long as you step carefully, i.e. checking for gi is semantically incorrect even if it sometimes works by accident.

The correct way to check is, of course, to actually run some test code from the gi module to verify that it's in good working order. The modules: kwarg does not work for gi, and never will.

Martin Pitt · Answer 21 · Sun Apr 16 2023 15:19:20 GMT+0800 (China Standard Time)

FTR: I'm not going to do anything about this myself. I don't understand what you mean/want in the above comments. I'll be happy to review a PR of course. Thanks!