pydoit / doit

task management & automation tool

Home Page:http://pydoit.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

0.36.0: strace test is failing

kloczek opened this issue · comments

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

  • python3 -sBm build -w --no-isolation
  • because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
  • install .whl file in </install/prefix>
  • run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-doit-0.36.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-doit-0.36.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/doit-0.36.0
plugins: cov-3.0.0, hypothesis-6.41.0, mock-3.7.0
collected 794 items

tests/test___init__.py .                                                                                                                                             [  0%]
tests/test___main__.py F                                                                                                                                             [  0%]
tests/test_action.py ....................................................................................................                                            [ 12%]
tests/test_api.py ....                                                                                                                                               [ 13%]
tests/test_cmd_base.py ............................                                                                                                                  [ 16%]
tests/test_cmd_clean.py ............                                                                                                                                 [ 18%]
tests/test_cmd_completion.py ........                                                                                                                                [ 19%]
tests/test_cmd_dumpdb.py .                                                                                                                                           [ 19%]
tests/test_cmd_forget.py ........                                                                                                                                    [ 20%]
tests/test_cmd_help.py ........                                                                                                                                      [ 21%]
tests/test_cmd_ignore.py .....                                                                                                                                       [ 22%]
tests/test_cmd_info.py ......                                                                                                                                        [ 22%]
tests/test_cmd_list.py .................                                                                                                                             [ 25%]
tests/test_cmd_resetdep.py .........                                                                                                                                 [ 26%]
tests/test_cmd_run.py ..............                                                                                                                                 [ 27%]
tests/test_cmd_strace.py F..F..                                                                                                                                      [ 28%]
tests/test_cmdparse.py ........................................                                                                                                      [ 33%]
tests/test_control.py ............................................................                                                                                   [ 41%]
tests/test_dependency.py .............s.sss......................................................................................................................... [ 58%]
..................                                                                                                                                                   [ 61%]
tests/test_doit_cmd.py ....................                                                                                                                          [ 63%]
tests/test_exceptions.py .........                                                                                                                                   [ 64%]
tests/test_loader.py .......................................................                                                                                         [ 71%]
tests/test_plugin.py .........                                                                                                                                       [ 72%]
tests/test_reporter.py ................................                                                                                                              [ 76%]
tests/test_runner.py ............................................................................                                                                    [ 86%]
tests/test_task.py ................................................................................                                                                  [ 96%]
tests/test_tools.py ............................                                                                                                                     [100%]

================================================================================= FAILURES =================================================================================
_______________________________________________________________________________ test_execute _______________________________________________________________________________

depfile_name = '/tmp/pytest-of-tkloczko/pytest-41/x0/testdb'

    def test_execute(depfile_name):
>       assert 0 == subprocess.call([executable, '-m', 'doit', 'list',
                                     '--db-file', depfile_name])
E       AssertionError: assert 0 == 3
E        +  where 3 = <function call at 0x7f58a8ce5670>(['/usr/bin/python3', '-m', 'doit', 'list', '--db-file', '/tmp/pytest-of-tkloczko/pytest-41/x0/testdb'])
E        +    where <function call at 0x7f58a8ce5670> = subprocess.call

tests/test___main__.py:6: AssertionError
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/doit/doit_cmd.py", line 294, in run
    return command.parse_execute(args)
  File "/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/doit/cmd_base.py", line 150, in parse_execute
    return self.execute(params, args)
  File "/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/doit/cmd_base.py", line 524, in execute
    self.loader.setup(params)
  File "/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/doit/cmd_base.py", line 394, in setup
    self.namespace = dict(inspect.getmembers(loader.get_module(
  File "/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/doit/loader.py", line 96, in get_module
    return importlib.import_module(os.path.splitext(file_name)[0])
  File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/dodo.py", line 7, in <module>
    from doitpy.pyflakes import Pyflakes
ModuleNotFoundError: No module named 'doitpy'
__________________________________________________________________________ TestCmdStrace.test_dep __________________________________________________________________________

self = <tests.test_cmd_strace.TestCmdStrace object at 0x7f58a764c5e0>, dependency1 = '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1'
depfile_name = '/tmp/pytest-of-tkloczko/pytest-41/x60/testdb'

    def test_dep(self, dependency1, depfile_name):
        output = StringIO()
        task = Task("tt", ["cat %(dependencies)s"],
                    file_dep=['tests/data/dependency1'])
        cmd = CmdFactory(Strace, outstream=output)
        cmd.loader = self.loader_for_task(task)
        params = DefaultUpdate(dep_file=depfile_name, show_all=False,
                               keep_trace=False, backend='dbm',
                               check_file_uptodate='md5', codec_cls=JSONCodec)
        result = cmd.execute(params, ['tt'])
        assert 0 == result
        got = output.getvalue().split("\n")
        dep_path = os.path.abspath("tests/data/dependency1")
>       assert "R %s" % dep_path in got[0]
E       AssertionError: assert ('R %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R /home/tkloczko/rpmbuild/BUILD/doit-0.36.0'

tests/test_cmd_strace.py:43: AssertionError
-------------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------------
strace -- version 5.17 strace-devel@lists.strace.io
Copyright (c) 1991-2022 The strace developers <>.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Optional features enabled: stack-trace=libdw stack-demangle m32-mpers mx32-mpers
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
.  tt
.  strace_report
________________________________________________________________________ TestCmdStrace.test_target _________________________________________________________________________

self = <tests.test_cmd_strace.TestCmdStrace object at 0x7f58a764cd60>, dependency1 = '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1'
depfile_name = '/tmp/pytest-of-tkloczko/pytest-41/x63/testdb'

    def test_target(self, dependency1, depfile_name):
        output = StringIO()
        task = Task("tt", ["touch %(targets)s"],
                    targets=['tests/data/dependency1'])
        cmd = CmdFactory(Strace, outstream=output)
        cmd.loader = self.loader_for_task(task)
        params = DefaultUpdate(dep_file=depfile_name, show_all=False,
                               keep_trace=False, backend='dbm',
                               check_file_uptodate='md5', codec_cls=JSONCodec)
        result = cmd.execute(params, ['tt'])
        assert 0 == result
        got = output.getvalue().split("\n")
        tgt_path = os.path.abspath("tests/data/dependency1")
>       assert "W %s" % tgt_path in got[0]
E       AssertionError: assert ('W %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R /home/tkloczko/rpmbuild/BUILD/doit-0.36.0'

tests/test_cmd_strace.py:90: AssertionError
-------------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------------
strace -- version 5.17 strace-devel@lists.strace.io
Copyright (c) 1991-2022 The strace developers <>.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Optional features enabled: stack-trace=libdw stack-demangle m32-mpers mx32-mpers
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
.  tt
.  strace_report
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_dependency.py:98: dumbdbm too dumb to detect db corruption
SKIPPED [2] tests/test_dependency.py:111: test doesnt apply to non DBM DB
SKIPPED [1] tests/test_dependency.py:113: dumbdbm too dumb to detect db corruption
FAILED tests/test___main__.py::test_execute - AssertionError: assert 0 == 3
FAILED tests/test_cmd_strace.py::TestCmdStrace::test_dep - AssertionError: assert ('R %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R /ho...
FAILED tests/test_cmd_strace.py::TestCmdStrace::test_target - AssertionError: assert ('W %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R ...
================================================================ 3 failed, 787 passed, 4 skipped in 10.12s =================================================================

Please make sure you install the dependencies from dev_requirements.txt.

Can you please tell me which version of strace is installed. And provide strace output file from the failing test.

Please make sure you install the dependencies from dev_requirements.txt.

After add doit-py to build env

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-doit-0.36.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-doit-0.36.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/doit-0.36.0
collected 794 items

tests/test___init__.py .                                                                                                                                             [  0%]
tests/test___main__.py .                                                                                                                                             [  0%]
tests/test_action.py ....................................................................................................                                            [ 12%]
tests/test_api.py ....                                                                                                                                               [ 13%]
tests/test_cmd_base.py ............................                                                                                                                  [ 16%]
tests/test_cmd_clean.py ............                                                                                                                                 [ 18%]
tests/test_cmd_completion.py ........                                                                                                                                [ 19%]
tests/test_cmd_dumpdb.py .                                                                                                                                           [ 19%]
tests/test_cmd_forget.py ........                                                                                                                                    [ 20%]
tests/test_cmd_help.py ........                                                                                                                                      [ 21%]
tests/test_cmd_ignore.py .....                                                                                                                                       [ 22%]
tests/test_cmd_info.py ......                                                                                                                                        [ 22%]
tests/test_cmd_list.py .................                                                                                                                             [ 25%]
tests/test_cmd_resetdep.py .........                                                                                                                                 [ 26%]
tests/test_cmd_run.py ..............                                                                                                                                 [ 27%]
tests/test_cmd_strace.py F..F..                                                                                                                                      [ 28%]
tests/test_cmdparse.py ........................................                                                                                                      [ 33%]
tests/test_control.py ............................................................                                                                                   [ 41%]
tests/test_dependency.py .............s.sss......................................................................................................................... [ 58%]
..................                                                                                                                                                   [ 61%]
tests/test_doit_cmd.py ....................                                                                                                                          [ 63%]
tests/test_exceptions.py .........                                                                                                                                   [ 64%]
tests/test_loader.py .......................................................                                                                                         [ 71%]
tests/test_plugin.py .........                                                                                                                                       [ 72%]
tests/test_reporter.py ................................                                                                                                              [ 76%]
tests/test_runner.py ............................................................................                                                                    [ 86%]
tests/test_task.py ................................................................................                                                                  [ 96%]
tests/test_tools.py ............................                                                                                                                     [100%]

================================================================================= FAILURES =================================================================================
__________________________________________________________________________ TestCmdStrace.test_dep __________________________________________________________________________

self = <tests.test_cmd_strace.TestCmdStrace object at 0x7f8f863d9b80>, dependency1 = '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1'
depfile_name = '/tmp/pytest-of-tkloczko/pytest-53/x60/testdb'

    def test_dep(self, dependency1, depfile_name):
        output = StringIO()
        task = Task("tt", ["cat %(dependencies)s"],
                    file_dep=['tests/data/dependency1'])
        cmd = CmdFactory(Strace, outstream=output)
        cmd.loader = self.loader_for_task(task)
        params = DefaultUpdate(dep_file=depfile_name, show_all=False,
                               keep_trace=False, backend='dbm',
                               check_file_uptodate='md5', codec_cls=JSONCodec)
        result = cmd.execute(params, ['tt'])
        assert 0 == result
        got = output.getvalue().split("\n")
        dep_path = os.path.abspath("tests/data/dependency1")
>       assert "R %s" % dep_path in got[0]
E       AssertionError: assert ('R %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R /home/tkloczko/rpmbuild/BUILD/doit-0.36.0'

tests/test_cmd_strace.py:43: AssertionError
-------------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------------
strace -- version 5.17 strace-devel@lists.strace.io
Copyright (c) 1991-2022 The strace developers <>.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Optional features enabled: stack-trace=libdw stack-demangle m32-mpers mx32-mpers
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
.  tt
.  strace_report
________________________________________________________________________ TestCmdStrace.test_target _________________________________________________________________________

self = <tests.test_cmd_strace.TestCmdStrace object at 0x7f8f863d99d0>, dependency1 = '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1'
depfile_name = '/tmp/pytest-of-tkloczko/pytest-53/x63/testdb'

    def test_target(self, dependency1, depfile_name):
        output = StringIO()
        task = Task("tt", ["touch %(targets)s"],
                    targets=['tests/data/dependency1'])
        cmd = CmdFactory(Strace, outstream=output)
        cmd.loader = self.loader_for_task(task)
        params = DefaultUpdate(dep_file=depfile_name, show_all=False,
                               keep_trace=False, backend='dbm',
                               check_file_uptodate='md5', codec_cls=JSONCodec)
        result = cmd.execute(params, ['tt'])
        assert 0 == result
        got = output.getvalue().split("\n")
        tgt_path = os.path.abspath("tests/data/dependency1")
>       assert "W %s" % tgt_path in got[0]
E       AssertionError: assert ('W %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R /home/tkloczko/rpmbuild/BUILD/doit-0.36.0'

tests/test_cmd_strace.py:90: AssertionError
-------------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------------
strace -- version 5.17 strace-devel@lists.strace.io
Copyright (c) 1991-2022 The strace developers <>.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Optional features enabled: stack-trace=libdw stack-demangle m32-mpers mx32-mpers
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
.  tt
.  strace_report
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_dependency.py:98: dumbdbm too dumb to detect db corruption
SKIPPED [2] tests/test_dependency.py:111: test doesnt apply to non DBM DB
SKIPPED [1] tests/test_dependency.py:113: dumbdbm too dumb to detect db corruption
FAILED tests/test_cmd_strace.py::TestCmdStrace::test_dep - AssertionError: assert ('R %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R /ho...
FAILED tests/test_cmd_strace.py::TestCmdStrace::test_target - AssertionError: assert ('W %s' % '/home/tkloczko/rpmbuild/BUILD/doit-0.36.0/tests/data/dependency1') in 'R ...
================================================================ 2 failed, 788 passed, 4 skipped in 10.15s =================================================================

Can you please tell me which version of strace is installed. And provide strace output file from the failing test.

Latest 5.17.

[tkloczko@devel-g2v tests]$ strace -V
strace -- version 5.17 strace-devel@lists.strace.io
Copyright (c) 1991-2022 The strace developers <>.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Optional features enabled: stack-trace=libdw stack-demangle m32-mpers mx32-mpers

I'm not sure is it wahat yuo are asking for .. output of the failing units in in pytest output 🤔

I faced this issue when upgrading to ubuntu 22.4 .
I guess the problem is that strace now also reports the dirs... so I just needed to amend the tests.

BTW is it any plan to release new version?

BTW is it any plan to release new version?

yes, soon. working on it.