Giters

g1itch / PyBitmessage

My development fork of PyBitmessage,

Home Page:https://pybitmessage-test.readthedocs.io/en/latest/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

New test failure

g1itch opened this issue · comments

commented

Interesting failure showed yesterday when I ran tests in the v0.6 tip (on py37):

FAIL: test_check_randomtrackingdict (src.tests.test_randomtrackingdict.TestRandomTrackingDict)
Check the logic of RandomTrackingDict class
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/glitch/test/PyBitmessage/src/tests/test_randomtrackingdict.py", line 49, in test_check_randomtrackingdict
    self.assertLess(a[x + 1] - a[x], 10)
AssertionError: 16.56256079673767 not less than 10
commented

This is a performance test. Maybe there should be a custom error message explaining this. I haven't tested it in python3. It could mean all kinds of things: an actual performance problem, or garbage collector kicking in at an inopportune time, or the machine running the test being overloaded, and maybe also other things I don't understand.

commented

This is a performance test.

So, the docstring is wrong? Checking the logic and doing a benchmark are the opposite things.

Maybe there should be a custom error message explaining this. I haven't tested it in python3.

It runs on python3 on your buildbot every time, when some PR updates.

It could mean all kinds of things: an actual performance problem, or garbage collector kicking in at an inopportune time, or the machine running the test being overloaded, and maybe also other things I don't understand.

I'm not surprised. My working PC is often overloaded. So I'm not sure if this is a bug at all.

commented

So, the docstring is wrong? Checking the logic and doing a benchmark are the opposite things.

This one test case has multiple checks, some are functional and one is performance. It also involves randomness, so the result won't be deterministic.

It runs on python3 on your buildbot every time, when some PR updates.

Yea, the issue here is when I was writing it I was investigating how python2's dict works internally, to avoid bottlenecks. I know dict it's been redesigned in python3 but I haven't investigated it in detail. The 10 second deadline is a very generous one, it would only trigger if there is something horribly wrong, but it doesn't trigger on minor performance degradation.So while it runs ok on python 3, for all I know it could be 10 times slower compared to python2. That's what I meant to say.

I'm not surprised. My working PC is often overloaded. So I'm not sure if this is a bug at all.

That's probably it.