Unit test crashes due to python3
linas opened this issue · comments
After pull req #3503 two unit tests have been crashing, fouling up circleci:
17 - AnaphoraTest (Failed)
26 - PythonModuleUTest (SEGFAULT)
My best guess is that AnaphoraTest needs to be ported to python3. Likewise for the other.
I looked around, I hit this:
Running cxxtest tests (4 tests)[2019-05-25 05:30:14:637] [DEBUG] registering opencog::Request "loadpy"
[2019-05-25 05:30:14:637] [DEBUG] Creating opencog::Request instance with "loadpy"
[2019-05-25 05:30:14:637] [DEBUG] [loadpy Request] execute
Exception ignored in: 'opencog.agent_finder.load_req_agent_module'
TypeError: Can't convert 'bytes' object to str implicitly
Got >>><<<
.Exception ignored in: 'opencog.agent_finder.load_req_agent_module'
TypeError: Can't convert 'bytes' object to str implicitly
[2019-05-25 05:30:14:637] [DEBUG] [Request] destructor
This typical python3 error is this:
TypeError: Can't convert 'bytes' object to str
The bug is probably located in opencog/cython/opencog/agent_finder.pyx somewhere after line 44.
I suggest disabling both unit tests. I don't think anyone is using agents. I don't really think agents should be fixed anyway, they're a goofy design that doesn't really make sense, so I'm fine with just disabling that code.
Anaphora -- well, at the current rate, it seems unlikely that anyone is going to be using that code any time soon. So, it too, is more-or-less an unmaintained experiment, and can be disabled. Surely the python3 port can't be hard... but why?
Ohhh ... the Anaophora code does use agents ... see `opencog/nlp/anaphora/agents
I'm investigating this since I've dealt with str/bytes difficulties and 2/3 incompatibility before.
Even though it's unrelated to what I should be doing, it's very difficult to make progress when a test suite is broken and you have no idea if incoming changes have broken stuff or if it's always been broken.
The test doesn't show any TypeError if you run it separately - though 3 of tests fail: test_conjunctions,test_pleonastic_if, and test_propose.
~/src/opencog $ python3 --version
Python 3.6.7
~/src/opencog $ export PYTHONPATH=opencog/nlp/anaphora:build/opencog/cython:../atomspace/build/opencog/cython/
~/src/opencog $ python3 tests/nlp/anaphora/test_anaphora.py
Which then caused me to check the full output of make test, only to find I don't get this TypeError.
@amebel This might be a circleci environment error?
Edit: read some of the linked issues.
Focus on PythonModuleUTest -- I think Anaphora failure is a side-effect of the PythonModuleUTest failure. (In terms of test sequencing, we should run PythonModuleUTest earlier, I guess ... alternately refactor anaphora to not use agents).
Can this be closed now? Github seemed to not automatically close it when "fixes #issue_number" is only in the PR title.
Github seemed to not automatically close it
It used to do that, but now it doesn't. Don't know why.
Can this be closed now?
No .. there are still four failing unit tests:
17 - AnaphoraTest (Failed)
19 - SuRealUTest (Failed)
20 - MicroplanningUTest (Failed)
30 - OpenPsiTest (Failed)
These used to pass, before #3503. Well, the first three used to pass; the fourth one is new, and has never passed ...
Ok my local dev environment experience is:
- when I finished #3518 it was just AnaphoraTest and OpenPsiTest failing.
- today those still fail and SuRealUTest and MicroplanningUTest pass.
- but alas MinerUTest and GhostSyntaxUTest now fail.
I switched to Cython 0.23.4 and Python 3.5 to try to match the CI but that doesn't appear to have helped the situation 😕
I had hoped that the agent fix would have fixed AnaphoraTest .. AnaphoraTest barely uses python at all, except as a test harness. Something about that test harness is not python3-ready.
SuRealUTest and MicroplanningUTest - no clue. If you are in shouting distance of @leungmanhin you should ask him. They seem to depend on ... stuff .. in some delicate way. Don't know why.
Failure of MinerUTest and GhostSyntaxUTest are a complete surprise. This might be due to a borked build env or install env ... sometimes an rm -r build is needed, and (much more rarely) an rm -r /usr/local/*/opencog and other less-obvious opencog install paths (e.g. wherever the python modules get written) .. so cd /usr/local; find |grep opencog to find them all. I do this whenever truly weird failures start happening, and it usually fixes them.
I fixed AnaphoraUTest here: #3527 - it now passes, without the need to skip any of the tests.
Not sure why SuRealUTest and MicroplanningUTest fail, they don't depend on Python though as far as I remember... I'll look into it later
@stellarspot is looking to OpenPsiTest
Closing; as everything now seems to work, except for SuReal/Microplanning and issue #3544 covers those.