Running all tests without requiring bazel? (test discovery)
taiya opened this issue · comments
abseil doesn't have special logic for test discovery. Since absl.testing.absltest
is built on top of unittest
, if you just use absl.testing.absltest.TestCase
over unittest.TestCase
in your code, python -m unittest
would also work.
But if your test or your test dependencies also define flags with absl.flags
, python -m unittest
will likely fail because absltest.main()
is not called (it parses flags and does some extra things).
We usually define a test module with the following main block:
from absl.testing import absltest
if __name__ == '__main__':
absltest.main() # Parses flags and does some extra things
and execute the modules separately as main programs, and it doesn't require bazel.
Does this answer your question?
I think so, as far as I add the main block above, then I can recycle the unittest logic.
Let me test it (aha!) and I will close it if it works well.
@julienvalentin for your reference
This works
(python3) ~/dev/graphics/tensorflow_graphics/nn/metric: python -m unittest discover --pattern *_test.py --failfast -v
test_evaluate_preset((0, 1, 1, 1, 1), (1, 1, 0, 0, 0), 0.3333333333333333) (tests.fscore_test.FscoreTest)
[...]
----------------------------------------------------------------------
Ran 50 tests in 1.273s
OK (skipped=4)
And this one works too, as pytest is unittest compatible:
===================================================================================================== test session starts =====================================================================================================
platform linux -- Python 3.6.9, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /home/ataiya/dev/graphics
collected 50 items
tests/fscore_test.py ........s [ 18%]
tests/intersection_over_union_test.py ............s [ 44%]
tests/precision_test.py .............s [ 72%]
tests/recall_test.py .............s [100%]
====================================================================================================== warnings summary =======================================================================================================
/home/ataiya/Envs/python3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py:15
/home/ataiya/Envs/python3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================================================================================== 46 passed, 4 skipped, 1 warning in 3.70s ===========================================================================================
Worked like a charm:
tensorflow/graphics#296
Closing the issue.
Re #138 (comment): as @yilei said --
But if your test or your test dependencies also define flags with
absl.flags
,python -m unittest
will likely fail becauseabsltest.main()
is not called (it parses flags and does some extra things).
While python -m unittest
and pytest
may seem to work, they may have unexpected side affects as the absltest.main()
is not called to parse the flags etc.
Instead, a better workaround would be to trigger the test discovery within the unittest.TestProgram
that absltest.main()
uses, simply by:
absltest.main(module=None)
And then call the above line e.g. via entry point script, or a __main__.py
within the test directory.