orchest / orchest

Build data pipelines, the easy way 🛠️

Home Page:https://orchest.readthedocs.io/en/stable/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Run static typing checks

astrojuanlu opened this issue · comments

#1216 started adding some type hints to the codebase, as requested in #450. However, to my knowledge, we are not actually verifying if those hints are correct.

One of the possible type checkers is MyPy (Dropbox). It would be nice to add MyPy as part of pre-commit - however, as far as I understand, each hook runs in its own virtual environment, and therefore doesn't have the dependencies installed. This is problematic to run MyPy in "strict" mode, since one would have to --ignore-imports. There's more background here https://jaredkhan.com/blog/mypy-pre-commit

At some point I was using pytest-mypy, but I was hitting realpython/pytest-mypy#123 too often so now I default to running mypy src tests just before pytest.

Other checkers:

I personally only have experience with MyPy.

I use pyright as my language server and have only occasionally used it as a type checker. I know that pyright can sometimes be rather slow so it would be something we have to investigate.

On another note, I do like the benefit of type checkers but I don't want to be in a spot where we are spending more time fixing types than improving the product. I guess we have to try it before we know it.

On another note, I do like the benefit of type checkers but I don't want to be in a spot where we are spending more time fixing types than improving the product. I guess we have to try it before we know it.

This is a very real concern. In my experience, MyPy in strict mode can be pretty picky and annoying. The good news is that these tools are usually highly configurable, it's a matter of finding the sweet spot.