Switch to TensorFlow 2.0 and tf.keras
de-code opened this issue · comments
From the Keras site:
The current release is Keras 2.3.0, which makes significant API changes and add support for TensorFlow 2.0. The 2.3.0 release will be the last major release of multi-backend Keras. Multi-backend Keras is superseded by
tf.keras.
Maybe it's time to upgrade?
I'd say yes! but after #82
Hello ! I've already started to look at it (it's in the TODO list at the end of the README), but it's complicated due to TF dependencies to BERT and ELMo (also the FLAIR implementation I am exploring at https://github.com/kensho-technologies/bubs).
Any more up-to-date thoughts on this? I am not sure about the current or near future support situation of TensorFlow 1.x. It seems on its way out. For some things there are backwards compatible APIs.
Still no ELMo compatible TF2, so not really possible. For BERT there's now https://github.com/kpe/bert-for-tf2 but it would probably involve quite a lot of effort for a change, because it's a rewrite from scratch and I had to introduced a lot of modifications linked to the original BERT to get the sequence labeling working efficiently and with a CRF layer (the original BERT repo was not supporting sequence labeling).
Last release of TF 1.15 is 3 months ago, so I am not worry for the moment :)
Last release of TF 1.15 is 3 months ago, so I am not worry for the moment :)
From the TensorFlow 1.15.0 release (Oct 16, 2019):
This is the last 1.x release for TensorFlow. We do not expect to update the 1.x branch with features, although we will issue patch releases to fix vulnerabilities for at least one year.
That is now more than a year ago. I didn't find the planned support for it.
On one of our repos we got the automatic low severity security notifications:
Those were fixed in TensorFlow 2.4.0 (Dec 14, 2020). There wasn't a related TensorFlow 1.x release with those fixes. As far as I can tell, the security advisories do not mention it not affecting 1.x (just that it affects < 2.4.0).
Additionally TensorFlow 1.15.4 doesn't support Python 3.8+ (according to PyPI). Whereas TensorFlow 2.4.0 supports 3.8, but not Python 2 (see PyPI).
Which makes me think that it may not be far off before the support is officially dropped.
Ok now I am slowly starting to worry :D
I think there's no real solution except skipping ELMo entirely or moving to pytorch, which would be almost like starting a new project.
I think there's no real solution except skipping ELMo entirely or moving to pytorch, which would be almost like starting a new project.
I am not using ELMo (or BERT) at the moment. I guess you are using it? What do we use for ELMo, I didn't see it in the requirements. In general the upgrade shouldn't be too difficult in particular when using the compatibility layer first. Converting it to be proper TensorFlow 2.0 might require more of a re-think.
Yes, switching to pytorch should probably be a separate project. I am not sure what the benefits would be for this project.
BTW pandas 1.2.0 dropped support for Python 3.6 (release notes, GitHub issue). Narrowing the Python version to 3.7, supported by both pandas 1.2.0+ and TensorFlow 1.x (if you did decide to keep your other dependencies up-to-date).
Any thought of adding TensorFlow 2.x support any time soon?
I am hoping to work on this and on ONNX support during the end of year holidays.
Closing the issue so silently? It almost deserves a big announcement. Well done!
Thank you @de-code ! A lot's of rewriting everywhere but very cool exercise. We added support of the hugging face transformers library so we can integrate a lot of transformer model variants (we have tested bert and roberta families).