faustomorales / vit-keras

Keras implementation of ViT (Vision Transformer)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Gelu Warning

awsaf49 opened this issue · comments

commented

While trying to load the model I'm getting this Userwarning,

/opt/conda/lib/python3.7/site-packages/tensorflow_addons/activations/gelu.py:81: UserWarning: Default value of `approximate` is changed from `True` to `False`   
"Default value of `approximate` is changed from `True` to `False`

Is this expected or we can just avoid it.. ?

Ah, yeah, it's a harmless warning but I've fixed it in ed267c4 which is now published on PyPi as v0.0.13. The warning was indicating that the default gelu behavior was changed to disable approximation (i.e., in order to make the activation computation a little faster).