Issue with removing omega_tol
jamesgoulet opened this issue · comments
I have completed the implementation and tested the new formulation for all the mixture-based activation functions in all four files
activation.cpp
activation_fun_cpu.cpp
activation_fun.cu
activation_cuda.cu
that can be found in the branch: New-formulation-mixture-activations
I have found a bug in the existing MixtureSigmoid()
where the ma = ma/2
should have been done as a separate step. The result is that the number of epochs in the LSTM example test_lstm.py
needs to be reduced to avoid NaNs.
- I have re-updated the unit tests that were all minimally changed by the new formulation.
- I have tested with
test.py
all activation functions with CPU and GPU.
The issue that I am running into is that I am unable to remove omega_tol
from all the classes' inputs. I tried searching for all occurence in the cuTAGI
repo and removing them, but I am then unable to compile and I do not have the experience to understand what I am doing wrong. If you can provide me with guidance with how to do so, I can finish this implementation. You will find below a screenshot of the errors I am running into when trying to compile after having remove the omega_tol
everywhere:
@jamesgoulet Did you push the latest code to your branch New-formulation-mixture-activations
? It compiled just fine on my side
@lhnguyen102 No, have deleted because I was afraid I would break something... I can redo it if you want?
@jamesgoulet could you push the latest code that caused this error to your New-formulation-mixture-activations
? Otherwise I wont be able to troubleshoot the issues. As long as we dont merge to main
, it wont break main branch
I re-did it from scratch and now it compiles. I will prepare the PR.