Other variants of Relu can be added in the script activation.py

Question

Other variants of Relu can be added in the script activation.py

parva-jain opened this issue 4 years ago · comments

Parva Jain commented 4 years ago

I'm willing to add other variants of the Relu activation function for a wide range of applications of this package.

Kwanit Gupta commented 3 years ago

LGTM too.

Rohan Singh · Answer 1 · Fri Dec 04 2020 02:58:47 GMT+0800 (China Standard Time)

@parva-jain Sure you can work on it. Before that can you please give a brief description of what all you are going to add?

Parva Jain · Answer 2 · Fri Dec 04 2020 15:52:41 GMT+0800 (China Standard Time)

I have heard of Parametric Relu(PRelu) and Scaled ELU(SELU) functions but only able to find out their Keras implementation till now.
Also, there can be added more activation functions like a binary step function and a swish function.

Rohan Singh · Answer 3 · Sun Dec 06 2020 04:50:25 GMT+0800 (China Standard Time)

@parva-jain sure go ahead, you can work on them. Make sure to follow PEP 8 formatting and correctness of your code.