Addition of 1. in the gating blocks

[Sayak Paul]

Why is there an additional of 1 to v in the gating blocks?

An example:

maxim/maxim/models/maxim.py

Line 149 in ae458a6

 return u * (v + 1.) 

@vztu

[Zhengzhong Tu]

It's like adding a skip connection: x = u * v + u. Actually doesn't matter that much in terms of performance.

[Sayak Paul]

Ah. Didn't even notice that. Pardon my stupidity.