Why convolute on the input_batch twice with different activation function, rather than convolute once and activate with different function

[Zhipeng Han]

Why the implement convolute on the input_batch twice with tanh and sigmod activation function, as follows:

conv_filter = causal_conv(input_batch, weights_filter, dilation)
conv_gate = causal_conv(input_batch, weights_gate, dilation)

I want to know whether we can convolution once time, and activate it with different function. I think this can reduce some computation. Pseudo-code in keras as follows:

conv = causal_conv(input_batch, filter_size)
tanh_out = Activation('tanh')(conv)
sigmod_out = Activation('sigmod')(conv)

[saddlekiller]

No, your pseudo-code is different from what described in paper. But you can do the convolution once and then split them into two components, one for "tanh" and the other for "sigmoid".

[Zhipeng Han]

Thanks for your reply.
I think the Eq.2 z=tanh(Wf,k ∗x)⊙σ(Wg,k ∗x) supports what your said.

But the Figure. 4 in the paper does not have the meaning as same as the equation, since it draw only one dilated convolutions and then split into two components.

I think the equation is more convincing.

[Zhipeng Han]

[saddlekiller]

If you use the same weights in these two components, the output will always close to 1 when output get larger and close to 0 when it goes smaller. I don’t think that’s the “gate” works for.

[Zhipeng Han]

Got it. Thanks.