ZhixiuYe / MLMAN

ACL 2019 paper:Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification

Home Page:https://www.aclweb.org/anthology/P19-1277

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

请问能解释一下全连接层和lstm这样初始化的原因吗?

qq31415926 opened this issue · comments

全连接层初始化代码
bias = np.sqrt(6.0 / (input_linear.weight.size(0) + input_linear.weight.size(1))) nn.init.uniform_(input_linear.weight, -bias, bias) if input_linear.bias is not None: input_linear.bias.data.zero_()
lstm层初始化代码
for ind in range(0, input_lstm.num_layers): weight = eval('input_lstm.weight_ih_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias) weight = eval('input_lstm.weight_hh_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias)