Two convolutional layer in the bottleneck block share parameters, which can be formulated as:
Such pattern can be extended into FFN or MOE FFN of large language models to reduce parameters and accelerate model traning and inference.
Two convolutional layer in the bottleneck block share parameters, which can be formulated as:
Such pattern can be extended into FFN or MOE FFN of large language models to reduce parameters and accelerate model traning and inference.
MIT License