Giters

tensorops / TransformerX

Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

New residual and residual gate layers

soran-ghaderi opened this issue · comments

commented

A list of new embedding layers to be added

To contribute, please:

  1. Create a new issue, copy an available (not closed and not opened by someone else) subtask name and paste it in the title followed by the reference of the task. (ex. subtask_name #source_issue_number). Make sure the subtask is not already opened by someone else. Another way is to hover over the subtasks and click on the Open convert to issue
  2. Copy and paste your new issue link here in the comments.
  3. Fork the repository
  4. Add your changes
  5. Create a pull request and mention your issue link
  • Residual
  • GRU Gating
  • Inverted Residual Block
  • Bottleneck Residual Block