Table of Contents
BERT implementation involved constructing Transformer components (encoder, multi-head attention) from scratch to grasp their core functionality. This included developing Segment, Token, and Position Embeddings. Additionally, an Attention Visualizer was crafted using the BertViz library. The dataset used for training is the IMDB movie reviews dataset which is available in this link. The implementation is in transformer.ipynb
The programming language, frameworks, and technologies used in the project are listed here:
- Python
- Tensorflow
Some useful links and tutorials about this project can be found here:
Mehrdad Nourbakhsh - mehrdad.nb4@gmail.com