shortthirdman / Transformer-Attention-Mechanism

Diving Into the Transformer Attention Mechanism: Building a Minimal Transformer in Pure Python

Transformer Attention Mechanism

Diving Into the Transformer Attention Mechanism: Building a Minimal Transformer in Pure Python

---

Local Development

• Create a Python virtual environment and activate

  $ python -m venv --upgrade-deps --clear dev
  $ export PIP_CONFIG_FILE=".\pip.conf"

  # PowerShell
  $ .\dev\Scripts\Activate.ps1

  # Linux/macOS
  $ source dev/bin/activate

  # Windows Command Prompt
  $ .\dev\Scripts\activate.bat

• Install the packages and dependencies as listed in requirements file

  $ pip install -U -r requirements.txt --no-cache-dir --disable-pip-version-check

• Start your development Jupyter Notebook or Jupyter Lab server

  $ jupyter lab --notebook-dir=.\notebooks --no-browser

• Install the below packages to run the Python script

  $ pip install -q numpy pandas matplotlib scikit-learn torch

---