A PyTorch starter for doing something actually interesting.
This project follows these principles:
- Explicit, because you shouldn't have to guess
- Dependency-light, because dependencies break
- Concise, because verbose code is skimmed & ignored
python3 -m venv .venv
# Add to the end of .venv/bin/activate
# export PYTHONPATH="${PYTHONPATH}:$(dirname ${VIRTUAL_ENV})"
source .venv/bin/activate
pip install wheel
# pip install torch --index-url https://download.pytorch.org/whl/cpu # for CPU
pip install -r requirements-dev.txt
./dev # run CI checks
- WikiText-103 dataset, Pointer Sentinel Mixture Models (2016), Stephen Merity, Caiming Xiong, James Bradbury, Richard Socher.
- Transformer, Attention Is All You Need (2017), Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin.