This repository implements Hawk and Griffin blocks from Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models using Accelerated Scan and Flash Attention for PyTorch.
pip install hippogriff
Griffin MQA + Hawk Linear RNN Hybrid
This repository implements Hawk and Griffin blocks from Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models using Accelerated Scan and Flash Attention for PyTorch.
pip install hippogriff
Griffin MQA + Hawk Linear RNN Hybrid
https://pypi.org/project/hippogriff
MIT License