andrewgcodes / FalconStreaming

Falcon40B and 7B (Instruct) with streaming, top-k, and beam search

FalconStreaming

Falcon40B and 7B (Instruct) with streaming, top-k, and beam search 4bit

You must change the runtime type for Colab to run both 7B and 40B!

Adapted from Serin32's starter code on huggingface: https://huggingface.co/tiiuae/falcon-40b/discussions/38