A basic ui for running gpt neo 2.7B on low vram (3 gb Vram minimum)
Expected speed on pcie-3 with 3gb vram is 0.8s/token or 20s for 25 tokens
Expected speed on pcie-3 with 8gb vram is 0.4s/token or 10s for 25 tokens
(with a 2000 token input)
A basic ui for running gpt neo 2.7B on low vram (3 gb Vram minimum)
A basic ui for running gpt neo 2.7B on low vram (3 gb Vram minimum)
Expected speed on pcie-3 with 3gb vram is 0.8s/token or 20s for 25 tokens
Expected speed on pcie-3 with 8gb vram is 0.4s/token or 10s for 25 tokens
(with a 2000 token input)
A basic ui for running gpt neo 2.7B on low vram (3 gb Vram minimum)
Apache License 2.0