takanori-azegami-jp / FlexGen-docker

FlexGen（大規模言語モデル（LLM:）シングルGPU生成エンジン） with docker

FlexGen-docker

FlexGen with docker

Setup

Install Docker.

$ git clone https://github.com/karaage0703/FlexGen-docker
$ cd FlexGen-docker
$ docker build -t ubuntu:FlexGen .

Usage

Execute following command in FlexGen-docker directory.

$ docker run -it -v $(pwd):/root --gpus all ubuntu:FlexGen

In container, execute following commands.

root@hostname:/# cd FlexGen
root@hostname:/FlexGen# python3 apps/chatbot.py --model facebook/opt-6.7b --compress-weight

Reference

• https://agirobots.com/flexgen/

参考

自宅で動くChatGPTと噂のFlexGenをDockerで手軽に動かす