Giters

PromtEngineer / localGPT

Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

GPU layers/batch size/models selction

bp020108 opened this issue · comments

commented

Can you please help me to select GPU layers/batch size/models for GPU A100 80 GB?

Please help us for chunk size and token.

please help to maximize the performance. if there is no exact number then please provide range so I can check