This repository provides the implementation of LLMCompass from the following papers:
LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference
Hengrui Zhang, August Ning, Rohan Baskar Prabhakar, David Wentzlaff
In the Proceedings of the 51st Annual International Symposium on Computer Architecture:
@inproceedings{LLMCompass,
author = {Zhang, Hengrui and Ning, August and Prabhakar, Rohan Baskar and Wentzlaff, David},
title = {LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference},
year = {2024},
booktitle = {Proceedings of the 51st Annual International Symposium on Computer Architecture},
}
$ conda create -n llmcompass_ae python=3.9
$ conda activate llmcompass_ae
$ pip3 install scalesim
$ conda install pytorch==2.0.0 -c pytorch
$ pip3 install matplotlib
$ pip3 install seaborn
$ pip3 install scipy
$ git clone -b ISCA_AE https://github.com/PrincetonUniversity/LLMCompass
$ cd LLMCompass
$ git submodule init
$ git submodule update --recursive
Unzip the file and download from https://github.com/PrincetonUniversity/ttm-cas.git as cost_model\supply_chain
A Dockerfile has been provided (./Dockerfile), including all the software dependencies and the LLMCompass source code.
A docker image has been provided here.
# Figure 5 (around 100 min)
$ cd ae/figure5
$ bash run_figure5.sh
# Figure 6 (around 1 min)
$ cd ae/figure6
$ bash run_figure6.sh
# Figure 7 (around 20 min)
$ cd ae/figure7
$ bash run_figure7.sh
# Figure 8 (around 40 min)
$ cd ae/figure8
$ bash run_figure8.sh
# Figure 9 (around 30 min)
$ cd ae/figure9
$ bash run_figure9.sh
# Figure 10 (around 45 min)
$ cd ae/figure10
$ bash run_figure10.sh
# Figure 11 (around 5 min)
$ cd ae/figure11
$ bash run_figure11.sh
# Figure 12 (around 4 hours)
$ cd ae/figure12
$ bash run_figure12.sh
After running each script above, the corresponding figures will be generated under the corresponding directory as suggested by its name.
For comparison, a copy of the expected results can be found in ae\expected_results
A guide on "How to Run a LLMCompass Simulation" is shown here.