Primarily focussing on inference on user setup with a 16 GB graphic card.
python falcon_test.py --test-gen --bitwidth (8|32)
The text generation appears to be 4x facter in 8-bit on an NVIDIA Tesla T4
Primarily focussing on inference on user setup with a 16 GB graphic card.
python falcon_test.py --test-gen --bitwidth (8|32)
The text generation appears to be 4x facter in 8-bit on an NVIDIA Tesla T4