Giters

ucbrise / actnn

ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Transformer Benchmarks?

CrazySherman opened this issue · comments

commented

Just curious, have you tested this method on transformer-like benchmarks like BERT etc and test the quantization accuracy?

commented

This repository currently does not support transformers officially. In another project, we do test the method for BERT. The "L2" strategy should work losslessly.