Add support for int4 datatype for afcuda
WilliamTambellini opened this issue · comments
Add support for int4 datatype at least for afcuda creation and matmul.
Description
- What problem are you trying to solve? quantized matmul
- (Optional) API of new function: no new api, just a new dtype
- (Optional) Algorithms that could be used to implement this feature: no need for algorithm atm
- (Optional)Are there other libraries that implement this feature?
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#vector-types-alignment-requirements-in-device-code
https://arxiv.org/abs/2212.09720