Giters

OpenNMT / CTranslate2

Fast inference engine for Transformer models

Home Page:https://opennmt.net/CTranslate2

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature] CANN Backend support

3manifold opened this issue · comments

commented

CANN (Compute Architecture of Neural Networks), developed by Huawei, is a heterogeneous computing architecture for AI scenarios.
It provides multi-layer programming interfaces to help users quickly build AI applications and services based on the Ascend platform.

CANN backend in CTranslate2, enables running AI models on the Ascend NPU extending the existing CPU & CUDA workflows.
One can find more on Ascend NPU and CANN library here.

Examples of projects that support CANN include ONNX Runtime & OpenCV.

Implementation as well as detailed description of tests, environment setup & benchmarks can be found in pull request #1606 .

Notes

  • In the context of the development of this feature we also submitted issue #1583 .
  • In case CANN Backend support has increased demand, a new Pull request/subproject will be introduced contributing the respective CI involving dedicated Ascend hardware.