CPU and NPU Hybrid Training Model

[MansonHua]

Background:
Hello, I am a user of this repository and I am interested in the AI model training sample code provided here. I have noticed that the current sample codes in the repository mainly focus on training using a single type of hardware (such as NVIDIA GPU). However, I am interested in exploring more about hybrid training examples, especially combining Intel U-series chip NPU (Neural Processing Unit) with NVIDIA GPU.

Request Details:
I would like to inquire if it would be possible to provide some sample code on hybrid training in this repository. If possible, it would be great if the sample code could be demonstrated using both the Transformer library and PyTorch library, showing how to combine Intel U-series chip NPU and NVIDIA GPU to train AI models.

Expected Examples:
Here are the examples I would like to see:

It would be preferable if the sample code is demonstrated using both the Transformer library and PyTorch library.
The examples should demonstrate how to utilize Intel U-series chip NPU and NVIDIA GPU for hybrid training.
The examples could cover some common AI models, such as text generation or image processing.
The examples should include necessary explanations and comments to help users understand the code and try it themselves.
Additional Notes:

If there are any specific settings or configuration requirements, please provide instructions in the sample code.
These examples would be very useful for users who wish to explore hybrid training, and would help in better understanding how to utilize different hardware resources to accelerate model training.
If this request is feasible, I look forward to seeing these sample codes in the repository. Thank you very much for your contributions and support to the community!

Thanks a lot,
Manson Hua

[Alessandro Palla]

I agree that will be very cool. We are actively working on this