[Future Request] 5D Transpose support in the edgetpu compiler
PINTO0309 opened this issue · comments
1. Overview
There are many good quality models nowadays, including the 5D Transpose. For example, YOLOv4, YOLOv5, NanoDet, and so on. Currently, EdgeTPU Compiler does not support 5D transpose, so being able to do the transformations will benefit many engineers as it will allow them to generate very high performance edgetpu tflite models.
2. Details
2-1. Environment
- Ubuntu 18.04
- Edge TPU Compiler version 15.0.340273435
- TensorFlow version tf-nightly==2.5.0-dev20201204
- NanoDet (Super fast and lightweight anchor-free object detection model. Real-time on mobile devices)
2-2. NanoDet repository of citations
https://github.com/RangiLyu/nanodet.git
2-3. NanoDet Full Integer Quant tflite:
tflite: https://drive.google.com/file/d/1CXWOabh2Kml-0hAZrDg2fhqaUeOJ4JaL/view?usp=sharing
Model structure diagram of NanoDet Full Integer Quant tflite
2-4. NanoDet EdgetTPU tflite
tflite: https://drive.google.com/file/d/1qqtj-DndmFTIJBl4tDkuZPeBwA-eHV7T/view?usp=sharing
5D Transpose and later are ignored for conversion to custom op.
Model structure diagram of NanoDet EdgetTPU tflite
3. Benchmarks
Performance is measured on Kirin 980(4xA76+4xA55) ARM CPU based on ncnn. There is no doubt that if we were able to convert this model perfectly in EdgeTPU, the performance would be much higher than the benchmark using NCNN.
| Model | Resolution | COCO mAP | Latency(ARM 4xCore) | FLOPS | Params | Model Size(ncnn bin) |
|---|---|---|---|---|---|---|
| NanoDet-m | 320*320 | 20.6 | 10.23ms | 0.72B | 0.95M | 1.8mb |
| NanoDet-m | 416*416 | 21.7 | 16.44ms | 1.2B | 0.95M | 1.8mb |
| YoloV3-Tiny | 416*416 | 16.6 | 37.6ms | 5.62B | 8.86M | 33.7mb |
| YoloV4-Tiny | 416*416 | 21.7 | 32.81ms | 6.96B | 6.06M | 23.0mb |
4. Appendix
I have already been able to convert all OPs except 5D Transpose into OPs supported by the edgetpu compiler.
4-1. PyTorch -> ONNX -> OpenVINO -> TFLite, Conversion Tools
openvino2tensorflow: https://github.com/PINTO0309/openvino2tensorflow.git
4-2. The various models generated by the conversion work are committed to
https://github.com/PINTO0309/PINTO_model_zoo/tree/master/072_NanoDet
EdgeTPU Compiler v16
A lot more layers have been converted than when I posted the issue, but 5D Reshape and Transpose are still not converted.
@PINTO0309
Hi, Is 5D-transpose still unsupport in the edgetpu compiler?
I'm trying to convert a model from pytroch -> onnx -> tensorflow -> tflite,
but I found there's additional 5D transpose ops have been added during conversion.
So, If edgetpu compiler still not support 5D-transpose operation,
I guess it is impossible to run my model on coral tpu?
Thank you for your attention and patience!
@HeywardLiu @PINTO0309
Tensors have to be either 1-, 2-, or 3-dimensional. If a tensor has more than 3 dimensions, then only the 3 innermost dimensions may have a size greater than 1.
For example:
(1,320,320,3) and (1,3,320,320,1) --> supported
(320,1,320, 3) and (3,320,1,320) --> unsupported.
We don't have any plans to support tensors with more than 3 innermost dimensions.
Thanks!
@hjonnala
I appreciate your message.
Thank you for your quick reply!
Thanks.