google / XNNPACK

High-efficiency floating-point neural network inference operators for mobile, server, and Web

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Openvino Benchmarks

pablogh-2000 opened this issue · comments

Dear all,

I have been running some benchmarks in the following computer:

  • Amazon EC2 x86 T2.Large with Ubuntu 18.04.

DNN Models:

  • MobileNet V1
  • AlexNet

For the following inferences engines:

  • OpenVino
  • TensorFlowLite
  • XNNPACK 'pure'

We see that OpenVino outperforms TFL and XNNPACK by quite a lot:

  • OpenVino uses 50% of the CPU of XNNPACK pure and 25% of TFL's
  • OpenVino takes 50% latency of XNNPACK and 10% of TFL's

Now, I know that OpenVino has this special layer optimisation logic where they reduce a model to a few conv layers.

However, I was wondering if someone else has been running similar benchmarks? And advise will be welcomed, as I am not sure if the results I got make sense.

Thanks a lot.

Pablo.