TIDLExecutionProvider Unavailable compiling YOLOv5 Ti Lite ONNX and PROTOTXT

[David Hostage Peterson]

When I run onnxrt_ep.py in osrt_python I get the following error, how do I fix this? Why is TIDL not available?
I am trying to compile my YOLOv5 Ti Lite ONNX and PROTOTXT for use on the Beaglebone AI.

Available execution providers : ['CPUExecutionProvider']

Running 1 Models - ['yolov5s6_640_ti_lite']

Running_Model : yolov5s6_640_ti_lite

/home/mm282681/miniconda3/envs/tidl-tools/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:56: UserWarning: Specified provider 'TIDLExecutionProvider' is not in available provider names.Available providers: 'CPUExecutionProvider'
"Available providers: '{}'".format(name, ", ".join(available_provider_names)))
Process Process-1:
Traceback (most recent call last):
File "/home/mm282681/miniconda3/envs/tidl-tools/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/mm282681/miniconda3/envs/tidl-tools/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "onnxrt_ep.py", line 190, in run_model
sess = rt.InferenceSession(config['model_path'] ,providers=EP_list, provider_options=[delegate_options, {}], sess_options=so)
File "/home/mm282681/miniconda3/envs/tidl-tools/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/home/mm282681/miniconda3/envs/tidl-tools/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 379, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: Unknown Provider Type: TIDLExecutionProvider

[Kumar Desappan]

Looks like your installation is not complete, after successful installation the ONNXRT shall have below execution providers

Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

But your log shows only
Available execution providers : ['CPUExecutionProvider']

Please check below in your python, if this is not available. I would recomd a fresh installation by follwoing the user guide

pip3 show onnxruntime_tidl
Name: onnxruntime-tidl
Version: 1.7.0
Summary: ONNX Runtime is a runtime accelerator for Machine Learning models
Home-page: https://onnxruntime.ai
Author: Microsoft Corporation
Author-email: [e-mail]
License: MIT License
Location: /home/a0393754/miniconda/envs/benchmark/lib/python3.6/site-packages
Requires: numpy, protobuf
Required-by:

[David Hostage Peterson]

I got the exact output as shown when you ran pip show onnxruntime_tidl, is there any other way I can confirm why I do not have TIDL Execution/Compilation Provider?

I created a conda environment with Python 3.6, I am on a Linux PC with Intel CPU so I set the Device to am62, then I ran the setup script with no flags. Did I use the wrong device or did I miss a step in the setup? Could these be causes to the issue I am seeing? Should I be used a different script to compile my YOLOv5 Ti Lite model for deployment using my Linux PC?

[SAI RAMAN PATRA]

The thing is you have installed both onnxruntime-tidl (using setup.sh) as well as onnxruntime (using requirements.txt). The problem is whenever you try to import onnxruntime, by default python imports onnxruntime, which has only one service provider i.e. ['CPUExecutionProvider']

(venv) hp]:app$ pip list | grep onnxruntime
onnxruntime             1.10.0
onnxruntime-gpu         1.10.0
onnxruntime-tidl        1.7.0

Earlier I had three types of onnxruntime packages installed in my machine, and It was importing onnxruntime by default.

(venv) hp]:app$ python
Python 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime as rt
>>> rt.get_available_providers()
['CPUExecutionProvider']

Steps to overcome this:

1. Uninstall each variant of onnxruntime present in the system. pip uninstall onnxruntime / onnxruntime-gpu / onnxruntime-tidl
2. Run the setup.sh script to create the tidl tool folder and onnxruntime-tidl will get installed on your system.
3. Once the above step completes, check the installation using pip. Now you have all the required service providers.

(venv) hp]:app$ pip list | grep onnxruntime
onnxruntime-tidl        1.7.0
(venv) hp]:app$ python
Python 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime as rt
>>> rt.get_available_providers()
['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

[Kate Yurkova]

@srpraman setup.sh seems to install onnxruntime-tidl only on PC, how do I get onnxruntime-tidl on the chip?