Giters

xboot / libonnx

A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Python bindings

KOLANICH opened this issue · comments

commented

It'd be nice to have python bindings, since onnxruntime by Micro$oft has telemetry and so it is a bit unethical to depend on it. Fortunately there can be a thin abstraction layer.

To maximize compatibility to various python impls and to spare users from compiling the lib themselves it may make sense to implement it via ctypes. There are packages generating ctypes bindings from headers automatically, but usually they need extensive postprocessing.

commented

Maybe cython is better, it's much faster than ctypes, and I'll have a try when I'm free.

commented

Maybe cython is better, it's much faster than ctypes

I think the speed of transition between Python and lib code doesn't matter enough. The most of processing still has to be done in the lib itself. But ctypes allow packages be pure python requiring no compilation and be portable across implementations. So I prefer ones that are ctypes-based over cext-based. I don't like unneeded compilation, especially if it is not always possible (i.e. on Windows machines there is usually no compilers, and the same with Android in chroot)

commented

Github action can do that well, a simple script will allow uploading pre-compiled wheels to pypi.

commented
  1. One cannot precompile wheels for every platform possible, every OS possible, every Python version possible, and every lib version possible.
  2. Running wheels compiled by a third party is a trust and security issue;
  3. But a pure Python solution requires no compilation at all, the same wheel can be used everywhere. And if needed, the wheel can be generated locally from source code. Installation can be done from git cheaply enough.
commented

One cannot precompile wheels for every platform possible, every OS possible, every Python version possible, and every lib version possible.

With CI build wheels and github action's matrix, you can do that, for example, watchfiles

Running wheels compiled by a third party is a trust and security issue

If you are using ctypes for wrapping, without a compiler, you will have to install the underlying lib first, which is also compiled by a third party. But if you have a compiler, why not install from source?

But a pure Python solution requires no compilation at all, the same wheel can be used everywhere. And if needed, the wheel can be generated locally from source code. Installation can be done from git cheaply enough.

Extensions won't disable the ability to install from a git repo, thanks to setuptools.

commented

With CI build wheels and github action's matrix, you can do that, for example, watchfiles

Not for every.

you will have to install the underlying lib first, which is also compiled by a third party

That lib can be a shared one from a trusted party, for example from distro packages.

But if you have a compiler, why not install from source?

  1. compilation can be slow, or memory consuming.
  2. I prefer to avoid compilation.

And pure python ones don't require compilation.

Extensions won't disable the ability to install from a git repo, thanks to setuptools.

cexts require compilation, which can be infeasible. I prefer to stay away of compilation. Pure python packages in the most of cases require no compilation.

commented

Okey... So, maybe a multi-backend package which both use cython and ctypes and choose to use one of them when installed should be fine?

commented

multi-backend packages

are always welcome ❤️

If you are also a fan of multi-backend packages, you can find quite some in the orgs within my profile.

commented

Actually I always write extensions with multiple backends. You can also find them in my profile, usually with cython and cffi, one for extreme performance, and another for pypy. Now maybe a third one with ctypes.