ashwinvis / cython_capi

Where is __pyx_capi__?

Experiments with Cython C API and PyCapsules

Exploring different methods to activate the elusive __pyx_capi__ attribute in Cython modules.

While using pxd, it is important that:

1. the .pyx and .pxd files are built in-place, i.e. the source files are present where the .so file will appear.

2. .pyx, .pxd and .so files - all of them share the same name

See the differences by building and testing using_pxd/pkg and using_pxd/pkg_inplace by running make all_pkg and make all_pkg_inplace respectively.

Summary of the different attempts in this repo

Directory	Description	What works	What does not work
using_pxd	Demonstrates without using api keyword how __pyx_capi__ attribute is generated by Cython. How it requires pyx and pxd files to be in-place during build_ext step.	Simple build_ext without packaging, in-place build_ext with packaging (see: setup_pkg_inplace.py)	When build_ext is done with packaging using the Cython sources in some other directory (see setup_pkg.py).
using_api_cpp_classes	A Cython interface to a C++ class in Rectangle.cpp. Attempt to access a C++ bound method getArea.	Modified api can export PyCapsules for regular Cython functions alone, not for bound methods in Cython classes	Pythran cannot have a struct type as an argument; thus Pythran fails to compile.
using_api_keyword	A pyx file containing a function which adds 2 arrays / memory-views and another pyx file which adds integers. Also, Pythran file which includes similar add functions and a Pythran exported PyCapsule. PyCapsules exported using Cython (api) and Pythran is fed into the Pythran function.	For integer type: Pythran calling Pythran-PyCapsule and Cython-PyCapsule. For array type: Pythran calling Pythran-PyCapsule. Direct Cython function calls.	For array type: Pythran calling Cython-PyCapsule
using_cpython_pycapsule	Revisiting the Rectangle.cpp example, now with manually created PyCapsules through Cython.	A cpdef function which works as a wrapper to access the getArea C++ bound method: only has int as argument and return types. Different kinds of twice functions of unbound cdef type and bound cpdef and static types.	-no issues-
using_cpython_pycapsule_class	A cleaner version of the previous case, containing only "twice-functions"	-ditto-	-ditto- and an attempt to write a C code with Python API to call the PyCapsule.
using_cpython_pycapsule_pointers	Trying to use the Rectangle.cpp now with a bound-method with pointer arguments	-nothing works-	Unable to manually generate PyCapsules
using_cpython_pycapsule_class_array	"Twice-functions" for an array of floats instead of a scalar int	Manually creating PyCapsules using make_PyCapsule function	Calling unbound and bound methods of cdef/cpdef/static types with memoryview/ndarray argument types from Pythran with the PyCapsules
using_scipy_lowlevelcallable	Instead of Pythran, we try to call PyCapsules using scipy.LowLevelCallable feature. It extends the integrate.nquad example found in SciPy docs. We use Cython instead of ctypes to generate the LowLevelCallable for the same example	Cython function using double* argument	Cython function using a memoryview or numpy argument (Could be something related to SciPy internals, since scipy does a preliminary type checking using the PyCapsule's signature).

Similar Cython experiments by the community

• https://github.com/pelson/calling_publicly_declared_cython_from_c
• https://gist.github.com/insertinterestingnamehere/df7894b414a94a4456c5

References

• SciPy 2015 | PDF
• serge-sans-paille/pythran#743 and serge-sans-paille/pythran#746
• cython/cython#2027
• cython-devel mail, 2012 Feb
• cython-users mail, 2012 Dec
• stackexchange
• cython-docs