cvnp provides automatic cast between OpenCV matrices and numpy arrays when using pybind11:
cv::Matandcv::Mat_<Tp>: standard OpenCV matrices are transformed to numpy array with shared memory (i.e. modification to matrices elements made from python are immediately visible to C++, and vice-versa).- Sub-matrices:
- Sub-matrices created from C++ can also be shared to python
- Sub-matrices created from python will need to be transformed to a contiguous array before being shared to C++
cv::Matx: small matrices are transformed to numpy arrays without shared memory- Casts without shared memory for simple types, between
cv::Size,cv::Point,cv::Point3,cv::Scalar_<Tp>,cv::Rect_<Tp>and pythontuple
Note: The API evolved on May 4th 2023: see breaking changes
pybind11::array mat_to_nparray(const cv::Mat& m);
cv::Mat nparray_to_mat(pybind11::array& a);template<typename _Tp, int _rows, int _cols> pybind11::array matx_to_nparray(const cv::Matx<_Tp, _rows, _cols>& m);
template<typename _Tp, int _rows, int _cols> void nparray_to_matx(pybind11::array &a, cv::Matx<_Tp, _rows, _cols>& out_matrix);Since OpenCV supports a subset of numpy types, here is the table of supported types:
➜ python
>>> import cvnp
>>> cvnp.print_types_synonyms()
cv_depth cv_depth_name np_format np_format_long
0 CV_8U B np.uint8
1 CV_8S b np.int8
2 CV_16U H np.uint16
3 CV_16S h np.int16
4 CV_32S i np.int32
5 CV_32F f float
6 CV_64F d np.float64
- Add cvnp to your project. For example:
cd external
git submodule add https://github.com/pthom/cvnp.git- Link it to your python module:
In your python module CMakeLists, add:
add_subdirectory(path/to/cvnp)
target_link_libraries(your_target PRIVATE cvnp)- In your module, include cvnp:
#include "cvnp/cvnp.h"- (Optional) If you want to import the declared functions in your module:
Write this in your main module code:
void pydef_cvnp(pybind11::module& m);
PYBIND11_MODULE(your_module, m)
{
....
....
....
pydef_cvnp(m);
}You will get two simple functions:
- cvnp.list_types_synonyms()
- cvnp.print_types_synonyms()
import cvnp
import pprint
pprint.pprint(cvnp.list_types_synonyms(), indent=2, width=120)
[ {'cv_depth': 0, 'cv_depth_name': 'CV_8U', 'np_format': 'B', 'np_format_long': 'np.uint8'},
{'cv_depth': 1, 'cv_depth_name': 'CV_8S', 'np_format': 'b', 'np_format_long': 'np.int8'},
{'cv_depth': 2, 'cv_depth_name': 'CV_16U', 'np_format': 'H', 'np_format_long': 'np.uint16'},
{'cv_depth': 3, 'cv_depth_name': 'CV_16S', 'np_format': 'h', 'np_format_long': 'np.int16'},
{'cv_depth': 4, 'cv_depth_name': 'CV_32S', 'np_format': 'i', 'np_format_long': 'np.int32'},
{'cv_depth': 5, 'cv_depth_name': 'CV_32F', 'np_format': 'f', 'np_format_long': 'float'},
{'cv_depth': 6, 'cv_depth_name': 'CV_64F', 'np_format': 'd', 'np_format_long': 'np.float64'}]Below is on extract from the test test/test_cvnp.py:
def test_cpp_sub_matrices():
"""
We are playing with these bindings:
struct CvNp_TestHelper {
// m10 is a cv::Mat with 3 float channels
cv::Mat m10 = cv::Mat(cv::Size(100, 100), CV_32FC3, cv::Scalar(0.f, 0.f, 0.f));
// GetSubM10 returns a sub-matrix of m10
cv::Mat GetSubM10() { return m10(cv::Rect(1, 1, 3, 3)); }
// Utilities to trigger value changes made by C++ from python
void SetM10(int row, int col, cv::Vec3f v) { m10.at<cv::Vec3f>(row, col) = v; }
cv::Vec3f GetM10(int row, int col) { return m10.at<cv::Vec3f>(row, col); }
...
};
"""
o = CvNp_TestHelper()
#
# 1. Transform cv::Mat and sub-matrices into numpy arrays / check that reference counts are handled correctly
#
# Transform the cv::Mat m10 into a linked numpy array (with shared memory) and assert that m10 now has 2 references
m10: np.ndarray = o.m10
assert o.m10_refcount() == 2
# Also transform the m10's sub-matrix into a numpy array, and assert that m10's references count is increased
sub_m10 = o.GetSubM10()
assert o.m10_refcount() == 3
#
# 2. Modify values from C++ or python, and ensure that the data is shared
#
# Modify a value in m10 from C++, and ensure this is visible from python
val00 = np.array([1, 2, 3], np.float32)
o.SetM10(0, 0, val00)
assert (m10[0, 0] == val00).all()
# Modify a value in m10 from python and ensure this is visible from C++
val10 = np.array([4, 5, 6], np.float32)
o.m10[1, 1] = val10
assert (o.m10[1, 1] == val10).all()
#
# 3. Check that values in sub-matrices are also changed
#
# Check that the sub-matrix is changed
assert (sub_m10[0, 0] == val10).all()
# Change a value in the sub-matrix from python
val22 = np.array([7, 8, 9], np.float32)
sub_m10[1, 1] = val22
# And assert that the change propagated to the master matrix
assert (o.m10[2, 2] == val22).all()
#
# 4. del python numpy arrays and ensure that the reference count is updated
#
del m10
del sub_m10
assert o.m10_refcount() == 1
#
# 5. Sub-matrices are supported from C++ to python, but not from python to C++!
#
# i. create a numpy sub-matrix
full_matrix = np.ones([10, 10], np.float32)
sub_matrix = full_matrix[1:5, 2:4]
# ii. Try to copy it into a C++ matrix: this should raise a `ValueError`
with pytest.raises(ValueError):
o.m = sub_matrix
# iii. However, we can update the C++ matrix by using a contiguous copy of the sub-matrix
sub_matrix_clone = np.ascontiguousarray(sub_matrix)
o.m = sub_matrix_clone
assert o.m.shape == sub_matrix.shapeBelow is on extract from the test test/test_cvnp.py:
def test_matx_not_shared():
"""
We are playing with these elements
struct CvNp_TestHelper {
cv::Matx32d mx_ns = cv::Matx32d::eye();
void SetMX_ns(int row, int col, double v) { mx_ns(row, col) = v;}
...
};
"""
# create object
o = CvNp_TestHelper()
m_linked = o.mx_ns # Make a numy array that is a copy of mx_ns *without* shared memory
assert m_linked.shape == (3, 2) # check its shape
m_linked[1, 1] = 3 # a value change in the numpy array made from python
assert o.mx_ns[1, 1] != 3 # is not visible from C++!
o.SetMX_ns(2, 1, 15) # A C++ change a value in the matrix
assert not are_float_close(m_linked[2, 1], 15) # is not visible from python,
m_linked = o.mx_ns # but becomes visible after we re-create the numpy array from
assert are_float_close(m_linked[2, 1], 15) # the cv::MatxThese steps are only for development and testing of this package, they are not required in order to use it in a different project.
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txtYou will need to have OpenCV installed on your system (you can use vcpkg or your package manager).
You need to specify the path to the python executable:
mkdir build && cd build
cmake .. -DPython_EXECUTABLE=../venv/bin/python
makeIn the build dir, run:
cmake --build . --target test
(this will run native C++ tests and python tests)
rm -rf build
rm -rf venv
rm -rf .pytest_cache
rm *.so
rm *.pyd
Thanks to Dan Mašek who gave me some inspiration here: https://stackoverflow.com/questions/60949451/how-to-send-a-cvmat-to-python-over-shared-memory
This code is intended to be integrated into your own pip package. As such, no pip tooling is provided.
This library was updated in May 2023, with breaking changes from previous versions:
- Previously, it was required to use the (now defunct)
cvnp::Mat_sharedin order to share memory between cv::Mat and a numpy array. The memory is now always shared between cv::Mat and numpy arrays, since it was shown that this is faster and safe. - cv::Matx cannot share memory with a numpy array
For those interested in the previous API, it is still available in the original_api branch.