CVXPY on xilinx Arm based SoC

Question

CVXPY on xilinx Arm based SoC

fahadak69 opened this issue 5 months ago · comments

Hi

I am trying to install cvxpy on xilinx arm A5 based SoC running linux image built via xilinx petalinux tool. When I run "python3 -m pip install cvxpy", I get error (copied at the end). Its an embedded platform so I dont want to install all the C-compilers etc? Is there a way around it? FYI, I was able to successfully install scipy using pip

**Downloading cvxpy-1.4.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch 64.whl (1.1 MB)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 1.1/1.1 MB 693.5 kB/s eta 0:00:00
Collecting pybind11
Downloading pybind11-2.11.1-py3-none-any.whl (227 kB)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 227.7/227.7 kB 765.2 kB/s eta 0:00:00
Collecting osqp>=0.6.2
Downloading osqp-0.6.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch6 4.whl (294 kB)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 294.4/294.4 kB 751.8 kB/s eta 0:00:00
Collecting scs>=3.0
Downloading scs-3.2.4.post1.tar.gz (1.5 MB)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 1.5/1.5 MB 463.5 kB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
x exit code: 1
mq> [20 lines of output]
+ meson setup /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb1223 77 /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377/.mesonpy-nm4er tkw -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pi p-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377/.mesonpy-nm4ertkw/meson- python-native-file.ini
The Meson build system
Version: 1.3.1
Source dir: /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377
Build dir: /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377/ .mesonpy-nm4ertkw
Build type: native build
Project name: scs
Project version: undefined

  ../meson.build:1:0: ERROR: Unknown compiler(s): [['cc'], ['gcc'], ['clang'                                                                                                                                                             ], ['nvc'], ['pgcc'], ['icc'], ['icx']]
  The following exception(s) were encountered:
  Running `cc --version` gave "[Errno 2] No such file or directory: 'cc'"
  Running `gcc --version` gave "[Errno 2] No such file or directory: 'gcc'"
  Running `clang --version` gave "[Errno 2] No such file or directory: 'clan                                                                                                                                                             g'"
  Running `nvc --version` gave "[Errno 2] No such file or directory: 'nvc'"
  Running `pgcc --version` gave "[Errno 2] No such file or directory: 'pgcc'                                                                                                                                                             "
  Running `icc --version` gave "[Errno 2] No such file or directory: 'icc'"
  Running `icx --version` gave "[Errno 2] No such file or directory: 'icx'"

  A full log can be found at /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758                                                                                                                                                             b070aa4fb122377/.mesonpy-nm4ertkw/meson-logs/meson-log.txt
  [end of output]

note: This error originates from a subprocess, and is likely not a problem wit h pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
mq> See above for output.

note: This is an issue with the package mentioned above, not pip.**

Riley Murray · Answer 1 · Mon Jan 29 2024 13:06:35 GMT+0800 (China Standard Time)

The error you're seeing looks like it's coming from trying to build SCS. You can try installing cvxpy-base instead of cvxpy. If you can get that to work then you can at least isolate the question of how to install solvers. (Different solvers have different build requirements.)

…

On Sun, Jan 28, 2024 at 10:23 PM fahadak69 ***@***.***> wrote: Hi I am trying to install cvxpy on xilinx arm A5 based SoC running linux image built via xilinx petalinux tool. When I run "python3 -m pip install cvxpy", I get error (copied at the end). Its an embedded platform so I dont want to install all the C-compilers etc? Is there a way around it? FYI, I was able to successfully install scipy using pip **Downloading cvxpy-1.4.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch 64.whl (1.1 MB) qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 1.1/1.1 MB 693.5 kB/s eta 0:00:00 Collecting pybind11 Downloading pybind11-2.11.1-py3-none-any.whl (227 kB) qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 227.7/227.7 kB 765.2 kB/s eta 0:00:00 Collecting osqp>=0.6.2 Downloading osqp-0.6.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch6 4.whl (294 kB) qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 294.4/294.4 kB 751.8 kB/s eta 0:00:00 Collecting scs>=3.0 Downloading scs-3.2.4.post1.tar.gz (1.5 MB) qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq 1.5/1.5 MB 463.5 kB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error × Preparing metadata (pyproject.toml) did not run successfully. x exit code: 1 mq> [20 lines of output] + meson setup /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb1223 77 /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377/.mesonpy-nm4er tkw -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pi p-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377/.mesonpy-nm4ertkw/meson- python-native-file.ini The Meson build system Version: 1.3.1 Source dir: /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377 Build dir: /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758b070aa4fb122377/ .mesonpy-nm4ertkw Build type: native build Project name: scs Project version: undefined ../meson.build:1:0: ERROR: Unknown compiler(s): [['cc'], ['gcc'], ['clang' ], ['nvc'], ['pgcc'], ['icc'], ['icx']] The following exception(s) were encountered: Running `cc --version` gave "[Errno 2] No such file or directory: 'cc'" Running `gcc --version` gave "[Errno 2] No such file or directory: 'gcc'" Running `clang --version` gave "[Errno 2] No such file or directory: 'clan g'" Running `nvc --version` gave "[Errno 2] No such file or directory: 'nvc'" Running `pgcc --version` gave "[Errno 2] No such file or directory: 'pgcc' " Running `icc --version` gave "[Errno 2] No such file or directory: 'icc'" Running `icx --version` gave "[Errno 2] No such file or directory: 'icx'" A full log can be found at /tmp/pip-install-ojj2_cp8/scs_5116970da4824d758 b070aa4fb122377/.mesonpy-nm4ertkw/meson-logs/meson-log.txt [end of output] note: This error originates from a subprocess, and is likely not a problem wit h pip. error: metadata-generation-failed × Encountered error while generating package metadata. mq> See above for output. note: This is an issue with the package mentioned above, not pip.** — Reply to this email directly, view it on GitHub <#2337>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACRLIFG2BWSFJB6TKXTO4BTYQ4I3NAVCNFSM6AAAAABCOW3Y2WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEYDINJWGAYTOMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

enzbus · Answer 2 · Tue Jan 30 2024 15:33:14 GMT+0800 (China Standard Time)

It seems you need to install a C compiler, and for SCS specifically, also BLAS/LAPACK development libraries. In Debian-based Linux distros that should be sudo apt install gcc libopenblas-dev.

EDIT: Comment was confusing, removed most of it.

fahadak69 · Answer 3 · Tue Jan 30 2024 20:18:45 GMT+0800 (China Standard Time)

i was able to install cvxpy-base. But I need SCS solver. What is the instruction to just install solver on top of cvxpy-base using pip?

I will try to get lappack/blas install. It is quite a procees since apt install is not recognized as valid command on embedded linux imag. I will need to embed thesr libraries before linux image compilation (which has its own issues

Thanks alot

Riley Murray · Answer 4 · Tue Jan 30 2024 21:52:40 GMT+0800 (China Standard Time)

@enzbus what do you mean when you say that SCS is the most advanced solver among the CVXPY defaults?

…

On Tue, Jan 30, 2024 at 2:33 AM enzbus ***@***.***> wrote: SCS is the most advanced solver among the CVXPY default solvers' suite, and it requires some more libraries to compile. There are pre-built binaries for most architectures but arm linux is not one of those. What you need is a BLAS/LAPACK development library. I'm not sure about that Linux distribution, but in Debian-based that is usually done by sudo apt install libopenblas-dev. @rileyjmurray <https://github.com/rileyjmurray> maybe this should be mentioned in the installation instructions? PS this is the same issue as cvxgrp/scs#258 <cvxgrp/scs#258> — Reply to this email directly, view it on GitHub <#2337 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACRLIFEOQ6XML7MXXZIFFX3YRCO4JAVCNFSM6AAAAABCOW3Y2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJWGIZTIMJRGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

enzbus · Answer 5 · Tue Jan 30 2024 21:59:25 GMT+0800 (China Standard Time)

Nothing @rileyjmurray, only that it's complicated to ship binaries for it because of its dependence on BLAS, which it needs to solve SDPs. Edited my comment.

Steven Diamond · Answer 6 · Tue Jan 30 2024 22:20:43 GMT+0800 (China Standard Time)

i was able to install cvxpy-base. But I need SCS solver. What is the instruction to just install solver on top of cvxpy-base using pip?

I will try to get lappack/blas install. It is quite a procees since apt install is not recognized as valid command on embedded linux imag. I will need to embed thesr libraries before linux image compilation (which has its own issues

Thanks alot

Could you say more about why you need SCS? The Clarabel solver also solves SDPs.

fahadak69 · Answer 7 · Tue Jan 30 2024 22:29:23 GMT+0800 (China Standard Time)

i was able to install cvxpy-base. But I need SCS solver. What is the instruction to just install solver on top of cvxpy-base using pip?
I will try to get lappack/blas install. It is quite a procees since apt install is not recognized as valid command on embedded linux imag. I will need to embed thesr libraries before linux image compilation (which has its own issues
Thanks alot

Could you say more about why you need SCS? The Clarabel solver also solves SDPs.

i can try that. is it part of cvxpy-base package? if not, how can i install it?

fahadak69 · Answer 8 · Sun Feb 04 2024 21:36:53 GMT+0800 (China Standard Time)

i was able to install cvxpy-base. But I need SCS solver. What is the instruction to just install solver on top of cvxpy-base using pip?
I will try to get lappack/blas install. It is quite a procees since apt install is not recognized as valid command on embedded linux imag. I will need to embed thesr libraries before linux image compilation (which has its own issues
Thanks alot

Could you say more about why you need SCS? The Clarabel solver also solves SDPs.

i can try that. is it part of cvxpy-base package? if not, how can i install it?

Philipp Schiele · Answer 9 · Mon Feb 05 2024 01:56:46 GMT+0800 (China Standard Time)

i was able to install cvxpy-base. But I need SCS solver. What is the instruction to just install solver on top of cvxpy-base using pip?
I will try to get lappack/blas install. It is quite a procees since apt install is not recognized as valid command on embedded linux imag. I will need to embed thesr libraries before linux image compilation (which has its own issues
Thanks alot

Could you say more about why you need SCS? The Clarabel solver also solves SDPs.

i can try that. is it part of cvxpy-base package? if not, how can i install it?

You can simply run:

python3 -m pip install cvxpy-base clarabel

cvxpy-base simply excludes all solvers that usually come by default, so you can install one that suits your needs explicitly (in my example: clarabel).

fahadak69 · Answer 10 · Sun Feb 11 2024 01:22:42 GMT+0800 (China Standard Time)

i was able to run clarabel on my board, thank you everyone

Victor E Ottati A · Answer 11 · Thu Feb 15 2024 22:58:47 GMT+0800 (China Standard Time)

It seems you need to install a C compiler, and for SCS specifically, also BLAS/LAPACK development libraries. In Debian-based Linux distros that should be sudo apt install gcc libopenblas-dev.

EDIT: Comment was confusing, removed most of it.

I had a similar problem installing cvxpy inside of a container on a rpi4/8GB running balenalib/*:ubuntu-python:3.10-jammy-run. I'm not an expert, do we need documentation on the possible need to install gcc, libopenblas-dev on aarch64 devices?