Prebuilt binaries do not work with CPUs that do not have AVX instruction sets.
gunan opened this issue · comments
As announced in release notes, TensorFlow release binaries version 1.6 and higher are prebuilt with AVX instruction sets. This means on any CPU that do not have these instruction sets either CPU or GPU version of TF will fail to load with any of the following errors:
ImportError: DLL load failed:- A crash with return code 132
Our recommendation is to build TF from sources on these systems.
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu/windows/macos
- TensorFlow installed from (source or binary): binary
- TensorFlow version (use command below): 1.6 and up
- Python version: 2.7, 3.3, 3.4, 3.5, 3.6 and any newer
- Bazel version (if compiling from source): n/a
- GCC/Compiler version (if compiling from source): n/a
- CUDA/cuDNN version: any
- GPU model and memory: any
- Exact command to reproduce: python -c "import tensorflow as tf"
We encourage the community to build and share binaries for older CPU models.
Any chance for you to make the build for the community?
FYI, here is a Docker image that can build TensorFlow https://github.com/hadim/docker-tensorflow-builder. It can help to compile TF on a wide range of configurations as long as you have Docker installed on it.
We encourage community supported wheels because for the officially blessed binaries we would like to run rigorous tests.
For just building, we encourage the community to build and share.
@gunan Will this change with tensorflow/community#2?
In the short term that involves this immediate design, no.
However this design is a part of a larger effort that will bring back support for older CPUs.
That will likely take longer to implement.
CMake version on the other side on windows does not have AVX at all. I needed to enable it explictly locally.
I've gone ahead and compiled GPU only builds for Tensorflow v1.10.1 against CUDA 9.1 + cuDNN 7.1, and CUDA 9.2 + cuDNN 7.2 and made the wheels and build info available here.
Huge thanks to all of the contributors and the supporting open source community.
Special thanks to @hadim for making his docker-tensorflow-builder available - it served as a great basis to generate these wheels. 🎉
Is there any way to detect this and print an appropriate error message?
We do have some code for this, but unfortunately when compiled with AVX, the machine code generated uses AVX instructions. We are looking into alternatives.
Well, why just you do not build a matrix of combinations?
AVX -> AVX2 -> GPU -> CPU Only
If the builds take too much time, prepare docker images which are ready for building and people will build them themselves.
@kingofthebongo2008 please see #19584 (comment) above
I've built tf 1.13 for CPU without any fancy instructions here:
Python 2.7: yaroslavvb/tensorflow-community-wheels#97
Python 3.6: yaroslavvb/tensorflow-community-wheels#103
Hope it helps someone
I have the same issue. Tried to build binary with bazel but failed. Try this "pip install tensorflow==1.5". It works for me.
Here's a TensorFlow 1.13.1 binary package for Westmere CPUs (no AVX) and Python 2.7: yaroslavvb/tensorflow-community-wheels#105
Any compilation for Linux (no AVX) and Python 3.5 (Ubuntu 16.04)?
Are there plans to release binaries without AVX instructions set for Windows?
Do you mean without AVX?
Our prebuilt binaries should already be built with AVX.
Yes, many people complain about not being able to use latest versions of tensorflow because their CPUs don't support AVX. I created an issue on Anaconda github to ask them if they could provide the binaries that are suitable to the user's CPU.
We have no plans to build and release non-AVX binaries at the moment.
Too bad, that's prejudicial for many users.
I have the same problem under Debian 10 Buster, Python v2.7, tensorflow v1.14.0:
lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
python -c 'import tensorflow'
Illegal instruction
Error log:
Nov 4 15:59:00 moodle37 kernel: [9768885.859206] traps: python[2419] trap invalid opcode ip:7f2fc9fbba59 sp:7ffd402810b0 error:0 in libtensorflow_framework.so.1[7f2fc9896000+18f8000]
sudo lshw -class cpu
*-cpu
description: CPU
product: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
vendor: Intel Corp.
physical id: 4
bus info: cpu@0
version: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
slot: CPU #000
size: 2100MHz
capacity: 4230MHz
width: 64 bits
capabilities: lm fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp x86-64 constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc cpuid pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer hypervisor lahf_lm pti ssbd ibrs ibpb stibp tsc_adjust arat flush_l1d arch_capabilities
configuration: cores=1 enabledcores=1
python -V
Python 2.7.16
pip list
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Ple ase upgrade your Python as Python 2.7 won't be maintained after that date. A fut ure version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/relea se-process/#python-2-support
Package Version
-------------------- ---------
absl-py 0.8.1
asn1crypto 0.24.0
astor 0.8.0
backports.weakref 1.0.post1
bzr-etckeeper 0.0.0
configparser 3.5.0b2
cryptography 2.6.1
cycler 0.10.0
entrypoints 0.3
enum34 1.1.6
funcsigs 1.0.2
future 0.17.1
futures 3.3.0
gast 0.3.2
google-pasta 0.1.7
grpcio 1.24.3
h5py 2.10.0
ipaddress 1.0.17
Keras-Applications 1.0.6
Keras-Preprocessing 1.1.0
keyring 17.1.1
keyrings.alt 3.1.1
Markdown 3.1.1
matplotlib 1.5.3
mock 3.0.5
moodlemlbackend 1.0.1
numpy 1.16.5
pip 19.3.1
protobuf 3.9.1
pycrypto 2.6.1
PyGObject 3.30.4
pyparsing 2.4.2
python-dateutil 2.8.0
pytz 2019.2
pyxdg 0.25
scikit-learn 0.20.4
scipy 0.17.1
SecretStorage 2.3.1
setuptools 41.2.0
six 1.12.0
tensorboard 1.14.0
tensorflow 1.14.0
tensorflow-estimator 1.14.0
termcolor 1.1.0
Werkzeug 0.16.0
wheel 0.33.6
wrapt 1.11.2
hi! can i built tensorflow 2.0 without AVX for windows with docker-tensorflow-builder ?
@s-afanasiev for windows you can find wheel here https://github.com/fo40225/tensorflow-windows-wheel
Check this repo for more unofficial wheels: https://github.com/yaroslavvb/tensorflow-community-wheels/issues
I found the right one for our server. YAY!
Something really strange happened to me regarding this issue: I have an i5-3230M CPU, and I was using tensorflow 2 without any problem until yesterday, that I decided to reinstall ubuntu in the machine. As a result, now pip-installed tensorflow cannot be imported. Same machine, same CPU.
Does anyone know where we can track progress on the effort to support legacy CPUs mentioned earlier in this thread?