Cannot build docker/dev.dockerfile (due to not finding tf-nightly)

[Zsolt Safrany]

Repro:

Run docker build --tag tensorflow:reverb - < "$REVERB_DIR/docker/dev.dockerfile"

(on macOS 12.0.1)

Error:

#10 1.162 ERROR: Could not find a version that satisfies the requirement tf-nightly (from versions: none)
#10 1.162 ERROR: No matching distribution found for tf-nightly

Whole output:

[+] Building 2.3s (10/16)
 => [internal] load build definition from Dockerfile                                                                                                                                                                                   0.0s
 => => transferring dockerfile: 3.36kB                                                                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/ubuntu:18.04                                                                                                                                                                        0.6s
 => [ 1/13] FROM docker.io/library/ubuntu:18.04@sha256:0fedbd5bd9fb72089c7bbca476949e10593cebed9b1fb9edf5b79dbbacddd7d6                                                                                                                0.0s
 => CACHED [ 2/13] RUN apt-get -o Acquire::Retries=3 -y update && apt-get -o Acquire::Retries=3 -y install -y --no-install-recommends         software-properties-common         aria2         build-essential         curl         g  0.0s
 => CACHED [ 3/13] RUN curl -O https://bootstrap.pypa.io/get-pip.py                                                                                                                                                                    0.0s
 => CACHED [ 4/13] RUN mkdir /bazel &&     cd /bazel &&     curl -fSsL -O https://github.com/bazelbuild/bazel/releases/download/3.7.2/bazel-3.7.2-installer-linux-x86_64.sh &&     chmod +x bazel-*.sh &&     ./bazel-3.7.2-installer  0.0s
 => CACHED [ 5/13] RUN cp /usr/local/lib/bazel/bin/bazel-complete.bash /etc/bash_completion.d                                                                                                                                          0.0s
 => CACHED [ 6/13] RUN python3.7 get-pip.py                                                                                                                                                                                            0.0s
 => ERROR [ 7/13] RUN python3.7 -mpip --no-cache-dir install tf-nightly --upgrade                                                                                                                                                      1.6s
------
 > [ 7/13] RUN python3.7 -mpip --no-cache-dir install tf-nightly --upgrade:
#10 1.162 ERROR: Could not find a version that satisfies the requirement tf-nightly (from versions: none)
#10 1.162 ERROR: No matching distribution found for tf-nightly

[Toby Boyd]

I dislike giving this answer because if I find an error then I can fix something and be confident I am helping. Here is what I ran and maybe something unlucky happened or there is a cached layer (a very weak guess because I did not see anything obvious) that needed updated. Try running the command I ran below which will do a pull and no-cache on the build. That is all I can think of.

From my debian workstation I ran:

docker build --no-cache --pull --tag tensorflow:reverb - < "$REVERB_DIR/docker/dev.dockerfile"

I then went into the docker to do a second test because I wanted to find something wrong I could fix:

$  python3.7 -mpip install --force-reinstall tf-nightly

# Result
root@98dd10f5c7c1:/tmp/reverb# python3.7 -mpip install --force-reinstall tf-nightly
Collecting tf-nightly
  Downloading tf_nightly-2.8.0.dev20211221-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (492.4 MB)
`

[Zsolt Safrany]

Thank you for trying to help!

It still doesn't work but I think I know why.

If I run pip debug --verbose inside docker then I get this list:

#10 0.265 Compatible tags: 231
#10 0.265   cp37-cp37m-manylinux_2_27_aarch64
#10 0.265   cp37-cp37m-manylinux_2_26_aarch64
#10 0.265   cp37-cp37m-manylinux_2_25_aarch64
#10 0.265   cp37-cp37m-manylinux_2_24_aarch64
#10 0.265   cp37-cp37m-manylinux_2_23_aarch64
#10 0.265   cp37-cp37m-manylinux_2_22_aarch64
#10 0.265   cp37-cp37m-manylinux_2_21_aarch64
#10 0.265   cp37-cp37m-manylinux_2_20_aarch64
#10 0.265   cp37-cp37m-manylinux_2_19_aarch64
#10 0.265   cp37-cp37m-manylinux_2_18_aarch64
#10 0.265   cp37-cp37m-manylinux_2_17_aarch64
#10 0.265   cp37-cp37m-manylinux2014_aarch64
#10 0.265   cp37-cp37m-linux_aarch64
[...]

Full output:
https://gist.github.com/ZsoltSafrany/78323ebb985abcb156e02a69f78c4a0f

Every tag contains aarch64 which makes sense because my host machine is an Apple M1.

The problem is tensorflow only supports x86_64:

I think this means we cannot use Reverb on macOS not even with Docker if it doesn't have an Intel chip. If I google tensorflow aarch64 I can find some arm compilations of TF so maybe one of these should work.

Anyway, I think we can close the issue at least. Thank you!

[Zsolt Safrany]

TF issue for adding aarch64 wheels
tensorflow/tensorflow#52973

[Toby Boyd]

@ZsoltSafrany I totally forgot Docker doesn't emulate the underlying hardware nor did I think about the M1. Thanks for giving the M1 build a +1 on TF.