Giters

google-deepmind / reverb

Reverb is an efficient and easy-to-use data storage and transport system designed for machine learning research

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Installation documentation (manual) for NVIDIA Jetson

markub3327 opened this issue · comments

commented

Hi,

can you create the official installation manual for NVIDIA Jetson. The instructions are:

  1. Install dependences

    Tensorflow for JetPack, follow instructions here for installation.

  2. Install Reverb

    Download Bazel 3.7.2 for arm64, here

     mkdir ~/bin
 mv ~/Downloads/bazel-3.7.2-linux-arm64 ~/bin/bazel
 chmod +x ~/bin/bazel
 export PATH=$PATH:~/bin

    Clone Reverb with version that corespond with TF verion installed on NVIDIA Jetson !

    git clone https://github.com/deepmind/reverb
cd reverb/
git checkout r0.8.0

    Make changes in Reverb before building !

    In .bazelrc

    - build:manylinux2010 --crosstool_top=//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010:toolchain
+ # build:manylinux2010 --crosstool_top=//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010:toolchain

- build --copt=-mavx --copt=-DEIGEN_MAX_ALIGN_BYTES=64
+ build --copt=-DEIGEN_MAX_ALIGN_BYTES=64

    In WORKSPACE

    - PROTOC_SHA256 = "15e395b648a1a6dda8fd66868824a396e9d3e89bc2c8648e3b9ab9801bea5d55"
+ # PROTOC_SHA256 = "15e395b648a1a6dda8fd66868824a396e9d3e89bc2c8648e3b9ab9801bea5d55"
+ PROTOC_SHA256 = "7877fee5793c3aafd704e290230de9348d24e8612036f1d784c8863bc790082e"

    In oss_build.sh

    -  bazel test -c opt --copt=-mavx --config=manylinux2010 --test_output=errors //reverb/cc/...
+  bazel test -c opt --copt="-march=armv8-a+crypto" --test_output=errors //reverb/cc/...

# Builds Reverb and creates the wheel package.
-  bazel build -c opt --copt=-mavx $EXTRA_OPT --config=manylinux2010 reverb/pip_package:build_pip_package
+  bazel build -c opt --copt="-march=armv8-a+crypto" $EXTRA_OPT reverb/pip_package:build_pip_package

    In reverb/cc/platform/default/repo.bzl

    urls = [
   -        "https://github.com/protocolbuffers/protobuf/releases/download/v%s/protoc-%s-linux-x86_64.zip" % (version, version),
   +        "https://github.com/protocolbuffers/protobuf/releases/download/v%s/protoc-%s-linux-aarch_64.zip" % (version, version),
]

    In reverb/pip_package/build_pip_package.sh

    -  "${PYTHON_BIN_PATH}" setup.py bdist_wheel ${PKG_NAME_FLAG} ${RELEASE_FLAG} ${TF_VERSION_FLAG} --plat manylinux2010_x86_64 > /dev/null
+  "${PYTHON_BIN_PATH}" setup.py bdist_wheel ${PKG_NAME_FLAG} ${RELEASE_FLAG} ${TF_VERSION_FLAG}  > /dev/null

    Build and install

    bash oss_build.sh --clean true --tf_dep_override "tensorflow=2.9.1" --release --python "3.8"
bash ./bazel-bin/reverb/pip_package/build_pip_package --dst /tmp/reverb/dist/ --release
pip3 install /tmp/reverb/dist/dm_reverb-*

    Cleaning

    cd ../
rm -R reverb/

Or to simplify this process for newbie users you can create a new branch with these changes, named rX.X.X-Jetson.

commented

Hi,

Thank you so much for this contribution but I'm afraid I'm going to have to decline making this part of the core documentation. We don't have the manpower (or hardware) to ensure that Reverb continues to be compatible with Jetson so even if this became part of the documentation it would be doomed to become obsolete in the future.

Hopefully this issue can serve as reference to other users if they run into the same problem.