mozilla / DeepSpeech

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error in @com_google_absl when compiling from source with gcc 11

WorldSEnder opened this issue · comments

Trying to compile from source (including the linked tensorflow) with gcc 11, I get the following error:

ERROR: /home/martin/.cache/bazel/_bazel_martin/ac764ffa28199e46c3d3f694acbbfd90/external/com_google_absl/absl/synchronization/BUILD.bazel:30:11: C++ compilation of rule '@com_google_absl//absl/synchronization:graphcycles_internal' failed (Exit 1): clang failed: error executing command /usr/bin/clang -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 42 argument(s) skipped)
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:26: error: no member named 'numeric_limits' in namespace 'std'
  if (x->version == std::numeric_limits<uint32_t>::max()) {
                    ~~~~~^
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:41: error: unexpected type name 'uint32_t': expected expression
  if (x->version == std::numeric_limits<uint32_t>::max()) {
                                        ^
external/com_google_absl/absl/synchronization/internal/graphcycles.cc:451:52: error: no member named 'max' in the global namespace
  if (x->version == std::numeric_limits<uint32_t>::max()) {
                                                 ~~^
3 errors generated.
Target //native_client:libdeepspeech.so failed to build

Reproduction (on arch linux):

pkgbase=deepspeech
pkgname=('deepspeech' 'python-deepspeech')
_pkgname=DeepSpeech
pkgver=0.9.3
pkgrel=1
pkgdesc="A TensorFlow implementation of Baidu's DeepSpeech architecture"
arch=('x86_64')
url="https://github.com/mozilla/DeepSpeech"
license=('MPL2')
makedepends=('bazel<4' 'python-numpy' 'python-pip' 'python-wheel' 'git' 'sox' 'wget')
source=("${_pkgname}-${pkgver}::git+https://github.com/mozilla/DeepSpeech.git#tag=v${pkgver}")
sha256sums=('SKIP')

prepare()
{
  cd "$srcdir/${_pkgname}-${pkgver}"
  git submodule sync tensorflow/
  git submodule update --init tensorflow/
  cd tensorflow
  ln -sf ../native_client
  cd "$srcdir"
  # fix for issue with mainline python setuptools
  # see also https://github.com/psf/black/issues/1933
  # check back with python-pip for updates (currently flagged outdated) 
  python -m venv .env
  . .env/bin/activate
  python -m pip install --upgrade pip

}

build() {
  cd "$srcdir/${_pkgname}-${pkgver}/tensorflow"
  export USE_DEFAULT_PYTHON_LIB_PATH=1
  export TF_NEED_OPENCL_SYCL=0
  export TF_ENABLE_XLA=1
  export TF_NEED_MPI=0
  export TF_SET_ANDROID_WORKSPACE=0
  export TF_NEED_ROCM=0
  export TF_DOWNLOAD_CLANG=0
  export TF_NEED_CUDA=0
  export CC_OPT_FLAGS="-march=x86-64"
  # work around https://bugs.archlinux.org/task/70104?project=1&string=python-pip
  export NUMPY_BUILD_VERSION="==1.19.4"
  export NUMPY_DEP_VERSION=">=1.19.4"
  # use virtual env python
  export PYTHON_BIN_PATH=$(realpath "$srcdir/.env/bin/python")
  export BAZEL_SH=/bin/bash
  ./configure

  bazel build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --config=monolithic -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --copt=-fvisibility=hidden //native_client:libdeepspeech.so

  cd "${srcdir}/${_pkgname}-${pkgver}/native_client"
  make deepspeech SOX_LDFLAGS="-lsox -Wl,-no-undefined"
  make bindings -C python
  make bindings -C ctcdecode
}

package_deepspeech() {
  depends=('sox')
  cd "${srcdir}/${_pkgname}-${pkgver}/native_client"
  PREFIX="${pkgdir}"/usr make install
}

package_python-deepspeech() {
  pkgdesc="DeepSpeech Python bindings"
  depends=('deepspeech' 'python-numpy')
  cd "${srcdir}/${_pkgname}-${pkgver}/native_client"
  PIP_CONFIG_FILE=/dev/null pip install --isolated --root="$pkgdir" --ignore-installed --no-deps python/dist/deepspeech*.whl
  PIP_CONFIG_FILE=/dev/null pip install --isolated --root="$pkgdir" --ignore-installed --no-deps ctcdecode/dist/ds_ctcdecoder*.whl
  mv "$pkgdir/usr/bin/deepspeech" "$pkgdir/usr/bin/deepspeech_python"
  cp -rv "${srcdir}/${_pkgname}-${pkgver}/training/deepspeech_training" "$pkgdir"`python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"`
}

Relevant tool versions:

> bazel --version
bazel 3.7.2
> gcc --version
gcc (GCC) 11.1.0
> src/.env/bin/python --version
Python 3.9.5

Most likely the same issue as described in grpc/grpc#25114, i.e. caused by an outdated abseil-cpp used in the linked tensorflow fork. Since it isn't immediately clear to me what the purpose of https://github.com/mozilla/tensorflow is, I have not yet tried compiling with a different version of tensorflow.

gcc 11

I dont think this is supported by TensorFlow r2.3: https://github.com/mozilla/tensorflow/tree/r2.3

Since it isn't immediately clear to me what the purpose of https://github.com/mozilla/tensorflow is, I have not yet tried compiling with a different version of tensorflow.

As documented, this fork contains specific changes required to either fix build issues similar to yours, or fix upstream issues that have not yet been upstreamed, or specific hacking/tuning we require. Do not try to attempt to build with another version.

Thanks for clearing that up. Using gcc-8, i.e. adding

--- a/PKGBUILD
+++ b/PKGBUILD
@@ -35,2 +37,3 @@ export BAZEL_SH=/bin/bash
+  export CC="gcc-8"
   ./configure

seems to work fine for now.

Thanks for clearing that up. Using gcc-8, i.e. adding

--- a/PKGBUILD
+++ b/PKGBUILD
@@ -35,2 +37,3 @@ export BAZEL_SH=/bin/bash
+  export CC="gcc-8"
   ./configure

seems to work fine for now.

I'm assuming this means it is enough for you, closing the issue then.

If someone wants to send patches to support GCC-11 on the tensorflow side, it is welcome to file a new issue.