Mac failed to build tensorboard (Duplicated key "define" when creating dictionary)

Question

Mac failed to build tensorboard (Duplicated key "define" when creating dictionary)

BwCai opened this issue 7 years ago · comments

I used Anaconda as my python environment and simply followed the instruction 'Prepare environment for Mac OS X'
Then, I met error as follows

File "/Users/bowen/workspace/tensorboard-distro/tensorboard-build/tensorflow/tensorflow/core/kernels/BUILD", line 53
		config_setting(name = "xsmm_backward", values = {..."})
	File "/Users/bowen/workspace/tensorboard-distro/tensorboard-build/tensorflow/tensorflow/core/kernels/BUILD", line 58, in config_setting
		{"define": "tensorflow_xsmm=1", "define": "tensorflow_xsmm_backward=1"}
Duplicated key "define" when creating dictionary

My system and other library is listed:
System: macOs Sierra 10.12.6
bazel 0.6.1;
protobuf 3.4.0

I wonder whether there is any problem on my tensorflow or bazel version.
Thank you for your time to read my experience.

Zihao Zheng · Answer 1 · Tue Nov 07 2017 10:32:01 GMT+0800 (China Standard Time)

I notice your working dir is tensorboard-distro, not this repo.

Zihao Zheng · Answer 2 · Tue Nov 07 2017 10:34:47 GMT+0800 (China Standard Time)

Try this:

git clone --recursive https://github.com/dmlc/tensorboard.git

#cd tensorboard
bash installer.sh

BwCai · Answer 3 · Tue Nov 07 2017 11:10:33 GMT+0800 (China Standard Time)

Actually, I first try this repo https://github.com/dmlc/tensorboard.git, but failed with same error. Then, I used tensorboard-distro.
errors using repo dmlc/tensorboard, as follows:

ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/BUILD:86:1: Traceback (most recent call last):
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/BUILD", line 86
		tf_py_logged_benchmark(name = "rnn_op_benchmark", target ...")
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 52, in tf_py_logged_benchmark
		tf_cc_logged_benchmark(name = name, target = target, benchm..., <2 more arguments>)
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in tf_cc_logged_benchmark
		list((set(tags) + set(["benchmark-tes..."])))
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in list
		set(tags)
The `set` constructor for depsets is deprecated and will be removed. Please use the `depset` constructor instead. You can temporarily enable the deprecated `set` constructor by passing the flag --incompatible_disallow_set_constructor=false.
ERROR: package contains errors: tensorflow/tools/test.
ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD:58:14: Traceback (most recent call last):
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD", line 53
		config_setting(name = "xsmm_backward", values = {..."})
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD", line 58, in config_setting
		{"define": "tensorflow_xsmm=1", "define": "tensorflow_xsmm_backward=1"}
Duplicated key "define" when creating dictionary.

I have tried to replace 'set' to 'depset', but I don't know how to tackle the error 'the duplicated key define' .

Zihao Zheng · Answer 4 · Tue Nov 07 2017 11:55:12 GMT+0800 (China Standard Time)

Could you provide more log for me? I've never met this before.

(tensorboard) MacBook-Pro:~ zihao.zzh$ bazel version
..................
Build label: 0.6.1-homebrew
Build target: bazel-out/darwin_x86_64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Oct 6 02:36:58 2017 (1507257418)
Build timestamp: 1507257418
Build timestamp as int: 1507257418

This is my bazel version.

BwCai · Answer 5 · Tue Nov 07 2017 13:03:53 GMT+0800 (China Standard Time)

Of course.
bazel version:

.........
Build label: 0.6.1
Build target: bazel-out/darwin_x86_64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Oct 5 21:54:05 2017 (1507240445)
Build timestamp: 1507240445
Build timestamp as int: 1507240445

And the log error:

Please specify the location of python. [Default is /usr/bin/python]: /Users/bowen/anaconda/bin/python
Please specify optimization flags to use during compilation [Default is -march=native]:
Do you wish to use jemalloc as the malloc implementation? (Linux only) [Y/n]
jemalloc enabled on Linux
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N]
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with Hadoop File System support? [y/N]
No Hadoop File System support will be enabled for TensorFlow
Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
No XLA support will be enabled for TensorFlow
Found possible Python library paths:
  /Users/bowen/anaconda/lib/python2.7/site-packages
Please input the desired Python library path to use.  Default is [/Users/bowen/anaconda/lib/python2.7/site-packages]

Using python library path: /Users/bowen/anaconda/lib/python2.7/site-packages
Do you wish to build TensorFlow with OpenCL support? [y/N]
No OpenCL support will be enabled for TensorFlow
Do you wish to build TensorFlow with CUDA support? [y/N]
No CUDA support will be enabled for TensorFlow
Configuration finished
................................................................
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
..........................................................
DEBUG: /private/var/tmp/_bazel_root/4379f75ba180e7b5a02b2aba75a31dac/external/bazel_tools/tools/build_defs/pkg/pkg.bzl:197:9: @//tensorflow/tools/lib_package:cheaders: you provided a non dictionary to the pkg_tar `files` attribute. This attribute was renamed to `srcs`. Consider renaming it in your BUILD file.
DEBUG: /private/var/tmp/_bazel_root/4379f75ba180e7b5a02b2aba75a31dac/external/bazel_tools/tools/build_defs/pkg/pkg.bzl:197:9: @//tensorflow/tools/lib_package:clib: you provided a non dictionary to the pkg_tar `files` attribute. This attribute was renamed to `srcs`. Consider renaming it in your BUILD file.
DEBUG: /private/var/tmp/_bazel_root/4379f75ba180e7b5a02b2aba75a31dac/external/bazel_tools/tools/build_defs/pkg/pkg.bzl:197:9: @//tensorflow/tools/lib_package:clicenses: you provided a non dictionary to the pkg_tar `files` attribute. This attribute was renamed to `srcs`. Consider renaming it in your BUILD file.
ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/BUILD:81:1: Traceback (most recent call last):
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/BUILD", line 81
		tf_cc_logged_benchmark(name = "cast_op_benchmark", target...")
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in tf_cc_logged_benchmark
		list((set(tags) + set(["benchmark-tes..."])))
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in list
		set(tags)
The `set` constructor for depsets is deprecated and will be removed. Please use the `depset` constructor instead. You can temporarily enable the deprecated `set` constructor by passing the flag --incompatible_disallow_set_constructor=false.
ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/BUILD:86:1: Traceback (most recent call last):
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/BUILD", line 86
		tf_py_logged_benchmark(name = "rnn_op_benchmark", target ...")
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 52, in tf_py_logged_benchmark
		tf_cc_logged_benchmark(name = name, target = target, benchm..., <2 more arguments>)
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in tf_cc_logged_benchmark
		list((set(tags) + set(["benchmark-tes..."])))
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/tools/test/performance.bzl", line 23, in list
		set(tags)
The `set` constructor for depsets is deprecated and will be removed. Please use the `depset` constructor instead. You can temporarily enable the deprecated `set` constructor by passing the flag --incompatible_disallow_set_constructor=false.
ERROR: package contains errors: tensorflow/tools/test.
ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD:58:14: Traceback (most recent call last):
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD", line 53
		config_setting(name = "xsmm_backward", values = {..."})
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD", line 58, in config_setting
		{"define": "tensorflow_xsmm=1", "define": "tensorflow_xsmm_backward=1"}
Duplicated key "define" when creating dictionary.
ERROR: package contains errors: tensorflow/core/kernels.
ERROR: error loading package 'tensorflow/tools/test': Package 'tensorflow/tools/test' contains errors.
ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD:58:14: Traceback (most recent call last):
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD", line 53
		config_setting(name = "xsmm_backward", values = {..."})
	File "/Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD", line 58, in config_setting
		{"define": "tensorflow_xsmm=1", "define": "tensorflow_xsmm_backward=1"}
Duplicated key "define" when creating dictionary.
ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/kernels/BUILD:359:1: Target '//tensorflow/core/kernels:bounds_check.h' contains an error and its package is in error and referenced by '//tensorflow/core/kernels:bounds_check'.
ERROR: /Users/bowen/workspace/tensorboard/tensorflow/tensorflow/core/BUILD:1235:1: Target '//tensorflow/core/kernels:bounds_check' contains an error and its package is in error and referenced by '//tensorflow/core:framework_internal'.
ERROR: Analysis of target '//tensorflow/tensorboard:tensorboard' failed; build aborted.
INFO: Elapsed time: 73.888s
cp: bazel-bin/tensorflow/tools: No such file or directory
cp: ../tools/pip_package: unable to copy extended attributes to bazel-bin/tensorflow/tools: No such file or directory
cp: bazel-bin/tensorflow/tools/build_pip_package.sh: No such file or directory
cp: bazel-bin/tensorflow/tools/MANIFEST.in: No such file or directory
cp: bazel-bin/tensorflow/tools/README: No such file or directory
bash: bazel-bin/tensorflow/tools/pip_package/build_pip_package.sh: No such file or directory
./installer.sh: line 23: pip: command not found
Now you can remove tensorflow with rm -rf tensorflow

Thanks for your help.

Zihao Zheng · Answer 6 · Tue Nov 07 2017 14:37:48 GMT+0800 (China Standard Time)

@bowcai Did you use --recursive when cloning the code?

The format of configuration looks different with mine.

Zihao Zheng · Answer 7 · Tue Nov 07 2017 14:40:10 GMT+0800 (China Standard Time)

It should be

Found possible Python library paths:
  /home/zihao.zzh/anaconda2/envs/tensorboard/lib/python2.7/site-packages
Please input the desired Python library path to use.  Default is [/home/zihao.zzh/anaconda2/envs/tensorboard/lib/python2.7/site-packages]

Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]: n
No jemalloc as malloc support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: n
No Google Cloud Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: n
No Hadoop File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [y/N]: n
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GDR support? [y/N]: n
No GDR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with VERBS support? [y/N]: n
No VERBS support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL support? [y/N]: n
No OpenCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: n
No CUDA support will be enabled for TensorFlow.

Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.

BwCai · Answer 8 · Tue Nov 07 2017 16:34:20 GMT+0800 (China Standard Time)

My fault. It seems that I used an out-of-date version.
Here is my installer.sh

#!/usr/bin/env bash

# make protobufs for logging part first
make all

# get tensorflow
git clone https://github.com/tensorflow/tensorflow
cd tensorflow
git checkout -b v1.0.0-rc1 v1.0.0-rc1

# run configuration.
bash configure
# build tensorboard
bazel build tensorflow/tensorboard:tensorboard

# prepare pip installation package
cp -r ../tools/* bazel-bin/tensorflow/tools/
# get .whl file in python/dist/
bash bazel-bin/tensorflow/tools/pip_package/build_pip_package.sh ../python/dist/

# install tensorboard package from .whl file
cd ..
pip install python/dist/*.whl

# clean up
echo 'Now you can remove tensorflow with rm -rf tensorflow'
#rm -rf tensorflow/

I will to try to redo git clone --recursive , thank you

Zihao Zheng · Answer 9 · Tue Nov 07 2017 17:32:16 GMT+0800 (China Standard Time)

Yep. As we're going to have a major update recently, we changed the backend and added more features.

BwCai · Answer 10 · Tue Nov 07 2017 21:01:28 GMT+0800 (China Standard Time)

I get it. Thanks for your help.