Giters

buildbuddy-io / buildbuddy

BuildBuddy is an open source Bazel build event viewer, result store, remote cache, and remote build execution platform.

Home Page:https://buildbuddy.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[CLI] bb CLI very slow/does not run if using env -i in Bazel wrapper

erikkerber opened this issue · comments

commented

Describe the bug

Using a minimal Bazel wrapper (tools/bazel), invoking bazel/bb is extremely slow (~30 seconds to minutes). It typically errors out with:

could not run Bazel: could not start Bazel: fork/exec /Users/ekerber/Developer/slack/slack-objc/tools/bazel: resource temporarily unavailable

To Reproduce

Steps to reproduce the behavior:

  1. Install the latest bb CLI via Bazelisk
  2. Use a minimal Bazel wrapper that wraps invocations in env -i
#!/bin/bash

SANITIZED_ENVS=(
  PATH=/usr/bin:/bin
  HOME="$HOME"
)

env -i \
  "${SANITIZED_ENVS[@]}" \
  "${BAZEL_REAL}" "$@"

If I comment out env -i and "${SANITIZED_ENVS[@]}", the CLi behaves as expected.

Expected behavior

Typing bazel results in showing help for Bazel and bb in milliseconds.

commented

My current guess is between bb, bazelisk and tools/bazel, there is a cyclic dependency somewhere.

 | | \-+= 57886 sluongng bb build --config=x
 | |   |--= 57903 sluongng bb --bes_backend=grpcs://buildbuddy.buildbuddy.io --cache.zstd_transcoding_enabled=true --build_event_proxy.buffer_size=500000 --li
 | |   |--= 57905 sluongng /opt/homebrew/Cellar/python@3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python ./hand
 | |   \-+- 57908 sluongng /bin/bash /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel --digest_function=BLAKE3 --ignore_all_rc_files build --curses=yes
 | |     \-+- 57909 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --digest_function=BLAKE3 --ig
 | |       |--= 57926 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --bes_backend=unix:///var/f
 | |       |--= 57927 sluongng /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Pytho
 | |       \-+- 57928 sluongng /bin/bash /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel --digest_function=BLAKE3 --ignore_all_rc_files build --curses=
 | |         \-+- 57930 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --digest_function=BLAKE3
 | |           |--= 57947 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --bes_backend=unix:///t
 | |           |--= 57948 sluongng /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/P
 | |           \-+- 57949 sluongng /bin/bash /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel --digest_function=BLAKE3 --ignore_all_rc_files build --cur
 | |             \-+- 57951 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --digest_function=BLA
 | |               |--= 57968 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --bes_backend=unix:
 | |               |--= 58009 sluongng /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents/Mac
 | |               \-+- 58010 sluongng /bin/bash /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel --digest_function=BLAKE3 --ignore_all_rc_files build -
 | |                 \-+- 58012 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --digest_function
 | |                   |--= 58047 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --bes_backend=u
 | |                   |--= 58067 sluongng /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Contents
 | |                   \-+- 58068 sluongng /bin/bash /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel --digest_function=BLAKE3 --ignore_all_rc_files bui
 | |                     \-+- 58070 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --digest_func
 | |                       |--= 58087 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --bes_backe
 | |                       |--= 58089 sluongng /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/Cont
 | |                       \-+- 58090 sluongng /bin/bash /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel --digest_function=BLAKE3 --ignore_all_rc_files
 | |                         \-+- 58092 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --digest_
 | |                           |--= 58109 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --bes_b
 | |                           |--= 58114 sluongng /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/Resources/Python.app/
 | |                           \-+- 58115 sluongng /bin/bash /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel --digest_function=BLAKE3 --ignore_all_rc_f
 | |                             \-+- 58117 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --dig
 | |                               \--= 58134 sluongng /Users/sluongng/Library/Caches/bazelisk/downloads/buildbuddy-io/bazel-5.0.22-darwin-arm64/bin/bazel --b

I can reproduce it in our repo with these

!?master ~/work/buildbuddy/buildbuddy> cat tools/bazel
#!/bin/bash

SANITIZED_ENVS=(
  PATH=/usr/bin:/bin
  HOME="$HOME"
)

env -i \
  "${SANITIZED_ENVS[@]}" \
  "${BAZEL_REAL}" "$@"
!?master ~/work/buildbuddy/buildbuddy> cat .bazelversion
buildbuddy-io/5.0.22
6.4.0rc1

Eventually the cyclic call ended with could not run Bazel: could not start Bazel: fork/exec /Users/sluongng/work/buildbuddy/buildbuddy/tools/bazel: argument list too long

commented

Adding BAZELISK_SKIP_WRAPPER into the sanitized envs seems to solve the issue on my end

SANITIZED_ENVS=(
  PATH=/usr/bin:/bin
  HOME="$HOME"
  BAZELISK_SKIP_WRAPPER="$BAZELISK_SKIP_WRAPPER"
)

See https://github.com/bazelbuild/bazelisk/?tab=readme-ov-file#ensuring-that-your-developers-use-bazelisk-rather-than-bazel for more info.

commented

Yeah, I forgot about that. Here is the rules_xcodeproj change for the same reason: MobileNativeFoundation/rules_xcodeproj@3c6c14e

commented

Late here (we had a maelstrom of team offsites) but this works great. Thanks @sluongng