grpcio fails to install on Apple Silicon

Question

grpcio fails to install on Apple Silicon

yoav-orca opened this issue 3 years ago · comments

Yoav Alon commented 3 years ago

What version of gRPC and what language are you using?

grpcio-1.34.0

What operating system (Linux, Windows,...) and version?

macOS big Sur 11.0.1, running on M1 Apple Silicon

What runtime / compiler are you using (e.g. python version or version of gcc)

Python 3.8.7 (default, Dec 30 2020, 02:09:32)
[Clang 12.0.0 (clang-1200.0.32.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

$ clang --version
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: arm64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

What did you do?

Please provide either 1) A unit test for reproducing the bug or 2) Specific steps for us to follow to reproduce the bug. If there’s not enough information to debug the problem, gRPC team may close the issue at their discretion. You’re welcome to re-open the issue once you have a reproduction.

pip install grpcio==1.34.0

What did you expect to see?

That grpcio will be successfully installed

What did you see instead?

Collecting grpcio
  Using cached grpcio-1.34.0.tar.gz (21.0 MB)
    ERROR: Command errored out with exit status 1:
     command: /Users/yoav/Library/Caches/pypoetry/virtualenvs/orca-g6p4a6cZ-py3.8/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/setup.py'"'"'; __file__='"'"'/private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-pip-egg-info-z41yqiw6
         cwd: /private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/jj/r3j0p91s2qgdpcpf5ptqr9qm0000gn/T/pip-install-n05vw0u0/grpcio_103d947f24954cde8b8a019b4c9f88fe/setup.py", line 359, in <module>
        if mac_target and (pkg_resources.parse_version(mac_target) <
      File "/Users/yoav/Library/Caches/pypoetry/virtualenvs/orca-g6p4a6cZ-py3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 113, in parse_version
        return packaging.version.Version(v)
      File "/Users/yoav/Library/Caches/pypoetry/virtualenvs/orca-g6p4a6cZ-py3.8/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/version.py", line 275, in __init__
        match = self._regex.search(version)
    TypeError: expected string or bytes-like object
    ASM Builds for BoringSSL currently not supported on: macosx-11-arm64
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

See TROUBLESHOOTING.md for how to diagnose problems better.

Anything else we should know about your project / environment?

Jeppe Fihl-Pearson · Answer 1 · Tue Jan 05 2021 19:00:49 GMT+0800 (China Standard Time)

I managed to get the grpcio Python package installed on Python 3.9.1 installed via Pyenv on a M1 macOS:

Python 3.9.1 (default, Jan  5 2021, 10:39:36)
[Clang 12.0.0 (clang-1200.0.32.28)] on darwin

Native arm64:

$ lipo -archs $(which python)
arm64

I had to set a few environment variables to get it to work:

GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

to make the installation process use the libraries I've already got on the system, and then the following to point it at OpenSSL installed via Homebrew:

CFLAGS="-I/opt/homebrew/opt/openssl/include"
LDFLAGS="-L/opt/homebrew/opt/openssl/lib"

洛OvO · Answer 2 · Tue Jan 05 2021 23:46:35 GMT+0800 (China Standard Time)

you can use system python3 (/usr/bin/python3)

mkdir grpctest
cd grpctest
/usr/bin/python3 -m pip install virtualenv
/usr/bin/python3 -m virtualenv venv
source venv/bin/activate __
arch -x86_64 pip install --upgrade pip setuptools
arch -x86_64 pip install grpcio
arch -x86_64 pip install grpcio-tools
touch test.proto
...
arch -x86_64 python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. test.proto
touch main.py
arch -x86_64 python main.py

attention

arch -x86_64 should be used before the command

Lidi Zheng · Answer 3 · Wed Jan 06 2021 07:21:07 GMT+0800 (China Standard Time)

@yoav-orca We currently doesn't have ARM64 machines to produce binary wheels, but we are working on a plan.

Back to the build, the failure is not caused by the ASM optimization. Can you check the result of sysconfig.get_config_var('MACOSX_DEPLOYMENT_TARGET') on your Mac? It seems returning a number or some sort, which supposed to be a string (or bytes) of targeted macOS version.

EDIT: if none presented, you could try to set it to your macOS version, e.g. Big Sur is 11.0.

Jeppe Fihl-Pearson · Answer 4 · Thu Jan 14 2021 03:24:35 GMT+0800 (China Standard Time)

I just tried to install the library from the master branch with the fix included, but it still fails out of the box.

Is the version of BoringSSL and zlib referenced in this repository supposed to be working on ARM macOS? If not it might be better to try and detect when running on a ARM Mac (platform.machine() can do this) and set the equivalent of the following environment variables automatically:

GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

Does that sound reasonable? If so, I don't mind trying to put a PR together to do that.

Lidi Zheng · Answer 5 · Thu Jan 14 2021 03:33:22 GMT+0800 (China Standard Time)

@Tenzer That would be great if you can put up a PR, and you can ensure it works on the target machine (I don't have an Apple M1).

Joongi Kim · Answer 6 · Thu Jan 14 2021 13:24:41 GMT+0800 (China Standard Time)

I got a successful build using @Tenzer 's solution, with addition of CFLAGS and LDFLAGS using brew --prefix protobuf, brew --prefix openssl, brew --prefix zlib, and setting the GRPC_PYTHON_BUILD_SYSTEM_* environment variables, with arm64 homebrew and Python 3.9.1 built via pyenv's latest master.

Dylan Vanmali · Answer 7 · Sat Jan 16 2021 10:39:27 GMT+0800 (China Standard Time)

Thanks @Tenzer and @achimnol! This worked for me as well.

Note if the following issue occurs after those recommendations:

File "____/venv/lib/python3.9/site-packages/pkg_resources/_vendor/packaging/version.py", line 275, in __init__
    match = self._regex.search(version)
TypeError: expected string or bytes-like object

Edit version.py line 275 on your system to:

match = self._regex.search(str(version))

Alexander L · Answer 8 · Sat Feb 13 2021 03:11:12 GMT+0800 (China Standard Time)

Setting the environment variables as recommended by @Tenzer works perfectly for me, too! Thank you! I used it to install stable_baselines3 on my Apple Silicon M1 🙂

I simply set the environment variables first:

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

Then installed the grpcio package with:

pip install grpcio

At the time of writing this installs and works with grpcio-1.35.0

@lidizheng @Tenzer Since it seems like this problem is not yet resolved I would offer help creating a PR, however I am completely new to this kind of stuff (pip install, setuptools etc.) and don't really know where I would have to put the code. This is what I think would have to go somewhere according to your contributions:

import platform
import os
from distutils import util

if 'arm64' in platform.machine() and 'mac' in util.get_platform():
    os.environ["GRPC_PYTHON_BUILD_SYSTEM_OPENSSL"] = "1"
    os.environ["GRPC_PYTHON_BUILD_SYSTEM_ZLIB"] = "1"
else:
    pass

Maybe in setup.py around line 82? Of course the required import statements are there already.

My Setup

Device: Apple M1 MacBook Pro running macOS Big Sur 11.2
Python:

Python 3.9.1 | packaged by conda-forge | (default, Jan 26 2021, 01:30:54) 
[Clang 11.0.1 ] on darwin

Let me know if there is any more info that could be helpful!

I also added the wheel generated by pip if that is helpful to anyone.
grpcio-1.35.0-cp39-cp39-macosx_11_0_arm64.whl.zip

Andrea Estrada · Answer 9 · Fri Mar 05 2021 20:30:44 GMT+0800 (China Standard Time)

I tried doing many of the above, and unfortunately all failed for me. Ended up getting it to work by following the steps here (but for grpcio) https://stackoverflow.com/questions/65564673/how-do-i-import-pandas-on-apple-m1-chip. Posting in case it's helpful to anyone in the future.

Finn Brown · Answer 10 · Wed Jun 30 2021 04:25:02 GMT+0800 (China Standard Time)

Still having this problem after having tried the above. Also, the listed solutions are pretty involved workarounds. Is someone working on a more permanent fix?

Dewey Gaedcke · Answer 11 · Thu Jul 08 2021 08:23:54 GMT+0800 (China Standard Time)

Yes, someone please respond ... I've already burned over 2 days on this error!!

I've tried Homebrew w Py3.8, Py3.9 and Conda-miniforge with both as well
I've tried bash arm64, bash x86, zsh arm64

I've even tried the Panda's example above but it didn't solve the problem for me...

How do I install and run this library?? The failure stacktrace is below:

import google.cloud.ndb as ndb
File "/opt/homebrew/Caskroom/miniforge/base/envs/env/lib/python3.8/site-packages/google/cloud/ndb/init.py", line 28, in
from google.cloud.ndb.client import Client
File "/opt/homebrew/Caskroom/miniforge/base/envs/env/lib/python3.8/site-packages/google/cloud/ndb/client.py", line 18, in
import grpc
File "/opt/homebrew/Caskroom/miniforge/base/envs/env/lib/python3.8/site-packages/grpc/init.py", line 23, in
from grpc._cython import cygrpc as _cygrpc
ImportError: dlopen(/opt/homebrew/Caskroom/miniforge/base/envs/env/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-darwin.so, 2): Symbol not found: _CFRelease
Referenced from: /opt/homebrew/Caskroom/miniforge/base/envs/env/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-darwin.so
Expected in: flat namespace
in /opt/homebrew/Caskroom/miniforge/base/envs/env/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-darwin.so

Lidi Zheng · Answer 12 · Thu Jul 08 2021 13:54:40 GMT+0800 (China Standard Time)

@dgaedcke I don't have a M1 machine. The answer is based on what I read.

Have you try to build your environment with emulated x86?

arch -x86_64 /bin/bash

Then you should be able to use x86 Python, and x86 grpcio binary wheel, no compilation needed.

Dewey Gaedcke · Answer 13 · Thu Jul 08 2021 22:43:16 GMT+0800 (China Standard Time)

Thanks for your response.
Your suggestion is a really good one!!!
And I already got this fixed by using the Conda miniforge build of grpcio
I suspect this will be fixed in this repo when they release 1.39.1 but I'm not sure!!
really appreciate your help!

Bashar Bhuiyan · Answer 14 · Sat Jul 10 2021 19:41:22 GMT+0800 (China Standard Time)

arch -x86_64 pip install grpcio==1.34.0

Thank you so much for this. Worked like a charm in my Macbook Pro M1 with Big Sur (11.4)

Finn Brown · Answer 15 · Sun Jul 11 2021 01:08:07 GMT+0800 (China Standard Time)

My related post on stack overflow. I found a different solution. https://stackoverflow.com/questions/68181171/install-of-python-firebase-admin-on-apple-silicon-m1-fails-grpico-fails

Marcel Kurovski · Answer 16 · Mon Jul 12 2021 21:17:12 GMT+0800 (China Standard Time)

Running Python 3.8.8 within a miniforge conda (v4.10.3) environment:
Setting the environment variables didn't resolve the problem for me - each time pip install grpcio failed:

conda install grpcio directly worked

Vadim · Answer 17 · Tue Jul 13 2021 16:11:09 GMT+0800 (China Standard Time)

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

pip3 install firebase-admin

God bless you @Tenzer - it works now! I've wasted so much time googling the solution. Thank you so much. <3

rimager · Answer 18 · Wed Jul 14 2021 01:03:51 GMT+0800 (China Standard Time)

I have these set:
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
GRPC_BUILD_WITH_BORING_SSL_ASM=

Still getting ERROR: Failed building wheel for grpcio

Grateful for any pointers!!

Jeppe Fihl-Pearson · Answer 19 · Wed Jul 14 2021 17:19:10 GMT+0800 (China Standard Time)

@rimager You would have to provide more information than that. The message you included just tells us the installation failed but no information about why. Could you take the entire output of the command you ran and put in a gist and post the link here? That would make it easier for other people to help you out.

rimager · Answer 20 · Wed Jul 14 2021 19:25:02 GMT+0800 (China Standard Time)

It was a long long error about the wheel. I resolved it by removing this flag:
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1

For some reason it wanted to use another ssl library.

Vadim · Answer 21 · Mon Jul 26 2021 06:03:00 GMT+0800 (China Standard Time)

grpcio 1.38.1 is used to have the issue but it seems that it's finally fixed in 1.39.0 \m/

chrisjune · Answer 22 · Sat Sep 11 2021 14:04:22 GMT+0800 (China Standard Time)

Just simply export and install through pip package installer

>> export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
>> export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

>> pip install grpcio

Ben Rapaport · Answer 23 · Sat Oct 02 2021 04:12:17 GMT+0800 (China Standard Time)

I got a successful build using @Tenzer 's solution, with addition of CFLAGS and LDFLAGS using brew --prefix protobuf, brew --prefix openssl, brew --prefix zlib, and setting the GRPC_PYTHON_BUILD_SYSTEM_* environment variables, with arm64 homebrew and Python 3.9.1 built via pyenv's latest master.

Can you be more explicit with exactly which variables you set, and with which values?

Jeppe Fihl-Pearson · Answer 24 · Sat Oct 02 2021 04:24:02 GMT+0800 (China Standard Time)

See the first comment on this issue: #25082 (comment).

Ben Rapaport · Answer 25 · Sat Oct 02 2021 04:34:39 GMT+0800 (China Standard Time)

I have tried with this (in fish shell)

> set -x LDFLAGS "-L/opt/homebrew/opt/openssl/lib -L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/protobuf/lib"
> set -x CFLAGS "-I/opt/homebrew/opt/openssl/include -I/opt/homebrew/opt/zlib/include -I/opt/homebrew/opt/protobuf/include"
> set -x GRPC_PYTHON_BUILD_SYSTEM_OPENSSL 1
> set -x GRPC_PYTHON_BUILD_SYSTEM_ZLIB 1

But I still cannot install grpcio 1.41.0 (as a dep firebase-admin)
Python 3.9.7

I believe the first errors are

error: no member named 'signbit' in the global namespace
    using ::signbit;
          ~~^
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/cmath:322:9: error: no member named 'fpclassify' in the global namespace
    using ::fpclassify;
          ~~^
    /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/cmath:323:9: error: no member named 'isfinite' in the global namespace
    using ::isfinite;

Ben Rapaport · Answer 26 · Sat Oct 02 2021 04:51:38 GMT+0800 (China Standard Time)

I was able to resolve this error via

> sudo rm -rf /Library/Developer/CommandLineTools
> sudo xcode-select -s /Applications/Xcode.app

Anh Tran Nhat · Answer 27 · Wed Dec 08 2021 11:26:47 GMT+0800 (China Standard Time)

This worked for me (Python 3.9, Big Sur), native compiled, no need to add arch -x86_64 before pip command

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

pip install grpcio

Son H. Nguyen · Answer 28 · Mon Jan 10 2022 19:27:37 GMT+0800 (China Standard Time)

This works for me (M1 Air)

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

And change the default CC to clang inorder to fix gcc: error: unrecognized command-line option '-stdlib=libc++' error

export CC=/usr/bin/clang     
export CXX=/usr/bin/clang++

matanby · Answer 29 · Sun Jan 23 2022 21:25:04 GMT+0800 (China Standard Time)

If anyone still encounters compilation errors when trying to pip install grpcio on a M1 Mac, and none of the above suggestions work, try this:

sudo xcode-select -s /Library/Developer/CommandLineTools

This solved the problem in my case.

Also, another piece of information that might be helpful in solving this: I was able to successfully install grpcio before I had installed xcode. Seems to me like there is a mismatch in the binary tools used for the compilation; the ones that come in /Library/Developer/CommandLineTools work file, while those that come bundled with xcode do not.

Chris Hunter · Answer 30 · Fri Mar 11 2022 04:13:50 GMT+0800 (China Standard Time)

Hey guys, I am appreciative of all of the advice here but unfortunately I'm having the same issue again, I don't know whether to open a new story or keep commenting on this one.

OSX 12.2.1
M1 Chip
PyEnv install of Python 3.9.5
All dev tools up to date with the latest versions (Xcode command line tools/XCode)
Brew installed latest OpenSSL, re2 etc
Installing grpcio==1.44.0

Env vars set to the following:

export GRPC_BUILD_WITH_BORING_SSL_ASM=""
export GRPC_PYTHON_BUILD_SYSTEM_RE2="1"
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL="1"
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB="1"
export LDFLAGS="-L/opt/homebrew/opt/openssl@1.1/lib"
export CPPFLAGS="-I/opt/homebrew/opt/openssl@1.1/include"

I've spent several hours reading and trying things to no avail.

I'm trying to install grpcio=1.44.0 and it is failing with the following:

      
      Traceback (most recent call last):
        File "/private/var/folders/r8/xs35x2dx48b88y7msj4nybvw0000gp/T/pip-install-85s2slvh/grpcio_c2ecc2b58cd44f59b3dd132d0e055106/src/python/grpcio/commands.py", line 285, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/distutils/command/build_ext.py", line 449, in build_extensions
          self._build_extensions_serial()
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/distutils/command/build_ext.py", line 474, in _build_extensions_serial
          self.build_extension(ext)
        File "/Users/<user>/.pyenv/versions/motoapi/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
          _build_ext.build_extension(self, ext)
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/distutils/command/build_ext.py", line 529, in build_extension
          objects = self.compiler.compile(sources,
        File "/private/var/folders/r8/xs35x2dx48b88y7msj4nybvw0000gp/T/pip-install-85s2slvh/grpcio_c2ecc2b58cd44f59b3dd132d0e055106/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
          multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 364, in map
          return self._map_async(func, iterable, mapstar, chunksize).get()
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 771, in get
          raise self._value
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
          return list(map(*args))
        File "/private/var/folders/r8/xs35x2dx48b88y7msj4nybvw0000gp/T/pip-install-85s2slvh/grpcio_c2ecc2b58cd44f59b3dd132d0e055106/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/private/var/folders/r8/xs35x2dx48b88y7msj4nybvw0000gp/T/pip-install-85s2slvh/grpcio_c2ecc2b58cd44f59b3dd132d0e055106/src/python/grpcio/commands.py", line 268, in new_compile
          return old_compile(obj, src, ext, cc_args, extra_postargs,
        File "/Users/<user>/.pyenv/versions/3.9.5/lib/python3.9/distutils/unixccompiler.py", line 120, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/Library/Developer/CommandLineTools/usr/bin/clang' failed with exit code 1
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> grpcio

Alejandro Flores · Answer 31 · Fri Mar 11 2022 07:37:27 GMT+0800 (China Standard Time)

@hunt3r Same issue, M1 Monterey, Python 3.9.10

Fixed it

Inside virtualenv:

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

and then

pip install grpcio

It works

mojoee · Answer 32 · Wed Mar 16 2022 09:45:39 GMT+0800 (China Standard Time)

Also works with python 3.8.11 in a conda environment if followed the above steps!

Chris Hunter · Answer 33 · Tue Mar 29 2022 19:30:13 GMT+0800 (China Standard Time)

Yeah I had to do it with Conda, a pyenv environment did not work for me. Feels weird but I got past the issue for now.

Pedro Lucas · Answer 34 · Wed Mar 30 2022 01:23:03 GMT+0800 (China Standard Time)

Resolved install python 3.9.8 or superior :D 🚀

Pietro De Nicolao · Answer 35 · Wed Mar 30 2022 03:52:06 GMT+0800 (China Standard Time)

Here's what worked on my setup.

My configuration:

MacBook Pro (16-inch, 2021), with M1 Pro
macOS 12.3
Homebrew installed into /opt/homebrew

What worked was a combination of @matanby's comment and some CFLAGS / LDFLAGS options that worked in the past from my colleagues.

I verified and both are needed: removing either makes the compilation fail.
Other flags and compilation options weren't useful.

With this combination of things, grpcio correctly compiles on Python 3.8, 3.9, and 3.10 (installed via pyenv).

# Xcode tools made compilation crash
sudo xcode-select -s /Library/Developer/CommandLineTools
brew install pyenv  # pyenv shell setup omitted...

# Python must be compiled with these options!
CFLAGS="-I$(brew --prefix xz)/include" LDFLAGS="-L$(brew --prefix xz)/lib" pyenv install 3.10.3
pyenv global 3.10.3

# Install grpcio from source tarball
pip install -v https://files.pythonhosted.org/packages/65/75/8b706e1170e2c7b6242b1675259e47986bb4fc490f29387989a965972e6e/grpcio-1.44.0.tar.gz

Pietro De Nicolao · Answer 36 · Thu Mar 31 2022 04:03:35 GMT+0800 (China Standard Time)

@yoav-orca We currently doesn't have ARM64 machines to produce binary wheels, but we are working on a plan.

@lidizheng it is not necessary to have an ARM64 machine to compile grpcio wheels for Apple Silicon: see this solution for a way to cross-compile from Intel. 🙂

Nathan Barreto · Answer 37 · Tue Apr 05 2022 21:45:38 GMT+0800 (China Standard Time)

none of the solutions above worked for me. It ended up being setuptools version. grpcio installation would fail with setuptools==60+ after changiing the env setuptools version to 57.5.0, grpcio installs fine.

Reese · Answer 38 · Fri Apr 15 2022 03:46:05 GMT+0800 (China Standard Time)

Adding onto @pietrodn's solution above: when installing grcpio from the source tarball, make sure to pip uninstall any existing grcpio versions first, otherwise pip may skip the build stage.

Your output for pip install should include:

  running build_py
  running build_project_metadata
  creating python_build
  creating python_build/lib.macosx-11.3-arm64-3.10
  creating python_build/lib.macosx-11.3-arm64-3.10/grpc
  copying src/python/grpcio/grpc/_compression.py -> python_build/lib.macosx-11.3-arm64-3.10/grpc
  copying src/python/grpcio/grpc/_channel.py -> python_build/lib.macosx-11.3-arm64-3.10/grpc
  copying src/python/grpcio/grpc/_common.py -> python_build/lib.macosx-11.3-arm64-3.10/grpc
  (.....)

steckoverflow · Answer 39 · Sun Apr 24 2022 16:52:43 GMT+0800 (China Standard Time)

+1 with this issue on Macbook Pro 12.2.1 version. Tried every possible "fix" suggested here, the only thing that finally worked was installing it via conda rather than pypy

Serhat Cillidag · Answer 40 · Tue Apr 26 2022 13:15:27 GMT+0800 (China Standard Time)

I was also not able to fix this problem. The installation would be successful but importing grpc would fail (shown below)

Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:25:14)
[Clang 12.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import grpc
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/private/var/tmp/_bazel_serhat/27cfb29c86a835e3df3ccb992f3eed84/execroot/core/bazel-out/darwin_arm64-fastbuild/bin/src/python/core/main/shell.runfiles/pypi__39__grpcio_1_41_0/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/private/var/tmp/_bazel_serhat/27cfb29c86a835e3df3ccb992f3eed84/execroot/intenseye/bazel-out/darwin_arm64-fastbuild/bin/src/python/core/main/shell.runfiles/pypi__39__grpcio_1_41_0/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/private/var/tmp/_bazel_serhat/27cfb29c86a835e3df3ccb992f3eed84/execroot/core/bazel-out/darwin_arm64-fastbuild/bin/src/python/core/main/shell.runfiles/pypi__39__grpcio_1_41_0/grpc/_cython/cygrpc.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'

However, installing from conda would be just fine, so I checked their instructions. I was missing some build flags via imports. The following export statements worked for me, and hope they would be helpful for you as well. Taken from here.

export GRPC_BUILD_WITH_BORING_SSL_ASM=""
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB="True"
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL="True"
export GRPC_PYTHON_BUILD_SYSTEM_CARES="False"
export GRPC_PYTHON_USE_PREBUILT_GRPC_CORE=""
export GRPC_PYTHON_BUILD_WITH_CYTHON="True"
export GRPC_PYTHON_LDFLAGS=" -framework CoreFoundation"

Chris Histe · Answer 41 · Thu May 05 2022 22:05:00 GMT+0800 (China Standard Time)

I lost a full day on this. Here's how I managed to make it work.

If you install python while running in arm mode it won't work. I've tried many Python versions with conda, pipenv, venv, Poetry.

You need to compile Python in x86_64 mode.

arch -x86_64 zsh
asdf install python 3.10.4

I use asdf but it should work with pyenv and similar tools.

I hope this helps.

Osvaldo Santana Neto · Answer 42 · Fri May 13 2022 23:15:37 GMT+0800 (China Standard Time)

Just for reference...

If you need to install grpcio through poetry you need to force the installation of the source package. And to make the compilation work you need to set up the following envvars before poetry install:

export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1

or directly with poetry install:

GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 poetry install

The content of pyproject.toml will need the following session to force the use of the source package in Apple M1 (arm64) architechture:

[tool.poetry.dependencies.grpcio]
python = "^3.10.0"
markers = "platform_machine == 'arm64'"
url = "{url-for-the-source-package}"  # eg. https://files.pythonhosted.org/packages/[...]/grpcio-1.46.1.tar.gz

adam lazur · Answer 43 · Tue Jun 07 2022 13:24:34 GMT+0800 (China Standard Time)

I ran into this (again) today so I came back to check out the bug.

@gnossen can you provide guidance on when a fix will make it into a release so pip users will no longer encounter this?

최준영 · Answer 44 · Tue Jun 07 2022 13:28:49 GMT+0800 (China Standard Time)

If you have any trouble about installing python 3.x under 3.9 version`,
please check this article.
https://chrisjune-13837.medium.com/how-to-install-python-3-x-on-apple-m1-9e77ff94266a

GalaxyStar · Answer 45 · Mon Jun 27 2022 17:43:52 GMT+0800 (China Standard Time)

I was also not able to fix this problem. The installation would be successful but importing grpc would fail (shown below)

Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:25:14)
[Clang 12.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import grpc
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/private/var/tmp/_bazel_serhat/27cfb29c86a835e3df3ccb992f3eed84/execroot/core/bazel-out/darwin_arm64-fastbuild/bin/src/python/core/main/shell.runfiles/pypi__39__grpcio_1_41_0/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/private/var/tmp/_bazel_serhat/27cfb29c86a835e3df3ccb992f3eed84/execroot/intenseye/bazel-out/darwin_arm64-fastbuild/bin/src/python/core/main/shell.runfiles/pypi__39__grpcio_1_41_0/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/private/var/tmp/_bazel_serhat/27cfb29c86a835e3df3ccb992f3eed84/execroot/core/bazel-out/darwin_arm64-fastbuild/bin/src/python/core/main/shell.runfiles/pypi__39__grpcio_1_41_0/grpc/_cython/cygrpc.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'

However, installing from conda would be just fine, so I checked their instructions. I was missing some build flags via imports. The following export statements worked for me, and hope they would be helpful for you as well. Taken from here.

export GRPC_BUILD_WITH_BORING_SSL_ASM=""
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB="True"
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL="True"
export GRPC_PYTHON_BUILD_SYSTEM_CARES="False"
export GRPC_PYTHON_USE_PREBUILT_GRPC_CORE=""
export GRPC_PYTHON_BUILD_WITH_CYTHON="True"
export GRPC_PYTHON_LDFLAGS=" -framework CoreFoundation"

It works for me.

Ulises Castro · Answer 46 · Wed Jun 29 2022 10:55:11 GMT+0800 (China Standard Time)

On my case I only be able to so using poetry even on Docker

adam lazur · Answer 47 · Wed Jun 29 2022 14:11:54 GMT+0800 (China Standard Time)

Looks like it was fixed in release v1.47.0 thanks to #30026 (and issue #28387 )

The "universal" darwin wheel is now labeled x86_64, so arm64 mac users will automatically download and compile grpcio from source

ElliottWilson · Answer 48 · Wed Jun 29 2022 18:16:46 GMT+0800 (China Standard Time)

I am getting this error today it worked last night. I have tried all the above solution and still seem to get the same thing:

MacBook Pro (13-inch, M1, 2020)
Chip M1

Richard Belleville · Answer 49 · Sat Jul 02 2022 01:11:08 GMT+0800 (China Standard Time)

@ElliottWilson This is likely a build failure while installing from source. First, make sure that a build toolchain is installed (i.e. XCode). It seems there is currently an issue building from source when gRPC is also installed via homebrew, so make sure that that is not the case either.

If the install still fails, please open a new issue with full logs included.

Joongi Kim · Answer 50 · Sun Jul 03 2022 10:26:42 GMT+0800 (China Standard Time)

@gnossen I and my team has experienced gprcio source build failures many times and discovered that depending on the development stack configurations the required build flags are vastly different laptop by laptop. Just installing XCode with command line tools and latest homebrew did not solve the problem and sometimes we could not figure out what installation/configuration triggers such difference.

It would be nice if we have a concrete guide to how to set up the grpcio build environment on macOS and/or the pre-built binary wheel distribution for Apple Silicon.

Russ Bubley · Answer 51 · Thu Jul 14 2022 01:53:06 GMT+0800 (China Standard Time)

It's not quite a guide, but after lots of trial and error the invocation that worked for me was:

CC=/usr/bin/clang GRPC_BUILD_WITH_BORING_SSL_ASM="" GRPC_PYTHON_BUILD_SYSTEM_RE2=true GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio==1.47.0

YMMV!

Vibha · Answer 52 · Mon Jul 25 2022 19:12:53 GMT+0800 (China Standard Time)

I am still having this issue :( used this
CC=/usr/bin/clang GRPC_BUILD_WITH_BORING_SSL_ASM="" GRPC_PYTHON_BUILD_SYSTEM_RE2=true GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio==1.47.0
Basically unable to install snorkel because of that :(
Apple Silicon Chip

Reetika Roy · Answer 53 · Wed Aug 03 2022 12:18:53 GMT+0800 (China Standard Time)

This did not work for me either for grpcio==1.47.0, but it does for 1.44.0

Tommy Carpenter · Answer 54 · Wed Aug 10 2022 20:43:52 GMT+0800 (China Standard Time)

@reetika-roy answer is the only thing that saved me in tox. We have grpcio transitively as a dependency of firebase and I tried nearly everything, including env variables - but alas pinning grpcio==1.44.0 in our setup.py made it work, thanks much!

Manoj Kumar P · Answer 55 · Wed Aug 24 2022 19:53:11 GMT+0800 (China Standard Time)

# Install grpcio from source tarball
pip install -v https://files.pythonhosted.org/packages/65/75/8b706e1170e2c7b6242b1675259e47986bb4fc490f29387989a965972e6e/grpcio-1.44.0.tar.gz

It does not work otherwise. what is the difference from pip install grpcio that fails to find wheel and compiles? @pietrodn

Pietro De Nicolao · Answer 56 · Wed Aug 24 2022 19:56:55 GMT+0800 (China Standard Time)

@pmontu in theory nothing should change in your case. If pip fails to find a compatible binary wheel, it uses the source tarball as last resort.
However keep in mind that that link is of an older version.

What are you experiencing exactly?
pip install grpcio==1.44.0 fails and pip install https://files.pythonhosted.org/packages/65/75/8b706e1170e2c7b6242b1675259e47986bb4fc490f29387989a965972e6e/grpcio-1.44.0.tar.gz works?

Zhamal Toktamysova · Answer 57 · Sat Aug 27 2022 00:17:14 GMT+0800 (China Standard Time)

I had a similar issue with installing grpcio=1.47.0 but I used pipenv and Pipfile.lock
What helped me - do uninstall pipenv previously installed with pip and install it again but with homebrew:
brew install pipenv
Then, pipenv could install grpcio without any problems

Yury V. Zaytsev · Answer 58 · Fri Sep 02 2022 22:54:25 GMT+0800 (China Standard Time)

@lidizheng so we are still facing this issue with grpc 1.48.1 on macOS 12 on M1.

The installation via pip install grpcio works, but after successful installation the resulting shared object cannot be loaded:

ImportError: dlopen(~/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_CFRelease'

The wheel is not used during the installation, so the artefact is built on the local machine, but the results are busted.

If we export magic variables from the conda recipe, keep conda libraries installed and install Cython, then pip install grpcio produces a working artefact:

export GRPC_BUILD_WITH_BORING_SSL_ASM=""
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB="True"
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL="True"
export GRPC_PYTHON_BUILD_SYSTEM_CARES="True"
export GRPC_PYTHON_USE_PREBUILT_GRPC_CORE=""
export GRPC_PYTHON_BUILD_WITH_CYTHON="True"
export GRPC_PYTHON_LDFLAGS=" -framework CoreFoundation"

With system libraries without conda provided libraries the installation fails.

We would really appreciate if you guys can manage to upload arm64 wheels and/or at least fix your installation process to produce working artefacts on M1 by default without dark conda recipe environment variable magic.

/cc @mm-ivanovig @marns93

Yury V. Zaytsev · Answer 59 · Sat Sep 03 2022 21:54:22 GMT+0800 (China Standard Time)

Binary wheels are available here: https://github.com/pietrodn/grpcio-mac-arm-build thanks to #29262 /cc @mm-ivanovig @marns93

Riccardo Carlesso · Answer 60 · Sat Sep 17 2022 04:53:06 GMT+0800 (China Standard Time)

Folks, I'm still having the issue and I believe this should be open until:

everyone has a working release, OR
there's an easy local fix.

Amchii · Answer 61 · Mon Sep 19 2022 14:50:30 GMT+0800 (China Standard Time)

Folks, I'm still having the issue and I believe this should be open until:

everyone has a working release, OR

there's an easy local fix.

Yes, it's still need to set env vars explicitly to install grpcio for me:

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

python version: 3.9.13
grpcio version: 1.49.0

Knut Vågnes Eriksen · Answer 62 · Tue Oct 04 2022 21:08:12 GMT+0800 (China Standard Time)

I ran into the same problem today with the following specs:

python version: 3.10.6
grpcio version: 1.49.1
os: mac os monterey 12.6 (M1 PRO)

Solved it by uninstalling, cleaning the conda and pip cache and then reinstalling grpcio again:

pip uninstall grpcio
conda clean --all
pip cache purge
pip install grpcio

Tom Potter · Answer 63 · Thu Oct 06 2022 07:20:38 GMT+0800 (China Standard Time)

I couldn't easily get a build process working on my Monterey M1, but the binary worked.

Details:

Setting the two environment variables still left me with

Traceback (most recent call last):
  File "/Users/tom/code_tmp/clone/grpc/examples/python/helloworld/./greeter_server.py", line 19, in <module>
    import grpc
  File "/Users/tom/opt/anaconda3/envs/basemod/lib/python3.9/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/Users/tom/opt/anaconda3/envs/basemod/lib/python3.9/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/Users/tom/opt/anaconda3/envs/basemod/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace (_CFRelease)

when trying to import grpc.

Maybe "If we export magic variables from the conda recipe, keep conda libraries installed and install Cython, then" would have worked, but I don't understand this enough to do it confidently (keep what installed how?). I haven't read the whole thread.

conda install -c conda-forge grpcio gave me

(basemod) tom@MacBook-Pro helloworld % conda install grpcio -c conda-forge
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/common/logic.py", line 125, in _convert
        return self.names[name]
    KeyError: 'conda-forge/osx-arm64::certifi-2022.6.15-py38h10201cd_0'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/exceptions.py", line 1114, in __call__
        return func(*args, **kwargs)
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/cli/main.py", line 86, in main_subshell
        exit_code = do_call(args, p)
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 90, in do_call
        return getattr(module, func_name)(args, parser)
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/cli/main_install.py", line 20, in execute
        install(args, parser, 'install')
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/cli/install.py", line 259, in install
        unlink_link_transaction = solver.solve_for_transaction(
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/core/solve.py", line 152, in solve_for_transaction
        unlink_precs, link_precs = self.solve_for_diff(update_modifier, deps_modifier,
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/core/solve.py", line 195, in solve_for_diff
        final_precs = self.solve_final_state(update_modifier, deps_modifier, prune, ignore_pinned,
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/core/solve.py", line 313, in solve_final_state
        ssc = self._add_specs(ssc)
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/core/solve.py", line 605, in _add_specs
        conflict_specs = ssc.r.get_conflicting_specs(tuple(concatv(
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/resolve.py", line 1109, in get_conflicting_specs
        C = r2.gen_clauses()
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/common/io.py", line 86, in decorated
        return f(*args, **kwds)
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/resolve.py", line 911, in gen_clauses
        nkey = C.Not(self.to_sat_name(prec))
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/common/logic.py", line 144, in Not
        return self._eval(self._clauses.Not, (x,), (), polarity, name)
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/common/logic.py", line 130, in _eval
        args = self._convert(args)
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/common/logic.py", line 120, in _convert
        return type(x)(map(self._convert, x))
      File "/Users/tom/opt/anaconda3/lib/python3.9/site-packages/conda/common/logic.py", line 127, in _convert
        raise ValueError("Unregistered SAT variable name: {}".format(name))
    ValueError: Unregistered SAT variable name: conda-forge/osx-arm64::certifi-2022.6.15-py38h10201cd_0

`$ /Users/tom/opt/anaconda3/bin/conda install grpcio -c conda-forge`

  environment variables:
                 CIO_TEST=<not set>
        CONDA_DEFAULT_ENV=basemod
                CONDA_EXE=/Users/tom/opt/anaconda3/bin/conda
             CONDA_PREFIX=/Users/tom/opt/anaconda3/envs/basemod
    CONDA_PROMPT_MODIFIER=(basemod)
         CONDA_PYTHON_EXE=/Users/tom/opt/anaconda3/bin/python
               CONDA_ROOT=/Users/tom/opt/anaconda3
              CONDA_SHLVL=1
           CURL_CA_BUNDLE=<not set>
                 INFOPATH=/opt/homebrew/share/info:
                  MANPATH=/Users/tom/.nvm/versions/node/v17.6.0/share/man:/Users/tom/software/gi
                          t-subrepo/man:/opt/homebrew/share/man::
                     PATH=/Users/tom/opt/anaconda3/bin:/Users/tom/opt/anaconda3/envs/basemod/bin
                          :/Users/tom/opt/anaconda3/condabin:/Users/tom/.nvm/versions/node/v17.6
                          .0/bin:/Users/tom/.poetry/bin:/opt/homebrew/opt/openssl@1.1/bin:/Users
                          /tom/software/git-subrepo/lib:/Users/tom/Library/Jupyter/kernels:/User
                          s/tom/Documents/code/mon/scripts:/Users/tom/.nimble/bin:/Users/tom/ins
                          tall/nim/bin:/Users/tom/Applications/Sublime Text.app/Contents/SharedS
                          upport/bin:/Users/tom/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/lo
                          cal/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion Tech
                          Preview.app/Contents/Public:/usr/local/zfs/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : basemod
    active env location : /Users/tom/opt/anaconda3/envs/basemod
            shell level : 1
       user config file : /Users/tom/.condarc
 populated config files : /Users/tom/.condarc
          conda version : 4.13.0
    conda-build version : 3.21.9
         python version : 3.9.12.final.0
       virtual packages : __osx=12.5.1=0
                          __unix=0=0
                          __archspec=1=arm64
       base environment : /Users/tom/opt/anaconda3  (writable)
      conda av data dir : /Users/tom/opt/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/anaconda/osx-arm64
                          https://conda.anaconda.org/anaconda/noarch
          package cache : /Users/tom/opt/anaconda3/pkgs
                          /Users/tom/.conda/pkgs
       envs directories : /Users/tom/opt/anaconda3/envs
                          /Users/tom/.conda/envs
               platform : osx-arm64
             user-agent : conda/4.13.0 requests/2.27.1 CPython/3.9.12 Darwin/21.6.0 OSX/12.5.1
                UID:GID : 501:20
             netrc file : None
           offline mode : False

Installing the binary wheel from the latest artifact at https://github.com/pietrodn/grpcio-mac-arm-build/actions by downloading it and

pip uninstall grpcio
cd ~/Downloads/artifacts/
pip install ./grpcio-1.48.1-cp39-cp39-macosx_11_0_arm64.whl

worked a treat. Thank you so much!! <3

Joe Goulet · Answer 64 · Thu Oct 06 2022 15:40:48 GMT+0800 (China Standard Time)

I ran into the same problem today with the following specs:

python version: 3.10.6 grpcio version: 1.49.1 os: mac os monterey 12.6 (M1 PRO)

Solved it by uninstalling, cleaning the conda and pip cache and then reinstalling grpcio again:
pip uninstall grpcio
conda clean --all
pip cache purge
pip install grpcio

Confirming python 3.10.6 works on Monterey 12.5 (M1 Pro) with no env vars set. 3.9 did not seem to work for me with any of the incantations above.

elementljarredc · Answer 65 · Tue Oct 18 2022 07:50:51 GMT+0800 (China Standard Time)

Would it be fair to say that this issue is not resolved? Should there be another one opened, or this one perhaps? We've got OSS users that encounter this problem somewhat regularly

edvard-bjarnason · Answer 66 · Thu Oct 20 2022 20:00:49 GMT+0800 (China Standard Time)

Of course the issue should be open, it is not fixed. I don't understand the trend on GitHub to close issues before a fix has been created, it is not like the issue goes away just by marking the issue closed:( Now it is almost 2 years since the issue was created and nobody cares enough about this to actually do the work needed to fix it. It is really frustrating that an important package like this can not be installed by pip without problems.

On my Mac m1 12.6 python 3.10.6 I was not able to install 1.50.0 I simply gave up trying get it compiled :( and installed 1.46.3 with the following env vars set:
GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
GRPC_BUILD_WITH_BORING_SSL_ASM=
GRPC_PYTHON_BUILD_SYSTEM_CARES=True
GRPC_PYTHON_USE_PREBUILT_GRPC_CORE=
GRPC_PYTHON_BUILD_WITH_CYTHON=True
GRPC_PYTHON_LDFLAGS= -framework CoreFoundation

Not sure if the env vars matter though

edvard-bjarnason · Answer 67 · Thu Oct 20 2022 22:05:18 GMT+0800 (China Standard Time)

Just to follow up on my previous comment, it turned out I could not use version 1.46.3. Anyway I managed to install version grpcio version 1.50.0. My pyenv was pulling in libraries for macOS 12.4 and after I updated pyenv and reinstalled python versions 3.10.6 (which pyenv builds from source) I could install grpcio version 1.50.0.

Here are the commands I used:
brew upgrade pyenv
pyenv uninstall 3.10.6
pyenv install 3.10.6
pyenv virtualenv 3.10.6 google
pyenv activate google
pip install grpcio
...
Installing collected packages: six, grpcio
Running setup.py install for grpcio ... done
Successfully installed grpcio-1.50.0 six-1.16.0

Dmitriy Ignatenko · Answer 68 · Sat Oct 22 2022 01:44:21 GMT+0800 (China Standard Time)

Would it be fair to say that this issue is not resolved? Should there be another one opened, or this one perhaps? We've got OSS users that encounter this problem somewhat regularly

Yeah it is absolutely true, I spent like 2 hours trying to install it without venv, nothing really helps...

Pietro De Nicolao · Answer 69 · Sat Oct 22 2022 15:33:50 GMT+0800 (China Standard Time)

Hello, since Google is doing nothing about this, I decided to set up a periodic GitHub Actions workflow to provide daily builds of Apple Silicon wheels for Mac. It will automatically cut a new release the day a new grpcio version is released on PyPI.

In the releases page of this repo you can find and download the wheels for Python 3.8, 3.9, 3.10, 3.11.
(Use at your own risk, these are not tested.)
https://github.com/pietrodn/grpcio-mac-arm-build

Roy · Answer 70 · Thu Oct 27 2022 03:57:18 GMT+0800 (China Standard Time)

I found out that setuptools-65.5.0 seems not able to install the latest grpcio-2.10.0, so I choose to install grpcio2.9.0 and then it works.

Sang · Answer 71 · Tue Nov 08 2022 17:40:28 GMT+0800 (China Standard Time)

@pietrodn Thanks 👍👍

Samuel Bishop · Answer 72 · Sat Nov 12 2022 23:17:04 GMT+0800 (China Standard Time)

@pietrodn the issues appear to also affect the grpcio-tools package can you add that to your build example (and the repo where you've got working builds)

Pietro De Nicolao · Answer 73 · Sun Nov 13 2022 15:38:58 GMT+0800 (China Standard Time)

@techdragon ✅ Done! https://github.com/pietrodn/grpcio-mac-arm-build/releases

Levin Rickert · Answer 74 · Wed Jan 25 2023 21:17:33 GMT+0800 (China Standard Time)

Official macOS ARM builds seem to land in GRPC 1.52: #31747

Also GRPC 1.51 builds in under a minute from source for me on a M1 without any env variables set.

Richard Belleville · Answer 75 · Thu Jan 26 2023 02:11:16 GMT+0800 (China Standard Time)

@levrik Yes, to confirm, 1.52.0 includes prebuilt universal2 artifacts which work on both x86-64 and arm64 macs. The release candidate is already uploaded and the official release is scheduled for later this week.

Richard Belleville · Answer 76 · Tue Feb 07 2023 02:54:52 GMT+0800 (China Standard Time)

1.52.0 has been released with proper universal2 artifacts which support both x86 macs and M1/2 macs.

Luiz · Answer 77 · Mon Feb 13 2023 12:31:45 GMT+0800 (China Standard Time)

1.52.0 has been released with proper universal2 artifacts which support both x86 macs and M1/2 Macs.

1.52.0 is Yanked

John Anderson · Answer 78 · Fri Feb 17 2023 23:30:34 GMT+0800 (China Standard Time)

Can we re-open?

Marcel Arns · Answer 79 · Fri Feb 17 2023 23:37:32 GMT+0800 (China Standard Time)

Can we re-open?

I can see that there is a new release https://github.com/grpc/grpc/releases/tag/v1.52.1 and that was announced for today.
#32306 (comment)

But it is not published on PyPI....

Richard Belleville · Answer 80 · Sat Feb 18 2023 02:52:22 GMT+0800 (China Standard Time)

1.51.2 is an unrelated C++-only patch release for a security vulnerability affecting only the C++ API. The release process for 1.51.3, which backports universal2 artifacts, is in progress and is still estimated to be complete today.

Richard Belleville · Answer 81 · Sat Feb 18 2023 09:41:12 GMT+0800 (China Standard Time)

Due to an issue with the CI build, the estimate is now Tuesday, February 28th.

Richard Belleville · Answer 82 · Wed Feb 22 2023 07:27:35 GMT+0800 (China Standard Time)

1.51.3 has now been released with universal2 artifacts. I'll leave this issue open for a couple more days in case there are further issues.