How to Install Grpcio Pip Package on Apple M1

|

I spent a long time figuring out how to install the latest grpcio Pip package (version 1.37.1) on my Apple M1 Macbook.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
pip install grpcio

Looking in indexes: https://pypi.org/simple, https://artifactory.spotify.net/artifactory/api/pypi/pypi/simple
Collecting grpcio
  Downloading https://artifactory.spotify.net/artifactory/api/pypi/pypi/packages/packages/a0/d6/d04c6550debe23e2eaef0d9c4adccbb6e20d8cce6da40ae989fe8836e287/grpcio-1.37.1.tar.gz (21.7 MB)
     |████████████████████████████████| 21.7 MB 143 kB/s
Requirement already satisfied: six>=1.5.2 in ./.virtualenvs/spotify/lib/python3.9/site-packages (from grpcio) (1.12.0)
Building wheels for collected packages: grpcio
  Building wheel for grpcio (setup.py) ... error
  ERROR: Command errored out with exit status 1:
...

  third_party/zlib/gzlib.c:252:9: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
          LSEEK(state->fd, 0, SEEK_END);  /* so gzoffset() is correct */
          ^
  third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
  #  define LSEEK lseek
                  ^
  third_party/zlib/gzlib.c:252:9: note: did you mean 'fseek'?
  third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
  #  define LSEEK lseek
                  ^
  /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:162:6: note: 'fseek' declared here
  int      fseek(FILE *, long, int);
           ^
  third_party/zlib/gzlib.c:258:24: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
          state->start = LSEEK(state->fd, 0, SEEK_CUR);
                         ^
  third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
  #  define LSEEK lseek
                  ^
  third_party/zlib/gzlib.c:359:9: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
      if (LSEEK(state->fd, state->start, SEEK_SET) == -1)
          ^
  third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
  #  define LSEEK lseek
                  ^
  third_party/zlib/gzlib.c:400:15: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
          ret = LSEEK(state->fd, offset - state->x.have, SEEK_CUR);
                ^
  third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
  #  define LSEEK lseek
                  ^
  third_party/zlib/gzlib.c:496:14: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
      offset = LSEEK(state->fd, 0, SEEK_CUR);
               ^
  third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
  #  define LSEEK lseek
                  ^
  5 errors generated.

...

  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Users/dxia/.virtualenvs/spotify/include -I/Users/dxia/.pyenv/versions/3.9.1/include/python3.9 -c /var/folders/x1/f9sjnv7j43z73sdv5lsk3r8h0000gp/T/tmpyvic7ha6/a.c -o None/var/folders/x1/f9sjnv7j43z73sdv5lsk3r8h0000gp/T/tmpyvic7ha6/a.o
  Traceback (most recent call last):
    File "/Users/dxia/.pyenv/versions/3.9.1/lib/python3.9/distutils/unixccompiler.py", line 117, in _compile
      self.spawn(compiler_so + cc_args + [src, '-o', obj] +
    File "/private/var/folders/x1/f9sjnv7j43z73sdv5lsk3r8h0000gp/T/pip-install-1ha5py6y/grpcio_12658497b5464faa852de046ce91485a/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
      _classic_spawn(self, command)
    File "/Users/dxia/.pyenv/versions/3.9.1/lib/python3.9/distutils/ccompiler.py", line 910, in spawn
      spawn(cmd, dry_run=self.dry_run)
    File "/Users/dxia/.pyenv/versions/3.9.1/lib/python3.9/distutils/spawn.py", line 87, in spawn
      raise DistutilsExecError(
  distutils.errors.DistutilsExecError: command '/usr/bin/clang' failed with exit code 1

...

    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/dxia/.virtualenvs/spotify/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/x1/f9sjnv7j43z73sdv5lsk3r8h0000gp/T/pip-install-1ha5py6y/grpcio_12658497b5464faa852de046ce91485a/setup.py'"'"'; __file__='"'"'/private/var/folders/x1/f9sjnv7j43z73sdv5lsk3r8h0000gp/T/pip-install-1ha5py6y/grpcio_12658497b5464faa852de046ce91485a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/x1/f9sjnv7j43z73sdv5lsk3r8h0000gp/T/pip-record-n4ihfdh1/install-record.txt --single-version-externally-managed --compile --install-headers /Users/dxia/.virtualenvs/spotify/include/site/python3.9/grpcio Check the logs for full command output.

Fixed by setting the following (I use fish shell). I found the first four environment variables in this Github comment. The second two I knew to add because I was seeing errors about the compiler not being able to find the openssl.h and re.h header files.

1
2
3
4
5
6
set -x GRPC_BUILD_WITH_BORING_SSL_ASM ""
set -x GRPC_PYTHON_BUILD_SYSTEM_RE2 true
set -x GRPC_PYTHON_BUILD_SYSTEM_OPENSSL true
set -x GRPC_PYTHON_BUILD_SYSTEM_ZLIB true
set -x CPPFLAGS "-I"(brew --prefix openssl)"/include -I"(brew --prefix re2)"/include"
set -x LDFLAGS "-L"(brew --prefix openssl)"/lib -L"(brew --prefix re2)"/lib"

Comments