symbol not found on macOS 11

Question

symbol not found on macOS 11

BerryKeers opened this issue 3 years ago · comments

Describe the bug
I've tried installing keyring with pip3 globally and also with pipenv, both times whenever I try to use this module i'm getting the same error:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keyring/backends/macOS/__init__.py", line 38, in set_password
    api.set_generic_password(self.keychain, service, username, password)
NameError: name 'api' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/Berry/Projects/pipenv/IEX/creds.py", line 20, in <module>
    save_cred()
  File "/Users/Berry/Projects/pipenv/IEX/creds.py", line 9, in save_cred
    keyring.set_password(service_name, 'username', username)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keyring/core.py", line 60, in set_password
    get_keyring().set_password(service_name, username, password)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/keyring/backends/macOS/__init__.py", line 39, in set_password
    except api.KeychainDenied as e:
NameError: name 'api' is not defined

To Reproduce
My little test script is:

import getpass
import keyring

def save_cred():
    service_name = 'Python_IEX'
    username = input('What\'s your IEX username?\n')
    password = getpass.getpass(prompt='Password: ', stream=None)

    keyring.set_password(service_name, 'username', username)
    keyring.set_password(service_name, username, password)

    print(f'Credentials for user {username} saved successfully.')

def get_cred():
    username = keyring.get_password(service_name, 'username')
    password = keyring.get_password(service_name, 'password')

    return username, password

save_cred()

Expected behavior
A clear and concise description of what you expected to happen.

Environment

OS: [e.g. macOS]
MacOS Big Sur 11.5.2

$ pip list | grep keyring
keyring                           23.1.0
$ keyring --list-backends
dlsym(RTLD_DEFAULT, SecItemAdd): symbol not found
keyring.backends.chainer.ChainerBackend (priority: -1)
keyring.backends.macOS.Keyring (priority: 5)
keyring.backends.fail.Keyring (priority: 0)

Daniel Wallace · Answer 1 · Wed Sep 01 2021 01:39:07 GMT+0800 (China Standard Time)

I am also experiencing this symbol not found bug.

Shawn O'Grady · Answer 2 · Sat Sep 04 2021 02:40:40 GMT+0800 (China Standard Time)

I noticed this immediately after updating to big sur, appears to be related to https://stackoverflow.com/questions/62587131/macos-big-sur-python-ctypes-find-library-does-not-find-libraries-ssl-corefou

Dmitry Shachnev · Answer 3 · Sun Sep 05 2021 02:51:53 GMT+0800 (China Standard Time)

Can you paste an error that this command gives you: python3 -c 'import keyring.backends.macOS.api'?

Cc @jaraco because you are an actual macOS user unlike me.

Tony F · Answer 4 · Fri Sep 10 2021 22:12:54 GMT+0800 (China Standard Time)

Can you paste an error that this command gives you: python3 -c 'import keyring.backends.macOS.api'?

Cc @jaraco because you are an actual macOS user unlike me.

I just pip3 installed keyring and also receive this error.

user@user-MBP:~$ python3 -c 'import keyring.backends.macOS.api'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/keyring/backends/macOS/api.py", line 45, in <module>
    SecItemAdd = _sec.SecItemAdd
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, SecItemAdd): symbol not found

Tony F · Answer 5 · Fri Sep 10 2021 22:57:52 GMT+0800 (China Standard Time)

Upgrading to Python 3.9.7 resolved the issue for me. python3 -c 'import keyring.backends.macOS.api' returns no errors now

lukeallpress · Answer 6 · Sat Sep 11 2021 03:00:04 GMT+0800 (China Standard Time)

Same issue

user@mbp: ~ % python3 -c 'import keyring.backends.macOS.api'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.8/site-packages/keyring/backends/macOS/api.py", line 26, in <module>
    CFDictionaryCreate = _found.CFDictionaryCreate
  File "/opt/anaconda3/lib/python3.8/ctypes/__init__.py", line 386, in __getattr__
    func = self.__getitem__(name)
  File "/opt/anaconda3/lib/python3.8/ctypes/__init__.py", line 391, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(RTLD_DEFAULT, CFDictionaryCreate): symbol not found

I'm not interested in upgrading my python beyond what Anaconda has released.

Tony F · Answer 7 · Sat Sep 11 2021 04:30:05 GMT+0800 (China Standard Time)

I'm not interested in upgrading my python beyond what Anaconda has released.

I think it's out of the maintainers hands since this is an underlying issue with python's ffi library when used with OSX Big Sur. According to https://bugs.python.org/issue41100 python 3.8.10 will get the fix. According to https://docs.anaconda.com/anaconda/packages/py3.8_osx-64/ ships with python 3.8.8 currently.

lukeallpress · Answer 8 · Sat Sep 11 2021 05:06:40 GMT+0800 (China Standard Time)

Thanks for the clarity and doing my research for me! I hadn't put the pieces together.

Jason R. Coombs · Answer 9 · Sat Sep 11 2021 21:16:18 GMT+0800 (China Standard Time)

I can confirm the error is shown when running the tests on Python 3.8.5:

__________________________________________________________ ERROR collecting keyring/backends/macOS/api.py ___________________________________________________________
keyring/backends/macOS/api.py:45: in <module>
    SecItemAdd = _sec.SecItemAdd
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ctypes/__init__.py:386: in __getattr__
    func = self.__getitem__(name)
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ctypes/__init__.py:391: in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
E   AttributeError: dlsym(RTLD_DEFAULT, SecItemAdd): symbol not found

And also confirm that updating to Python 3.8.10 (universal binary) fixes the issue. Since 3.9.5 was released on the same date, I expect the fix to require that version as well.

Checking the changelog for 3.9.5, I don't see anything relevant. My guess is that the universal binary is required for Python 3.9.1 and 3.8.7 or later.

I don't believe this library can address the root cause, but it may be useful for this library to disable the backend when the API is unavailable (opening up the possibility of another macOS backend that bypasses the API).

Jason R. Coombs · Answer 10 · Sat Sep 11 2021 22:01:50 GMT+0800 (China Standard Time)

I filed #529 to track the possible enhancement to not fail with the error reported above.

Tony F · Answer 11 · Mon Sep 13 2021 00:02:09 GMT+0800 (China Standard Time)

Checking the changelog for 3.9.5, I don't see anything relevant.

It's there, but in the full HTML view as bpo-41100, https://docs.python.org/release/3.9.5/whatsnew/changelog.html#id33

Jason R. Coombs · Answer 12 · Mon Sep 13 2021 01:08:20 GMT+0800 (China Standard Time)

Right, so it's not a specific Python version that matters but you must have a universal binary, which was introduced in Python 3.9.1 and 3.8.7, but the error will still occur on the latest releases of Python 3.8 and 3.9 if not using the universal binary, as observed above with reports of the failure on Python 3.9.2 and 3.8.8.

David Marshall · Answer 13 · Mon Oct 11 2021 07:04:08 GMT+0800 (China Standard Time)

@jaraco is this an example of a universal binary installer that you'd expect to fix this issue?

Jason R. Coombs · Answer 14 · Mon Oct 11 2021 20:48:16 GMT+0800 (China Standard Time)

Yes. The “universal2” in the name is the clue. These installers were introduced as part of the ARM support, and they seem to have the necessary symbols. For Python 3.10 and maybe 3.9, it may be your only option.