octodns_ns1 isn't ready for python 3.10 yet
flz opened this issue · comments
Here's the error I'm getting:
2022-03-24T05:33:28 [140404123150144] INFO Manager __init__: config_file=/octodns/test.yaml
2022-03-24T05:33:28 [140404123150144] INFO Manager __init__: max_workers=2
2022-03-24T05:33:28 [140404123150144] INFO Manager __init__: include_meta=False
2022-03-24T05:33:28 [140404123150144] ERROR Manager _get_{}_class: Unable to import module octodns_ns1.Ns1Provider
Traceback (most recent call last):
File "/opt/octodns/env/src/octodns/octodns/manager.py", line 190, in _get_named_class
module = import_module(module_name)
File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/octodns/env/lib/python3.10/site-packages/octodns_ns1/__init__.py", line 7, in <module>
from collections import Mapping, OrderedDict, defaultdict
ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)
Traceback (most recent call last):
File "/opt/octodns/env/src/octodns/octodns/manager.py", line 190, in _get_named_class
module = import_module(module_name)
File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/octodns/env/lib/python3.10/site-packages/octodns_ns1/__init__.py", line 7, in <module>
from collections import Mapping, OrderedDict, defaultdict
ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/octodns/env/bin/octodns-sync", line 33, in <module>
sys.exit(load_entry_point('octodns', 'console_scripts', 'octodns-sync')())
File "/opt/octodns/env/src/octodns/octodns/cmds/sync.py", line 37, in main
manager = Manager(args.config_file)
File "/opt/octodns/env/src/octodns/octodns/manager.py", line 115, in __init__
_class = self._get_named_class('provider', _class)
File "/opt/octodns/env/src/octodns/octodns/manager.py", line 194, in _get_named_class
raise ManagerException(f'Unknown {_type} class: {_class}')
octodns.manager.ManagerException: Unknown provider class: octodns_ns1.Ns1Provider
Using python:3.9-slim instead works just fine. Code could be changed in octodns_ns1 to import Mapping from collections.abc (and fall back to collections in case ImportError is thrown).
CI runs octodns-ns1 against 3.10, https://github.com/octodns/octodns-ns1/runs/5613772684?check_suite_focus=true so I'm a bit surprised it's having a problem here.
Were you using the octoDNS pre-built docker images or rolling your own? I ask b/c if it's the latter this probably belongs over in https://github.com/octodns/octodns-ns1 (and I can move it.)
Just ran a quick check in python:3.10-slim and the tests pass:
root@bb4cd1f5c40b:~/foo/app# ./script/bootstrap
...
root@bb4cd1f5c40b:~/foo/app# ./script/test
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.10.3, pytest-7.0.1, pluggy-1.0.0
rootdir: /root/foo/app
plugins: cov-3.0.0, mock-3.7.0, network-0.0.1
collected 37 items
tests/test_provider_ns1.py ..................................... [100%]
============================================================================================ 37 passed in 0.25s ============================================================================================
root@bb4cd1f5c40b:~/foo/app# python --version
Python 3.10.3But trying to do the import line alone doesn't work:
root@bb4cd1f5c40b:~/foo/app# python
Python 3.10.3 (main, Mar 17 2022, 05:23:29) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import Mapping, OrderedDict, defaultdict
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)That line isn't in octodns-ns1 though, it's been updated https://github.com/octodns/octodns-ns1/blob/6692bc0f60eaee0b76cc936b0e8631ecc86b824a/octodns_ns1/__init__.py#L7-L8
from collections.abc import Mapping
from collections import OrderedDict, defaultdictWill keep digging into what's happening here, but I'm a little confused atm.
Looks like that fix happened between 0.0.1 and 0.0.2 https://github.com/octodns/octodns-ns1/blame/6692bc0f60eaee0b76cc936b0e8631ecc86b824a/octodns_ns1/__init__.py#L7-L8.
If I pull down the pre-built docker container and look at the top of the file what I get has the current lines:
$ sudo docker run --rm -ti octodns/ns1:2022.01-beta2 /bin/bash
...
root@cdf06ba57542:/opt/octodns# head /opt/octodns/env/lib/python3.10/site-packages/octodns_ns1/__init__.py
#
#
#
from logging import getLogger
from itertools import chain
from collections.abc import Mapping
from collections import OrderedDict, defaultdict
from ns1 import NS1
from ns1.rest.errors import RateLimitException, ResourceExceptionNot sure what's up with the 2022.01-beta2 bit and why there's not a latest tag or anything: https://hub.docker.com/r/octodns/ns1/tags, but don't think that's related to this problem.
Can you provide reproduction steps since i can't figure out a way to hit the issue you're having?
Apologies for the somewhat sloppy late-night bug report, I should have checked octodns-ns1 for that particular line.
It looks like GitHub ate the top line of the code block so you couldn't see the command I used:
docker run -v "$(pwd):/octodns" --workdir /octodns octodns/octodns:2022.01-beta2 octodns-sync --config-file /octodns/test.yaml
Here's the test.yaml file I used, nothing particular about it:
---
manager:
max_workers: 2
providers:
yaml:
class: octodns.provider.yaml.YamlProvider
directory: ./yaml
default_ttl: 3600
enforce_order: True
ns1:
class: octodns_ns1.Ns1Provider
api_key: xxx
zones:
xxx.com.:
lenient: true
sources:
- ns1
targets:
- yaml
Trying with the octodns/ns1 flavor of the docker image, things are working as expected.
Trying with the octodns/ns1 flavor of the docker image, things are working as expected.
Ah. OK. This is a docker specific issue then. Looks like the base octodns variant hasn't been updated since ns1 was released 2022.01-beta2.
Will look into what's up there, or parkr may know off-hand.
Actions run to publish new docker images should be running momentarily and will push a fresh version of octodns/octodns.
Happy to rework the versions. Maybe we do: octodns/variant:variant_version and octodns/octodns:octodns_version.
Happy to rework the versions. Maybe we do: octodns/variant:variant_version and octodns/octodns:octodns_version.
I don't have strong opinions or really even useful thoughts on the versioning specifics. Date-based seems reasonable and can be tracked back to versions so long as they're picking up the latest as of the date. Since there's both a core octodns version and provider version I'm not sure there's a better way to signify both.
Does make me think that maybe we should include versions in the logging output of sync. I guess the manager can print out the octoDNS core version and the provider init's can include the module versions. I'll file something about that.
Edit: I think this issue can be closed. You good @flz?, doh already was. I just needed to refresh.
Filed octodns/octodns#891 which will start including the versions of things in the logs. I think this will be helpful in general and specifically for docker. See octodns/octodns#891 (comment) for an example.