Fix RecursionError with Python3.6

Question

Fix RecursionError with Python3.6

Lygoria opened this issue 4 years ago · comments

Camille Levavasseur commented 4 years ago

Hi,
With Python3.6, I got this error (contrail-api-cli 0.3.0):

File "/usr/lib/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
    [Previous line repeated 308 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object

After gevent update, I found this warning:
MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Ple ase monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util (/home/vagrant/env/lib/python3.6/site-packages/urllib3/util/__init__.py)', 'urllib3.util.ssl_ (/home/vagrant/env/lib/python3.6/site-p ackages/urllib3/util/ssl_.py)'].

Once those lines removed https://github.com/eonpatapon/contrail-api-cli/blob/master/contrail_api_cli/utils.py#L25-L26, my error was gone.

Could you integrate this fix ?

Jean-Philippe Braun · Answer 1 · Wed Feb 26 2020 00:44:06 GMT+0800 (China Standard Time)

What is your gevent version ?

Camille Levavasseur · Answer 2 · Wed Feb 26 2020 01:38:24 GMT+0800 (China Standard Time)

It was gevent 1.2 initially (as requested by setup.py) then I tried with 1.3 (and maybe 1.4)

Jean-Philippe Braun · Answer 3 · Wed Feb 26 2020 15:49:23 GMT+0800 (China Standard Time)

Can you try with this patch ?

Camille Levavasseur · Answer 4 · Wed Feb 26 2020 16:03:30 GMT+0800 (China Standard Time)

Same error:

Traceback (most recent call last):
  File "contrail_cleaner.py", line 161, in <module>
    cleaner = ContrailCleaner(**args)
  File "contrail_cleaner.py", line 74, in __init__
    Context().schema = DummySchema()
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/schema.py", line 214, in __init__
    DummyResourceSchema()
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/utils.py", line 117, in __call__
    cls.instance = super(Singleton, cls).__call__(*args, **kw)
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/schema.py", line 238, in __init__
    [c.type for c in RootCollection(fetch=True)]
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/resource.py", line 403, in __init__
    return super(RootCollection, self).__init__('', **kwargs)
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/resource.py", line 264, in __init__
    self.fetch(recursive=recursive)
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/resource.py", line 47, in wrapper
    return f(self, *args, **kwargs)
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/resource.py", line 374, in fetch
    data = self.session.get_json(self.href, **params)
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/client.py", line 22, in wrapper
    return f(*args, **kwargs)
  File "/home/vagrant/shared/tmp/contrail-api-cli/contrail_api_cli/client.py", line 130, in get_json
    return self.get(url, params=kwargs).json()
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 1123, in get
    return self.request(url, 'GET', **kwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 772, in request
    auth_headers = self.get_auth_headers(auth)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 1183, in get_auth_headers
    return auth.get_headers(self, **kwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/plugin.py", line 95, in get_headers
    token = self.get_token(session)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 88, in get_token
    return self.get_access(session).auth_token
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access
    self.auth_ref = self.get_auth_ref(session)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 184, in get_auth_ref
    authenticated=False, log=False, **rkwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 1131, in post
    return self.request(url, 'POST', **kwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 913, in request
    resp = send(**kwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 1004, in _send_request
    resp = self.session.request(method, url, **kwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/vagrant/env/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/vagrant/env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/home/vagrant/env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request
    self._validate_conn(conn)
  File "/home/vagrant/env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn
    conn.connect()
  File "/home/vagrant/env/lib/python3.6/site-packages/urllib3/connection.py", line 336, in connect
    cert_reqs=resolve_cert_reqs(self.cert_reqs),
  File "/home/vagrant/env/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 273, in create_urllib3_context
    context.options |= options
  File "/usr/lib/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/lib/python3.6/ssl.py", line 465, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 308 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object

Jean-Philippe Braun · Answer 5 · Wed Feb 26 2020 16:22:49 GMT+0800 (China Standard Time)

How are using contrail-api-cli ? As a lib ? Wonder how do you import it

Camille Levavasseur · Answer 6 · Wed Feb 26 2020 16:35:35 GMT+0800 (China Standard Time)

from contrail_api_cli.client import SessionLoader
from contrail_api_cli.context import Context
from contrail_api_cli.resource import Collection
from contrail_api_cli.schema import DummySchema


class ContrailCleaner(object):

    def __init__(self, [...]):

        Context.session = SessionLoader().make([...])
        Context().schema = DummySchema()

Following https://github.com/eonpatapon/contrail-api-cli/blob/master/doc/usage.rst#basic-auth-on-localhost8095

Jean-Philippe Braun · Answer 7 · Wed Feb 26 2020 16:45:03 GMT+0800 (China Standard Time)

Thanks, Do you make any imports before that ? Also are you connecting to an SSL enabled contrail-api ?

Camille Levavasseur · Answer 8 · Wed Feb 26 2020 16:59:16 GMT+0800 (China Standard Time)

Oh, you're right, urllib3 is imported so we can disable warnings about InsecureRequests:

import urllib3

from contrail_api_cli.client import SessionLoader
from contrail_api_cli.context import Context
from contrail_api_cli.resource import Collection
from contrail_api_cli.schema import DummySchema
from contrail_api_cli.exceptions import ChildrenExists, BackRefsExists, ResourceNotFound
from keystoneauth1.exceptions.http import Conflict

urllib3.disable_warnings()


class ContrailCleaner(object):

When removing

import urllib3
urllib3.disable_warnings()

it works (with a lot of warnings tho)

Jean-Philippe Braun · Answer 9 · Wed Feb 26 2020 17:26:20 GMT+0800 (China Standard Time)

Unfortunately I'm not able to reproduce the issue. Maybe you should import `urllib3` after `contrail_api_cli` imports ? I tried the following using python 3.6: ``` import urllib3 from contrail_api_cli.client import SessionLoader from contrail_api_cli.resource import Collection from contrail_api_cli.context import Context from contrail_api_cli.schema import DummySchema urllib3.disable_warnings() session = SessionLoader().make(host="localhost", port=8082, os_username="admin", os_password="password", os_cacert=None, os_cert=None, os_key=None, collect_timing=True, insecure=False, timeout=1) Context().session = session Context().schema = DummySchema() print(len(Collection('virtual-network'))) ```

Camille Levavasseur · Answer 10 · Wed Feb 26 2020 17:41:32 GMT+0800 (China Standard Time)

It works fine with your fix 3b4ceec and import urllib3 after contrail_api_cli imports

Thanks !