server_version requires Kubernetes config file

Question

server_version requires Kubernetes config file

NickG123 opened this issue 8 years ago · comments

Hi there,
I have another issue for you.

It appears that the server_version function requires a Kubernetes config file. This causes problems when trying to create objects that make use of the function with a K8sConfig object provided.

A simple test case:

from kubernetes import K8sConfig, K8sCronJob

cfg_cert = K8sConfig(kubeconfig=None, api_host="<redacted>", cert=("apiserver.pem", "apiserver-key.pem"))
cj = K8sCronJob(config=cfg_cert, name="test")
cj.create()

gives the stack trace:

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    cj = K8sCronJob(config=cfg_cert, name="test")
  File "<redacted>/env/lib/python2.7/site-packages/kubernetes/K8sCronJob.py", line 23, in __init__
    name=name
  File "<redacted>/env/lib/python2.7/site-packages/kubernetes/K8sObject.py", line 62, in __init__
    self.model = str_to_class(obj_type)
  File "<redacted>/env/lib/python2.7/site-packages/kubernetes/utils/Helpers.py", line 105, in str_to_class
    _class = getattr(module, obj_type)()
  File "<redacted>/env/lib/python2.7/site-packages/kubernetes/models/v2alpha1/CronJob.py", line 28, in __init__
    v = server_version()
  File "<redacted>/env/lib/python2.7/site-packages/kubernetes/utils/Helpers.py", line 113, in server_version
    obj = K8sObject(obj_type='Pod', name='obj')
  File "<redacted>/env/lib/python2.7/site-packages/kubernetes/K8sObject.py", line 54, in __init__
    config = K8sConfig()
  File "<redacted>/env/lib/python2.7/site-packages/kubernetes/K8sConfig.py", line 57, in __init__
    raise IOError('K8sConfig: kubeconfig: [ {0} ] doesn\'t exist.'.format(kubeconfig))
IOError: K8sConfig: kubeconfig: [ <redacted>/.kube/config ] doesn't exist.

Tested on kubernetes-py 1.4.7.17 and python 2.7.5.

Thanks,
Nick

Francis Roch · Answer 1 · Fri Mar 31 2017 00:08:35 GMT+0800 (China Standard Time)

Hi @NickG123, thanks for taking the time to raise this issue.

Can you please fetch version 1.4.7.18 and try again? The way we check for server_version for CronJobs and StatefulSets has been updated to break the assumption that a ~/.kube/config exists to fall back on.

Nick Gerow · Answer 2 · Fri Mar 31 2017 04:04:53 GMT+0800 (China Standard Time)

It looks like that solved the problem with looking for the config file, but I'm having some issues with the new is_reachable guard.

I noticed my server was always considered to be unreachable according to this code and did some digging and found that my config.api_host passed into is_reachable is of the format "https://<ip_address>:<port>".
This parameter eventually gets passed into socket.inet_aton, which simply expects an IP address and thus throws an exception.

Francis Roch · Answer 3 · Fri Mar 31 2017 04:19:17 GMT+0800 (China Standard Time)

i just pushed another build with a modification to the reachability helper. it uses requests.get() instead of attempting a socket connection. can you please try that one?

Francis Roch · Answer 4 · Fri Mar 31 2017 04:19:44 GMT+0800 (China Standard Time)

1.4.7.19, in fact.

Nick Gerow · Answer 5 · Fri Mar 31 2017 04:37:02 GMT+0800 (China Standard Time)

Closer, now I get certificate validation errors because my cert files are not passed in to the requests call

Nick Gerow · Answer 6 · Sat Apr 01 2017 01:38:55 GMT+0800 (China Standard Time)

After some further use of this version, this creates a lot more issues, because now is_reachable is failing for me throughout the code base.

Francis Roch · Answer 7 · Sat Apr 01 2017 01:43:22 GMT+0800 (China Standard Time)

Hi @NickG123, do feel free to contribute a pull request which addresses your specific use case. We'll review and integrate it as time allows.

Nick Gerow · Answer 8 · Sat Apr 01 2017 02:40:48 GMT+0800 (China Standard Time)

I would be happy to create a pull request, but I'm unsure what direction you would like to head in here.

In bd0f14a is_reachable was changed from socket.inet_aton(ip) to requests.get(ip). This completely changes the behaviour of the function.

Before the change, inet_aton(ip) was used, which simply converts a ipv4 style IP address to 32-bit packed binary format. It's possible the intent here was simply to validate that the IP address is valid, in which case I would argue that the function should be renamed to something more appropriate, and should not be used here since config.api_host is not necessarily an IP address.

After the change, requests.get(ip) is used, which actually performs a query against the provided URL. This has two problems. First, it does not make use of the certs provided by the user, unlike other calls to the kubernetes server. Second, it is used in cases that before were simply passing in IP addresses. For example, is_reachable is run on the the external IPs in a Service. As far as I know, external IPs are usually a simply IP address, not a URL, and requests.get does not work on an IP (at least in python 2.7):

>>> requests.get("127.0.0.1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\requests\api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 474, in request
    prep = self.prepare_request(req)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 407, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "C:\Python27\lib\site-packages\requests\models.py", line 302, in prepare
    self.prepare_url(url, params)
  File "C:\Python27\lib\site-packages\requests\models.py", line 382, in prepare_url
    raise MissingSchema(error)
requests.exceptions.MissingSchema: Invalid URL '127.0.0.1': No schema supplied. Perhaps you meant http://127.0.0.1?

My suggestion would be rolling back the behaviour of is_reachable to use inet_aton, and renaming it to is_valid_ip. Then, create a new function (possibly in K8sObject.py so it can take advantage of the existing request function there) called is_reachable that checks if the Kubernetes host is reachable, and then use that function instead of the newly added calls to is_reachable in K8sCronJob.py and K8sPetSet.py.

Alternatively, instead of checking that the server is reachable ahead of time, the call to temp.server_version could simply be wrapped in a try... Except: pass

Let me know if you like the idea and I'll get working on a merge request.
Nick

Francis Roch · Answer 9 · Sat Apr 08 2017 03:11:02 GMT+0800 (China Standard Time)

i've updated the behaviour of is_reachable(). it takes as parameter a K8sConfiguration() object. It first attempts a socket connection, and falls back on an HttpRequest helper object which handles any auth details present in the config. This should fix your problem.

Nick Gerow · Answer 10 · Tue Apr 11 2017 22:25:07 GMT+0800 (China Standard Time)

You're correct that you have fixed my problem.
However, (just FYI) I think you are confused as to the behaviour of socket.inet_aton(cfg.api_host). It does not attempt a socket connection.
Nick

Francis Roch · Answer 11 · Wed Apr 12 2017 23:12:14 GMT+0800 (China Standard Time)

You're quite right. Thanks for pointing that out. I've added a fix that addresses the problem.