CS client does not finish if there is only one element and fetch_list is used

[Daniel Herrmann]

When using a list API call (for example listVPCs) and using for example the ID as parameter (id=), I'd expect CS to return that single entry. However, the script times out, CS never finishes. When stopping the script, the following traceback is shown:

Traceback (most recent call last):
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./cloud_access_tool", line 321, in <module>
    main()
  File "./cloud_access_tool", line 312, in main
    at.action_migrate(args.database)
  File "/Users/dherrman/PycharmProjects/cloud_access_tool/AccessTool.py", line 710, in action_migrate
    acl_list = self.cs.listNetworkACLLists(fetch_list=True, id=acl)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/cs/client.py", line 103, in handler
    return self._request(command, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/cs/client.py", line 140, in _request
    **{kwarg: kwargs})
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/Users/dherrman/venc/cloud_access_tool/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1002, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 865, in read
    return self._sslobj.read(len, buffer)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 625, in read
    v = self._sslobj.read(len, buffer)
KeyboardInterrupt

When omitting the fetch_list=True parameter, the expected result is shown:

{'count': 1, 'networkacllist': [{'id': 'eafcf9a3-8b01-4cda-b206-c3ebfed2543c', 'name': 'dhe-test-migrate', 'description': '', 'vpcid': 'bfdfa866-e533-4270-b284-063ff0e1321a'}]}

[Daniel Herrmann]

Hm, looks like I have to take that back. The issue only happens with some API calles. When using listPublicIpAddresses for example, a List of length 1 is returned, as expected (using the id as match criterion). When using listNetworkACLLists, the issue happens as described above. Looks more like a server side issue to me...

[Yoan Blanc]

self.cs.listNetworkACLLists(fetch_list=True, id=acl)

• fetch_list wants to paginate
• id= wants exactly one item (according to the CloudStack philosophy)

Is it any better without fetch_list?

[Daniel Herrmann]

The funny thing is:

ip_addr = self.cs.listPublicIpAddresses(fetch_list=True, id=ip)

Works just fine and returns a List of length 1.

self.cs.listNetworkACLLists(fetch_list=True, id=acl)

times out.

self.cs.listNetworkACLLists(id=acl)

Works just fine.

[Yoan Blanc]

@waza-ari you may want to increase the timeout when creating the CloudStack client: self.cs = CloudStack(endpoint, key, secret, timeout=30). I guess that listNetworkACLLists is an heavy operation taking longer than the default 10 seconds timeout.

[Daniel Herrmann]

@greut thank you, I will try. When omitting the fetch_list parameter, the result is returned within parts of a second, but let me try and report back.

[Yoan Blanc]

probably closed by #65