New Python library version returning AttributeError: 'NoneType' object has no attribute '_composed_schemas'

Question

New Python library version returning AttributeError: 'NoneType' object has no attribute '_composed_schemas'

diego-duarte-wcar opened this issue 3 years ago · comments

Describe the bug

Error AttributeError: 'NoneType' object has no attribute '_composed_schemas' after upgrading DD Python libs to datadog==0.40.1 and datadog-api-client==1.0.0b6, when querying MetricsAPI.

To Reproduce
Steps to reproduce the behavior:

Create your new Venv
Install the latest DD libs (versions stated above)
Follow the exact documentation for search query, as here https://docs.datadoghq.com/api/latest/metrics/#search-metrics, but a slightly change:

with ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = metrics_api.MetricsApi(api_client)
    _from = datetime(2021, 2, 1).strftime('%s')  # int | Start of the queried time period, seconds since the Unix epoch.
    to = datetime(2021, 3, 1).strftime('%s')  # int | End of the queried time period, seconds since the Unix epoch.
    query = 'avg:aws.apigateway.latency{component:tag-tag-tag,sla:true}'  # str | Query string.

    print(_from, to)
    print(api_instance)
    # example passing only required values which don't have defaults set
    try:
        # Query timeseries points
        api_response = api_instance.query_metrics(_from, to, query)
    except ApiException as e:
        print("Exception when calling MetricsApi->query_metrics: %s\n" % e)

Expected behavior
The correct object returned in console

Screenshots
Error:

Traceback (most recent call last):
  File "ddn.py", line 30, in <module>
    pprint(api_response)
  File "/usr/lib/python3.8/pprint.py", line 53, in pprint
    printer.pprint(object)
  File "/usr/lib/python3.8/pprint.py", line 148, in pprint
    self._format(object, self._stream, 0, 0, {}, 0)
  File "/usr/lib/python3.8/pprint.py", line 170, in _format
    rep = self._repr(object, context, level)
  File "/usr/lib/python3.8/pprint.py", line 404, in _repr
    repr, readable, recursive = self.format(object, context.copy(),
  File "/usr/lib/python3.8/pprint.py", line 417, in format
    return _safe_repr(object, context, maxlevels, level, self._sort_dicts)
  File "/usr/lib/python3.8/pprint.py", line 569, in _safe_repr
    rep = repr(object)
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 135, in __repr__
    return self.to_str()
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 347, in to_str
    return pprint.pformat(self.to_dict())
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 343, in to_dict
    return model_to_dict(self, serialize=False)
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 1410, in model_to_dict
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 1410, in <listcomp>
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 1410, in model_to_dict
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 1410, in <listcomp>
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/home/ab000113/workspace/venv-test/lib/python3.8/site-packages/datadog_api_client/v1/model_utils.py", line 1395, in model_to_dict
    if model_instance._composed_schemas:
AttributeError: 'NoneType' object has no attribute '_composed_schemas'

Environment and Versions (please complete the following information):
A clear and precise description of your setup:
Working version as tested here: datadog==0.39.0, datadog-api-client==1.0.0b5
Broken version: datadog==0.40.1, datadog-api-client==1.0.0b6
Python ver: 3.8.5
Ubuntu: 20.04

Additional context
Add any other context about the problem here.

github-actions · Answer 1 · Sun Apr 18 2021 08:01:45 GMT+0800 (China Standard Time)

Thanks for your contribution!

This issue has been automatically marked as stale because it has not had activity in the last 30 days. Note that the issue will not be automatically closed, but this notification will remind us to investigate why there's been inactivity. Thank you for participating in the Datadog open source community.

If you would like this issue to remain open:

Verify that you can still reproduce the issue in the latest version of this project.
Comment that the issue is still reproducible and include updated details requested in the issue template.

Piotr Boruc · Answer 2 · Thu Jun 10 2021 20:05:34 GMT+0800 (China Standard Time)

Hey,

I've also hit this issue, but in my case it was one (maybe more) metric that is failing and others are working just fine. Problematic metrics is system.mem.used. I've tried with few others and same code works with i.e. system.mem.total system.cpu.user
Here is code it can be reproduced with. I'm on Python 3.9.5. By simply setting metric = "system.mem.used" you can get error. BTW, code assumes usage of ENV vars:

DD_SITE="datadoghq.eu"
DD_API_KEY="xxxxx"
DD_APP_KEY="xxxx"

import time
from datetime import datetime, timedelta
from datadog_api_client.v1 import ApiClient, ApiException, Configuration
from datadog_api_client.v1.api import metrics_api

configuration = Configuration()

metric = "system.mem.used"

# Enter a context with an instance of the API client
with ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = metrics_api.MetricsApi(api_client)

    last_hour_date_time = datetime.now() - timedelta(hours=1)
    unixtime_now = int(time.mktime(datetime.now().timetuple()))
    unixtime_last_hour = int(time.mktime(last_hour_date_time.timetuple()))

    _from = unixtime_last_hour  # int | Start of the queried time period, seconds since the Unix epoch.
    to = unixtime_now  # int | End of the queried time period, seconds since the Unix epoch.
    query = 'avg:' + metric + ' {*} by {application, host}'

    # example passing only required values which don't have defaults set
    try:
        # Query timeseries points
        api_response = api_instance.query_metrics(_from, to, query)
        data = api_response.to_dict()
        print(data)

    except ApiException as e:
        print("Exception when calling MetricsApi->query_metrics: %s\n" % e)

Error + stack trace:

Traceback (most recent call last):
  File "/Users/wasilp01/git/dps/datadog-scripts/metrics.py", line 27, in <module>
    data = api_response.to_dict()
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 344, in to_dict
    return model_to_dict(self, serialize=False)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1420, in model_to_dict
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1420, in <listcomp>
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1420, in model_to_dict
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1420, in <listcomp>
    result[attr] = [model_to_dict(x, serialize=serialize) for x in value]
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1405, in model_to_dict
    if model_instance._composed_schemas:
AttributeError: 'NoneType' object has no attribute '_composed_schemas'

Thomas Hervé · Answer 3 · Wed Jun 16 2021 00:44:11 GMT+0800 (China Standard Time)

Hi,

Sorry I found the issue, it's about the unit returned for the query. We'll have a fix for it soon, thanks for your patience.

Piotr Boruc · Answer 4 · Wed Jun 16 2021 17:21:43 GMT+0800 (China Standard Time)

Awesome, thanks! 👍

Thomas Hervé · Answer 5 · Mon Jun 21 2021 23:09:08 GMT+0800 (China Standard Time)

Should be fixed with #486, will be in next release. Thanks!

Piotr Boruc · Answer 6 · Fri Jul 09 2021 15:44:35 GMT+0800 (China Standard Time)

Hey,

Unfortunately now (using version 1.2.0) running exactly the same script (#321 (comment)) fails with brand new error with huge stack trace... please take a look:

Traceback (most recent call last):
  File "/Users/wasilp01/git/dps/datadog-scripts/test_githib.py", line 26, in <module>
    api_response = api_instance.query_metrics(_from, to, query)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/api/metrics_api.py", line 476, in query_metrics
    return self._query_metrics_endpoint.call_with_http_info(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/api_client.py", line 844, in call_with_http_info
    return self.api_client.call_api(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/api_client.py", line 386, in call_api
    return self.__call_api(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/api_client.py", line 214, in __call_api
    return_data = self.deserialize(response_data, response_type, _check_type)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/api_client.py", line 307, in deserialize
    deserialized_data = validate_and_convert_types(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1310, in validate_and_convert_types
    converted_instance = attempt_convert_item(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1205, in attempt_convert_item
    raise conversion_exc
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1197, in attempt_convert_item
    return deserialize_model(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1113, in deserialize_model
    return model_class(**kw_args)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1483, in wrapped_init
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model/metrics_query_response.py", line 194, in __init__
    setattr(self, var_name, var_value)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 143, in __setattr__
    self[attr] = value
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 316, in __setitem__
    self.set_attribute(name, value)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 119, in set_attribute
    value = validate_and_convert_types(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1360, in validate_and_convert_types
    validate_and_convert_types(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1310, in validate_and_convert_types
    converted_instance = attempt_convert_item(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1205, in attempt_convert_item
    raise conversion_exc
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1197, in attempt_convert_item
    return deserialize_model(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1113, in deserialize_model
    return model_class(**kw_args)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1483, in wrapped_init
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model/metrics_query_metadata.py", line 213, in __init__
    setattr(self, var_name, var_value)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 143, in __setattr__
    self[attr] = value
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 316, in __setitem__
    self.set_attribute(name, value)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 119, in set_attribute
    value = validate_and_convert_types(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1360, in validate_and_convert_types
    validate_and_convert_types(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1310, in validate_and_convert_types
    converted_instance = attempt_convert_item(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1205, in attempt_convert_item
    raise conversion_exc
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1197, in attempt_convert_item
    return deserialize_model(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1105, in deserialize_model
    return model_class(model_data, **kw_args)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1483, in wrapped_init
    return fn(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model/point.py", line 181, in __init__
    self.value = value
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 143, in __setattr__
    self[attr] = value
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 263, in __setitem__
    self.set_attribute(name, value)
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 119, in set_attribute
    value = validate_and_convert_types(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1360, in validate_and_convert_types
    validate_and_convert_types(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1310, in validate_and_convert_types
    converted_instance = attempt_convert_item(
  File "/usr/local/lib/python3.9/site-packages/datadog_api_client/v1/model_utils.py", line 1210, in attempt_convert_item
    raise get_type_error(input_value, path_to_item, valid_classes, key_type=key_type)
datadog_api_client.v1.exceptions.ApiTypeError: Invalid type for variable '1'. Required value type is float and passed type was NoneType at ['received_data']['series'][12]['pointlist'][0]['value'][1]

Not sure if it is related or should be considered separate issue, though :/

Thomas Hervé · Answer 7 · Mon Jul 19 2021 18:23:32 GMT+0800 (China Standard Time)

Hi, the last traceback is fixed by #516 and will be in the next release.

Piotr Boruc · Answer 8 · Tue Aug 17 2021 18:33:20 GMT+0800 (China Standard Time)

hey @therve 1.3.0 worked like a charm :D thanks!