datahub-project / datahub

Hi!,
when ingesting the audit data from Starburst, the integration fails silently with the following error:

- Error while parsing TrinoJoinedAccessEvent: 1 validation error for TrinoJoinedAccessEvent
create_time
  none is not an allowed value (type=type_error.none.not_allowed)

This integration executes a query that returns a create_time field with a DateTime format, but the code only returns the value if it is an string:

datahub/metadata-ingestion/src/datahub/ingestion/source/usage/starburst_trino_usage.py

Line 190 in d3fb698

def _convert_str_to_datetime(self, v):

To fix this, I have slightly modified the code to return a value if the value is a DateTime type:

def _convert_str_to_datetime(self, v):
        if isinstance(v, str):
            isodate = parser.parse(v)  # compatible with Python 3.6+
            return isodate
        if isinstance(v, datetime):
            return v

After this change, the integration works as expected.

Should I PR this or is it enough with this issue?

Thanks!

Hello Sergio.

Please open a PR. cc @hsheth2

Closed by #10345

Starburst/Trino integration not working properly