Integer Overflow Error on String Field

[Nick Valin]

If you have a dataset in Domo with a field of type string that contains numeric data with values larger than your system's signed integer maximum (i.e. some sort of 12+ digit numeric id value), the ds_get function will throw and integer overflow error.

This occurs because the Domo API exports this field in the csv with quotation marks to denote that it is a string, not a number, causing the pandas read_csv function to create the field with a dtype of "object". The ds_get function then (on line 195) attempts to interpret all object type fields as dates. Then, down the stack, the Pandas datetime parser tries to read the entire string as an integer. Because this happens in Cython, it throws an overflow error.

Here's the stack trace (minus my code):

[ERROR] OverflowError: signed integer is greater than maximum
Traceback (most recent call last):
File "/var/task/pydomo/init.py", line 195, in ds_get
df[col] = to_datetime(df[col])
File "/var/task/pandas/core/tools/datetimes.py", line 805, in to_datetime
values = convert_listlike(arg._values, format)
File "/var/task/pandas/core/tools/datetimes.py", line 472, in _convert_listlike_datetimes
allow_object=True,
File "/var/task/pandas/core/arrays/datetimes.py", line 2081, in objects_to_datetime64ns
require_iso8601=require_iso8601,
File "pandas/_libs/tslib.pyx", line 364, in pandas._libs.tslib.array_to_datetime
File "pandas/_libs/tslib.pyx", line 591, in pandas._libs.tslib.array_to_datetime
File "pandas/_libs/tslib.pyx", line 726, in pandas._libs.tslib.array_to_datetime_object
File "pandas/_libs/tslib.pyx", line 717, in pandas._libs.tslib.array_to_datetime_object
File "pandas/_libs/tslibs/parsing.pyx", line 243, in pandas._libs.tslibs.parsing.parse_datetime_string
File "/var/task/dateutil/parser/_parser.py", line 1374, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/var/task/dateutil/parser/_parser.py", line 655, in parse
ret = self._build_naive(res, default)
File "/var/task/dateutil/parser/_parser.py", line 1241, in _build_naive
naive = default.replace(**repl)