Timestamp Encoding Discrepancy: Julian vs. Unix Epoch

[sbertrand]

What happened?

I've encountered an issue with the way the BigQuery emulator I'm using integrates dates into its database and returns them in query results. Specifically, it seems that the emulator encodes TIMESTAMP values in the Julian format, whereas BigQuery uses the Unix Epoch system. This discrepancy is causing problems because the BigQuery libraries (particularly the Python library) expect to receive an integer in Unix Epoch format, but they are getting a floating-point number in Julian format.

What did you expect to happen?

The returned TIMESTAMP value should be in Unix Epoch format, which is the expected format for compatibility with BigQuery libraries.

How can we reproduce it (as minimally and precisely as possible)?

Steps to Reproduce

• Insert a TIMESTAMP value.
• Query the TIMESTAMP value.
• Observe that the returned value is in Julian format (floating-point) instead of Unix Epoch format (integer).

Anything else we need to know?

Actual Behavior

The returned TIMESTAMP value is in Julian format, causing compatibility issues with the BigQuery Python library.
Partial log error :
File "/Volumes/Shared/Dev/TrainIC/outillage/toolbox-api/.venv/lib/python3.8/site-packages/google/cloud/bigquery/_helpers.py", line 204, in _timestamp_from_json return _datetime_from_microseconds(int(value)) ValueError: invalid literal for int() with base 10: '1721123541.897000' ", "severity": "ERROR", "time": "2024-07-26T18:44:19.443"} {"message": "Catching global error: invalid literal for int() with base 10: '1721123541.897000' (500)", "severity": "ERROR", "time": "2024-07-26T18:44:19.443"}

[Dan Hansen]

Can you provide a reproduction case using the Python client?
#32 contains some more context around how the BigQuery API differs its timestamp output based on request parameters

[sbertrand]

Ok i'l try to use query_and_wait instead of query. Do you think it's possible to use query() with formatOptions.useInt64Timestamp=true ?

[Dan Hansen]

I'd imagine its possible, but I haven't reproduced the issue yet. We use BigQueryClient.query() in our project without issue.

Could you provide a test case that uses the Python client?