Timezone-aware datetimes are no longer supported

[Ashwin Madavan]

I am trying to upgrade from 1.5.0 to 1.8.0, and my unit tests are failing because of a "TypeError: Type datetime64[ns, Timezone('UTC')] was not understood." on a non-index column. Here's a quick reproduction. This works on 1.5.0 and fails on 1.6.0 and 1.8.0. (pyspark 3.0.2 and pandas 1.1.5)

from databricks import koalas
import datetime
import pandas

df = pandas.DataFrame({
    "time": [datetime.datetime.now(tz=datetime.timezone.utc)],
})

koalas.from_pandas(df)
# TypeError: Type datetime64[ns, UTC] was not understood.

Originally posted by @ashwin153 in #2102 (comment)

[Ashwin Madavan]

IMO this feels more like a regression than an enhancement, since this worked in previous versions.

[Ashwin Madavan]

Any update on this?

[Hyukjin Kwon]

The previous implementation was not really correct. So this is disabled for now. To mimic the previous behaviour, you can manually localize it by converting UTC to your local timezone (without timezone), and use it for now.