XMLGregorianCalendar type blocks this lib in spark projects
atais opened this issue · comments
steps
define schema A that has type="dt:date"
element
compile them.
problem
No Encoder found for javax.xml.datatype.XMLGregorianCalendar
expectation
Spark Encoders are actually a bit of pain in the ***, because there is not a lot of them right now:
https://spark.apache.org/docs/2.1.0/api/java/org/apache/spark/sql/Encoders.html
So even the types prosed in #376
will not solve the issue, because there is only an Encoder for java.sql.Date
and java.sql.Timestamp
I know it sounds medieval but we basically used String
for that fields 😢
So I guess, the most flexible and easy solution would be to have a field:
dateTypeClass where user could provide a full path to the desired class. Let it be current Calendar, Date or String.
WDYT?
Could you comment on #376 please?