Error org.apache.spark.SparkSQLException: Unsupported type TIMESTAMP_WITH_TIMEZONE

[KaushikMaji]

Hi Guys,

I am trying to load data from a custom JDBC data source like below. I am getting this error, in spite of specifying "customSchema" attribute on the timestamp column. Do you know how to resolve this?

material_master = spark.read \

.format("jdbc") \

.option("url","<my jdbc url>") \

.option("driver", "<my jdbc driver>") \

.option("customSchema", "timestamp STRING") \

.option("query","SELECT CONO, STAT, ITNO, timestamp FROM MITMAS LIMIT 10") \

.load()

[RaghavendraY]

May try cast timestamp, like "SELECT CONO, STAT, ITNO, cast(timestamp as varchar) FROM MITMAS LIMIT 10"

[Anonymous]

Hi @Kaushik Maji​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!