Error org.apache.spark.SparkSQLException: Unsupported type TIMESTAMP_WITH_TIMEZONE

KaushikMaji
New Contributor II

Hi Guys,

I am trying to load data from a custom JDBC data source like below. I am getting this error, in spite of specifying "customSchema" attribute on the timestamp column. Do you know how to resolve this?

material_master = spark.read \

.format("jdbc") \

.option("url","<my jdbc url>") \

.option("driver", "<my jdbc driver>") \

.option("customSchema", "timestamp STRING") \

.option("query","SELECT CONO, STAT, ITNO, timestamp FROM MITMAS LIMIT 10") \

.load()