cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to read enum[] (enum of array) datatype from postgres using spark

RamyaN
New Contributor II

We are trying to read a column which is enum of array datatype from postgres as string datatype to target. We could able to achieve this by expilcitly using concat function while extracting like below

val jdbcDF3 = spark.read

 .format("jdbc")

 .option("url", <jdbc url>)

 .option("query", "SELECT concat(colname) as colname FROM <tablename> ")

 .load()

Does spark not support reading array datatype as string by default without using concat function?

We are getting the "SQLException: Unsupported type ARRAY" error if we dont use concat function while extracting

2 REPLIES 2

Hubert-Dudek
Esteemed Contributor III

You can try custom schema for JDBC read

.option("customSchema", "colname STRING") 

RamyaN
New Contributor II

Tried.. No luck @Hubert Dudek​ 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group