How to read enum[] (enum of array) datatype from postgres using spark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-05-2023 04:29 AM
We are trying to read a column which is enum of array datatype from postgres as string datatype to target. We could able to achieve this by expilcitly using concat function while extracting like below
val jdbcDF3 = spark.read
.format("jdbc")
.option("url", <jdbc url>)
.option("query", "SELECT concat(colname) as colname FROM <tablename> ")
.load()
Does spark not support reading array datatype as string by default without using concat function?
We are getting the "SQLException: Unsupported type ARRAY" error if we dont use concat function while extracting
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-05-2023 05:12 AM
You can try custom schema for JDBC read
.option("customSchema", "colname STRING")
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-05-2023 08:01 PM
Tried.. No luck @Hubert Dudek

