Resolved! Vectorized reading of parquet file containing decimal type column(s)
I was trying to read a parquet file, and write to a delta table, with a parquet file that contains decimal type columns. I encountered a problem that is pretty neatly described by this kb.databricks article, and which I solved by disabling the vector...
- 2335 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
@Alberte Mørk​ :The behavior you observed is due to a known issue in Apache Spark when vectorized reading is used with Parquet files that contain decimal type columns. As you mentioned, the issue can be resolved by disabling vectorized reading for th...
- 2 kudos