Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-25-2022 07:12 PM
I'm using the Databricks SQL Endpoint and I'm attempting to create an external table on top of an existing parquet file. I can do this so long as my table definition does not include a reference to a decimal or timestamp/date datatype.
ex. This works
CREATE EXTERNAL TABLE sampleTable (
ColA STRING,
ColB INT)
LOCATION 'storagelocation' STORED AS PARQUETWhen I need to reference a timestamp column "ColC TIMESTAMP", I receive the following error..."Parquet does not support timestamp. See HIVE-6384"
I found in Databricks you can specify the hive version by using the following, but this did not resolve my issue.
spark.sql.hive.metastore.version 1.2.1
spark.sql.hive.metastore.jars builtinAny ideas on a way around this?
