Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-25-2022 07:12 PM
I'm using the Databricks SQL Endpoint and I'm attempting to create an external table on top of an existing parquet file. I can do this so long as my table definition does not include a reference to a decimal or timestamp/date datatype.
ex. This works
CREATE EXTERNAL TABLE sampleTable (
ColA STRING,
ColB INT)
LOCATION 'storagelocation' STORED AS PARQUET
When I need to reference a timestamp column "ColC TIMESTAMP", I receive the following error..."Parquet does not support timestamp. See HIVE-6384"
I found in Databricks you can specify the hive version by using the following, but this did not resolve my issue.
spark.sql.hive.metastore.version 1.2.1
spark.sql.hive.metastore.jars builtin
Any ideas on a way around this?
- Labels:
-
Endpoint
-
Int
-
Parquet File
-
SQL
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-25-2022 09:31 AM
Hey there @T A
Hope everything is going great!
Does @Kaniz Fatma's response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly? If not, would you be happy to give us more information?
We'd love to hear from you.
Thanks!