Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype

ta_db — Thu, 26 May 2022 02:12:11 GMT

I'm using the Databricks SQL Endpoint and I'm attempting to create an external table on top of an existing parquet file. I can do this so long as my table definition does not include a reference to a decimal or timestamp/date datatype.

ex. This works

CREATE EXTERNAL TABLE sampleTable (  
ColA STRING,  
ColB INT)  
LOCATION 'storagelocation' STORED AS PARQUET

When I need to reference a timestamp column "ColC TIMESTAMP", I receive the following error..."Parquet does not support timestamp. See HIVE-6384"

I found in Databricks you can specify the hive version by using the following, but this did not resolve my issue.

spark.sql.hive.metastore.version 1.2.1 
spark.sql.hive.metastore.jars builtin

Any ideas on a way around this?

Re: Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype

Anonymous — Mon, 25 Jul 2022 16:31:29 GMT

Hey there @T A

Hope everything is going great!

Does @Kaniz Fatma's response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly? If not, would you be happy to give us more information?

We'd love to hear from you.

Thanks!

topic Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype in Data Engineering

Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype

Re: Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype