Databricks SQL Endpoint Failing to create an external table on a parquet file with Decimal or Timestamp datatype

[ta_db]

I'm using the Databricks SQL Endpoint and I'm attempting to create an external table on top of an existing parquet file. I can do this so long as my table definition does not include a reference to a decimal or timestamp/date datatype.

ex. This works

CREATE EXTERNAL TABLE sampleTable (  
ColA STRING,  
ColB INT)  
LOCATION 'storagelocation' STORED AS PARQUET

When I need to reference a timestamp column "ColC TIMESTAMP", I receive the following error..."Parquet does not support timestamp. See HIVE-6384"

I found in Databricks you can specify the hive version by using the following, but this did not resolve my issue.

spark.sql.hive.metastore.version 1.2.1 
spark.sql.hive.metastore.jars builtin

Any ideas on a way around this?

[Kaniz]

Hi @T A​, Which Databricks Runtime version are you using? Add the settings according to your DBR runtime.

Databricks Runtime 6.6 and below

• spark.sql.hive.metastore.version 1.2.1
• spark.sql.hive.metastore.jars builtin

Databricks Runtime 7.0 and above

• spark.sql.hive.metastore.jars /dbfs <path-to-downloaded-jars>
• spark.sql.hive.metastore.version 1.2.1

Here is the KB on how to use the TIMESTAMP with a Parquet table.

[Anonymous]

Hey there @T A​ 

Hope everything is going great!

Does @Kaniz Fatma​'s response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly? If not, would you be happy to give us more information?

We'd love to hear from you.

Thanks!