unable to create delta tables in aws glue catalog

[727123]

unable to create delta tables in aws glue catalog

The project requires that we integrate with the AWS Glue catalog.

We would like to be able to create tables in delta format in the glue catalog.

To test this functionality. We did the following

Created the cluster with access to the Glue catalog as described here https://docs.databricks.com/data/metastores/aws-glue-metastore.html

Created a database from AWS Athena

Created a table in Athena based on a csv file in s3

Launched the cluster with glue access in databricks

Ran sql to read table created in step 3 .This worked

Now tried to run a CREATE TABLE AS SELECT .. this failed (error attached.)

I tried to create a parquet table as shown

create table sales222

STORED AS parquet

LOCATION 's3://pla-sc-357559-44-aws-databricks-s3/test/'

as select * from sales_orders

This worked

8 Ran this to convert to delta

convert to delta parquet.`s3://pla-sc-357559-44-aws- databricks-s3/test`

This succeeded. But i noticed that glue catalog ddl still shows format as parquet

Is this the only way to create delta tables on the AWS glue catalog ?

If you can assist with a short demo that would be great

We would like to do this in SQL only as we are integrating with dbt

[Kaniz_Fatma · Accepted solution]

Hi @extCheeren.John​, Watch the demo video and see how to update the Databricks Cluster Spark Configuration properties.

[Kaniz_Fatma]

Hi @extCheeren.John​, We haven't heard from you since my last response, and I was checking back to see if my suggestions helped you.

Or else, If you have any solution, please share it with the community as it can be helpful to others.

Also, please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

[jose_gonzalez]

Hi @extCheeren.John​,

Just a friendly follow-up. Do you still are looking for help of any of the responses helped you to resolve your questions? please let us know.