unable to create delta tables in aws glue catalog

727123 — Fri, 01 Jul 2022 14:55:33 GMT

The project requires that we integrate with the AWS Glue catalog.

We would like to be able to create tables in delta format in the glue catalog.

To test this functionality. We did the following

Created the cluster with access to the Glue catalog as described here https://docs.databricks.com/data/metastores/aws-glue-metastore.html

Created a database from AWS Athena

Created a table in Athena based on a csv file in s3

Launched the cluster with glue access in databricks

Ran sql to read table created in step 3 .This worked

Now tried to run a CREATE TABLE AS SELECT .. this failed (error attached.)

I tried to create a parquet table as shown

create table sales222

STORED AS parquet

LOCATION 's3://pla-sc-357559-44-aws-databricks-s3/test/'

as select * from sales_orders

This worked

8 Ran this to convert to delta

convert to delta parquet.`s3://pla-sc-357559-44-aws- databricks-s3/test`

This succeeded. But i noticed that glue catalog ddl still shows format as parquet

Is this the only way to create delta tables on the AWS glue catalog ?

If you can assist with a short demo that would be great

We would like to do this in SQL only as we are integrating with dbt

Re: unable to create delta tables in aws glue catalog

jose_gonzalez — Wed, 20 Jul 2022 23:36:32 GMT

Hi @extCheeren.John,

Just a friendly follow-up. Do you still are looking for help of any of the responses helped you to resolve your questions? please let us know.