AWS Databricks external tables are delta tables?

[Akshay_Petkar]

If I create an external table on AWS Databricks, will it be a Delta table? If not, is there a way to make it a Delta table, or is there no Delta capability for external tables?

[Kaniz_Fatma · Accepted solution]

Hi @Akshay_Petkar, Unity Catalog external tables in Databricks can be Delta tables, but they are not required to be. When you create an external table, it’s possible to choose whether or not to use Delta Lake. These external tables do not provide the transactional guarantees or optimized performance of Delta ...¹. If you want to create a Delta table explicitly, you can follow these steps:

1. Create an External Table: You can create an external table in Databricks using SQL commands or DataFrame write operations. Ensure that you have the necessary permissions, including the CREATE EXTERNAL TABLE privilege on the...².

2. Choose the File Format: External tables can use various file formats, including DELTA, CSV, JSON, AVRO, PARQUET, and ORC. To create a Delta table, specify the DELTA format when creating the external table.

3. Register Data Files: When creating the external table, you can either register an existing directory of data files as a table or provide a path to create new data files. The table registration in Unity Catalog is essentially a pointer to the data files, and Databricks only manages the metadata for external tables without affecting the data files themselves.

Remember that Delta tables offer additional features beyond traditional external tables. They allow direct modification through inserts, updates, deletes, and merges (CRUD operations), and they can be used as batch tables or streaming sources and sinks. Delta Lake also ensures exactly-once processing during multiple streaming or concurrent batch jobs³. If you need these capabilities, consider using Delta Tables instead of plain external tables. 😊