cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

What is "External tables backed by Delta Lake"?

tana_sakakimiya
New Contributor III

Goal: event-driven without implementing job triggereed on file arrival

I see hope to incrementally update materialized views which have external tables as their sources.
This is quite a game changer if it works for various data formats.
(since MV started supporting trigger on update recently) 

Incremental refresh for materialized views | Databricks on AWS
it mentioned that materialized view support "external table backed by delta lake"

  • Delta tables, including Unity Catalog managed tables and external tables backed by Delta Lake.

does it mean external source data required to be in delta format? or parquet format data source can be external table backed by delta lake if it is declared as an external table properly?

1 ACCEPTED SOLUTION

Accepted Solutions

szymon_dybczak
Esteemed Contributor III

Hi @tana_sakakimiya ,

Yes, only external tables that are in delta format are supported. Databricks supports other table formats, but to be able to use this particular feature, your table needs to be in Delta format.

szymon_dybczak_0-1757920817290.png

 


But if you have parquet files it's really easy to convert them to delta format, so that shouldn't be an issue here.

View solution in original post

1 REPLY 1

szymon_dybczak
Esteemed Contributor III

Hi @tana_sakakimiya ,

Yes, only external tables that are in delta format are supported. Databricks supports other table formats, but to be able to use this particular feature, your table needs to be in Delta format.

szymon_dybczak_0-1757920817290.png

 


But if you have parquet files it's really easy to convert them to delta format, so that shouldn't be an issue here.