- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ03-26-2022 04:18 AM
If I create delta table the table is stored in parque format in DBFS location ?
and please share how the parque files supports schema evolution if i do DML operation.
As per my understanding : we read data from data lake first in data frame and try to write the dataframe to delta tables and these delta tables when created as tables are stored in parque format ??
- Labels:
-
Delta
-
Delta table
-
DML
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ03-28-2022 01:24 AM
delta lake is parquet on steroids. The actual data is stored in parquet files, but you get a bunch of extra functionalities (time traveling, ACID, optimized writes, MERGE etc).
check this page for lots of info.
Delta lake does support schema evolution but it has itยดs limitations.
About the location: they will indeed be stored in dbfs, but dbfs is a 'virtual' layer on top of your actual storage. So if you mount your data lake into dbfs, dbfs will contain your data (it will link it, not copy it).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ03-28-2022 01:24 AM
delta lake is parquet on steroids. The actual data is stored in parquet files, but you get a bunch of extra functionalities (time traveling, ACID, optimized writes, MERGE etc).
check this page for lots of info.
Delta lake does support schema evolution but it has itยดs limitations.
About the location: they will indeed be stored in dbfs, but dbfs is a 'virtual' layer on top of your actual storage. So if you mount your data lake into dbfs, dbfs will contain your data (it will link it, not copy it).

