- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-06-2023 02:24 AM
Hi,
Looking at the delta tables and how they are stored I have a question.
If the delta tables are stored as parquet files in ADLS why if I copy/paste/rename a folder that corresponds to an existing table in the same location/database this does not generate a copy of the previous table but with another name? What happens in the backgroud?
Thank you in advanced!
- Labels:
-
Delta
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-06-2023 07:02 PM
@Sara Corral :
When you copy/paste/rename a folder that corresponds to an existing delta table in the same location/database, it does not generate a copy of the previous table because delta tables are not just plain parquet files. They have additional metadata that is stored in a hidden folder called "_delta_log". This metadata includes information about the transaction log, version history, and other metadata that allows Delta Lake to perform ACID transactions on the table.
When you copy/paste/rename a folder, you are essentially creating a new folder with a new name, but you are not copying the "_delta_log" folder that contains the table's metadata. Without this metadata, the new folder is not recognized as a Delta table, and you cannot query it or interact with it as a Delta table. In order to create a copy of a Delta table with a different name, you need to use the Delta Lake
COPY command or other Delta-specific tools to ensure that the metadata is also copied and updated appropriately. Simply copying the folder will not work.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-06-2023 07:02 PM
@Sara Corral :
When you copy/paste/rename a folder that corresponds to an existing delta table in the same location/database, it does not generate a copy of the previous table because delta tables are not just plain parquet files. They have additional metadata that is stored in a hidden folder called "_delta_log". This metadata includes information about the transaction log, version history, and other metadata that allows Delta Lake to perform ACID transactions on the table.
When you copy/paste/rename a folder, you are essentially creating a new folder with a new name, but you are not copying the "_delta_log" folder that contains the table's metadata. Without this metadata, the new folder is not recognized as a Delta table, and you cannot query it or interact with it as a Delta table. In order to create a copy of a Delta table with a different name, you need to use the Delta Lake
COPY command or other Delta-specific tools to ensure that the metadata is also copied and updated appropriately. Simply copying the folder will not work.

