- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-09-2025 05:13 AM
@Brianben Correct, apologies if my previous response is introducing further confusion. I had to go back to Azure's documentation to get more in context, initially I was considering timestamps inside the files, instead of the files metadata timestamp.
Anyways, you're absolutely correct. To clarify:
The delayed archival will happen when operations like updates, deletes, or compactions modify a file, resetting its "last modified" or "creation date." Example: A file originally written on Jan 1, 2018 and updated on Jan 1, 2023 will now have a "last modified date" of Jan 1, 2023, delaying archival until Jan 1, 2028 (5 years later). Data from 2018 will effectively be archived 10 years after its original creation. And then with respect to the premature archival clarification, because the Lifecycle policies only act on blob metadata (e.g., creation or modification dates), not the timestamps of individual rows inside the Parquet files, the risk of premature archival for "newer" data inside a file does not exist when using standard lifecycle policies.