- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 03:38 PM
I never ran VACUUM on the Delta table. Will Delta perform direct listing on those directories, I am afraid the query performance is going to get degraded over time
How about the log directories. I have more than 100k JSON files in the log directory
- Labels:
-
Data Directories
-
Delta
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 03:39 PM
For both Data and logs, Delta does not perform listing. The transaction logs have the details of the files and the objects are directly accessed.
Similarly with in the Delta log directory listing is performed. However, it's performed using a continuation token. Hence only partial listing is done.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 03:39 PM
For both Data and logs, Delta does not perform listing. The transaction logs have the details of the files and the objects are directly accessed.
Similarly with in the Delta log directory listing is performed. However, it's performed using a continuation token. Hence only partial listing is done.

