Get total number of files of a Delta table
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2023 01:41 AM
I'm looking to know programatically how many files a delta table is made of.
I know I can do
%sql
DESCRIBE DETAIL my_table
But that would only give me the number of files of the current version. I am looking to know the total number of files (basically what I would get If I ran the following command in a terminal in the folder of my table.
> ls *.parquet | wc -l
Is there a way to achieve this ? The end goal is to build a dashboard showing, among other things, the proportion of "old parquet files" compared to the current version for a given table.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2023 08:55 AM
Yes I think that solution is good, thank you !
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-07-2024 04:30 AM
What was the solution?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-20-2026 12:58 PM
The best way to get this is executing the following statement:
ANALYZE TABLE [table_name] COMPUTE STORAGE METRICS;
Applies to: Databricks Runtime 18.0 and above
