Get total number of files of a Delta table
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2023 01:41 AM
I'm looking to know programatically how many files a delta table is made of.
I know I can do
%sql
DESCRIBE DETAIL my_table
But that would only give me the number of files of the current version. I am looking to know the total number of files (basically what I would get If I ran the following command in a terminal in the folder of my table.
> ls *.parquet | wc -l
Is there a way to achieve this ? The end goal is to build a dashboard showing, among other things, the proportion of "old parquet files" compared to the current version for a given table.
