topic Re: Get total number of files of a Delta table in Data Engineering

Get total number of files of a Delta table

erigaud — Thu, 10 Aug 2023 08:41:51 GMT

I'm looking to know programatically how many files a delta table is made of.

I know I can do

%sql
DESCRIBE DETAIL my_table

But that would only give me the number of files of the current version. I am looking to know the total number of files (basically what I would get If I ran the following command in a terminal in the folder of my table.

> ls *.parquet | wc -l

Is there a way to achieve this ? The end goal is to build a dashboard showing, among other things, the proportion of "old parquet files" compared to the current version for a given table.

Re: Get total number of files of a Delta table

erigaud — Thu, 10 Aug 2023 15:55:57 GMT

Yes I think that solution is good, thank you !

Re: Get total number of files of a Delta table

ADavid — Wed, 07 Aug 2024 11:30:54 GMT

What was the solution?

Re: Get total number of files of a Delta table

gmiguel — Wed, 20 May 2026 19:58:56 GMT

The best way to get this is executing the following statement:

ANALYZE TABLE [table_name] COMPUTE STORAGE METRICS;

Applies to: Databricks Runtime 18.0 and above