cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

how to find the size of a table in python or sql?

ramankr48
Contributor II

let's suppose there is a database db, inside that so many tables are there and , i want to get the size of tables . how to get in either sql, python, pyspark.

even if i have to get one by one it's fine.

1 ACCEPTED SOLUTION

Accepted Solutions

shan_chandra
Honored Contributor III
Honored Contributor III

@Raman Guptaโ€‹ - could you please try the below

%python
spark.sql("describe detail delta-table-name").select("sizeInBytes").collect()

View solution in original post

6 REPLIES 6

elgeo
Valued Contributor II

DESCRIBE DETAIL table_name returns the sizeInBytes

youssefmrini
Honored Contributor III
Honored Contributor III

Describe detail give you only the size of the latest snapshot. It's worth running a dbutils.fs.ls

shan_chandra
Honored Contributor III
Honored Contributor III

@Raman Guptaโ€‹ - Please refer to the below

Calculate the size of the Delta table:

%scala 
import com.databricks.sql.transaction.tahoe._
val deltaLog = DeltaLog.forTable(spark, "dbfs:/delta-table-path")
val snapshot = deltaLog.snapshot                     // the current delta table snapshot
println(s"Total file size (bytes): ${deltaLog.snapshot.sizeInBytes}"

calculate the size of the non delta table:

%scala
spark.read.table("non-delta-table-name").queryExecution.analyzed.stats

Already know in scala.

I wanted this in either python or sql

scala link: https://kb.databricks.com/sql/find-size-of-table.html#:~:text=To%20find%20the%20size%20of%20a%20delt...

shan_chandra
Honored Contributor III
Honored Contributor III

@Raman Guptaโ€‹ - could you please try the below

%python
spark.sql("describe detail delta-table-name").select("sizeInBytes").collect()

Thanks @Shanmugavel Chandrakasuโ€‹ 

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.