cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

how to find the size of a table in python or sql?

ramankr48
Contributor II

let's suppose there is a database db, inside that so many tables are there and , i want to get the size of tables . how to get in either sql, python, pyspark.

even if i have to get one by one it's fine.

1 ACCEPTED SOLUTION

Accepted Solutions

shan_chandra
Honored Contributor III
Honored Contributor III

@Raman Gupta​ - could you please try the below

%python
spark.sql("describe detail delta-table-name").select("sizeInBytes").collect()

View solution in original post

6 REPLIES 6

elgeo
Valued Contributor II

DESCRIBE DETAIL table_name returns the sizeInBytes

youssefmrini
Honored Contributor III
Honored Contributor III

Describe detail give you only the size of the latest snapshot. It's worth running a dbutils.fs.ls

shan_chandra
Honored Contributor III
Honored Contributor III

@Raman Gupta​ - Please refer to the below

Calculate the size of the Delta table:

%scala 
import com.databricks.sql.transaction.tahoe._
val deltaLog = DeltaLog.forTable(spark, "dbfs:/delta-table-path")
val snapshot = deltaLog.snapshot                     // the current delta table snapshot
println(s"Total file size (bytes): ${deltaLog.snapshot.sizeInBytes}"

calculate the size of the non delta table:

%scala
spark.read.table("non-delta-table-name").queryExecution.analyzed.stats

shan_chandra
Honored Contributor III
Honored Contributor III

@Raman Gupta​ - could you please try the below

%python
spark.sql("describe detail delta-table-name").select("sizeInBytes").collect()

Thanks @Shanmugavel Chandrakasu​ 

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.