topic How to quickly check if Delta Table is Empty in Data Engineering

How to quickly check if Delta Table is Empty

Retko — Thu, 09 Mar 2023 07:47:18 GMT

Hi,

I need some quick way to return True if Delta Table is Empty.

Tried this, but is is quite slow when checking more tables.

spark.read.table("table_name").count()

spark.read.table("table_name").rdd.isEmpty()

len(spark.read.table("table_name").head(1)) == 0

Thank you.

Re: How to quickly check if Delta Table is Empty

Anonymous — Mon, 13 Mar 2023 08:10:59 GMT

@Retko Okter :

Use the isEmpty method of the DataFrame API in Spark to check if a Delta table is empty

from delta.tables import DeltaTable
 
def is_delta_table_empty(table_name):
  delta_table = DeltaTable.forPath(spark, table_name)
  return delta_table.toDF().isEmpty()
 
# Usage example
if is_delta_table_empty("table_name"):
  print("Table is empty")
else:
  print("Table is not empty")

Re: How to quickly check if Delta Table is Empty

Vartika — Fri, 31 Mar 2023 09:59:44 GMT

Hi @Retko Okter

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best? If not, please tell us so we can help you.

Thanks!