https://community.databricks.com/t5/data-engineering/how-to-get-the-total-number-of-records-in-a-delta-table-from-the/m-p/20441#M13794 Thu, 24 Jun 2021 18:26:12 GMT brickster_2018 2021-06-24T18:26:12Z https://community.databricks.com/t5/data-engineering/how-to-get-the-total-number-of-records-in-a-delta-table-from-the/m-p/20441#M13794 Thu, 24 Jun 2021 18:26:12 GMT https://community.databricks.com/t5/data-engineering/how-to-get-the-total-number-of-records-in-a-delta-table-from-the/m-p/20441#M13794 brickster_2018 2021-06-24T18:26:12Z https://community.databricks.com/t5/data-engineering/how-to-get-the-total-number-of-records-in-a-delta-table-from-the/m-p/20442#M13795 <P>The below code can be used to get the number of records in a Delta table without querying it</P><PRE><CODE>%scala import com.databricks.sql.transaction.tahoe.DeltaLog import org.apache.hadoop.fs.Path import org.apache.spark.sql.DataFrame import org.apache.spark.sql.functions._ &nbsp; val deltaTablePath = "location of the table" &nbsp; def getRecordCount(deltaTablePath: String): Any = { val snapshot = DeltaLog.forTable(spark, new Path(deltaTablePath)).update() val statsSchema = snapshot.statsSchema var files = snapshot.allFiles.withColumn("stats", from_json($"stats", statsSchema)) val dfWithNumRecords = files.select($"path", $"stats.numRecords".as("numRecords")) val totalCount = dfWithNumRecords.select(sum($"numRecords")).first().get(0) return totalCount } println(getRecordCount(deltaTablePath))</CODE></PRE><P></P> Thu, 24 Jun 2021 18:27:27 GMT https://community.databricks.com/t5/data-engineering/how-to-get-the-total-number-of-records-in-a-delta-table-from-the/m-p/20442#M13795 brickster_2018 2021-06-24T18:27:27Z