How much space does the metadata for a Delta table take up?

[User16826992666]

If you have a lot of transactions in a table it seems like the Delta log keeping track of all those transactions would get pretty large. Does the size of the metadata become a problem over time?

[Ryan_Chynoweth · Accepted solution]

Yes, the size of the metadata can become a problem over time but not because of performance but because of storage costs. Delta performance will not degrade due to the size of the metadata, but your cloud storage bill can increase.

By default Delta history is only maintained for 30 days: https://docs.databricks.com/delta/delta-utility.html#retrieve-delta-table-history

This can be changed by setting the "delta.logRetentionDuration" to a longer period.