- 4390 Views
- 3 replies
- 5 kudos
Resolved! Delta Table - Reduce time travel storage size
Hello! I am trying to understand time travel feature. I see with "DESCRIBE HISTORY" command that all the transaction history on a specific table is recorded by version and timestamp. However, I understand that this occupies a lot of storage especiall...
- 4390 Views
- 3 replies
- 5 kudos
- 5 kudos
Thank you @Werner Stinckens​ for your reply. However I still haven't managed to delete history even after setting the below. The number of history rows remains the same when running "DESCRIBE HISTORY".SET spark.databricks.delta.retentionDurationCheck...
- 5 kudos
- 10120 Views
- 7 replies
- 10 kudos
Resolved! Delta Table storage best practices
Hi!We have a project where we do some Data Engineering for a client. I implemented a scheduled batch processing of Databricks' autoloader (stream w/ availableNow) since they primarily have numerous file exports from several sources. We wanted to foll...
- 10120 Views
- 7 replies
- 10 kudos
- 10 kudos
Hi @Gimwell Young​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 10 kudos