Dear all,
we are monitoring the size of managed storage accounts associated with our deployed Azure databricks instances.
We have 5 databricks instances for specific components of our platform replicated in 4 environments (DEV, TEST, PREPROD, PROD).
During our analysis we observed Storage Account sizes ranging from some MBytes to a couple of TBytes. Note, that we do not store any production tables on managed storage accounts nor do we upload any form of data. Production instances usually have the larger volume.
Note, that the size is comparable to our production tables (some TBytes).
Our main questions are:
- What do these Storage Accounts contain?
- What is the best way to reduce the size?
- How can we manually delete not important (e.g. logs) files?
- Can we automate the process on #3?
Thanks a lot,
Kind regards,
The European Dynamics team