topic Re: Cloud storage - enabling object versioning? in Data Engineering

Cloud storage - enabling object versioning?

646901 — Wed, 01 Feb 2023 13:15:21 GMT

So i am going to keep this generic as to all cloud provider storage options as its relevant across the board, (GCS, S3 and blob store). Nothing is mentioned in docs as far as i can see.

Is there a use case against enabling object versioning in cloud storage for the databricks file system? I have a use case for requiring object versions to protect against accidental deletion and keeping filestore history in the case of overwrites.

Is enabling object storage suitable, recommended and practical on the databricks main bucket?

Re: Cloud storage - enabling object versioning?

-werners- — Wed, 01 Feb 2023 13:59:16 GMT

it certainly is.

That is the reason why projects like LakeFS and Project Nessie are soaring.

Re: Cloud storage - enabling object versioning?

Anonymous — Thu, 06 Apr 2023 10:35:57 GMT

Hi @Matt User

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!