Can someone post about how the ci/cd process for a lakehouse fits into a ring environment? What is the recommended way for someone to synchronize the lakehouse in the production ring with any of the lower ring environments?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-20-2022 12:14 PM
For example, with traditional sql databases, we just take a production backup and restore to whatever environment as needed. I have not seen similar functionality in regards to the Lakehouse/spark tables.
- Labels:
-
CICD
-
Traditional Sql Databases
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-31-2022 01:04 PM
Customers typically use Delta Cloning to "clone" Delta Lake datasets/sub-datasets to other environments for development, testing, etc. See this blog: https://www.databricks.com/blog/2020/09/15/easily-clone-your-delta-lake-for-testing-sharing-and-ml-r...
If customers don't want to move the data, but allow it to be securely accessed across environments, maybe even for a specific time window, they might also use Delta Sharing. https://learn.microsoft.com/en-us/azure/databricks/data-sharing/delta-sharing/

