cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

git integration with volumes?

-werners-
Esteemed Contributor III

Volumes in unity are said to be ideal for "storing library and config files of arbitrary formats such as .whl or .txt centrally and providing secure access across workspaces to it".

So basically this is the same we had with dbfs but with decent access control.
But if we want to use volumes for our libraries and config files, I am missing git integration.
How do we handle automated scenarios where a git provider automatically puts updated config files in the volumes on Unity?
In dbfs we could create a pipeline in f.e. Azure devops that runs when a merge is executed on main and copies all the concerning files to dbfs.
However for Volumes it is not clear if this is possible.
Anyone an idea/experience?

1 ACCEPTED SOLUTION

Accepted Solutions

Kaniz
Community Manager
Community Manager

Hi @-werners-Based on the provided information, there isn't a direct mention of Git integration or automated pipeline setup for Unity Catalog volumes. The Unity Catalog volumes are designed to provide capabilities for accessing, storing, governing, and organizing files, including libraries and configuration files of various formats like .whl or .txt. However, the automated updating of these files using a Git provider is not explicitly discussed in the provided sources. For DBFS, you mentioned setting up a pipeline in Azure DevOps that triggers a merge to the main branch and copies the relevant files to DBFS. The sources do not clearly outline a similar automated process for Unity volumes. Therefore, it's impossible to provide a definitive answer without additional information. Please feel free to raise a support ticket.

View solution in original post

2 REPLIES 2

Kaniz
Community Manager
Community Manager

Hi @-werners-Based on the provided information, there isn't a direct mention of Git integration or automated pipeline setup for Unity Catalog volumes. The Unity Catalog volumes are designed to provide capabilities for accessing, storing, governing, and organizing files, including libraries and configuration files of various formats like .whl or .txt. However, the automated updating of these files using a Git provider is not explicitly discussed in the provided sources. For DBFS, you mentioned setting up a pipeline in Azure DevOps that triggers a merge to the main branch and copies the relevant files to DBFS. The sources do not clearly outline a similar automated process for Unity volumes. Therefore, it's impossible to provide a definitive answer without additional information. Please feel free to raise a support ticket.

-werners-
Esteemed Contributor III

I suspected as much. Otherwise the docs would mention it.
It would be great though to have some kind of automation, especially when working with config/libs.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.