cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Is it possible to clone a private repository and use it in databricks Repos?

Andyfcx
New Contributor

As title, I need to clone code from my private git repo, and use it in my notebook,

I do something like

def cmd(command, cwd=None):
   process = subprocess.Popen(command.split(), stdout=subprocess.PIPE, cwd=cwd)
   output, error = process.communicate()
   print (output.decode())
   if error is not None:
      print ("----\n"+error.decode())

But I have to do this, redefine this function all the time.

I think this is cumbersome,

Is there a way that I could clone the repo into the "Repos" which databricks offered in the left side bar, and have full version control with databricks?

(I am not installing published packages from pip)

1 ACCEPTED SOLUTION

Accepted Solutions

Prabakar
Esteemed Contributor III
Esteemed Contributor III

Hi @Andy Huang​ , Yes, you can do it if it's accessible from Databricks. Please refer to:

https://docs.databricks.com/repos.html#repos-for-git-integration

Databricks does not support private Git servers, such as Git servers behind a VPN.

View solution in original post

3 REPLIES 3

Prabakar
Esteemed Contributor III
Esteemed Contributor III

Hi @Andy Huang​ , Yes, you can do it if it's accessible from Databricks. Please refer to:

https://docs.databricks.com/repos.html#repos-for-git-integration

Databricks does not support private Git servers, such as Git servers behind a VPN.

Hubert-Dudek
Esteemed Contributor III

I think it does support private git servers just for example in Azure you need to set network routing - vnet injection ( https://docs.microsoft.com/en-us/azure/databricks/administration-guide/cloud-configurations/azure/vn... ) and whitelist your internet gateway on git. So it will be seemed as any other git.

Kaniz
Community Manager
Community Manager

Hi @Andy Huang​ , Just a friendly follow-up. Do you still need help, or @Prabakar Ammeappin​ 's response help you to find the solution? Please let us know.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.