How can we do data copy from Databricks SQL using notebook?

Jayesh
New Contributor III

Hi Team, we have a scenario where we have to connect to the DataBricks SQL instance 1 from another DataBricks instance 2 using notebook or Azure Data Factory. Can you please help?

User16764241763
Databricks Employee
Databricks Employee

Hello @Jayesh Mehta​ 

You can using Python connector from notebook

https://docs.databricks.com/dev-tools/python-sql-connector.html

View solution in original post

Jayesh
New Contributor III

Thanks a lot 🙂

Hubert-Dudek
Databricks MVP
  • You can also mount on both databricks workspaces the same storage folder and use COPY INTO for example,
  • In ADF you can use a databricks/delta connector to read from one delta and insert it into another,
  • If it is the one-time copy (like migration) we can copy using ADF just files between storage accounts, it is the fastest copy

My blog: https://databrickster.medium.com/

artsheiko
Databricks Employee
Databricks Employee

Make sure that you need such an operation in principle : we have Clone use cases as well as Delta Sharing features that might be useful for you

Anonymous
Not applicable

Thanks for jumping in to help @Arvind Ravish​  @Hubert Dudek​ and @Artem Sheiko​ !