cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Need to extract data from delta tables and need to move it to on-prem, what's the best approach

Shivap
New Contributor III

I want to extract data from databricks delta tables and need to move it to on-prem what's the best way to accomplish it -

3 REPLIES 3

Master_DataBric
New Contributor II

You can extract the files and save them on cloud storage(s3/ADLS/GCS).
Then you can use the appropriate cloud CLI to read the files from cloud storage. 
I think this is more scalable and can be achieved programmatically(i.e via your batch job).  

sandeepmankikar
New Contributor II

If you are planning to load data into an on-premises database, you can write a PySpark JDBC script to efficiently transfer data from Databricks Delta tables to the target database. This method enables seamless integration between Databricks and on-prem databases

Stefan-Koch
Valued Contributor II

An easy way to do is to use Airbyte. You can run Airbyte locally, connect to databricks and copy the data to your OnPrem location.

https://docs.airbyte.com/integrations/destinations/databricks

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group