cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

We have hundreds of ETL process (Informatica) with a lot of logic pulling various data from applications into a relational db (Target DB). Can we use Delta Lake as the Target DB?

Disney
New Contributor II

Hi DB Support,

Can we use DB's Delta Lake as our Target DB? Here's our situation...

  • We have hundreds of ETL jobs pulling from these Sources. (SAP, Siebel/Oracle, Cognos, Postgres) .
  • Our ETL Process has all of the logic and our Target DB is an MPP system.
  • We have MSTR and Tableau for reporting.

Can we use Databrick's Delta Lake to replace our "Target DB"? We have over a thousand tables and views on top so a conversion tool is great to have? Does Databrick have an easier way to replace our MPP "Target DB"? Target DB can be a redshift, postgre, sqlserver or other mpp db.

Can you advise? Please send reply via email.

Thank you!

-william

1 ACCEPTED SOLUTION

Accepted Solutions

Hubert-Dudek
Esteemed Contributor III

Hi yes you can the best is to create sql endpoint in premium workspace and just write to delta lake as to sql.

This is community forum not support. You can contact databricks via https://databricks.com/company/contact or via AWS, Azure if you have support plan there (is the fastest)

View solution in original post

1 REPLY 1

Hubert-Dudek
Esteemed Contributor III

Hi yes you can the best is to create sql endpoint in premium workspace and just write to delta lake as to sql.

This is community forum not support. You can contact databricks via https://databricks.com/company/contact or via AWS, Azure if you have support plan there (is the fastest)

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group