Rehouse Postgresql DB into AWS Databricks options ?

RIDBX
Contributor

Rehouse Postgresql DB into AWS Databricks options ?

==========================================

Hi Community experts?

Thanks for replies to my threads.

We need to relocate the Postgresql DB (data + code + on-prem scheduled jobs) into Cloud Databricks datalake medallion architecture. Postgre DB sitting on Linux on-prem and getting data via ETL. This is happening in many places.


My google search returns a number of recommendations, including:


1) Utilizing DBT framework in SQL-based system
2) Applying DLT architecture
3) Coding manually with Python notebooks.


The problem we face is that we have a huge amount of postgresql code and related data.


What new capabilities does Databricks support in this task?


Is there any guide or document that I may go through on this topic?


Is there a strategy based on Gen AI to consider?

Thanks for your guidance.