Hello everyone,
I am responsible for designing and implementing a Lakehouse architecture in an industrial company.
I am currently facing some challenges regarding the initial ingestion of data from our onโpremise Oracle database into Databricks.
The data comes from production systems and is actively used by several applications. My main concern is that the initial load is very large, and Iโm worried about impacting database performance or even causing issues if we extract all the data at once.
For the ongoing ingestion, the data volume will be much smaller and continuous, so that part is not an issue.
However, I would really appreciate advice or best practices on how to safely handle the first largeโscale ingestion (initial load) without overloading or disrupting the Oracle database.
What approaches, tools, or patterns would you recommend in this situation?
Thank you in advance for your help.
