Hi all,
- migrating from an on-premise Oracle -

Currently on Oracle, I have a "library" of let's say 300 tables to load, sequentially, based on views (some tables being fed potentially by several views, therefore the number of underlying views is higher).
All of this is processed by Oracle via PL/SQL, dynamic SQLs, etc.. and all is working very well.
The volumetry per table is not big, as for example in 60% of the cases, the number of rows per table is lower than ..let's say 10 000. Queries are complex, yes, but volumetry is relatively low.
It runs approximately every 2 hours, and as we are logging all (start, end of each processes, errors of course, etc.), the number of "actions" (select, update, inserts, etc..) on the database is growing high very quickly in a short amount of time.
Again, this is the current situation in Oracle and all is fine.

Now in Databricks, is it possible to reproduce the same via PySpark (and things like "spark.sql(...)", etc..) and should I be aware of some potential issues on the performance side?
Assuming we will have a clean environment, with the opportunity to have some computes correctly "sized", can I assume that I don't risk much in terms of performance?
What would be the recommendations for this ?

I've already performed a small POC, reproducing what I'm doing in Oracle, but on a very small scale.
All is fine and working as expected.
But I can't reproduce a bigger workload.

Any opinion or advice on this?
Thanks!

Vincent