I recommend avoiding fetching from a Git provider directly to run code during workflows and instead have a task that updates a Git folder within your workspace during the job (article with more details below). That way you can use Databricks to mana...
The main remaining advantages of Scala are performance as there will always be some interoperation overhead when using PySpark. While I don't have any stats on me, I would assume the differences in performance are negligible at this point until very ...
