The issue is probably related to the self join between 100 million rows, I'm not positive without seeing the code and understanding the problem better but you may want to think about using windowing functions insteadhttps://blog.knoldus.com/using-win...
Unfortunately, the Git server must be accessible from Databricks.Here are the Repos Requirements where it currently states:The Git server must be accessible from Databricks. Databricks does not support private Git servers, such as Git servers behind ...