Hey @Dhruv-22 ! Thanks for the info! 

I will need to analyse this internally to pinpoint the exact root cause. I advise that you raise a support case with us to have a closer look. You can raise a support case using the link: https://help.databricks.com/  
and add a comment to assign it to me so that I can look and provide a detailed analysis and fix if any