@Debayan Mukherjee​ , We were able to fix the issue by changing the few guids that we used. Apparently the managed identity application id wasnt getting deployed appropriately by Terraform which caused this issue. It wasn't an issue related to Databricks Runtime.

View solution in original post