@FaizH - Try this : https://docs.databricks.com/en/partners/bi/power-bi.html and also explore this https://docs.databricks.com/en/integrations/compute-details.html make sure you are usign correct Authentication method . NOTE : Also may be your SPN is...
@sambgp Chekc this if this can help https://stackoverflow.com/questions/45873155/out-of-memory-error-when-collecting-data-out-of-spark-cluster ; https://stackoverflow.com/questions/26562033/how-to-set-apache-spark-executor-memory ; https://community....
@ramesitexp Yes @szymon_dybczak is correct for now only valid option is below : OAuth 2.0 with a Microsoft Entra ID service principalShared access signatures (SAS)Account keys For now we are using OAuth 2.0 with a Microsoft Entra ID service principal...
@sinclair - Can you please use num_workers=2 or 5 which will be constant instead of using auto scaling ? Meantime i am trying to reproduce the same issue in my end !
