07-05-2023 09:20 AM - edited 07-05-2023 09:21 AM
You can look at the sparkUI of the cluster to get what is happening on the cluster.
Debugging with the Apache Spark UI - Azure Databricks | Microsoft Learn
View solution in original post
