Databricks Community

The images below are DAG and text execution summary from Spark UI and I'm having hard time interpreting these logs. I have two questions below.1. In the Text Execution Summary, Duration total for WholeStageCodegen (2) says 38.3 m (0 ms, 2.7 m, 16.9 m...

I am a member of the development team in our company and we use Databricks as sort of like ETL tool. We utilize git integration for our program and run Workflow daily basis. Recently, we created another company internal private git repository and wan...

Is there any way to visualize logging such as execution runtime or memory usage of Spark Job graphically just like the image below by utilizing Databricks Partner Connect for free?Also, I'd appreciate to know any other ways to visualize the logging o...