Get Network usage read and write from a job cluster on databricks
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-03-2023 02:59 PM
Hi ! I am writing because I am trying to get the data from network usage from a databricks job cluster. I know that the data is on spark-ui / storage but when the job cluster is terminate I cannot acces to the information , and I trying to make some analysis about report that are consuming more network. Do you know if exist a way programatically that I can access to all the information of storage from a job cluster, by code.It can be useful to add into a table and compare performances. I attached the image of what im talking about. Hope you can helps me, thanks
Labels:
- Labels:
-
DatabricksJobCluster
-
sparkui