Databricks Community

Hi @mannepk85 ,You can take a look on jobs system table. Notice though, that it is in public preview now so use it with caution:  https://learn.microsoft.com/en-us/azure/databricks/admin/system-tables/jobs

I found the following IPs from the Cluster JSON file:"driver": {"private_ip": "10.*.*.*","public_dns": "172.*.*.*","node_id": "80*****",Similar the executors configuration"executors": [{"private_ip": "10.*.*.*","public_dns": "172.*.*.*","node_id": "7...

The below approach is working for me... I had to download the packages upfront and place it on a volume though.#!/bin/bashset -euxo pipefailecho 'debconf debconf/frontend select Noninteractive' | debconf-set-selectionssudo ACCEPT_EULA=Y dpkg -i odbci...

Why not use Substitutions and Custom variables that can be specified on command line using --var="<key>=<value>"?With all the features your databricks.yml would look as follows:variables:  git_branch:    default: maingit_source:  git_url: https://git...

Thanks @szymon_dybczak 

I am currently experiencing the same error, Azure DataBricks, Runtime version is 15.3 ML, default Notebook Editor.

@youcanlearn Have you explore this : https://docs.databricks.com/en/delta-live-tables/expectations.html  

 Hi @erwingm10 ,Unfortunately, currently that there is no direct endpoint in REST API to get cluster metrics. You can extract some ganglia metrics through custom scripting, but they're not so detailed like the one you looking for.Look at below links ...