Databricks Community

jakesippy

I'm looking to track how much time is being spent running updates for my DLT pipelines.When querying the list pipeline updates REST API endpoint I can see start and end times being returned, however, these fields are not listed in the documentation. ...

jakesippy

Originally went with the approach of exporting to and reading from the event log table, which has been helpful for getting other metrics as well.Also found today that there is a new system table is in public preview which exposes the durations I was ...

jakesippy

Thanks all, I am able to see the event logs for pipelines now, I wasn't aware of that!My hope though is that there is some way to fetch these details across an account directly, not just for pipelines I've personally created or for pipelines which ar...

Databricks Community

User Stats

User Activity

How to get pipeline update duration programmatically

Re: How to get pipeline update duration programmatically

Re: How to get pipeline update duration programmatically