DLT Pipeline Stats on Object level
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-29-2025 02:40 AM
Hi Guys,
I want to create a table where I want to store information about each DLT pipelines on object/table id level details about how much time it took for waiting for resources and how much time it took to run for each object and numbers or records upserted and these kind of information.
From where can I get these details? is there any tables in system catalog where I can get all the information related to DLT pipelines and DLT managed streaming tables and materialised views?
Appreciate your help on this.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-29-2025 03:56 AM
DLT Event logs helps you to gather most of the information you've mentioned above.
Below is the documentation to the DLT Event Logs:
https://docs.databricks.com/aws/en/dlt/observability
Let me know if any questions.
Best,