Dear all
we have around 5000+ finished data products (aka views) in several schemas of unity catalog. One question that comes from business users frequently is - how frequently these get refreshed?
for that the answer is not simpler as the underlying tables are across several layers - bronze, silver (these in turn read from bronze), and these get processed by at least 200+ Databricks jobs defined with a refresh schedule.
I plan to query the lineage of these 5000+ views, reach the bronze and silver layer tables and report the refresh schedule on the top of it.
Has anyone done something similar? Is there an OOTB script to do so or...
Appreciate the mind share
