SP_6721
Honored Contributor II

Hi @ankit001mittal 

Yes, you're right, when you enable the "Publish Event Log to Metastore" option for DLT pipelines, Databricks creates a separate event log table for each pipeline. So, if you have thousands of pipelines, you'll see thousands of log tables. There’s no built-in feature at the moment to automatically combine all logs into a single centralised table like the system tables.

Your approach is valid. If you want a unified view of all logs across pipelines, you can create a SQL view that uses UNION ALL across the individual event log tables. Or set up a streaming process that regularly consolidates logs into one Delta table for easier querying and dashboarding.