Lakeflow Ingestion Pipeline – Unable to Generate Event Log Table

agent007
Databricks Partner

For the DLT pipeline, we are successfully generating an event log table using the event_log configuration in the pipeline YAML.

However, for the Salesforce pipeline, which is an ingestion pipeline (via Lakeflow Connector), we are unable to create or access a similar event log table.


What We Tried

We attempted enabling event logs in the Salesforce pipeline YAML configuration.

However:

  • The event log table is not being created
  • We can see pipeline logs in the Databricks UI (screenshot attached)
  • But we cannot find or query any corresponding event log table


    Questions
    1. Is event log table creation supported for ingestion pipelines (Lakeflow Connector)?
    2. If yes, what is the correct configuration to enable and persist event logs as a Unity Catalog table?
    3. Is there anything we could find in system.lakeflow (just an assumption as we don't have access to it)
    4. If direct event log tables are not supported for ingestion pipelines, what would be the recommended alternative approach to expose pipeline execution logs for reporting in Power BI?

      Attached Image - 
    agent007_0-1770974343963.jpeg