Creating event_log for dlt pipeline using dabs

21f3001806 — Sun, 30 Mar 2025 09:21:00 GMT

Hi there,

i have dlt pipeline and I recently came to know about the event_log feature, i want to deploy my dlt pipeline along with the event_log using databricks asset bundles but i am not able to find any resources for it.
If anyone has tried it , your help would be appreciated.

Re: Creating event_log for dlt pipeline using dabs

ashraf1395 — Sun, 30 Mar 2025 09:50:22 GMT

Hi there @21f3001806 ,

I guess you are talking about the this : https://docs.databricks.com/api/workspace/pipelines/create#event_log
Its still in public preview
I tried creating it through UI or by chaing the pipeline settings it worked. But when I implemented it through UI , but when I tried the same code on dabs it didn't work.

bundle: name: event_log_dlt resources: pipelines: event_log_pipeline: name: event_log_pipeline libraries: - notebook: path: ./test.ipynb schema: default development: true catalog: retail_data serverless: true event_log: name: event_log_test schema: default catalog: retail_data

Though I found out the same is available in databricks SDK as well. Try it from that It will work and will be more secure

https://databricks-sdk-py.readthedocs.io/en/latest/dbdataclasses/pipelines.html#databricks.sdk.service.pipelines.PipelineSpec

topic Creating event_log for dlt pipeline using dabs in Data Engineering

Creating event_log for dlt pipeline using dabs

Re: Creating event_log for dlt pipeline using dabs