topic Not readable format of event_logs traceback in dlt pipeline in Data Engineering

Not readable format of event_logs traceback in dlt pipeline

ashraf1395 — Thu, 20 Feb 2025 09:40:27 GMT

This is my dlt pipeline event_log - why is it not in readable foramt how can i correct it.

This is my pipeline code :

import logging logger = logging.getLogger(__name__) logger.info("Error") raise "Error is error"

Re: Not readable format of event_logs traceback in dlt pipeline

ashraf1395 — Wed, 12 Mar 2025 05:38:08 GMT

Got the solution to this error,
It was because of multiple loggers initialized and all those handlers were not getting cleard - Databricks dlt pipeline handles, my custom code logs etc.

Once I added this command

# Remove existing handlers to avoid duplicates

if logger.hasHandlers():

logger.handlers.clear()

the format of the code got correct.

One more thing I found out was, this unknown formats were nothing but colour codes which we see in the dlt pipeline event logs. Since the handlers was not getting cleared when it was going in my custom code logs it was also trying to add colour code formats there which was causing trouble once I added clear handles before initializing it in my code, It worked

Re: Not readable format of event_logs traceback in dlt pipeline

jorperort — Sun, 30 Mar 2025 11:56:56 GMT

Hi @ashraf1395 ,

I'm working with Delta Live Tables (DLT) and the event_log table. I would like to know if it is possible to access the event handler that DLT uses to write custom logs and send them to this table when events are published.

If this is not possible, do you have any alternative ideas? For example:

Creating a custom handler with the necessary logic.
Inserting records directly into the event_log table.