cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Not readable format of event_logs traceback in dlt pipeline

ashraf1395
Valued Contributor III

This is my dlt pipeline event_log - why is it not in readable foramt how can i correct it.

ashraf1395_0-1740044260033.png

This is my pipeline code :

 

 

 

import logging

logger = logging.getLogger(__name__)

logger.info("Error")
raise "Error is error"

 

 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

ashraf1395
Valued Contributor III

Got the solution to this error, 
It was because of multiple loggers initialized and all those handlers were not getting cleard - Databricks dlt pipeline handles, my custom code logs etc. 

Once I added this command 

  # Remove existing handlers to avoid duplicates
    if logger.hasHandlers():
        logger.handlers.clear()

the format of the code got correct. 

One more thing I found out was, this unknown formats were nothing but colour codes which we see in the dlt pipeline event logs. Since the handlers was not getting cleared when it was going in my custom code logs it was also trying to add colour code formats there which was causing trouble once I added clear handles before initializing it in my code, It worked

View solution in original post

1 REPLY 1

ashraf1395
Valued Contributor III

Got the solution to this error, 
It was because of multiple loggers initialized and all those handlers were not getting cleard - Databricks dlt pipeline handles, my custom code logs etc. 

Once I added this command 

  # Remove existing handlers to avoid duplicates
    if logger.hasHandlers():
        logger.handlers.clear()

the format of the code got correct. 

One more thing I found out was, this unknown formats were nothing but colour codes which we see in the dlt pipeline event logs. Since the handlers was not getting cleared when it was going in my custom code logs it was also trying to add colour code formats there which was causing trouble once I added clear handles before initializing it in my code, It worked

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group