Why am I getting NameError name _all_timezones_unchecked' is not defined

[AgusBudianto]

I defined the following local time function get datetime:

def get_sysdate():

    jkt_tz = pytz.timezone('Asia/Jakarta')

    sysdate = datetime.now(jkt_tz).strftime('%Y-%m-%d %H:%M:%S')

    return sysdate

spark.udf.register("get_sysdate", get_sysdate)

But sometimes I get the error "_all_timezones_unchecked' is not defined," and the next run works fine.

[Khaja_Zaffer]

Hello @AgusBudianto 

can you this and check once? I got it from GPT

ensure the pytz library and its timezone data are fully initialized on every worker node before the UDF is called.

import pytz
from datetime import datetime

# Force pytz to load all timezones on the driver and workers
# This is a key step to prevent the race condition
pytz.timezone('UTC')

def get_sysdate():
jkt_tz = pytz.timezone('Asia/Jakarta')
sysdate = datetime.now(jkt_tz).strftime('%Y-%m-%d %H:%M:%S')
return sysdate

spark.udf.register("get_sysdate", get_sysdate)

I did run on my workspace and got these results. 

[AgusBudianto]

@Khaja_Zaffer, thank you for being willing to reply to my post I have initiated the function on the 2nd cell, and this problem only appears occasionally, and I am using General Compute

error

in function I have using library:

from datetime import datetime, timedelta

import pytz

Thank's

Anto

[Khaja_Zaffer]

Hello @AgusBudianto 

Thank you so much for your response. 

Can you try explicitly import inside the UDF, lets see if the error still persisting.