Hello @AgusBudianto 

can you this and check once? I got it from GPT

ensure the pytz library and its timezone data are fully initialized on every worker node before the UDF is called.

import pytz
from datetime import datetime

# Force pytz to load all timezones on the driver and workers
# This is a key step to prevent the race condition
pytz.timezone('UTC')

def get_sysdate():
jkt_tz = pytz.timezone('Asia/Jakarta')
sysdate = datetime.now(jkt_tz).strftime('%Y-%m-%d %H:%M:%S')
return sysdate

spark.udf.register("get_sysdate", get_sysdate)

I did run on my workspace and got these results. 

@Khaja_Zaffer, thank you for being willing to reply to my post I have initiated the function on the 2nd cell, and this problem only appears occasionally, and I am using General Compute

error

in function I have using library:

from datetime import datetime, timedelta

Thank's

Anto

Hello @AgusBudianto 

Thank you so much for your response. 

Can you try explicitly import inside the UDF, lets see if the error still persisting. 

Sorry I'm just replying, I think this might be due to the choice of compute specifications but I'm not sure about this