Converting utf-8 is not reflecting panda data frame to workspace for csv file?

[SivaPK]

Hi,

I would like to convert the specific column into utf-8 for all the country languages. After converting into and writing into workspace and download it to my local system and opening the excel file still other country character are not displayed properly. What i am missing here?

# Encode the "term" column to UTF-8

df_pandas['term'] = df_pandas['term'].apply(lambda x: x.encode('utf-8').decode('utf-8') if isinstance(x, str) else x)

%python

# Define the output path in DBFS

output_path = "./Files/output_utf_8.csv"

# Save DataFrame as CSV with UTF-8 encoding

df_pandas.to_csv(output_path, encoding='utf-8', index=False)

#df_pandas.to_csv(output_path)

print(f"File saved at: {output_path}")

Thank you.

[filipniziol]

Hi @SivaPK ,

It may happen that the CSV is fine, but Excel often does not automatically recognize CSV files as UTF-8 (it might assume ANSI or Windows-1252). This is why characters can appear incorrect.

Instead, open the CSV in a text editor like Notepad++ or VS Code. Then manually set the encoding to UTF-8 to confirm the characters display correctly.