topic Working with Pandas through Abfss in Data Engineering

Working with Pandas through Abfss

pradeepvatsvk — Thu, 23 Jan 2025 06:34:53 GMT

Hi,

I am unable to read and write pandas datframe through abfss protocol , Is there a work around for this , I do not want to store my files in dbfs

szymon_dybczak — Thu, 23 Jan 2025 07:59:02 GMT

It's well known limitation. The pandas dataframe does not support that protocol. Below you can find some workarounds:

Avinash_Narala — Thu, 23 Jan 2025 08:37:48 GMT

you can use volumes and mount the abfss location to unity catalog and can access the files present in azure in databricks.

Regards,

Avinash N