topic Re: deltaformat in Data Engineering

deltaformat

SwathiChidurala — Sun, 26 Jan 2025 01:28:38 GMT

Hi,

I am a student who learning databricks, In the below code I tried to write data in delta format to a gold layer. I authenticated using the service principle method to read, write and execute data , I assigned the storage blob contributor role, but still, I am having the following error the location and the folder in the container everything is right. Not sure what causing the error. Any suggestions and solutions is greatly appreciated!

silver = 'abfss://silver@synapsecoursedlstorag.dfs.core.windows.net'
gold = "abfss://gold@synapsecoursedlstorag.dfs.core.windows.net"

df_zone.write.format('delta').mode('append').option('path',f'{gold}/trip_zone').saveAsTable('gold.trip_zone')

Re: deltaformat

Takuya-Omi — Sun, 26 Jan 2025 11:50:54 GMT

@SwathiChidurala

Have you ensured that the external location was created correctly? Also, do you have the appropriate permissions to manage it?
You may find the following documentation helpful:
External Locations in Unity Catalog

Although the scenario is slightly different, I previously addressed a similar error in the following thread. I hope it can also be of help:
Community Thread on External Delta Table Error

Re: deltaformat

Avinash_Narala — Mon, 27 Jan 2025 06:19:11 GMT

Hi @SwathiChidurala ,

The error is because you don't have the folder trip_zone inside the gold folder, so you can try by removing the trip_zone from the location or adding the folder trip_zone inside the gold folder in adls and then try it again.

If this helps, mark it as solution.

Regards,

Avinash N