Save as Delta file in catalog

Hritik_Moon
Contributor II

Hello, I have created data frame on csv file when I try to write it as:

df_op_clean.write.format("delta").save("/Volumes/optimisation/trial")
I get this error :
Cannot access the UC Volume path from this location. Path was /Volumes/optimisation/trial/_delta_log
 
I am able to save as table on the same location.

-werners-
Esteemed Contributor III

When writing delta using UC (Volumes are part of UC) it expects a table.
So in your case you mix the two, you want to use Volumes to write files but UC expect a table when using delta.
Either save as a table or as a file (but not in delta format or outside of UC).
Exactly what you have tried already.

View solution in original post

-werners-
Esteemed Contributor III

Also to add on this:
avoid overlap between tables and Volumes.
Create a separate folder for tables and files.
Unity catalog does this too if you use managed tables/volumes.