topic Writing append blob files to unity catalog volum in Community Articles

Writing append blob files to unity catalog volum

Thusharr — Thu, 26 Sep 2024 06:57:25 GMT

The workspace is assigned to unity catalog, and all the access to the ADLS Gen2 is now handled via unity catalog only, means no SPN, no connection string, access keys etc. I have to create append blob files in a volume, Is this is possible in a workspace where unity catalog is enabled?

Re: Writing append blob files to unity catalog volum

gchandra — Mon, 30 Sep 2024 03:42:59 GMT

As long as you have necessary access to the catalog/schema / volume you should be able to write data to a Volume.

Re: Writing append blob files to unity catalog volum

Thusharr — Mon, 30 Sep 2024 07:01:57 GMT

I have access and I can write the files from a databrciks notebook to the volumes, but my ask is how can I create an append blob file? Earlier (before unity catalog) we used the libraries like azure-identity and azure-storage-blob but which is using the SPN and secrtes. But the usage of SPN is denied in by business case after the unity catalog.

Re: Writing append blob files to unity catalog volum

Witold — Mon, 30 Sep 2024 08:06:47 GMT

It makes sense that you learn how the security model of Unity Catalog works, including long-term storage credentials, external locations, etc: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/#other-securable-objects

Re: Writing append blob files to unity catalog volum

Thusharr — Mon, 30 Sep 2024 09:29:20 GMT

I didn't get you, sorry. My question is simple, I need to create an append blob file where the Unity catalog is enabled. This was possible earlier, but now it is not working.

Re: Writing append blob files to unity catalog volum

Witold — Mon, 30 Sep 2024 09:48:05 GMT

Now I got your point. No, you can't create Append Blob files directly in Volumes, as this is a native Azure functionality. A volume is basically just an abstraction over a native storage.

You will still need to use libraries like azure-storage-blob with a proper authentication to your storage.