Privileges

Tommabip
Databricks Partner

I'm working out a terraform script that  creates a job that needs to write files in the volume.

When the job runs as me (I have full privileges), it encounters the following error if running with job compute:  Permission error: operation not permitted

I managed to skip the error using serverless cluster on the same job but I need the job cluster for this task.
I succesfully run the notebook if I run it without the job, both using a serverless and interactive cluster

Tommabip
Databricks Partner

I think I found the solution, you need to specify the data_security_mode parameter as SINGLE_USER to grant access to the Unity Catalog

View solution in original post