topic Re: PermissionError / Operation not Permitted with Files-in-Repos in Data Engineering

PermissionError / Operation not Permitted with Files-in-Repos

jfarmer — Sat, 21 Jan 2023 19:18:59 GMT

I've been running a notebook using files-in-repo. Previously this has worked fine. I'm unsure what's changed (I was testing integration with DCS on older runtimes, but don't think I made any persistent changes)--but now it's throwing an error (always a variant of "Operation not permitted" or "PermissionError'). I'm unable to access a requirements.txt file or load modules in my repo.

As an example, I'll use the databrick files-in-repo notebook. Loading a module:

Loading a requirements.txt:

Some other information:

Running on a single-node cluster with DBR 12.0 and no DCS
Files-in-Repo is enabled for DBR 8.4+
The Python PWD is my repository path, and the and os path include my repository path.

I'm new to DB; is there something obvious I'm overlooking here?

Re: PermissionError / Operation not Permitted with Files-in-Repos

Hubert-Dudek — Mon, 23 Jan 2023 13:28:44 GMT

In my opinion, requirements.txt is just in another folder. So please save the requirements file in dbfs and put the full path. And also magic command pip has to be prefixed with a % sign.

%pip install -r /dbfs/requirements.txt

Re: PermissionError / Operation not Permitted with Files-in-Repos

jfarmer — Mon, 23 Jan 2023 15:05:31 GMT

Appreciate the advice, but using an absolute path and using % doesn't fix the problem.

It's almost like DB thinks files-in-repos is not enabled for this cluster.

Re: PermissionError / Operation not Permitted with Files-in-Repos

_carleto_ — Tue, 17 Oct 2023 14:14:16 GMT

Hi @jfarmer , did you solved this issue? I'm having exactly the same challenge.
Thanks!