cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

PermissionError / Operation not Permitted with Files-in-Repos

jfarmer
New Contributor II

I've been running a notebook using files-in-repo. Previously this has worked fine. I'm unsure what's changed (I was testing integration with DCS on older runtimes, but don't think I made any persistent changes)--but now it's throwing an error (always a variant of "Operation not permitted" or "PermissionError'). I'm unable to access a requirements.txt file or load modules in my repo.

As an example, I'll use the databrick files-in-repo notebook. Loading a module:

imageLoading a requirements.txt:

imageSome other information:

  • Running on a single-node cluster with DBR 12.0 and no DCS
  • Files-in-Repo is enabled for DBR 8.4+
  • The Python PWD is my repository path, and the and os path include my repository path.

I'm new to DB; is there something obvious I'm overlooking here?

3 REPLIES 3

Hubert-Dudek
Esteemed Contributor III

In my opinion, requirements.txt is just in another folder. So please save the requirements file in dbfs and put the full path. And also magic command pip has to be prefixed with a % sign.

%pip install -r /dbfs/requirements.txt

Appreciate the advice, but using an absolute path and using % doesn't fix the problem.

It's almost like DB thinks files-in-repos is not enabled for this cluster.

_carleto_
New Contributor II

Hi @jfarmer , did you solved this issue? I'm having exactly the same challenge.
Thanks!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group