cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Can't run .py file using workflows anymore

UdayPatel
New Contributor III

Dear all,

Greetings!

I have been trying to run a workflow job which runs successfully when a task is created using a Notebook file from a folder present in the Workspace but when the same task's type is changed to python script and a .py file is selected from the same folder, the workflow fails with the Resource Not Found error.  Both the files (notebook and .py) uses exactly same lines of code.

I have checked all the access rights from Workspace to File level, User and Groups, Cluster and Job level etc etc but can't find a single option to stop the job to execute a .py file but notebook at any level. The job is being run by a user which has full admin rights on the entire workspace.

Any help will be greatly appreciated!

Many thanks in advance!

Regards,

Uday

1 ACCEPTED SOLUTION

Accepted Solutions

UdayPatel
New Contributor III

Hi,

Have found the solution. It was due to following option being enabled under the Feature Enablement tab under  Databricks_Account_Console -- > Settings.

 

UdayPatel_0-1713796507608.png

 

Thank you for all your help and the try!

Regards,

Uday

View solution in original post

5 REPLIES 5

AmanSehgal
Honored Contributor III

I just did a test and it worked.

Is it possible for you to provide a sample of your notebook code and python code.

Also, if you could please share your JOB details (JSON format should do)

UdayPatel
New Contributor III

Hi Aman,

Thank you for your reply.

PFA the json, yaml and the error message with full details of the code etc. 

 

Regards,

Uday

AmanSehgal
Honored Contributor III

Hi @UdayPatel 

Thank you for sharing the job details. 
I tried the same approach like running a notebook and then a python file. It worked for me successfully.

At python file level, could you please check the permissions and see if the user running the script has 'Can Manage' privilege?

nb_permission.jpg

UdayPatel
New Contributor III

One update to this, when I removed the user from Admin group, the workflow could successfully run the .py file but when I re-added the user to Admin group, it started giving same error.. 😞 

In summary, a .py file owned by the user having Admin rights are not getting executed via Workflow but works fine when the admin rights are removed for that user.

UdayPatel
New Contributor III

Hi,

Have found the solution. It was due to following option being enabled under the Feature Enablement tab under  Databricks_Account_Console -- > Settings.

 

UdayPatel_0-1713796507608.png

 

Thank you for all your help and the try!

Regards,

Uday

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group